GStreamer is the future?

Hi, with today's news I though I'd share some sentiments I have about the future (and partly present) of multimedia (meaning only video and audio in this post) in linux.

First take a look of what we have now. We have two major libraries written to do the job and another two players doing the same, but with their own libraries integrated. I am talking about gstreamer, xine-lib, mplayer and vlc.

Let me state I was originally a huge fan of xine-lib but during some of the recent Fedoras (namely Werewolf and Sulphur) I slowly dissolved from it for various reasons I explain later. During the same time I became a frequent user of gstreaer and mplayer. I haven't ever been a fan of vlc and I don't like it's ui, but that's not what I'd like to talk about in this post.

So, with linux expanding to desktop machines we need to have also some good way of playing various multimedia ranging from youtube videos through full HD mpeg4-AVC/h264 to our favourite songs in mp3. Or we'd like to share our screencasts in theora, back-up favourite recordings saved in FLAC or burn an audio CD out of our vorbis encoded music collection. And to add to this lot something more - there are zillions of various containers from aging AVI, through "home-made" OGG or playstation 3 compatible MP4 to modern "I can do anything" matroska.

To be able to keep up with the world we need a solid framework that has good enough code, and is easilly expandable to contain new codecs. That's where mplayer or vlc fail - you cannot build up a serious app using their libraries (since they are shipped together with the player), you can just wrap up the player into nicer gui. Also the pace of adoption of new codecs is rather slow (as I'll explain later on some examples).

This is where gstreamer and xine-lib comes. Xine-lib is pretty interesting library, since it has similar capabilities to mplayer it's one of the "great three" of linux video players - mplayer, vlc and xine (which is official frontend to xine-lib). It has great amount of configurability, which makes it more geek-friendly, but everything has it's buts. The design of the library is not that great, as can be proved how much time it took before it could be reasonably split to free and nonfree (patented stuff) version, also it seems not be able to "keep pace" with the age (it took far too long before they get good enough pulseaudio audio output working), it seems to be more susceptible to crashes than gstreamer or mplayer and does not seem to be good on hardware (seem to need more power to play videos).

That, and the continual (and rather fast) improvement of gstreamer drove me away from xine-lib based applications (though I still keep gxine around, partly because I maintain gxine package for fedora). And why gstreamer? You *could* say it's rubbish, but cannot anymore. Yes, back in Fedora 6 it was the half-working default (implemented in totem). That it wasn't "good enough" was proved by most of users switching totem to xine-lib backend, installing xine, gxine or kaffeine, or going the mplayer/vlc way. Not anymore. Over the past few releases it wastly improved no only thanks to it's plugins system.

In gstreamer (as my understanding of the basics is) you have the basic framework and a lots of separately installable plugins (though batched together in logical packs) which can make it rock. Most likely you'll have installed only the base and good plugins which covers almost everything in the FLOSS codecs, containers, ... world. In addition to that, to be able to play more widely used content, you need to install bad and ugly plugins, or perhaps the ffmpeg plugin (ffmpeg is library for decoding/encoding using of the MPEG codecs and more). That makes the framework more robust, more configurable and also easiest to implement new codecs to.

One of the examples where you can find only a limited number of players being able to play it THEORA video in Matroska container. It's perfectly correct usage, adhers to both THEORA and Matroska specifications, but behold - xine-lib based players cannot play it, vlc cannot play it. And "rare combination" excuse does not stand a chance. It's working flawlessly with mplayer, it's working flawlessly with gstreamer - that's two working implementations already. GStreamer can even both encode theora video to matroska container and read it afterwards, mplayer (resp. mencoder) wasn't capable of muxing matroska last time I checked.

Now another example is adopting of brand new and partly "scientific research" codecs like Dirac or Schroedinger. Did you know that the only way how to encode video using schroedinger codec with fedora is using gstreamer? Did you know that the only way how to play Dirac/Schroedinger videos, be it either in OGG or Matroska, is to use gstreamer? Yes, there are patches for mplayer, xine-lib, ... but they're not in. Why? Because it's a patch and that means modifying source code and it needs to be accepted by the developers of xine-lib, resp. mplayer, first. In gstreamer though, the only thing you need to do is write a plugin - that means you do not need to wait till your code is accepted, it does not need to be, because it's standalone. It can be published and installed separately. And that is a great advantage.

And to add more fuel to the gstreamer camp, it seems there will be a way, thanks to gstreamer being plugin-ready, RPM being able to know what plugin handles what, PackageKit being able to install and depsolve all the needed plugins and plugin (yet-to-be-written) being able to tell PackageKit that gstreamer needs plugin to handle this or that, and given you have all necessary repositories installed (in Fedora case it's most likely livna, or rpmfusion in the future); the installation of codecs will be completely automatic. GStreamer sends a request to PackageKit to install missing plugin (more exactly to install something that provides the necessary functionality), PackageKit hopefully finds it in the repos (thanks to RPM being able to provide information about what content the plugin handles), resolve all the needed dependencies and tada - you can play the video.

I just can't wait to see this in actions!

The only thing that still keeps mplayer on my machine is its support for SSA/ASS styled subs. Not only it can parse it, but it can also render the advanced styling correctly. I wonder if there is a RFE for this in gstreamer bugzilla already, or whether I should move my butt and file one :-D

PS: I am too lazy to add references this time (partly due to me being ill), so use google to find what you'd like to read more about, e.g. googling for "dirac video" gives you on the first hit the homepage of the Dirac and Schroedinger codecs.