Fedora 10 and Multimedia - Is It Worse or Better?

Well, a hard question. At first I got thrilled by some new features, especially the automatic codec install, and pretty stable pace of improvements on gstreamer front, but lately I'm starting to notice issues as well.

First let me say that while both video and audio handling in gstreamer got generally improved I recently came across a clip (video in h264 and dual audio FLAC and mp3 in matroska + karaoke using styled ASS subs) that fails misserably pretty everywhere. Totem plays the audio fine, but the video gets pretty quickly out of sync, in gst-launch the video even eventually stops altogether. Gxine does not support styled subs, but other than that it plays it pretty well, though it picks wrong audio track. Mplayer plays it well, but the subtitles are totally out of sync. And that's probably the biggest woe of recent mplayer builds (where recent spans even to some time before Fedora 10 GA) - slightly more complex styled (ASS) subs are frequently out of sync or not displayed at all (usually when there are more subtitles at once).

So I was looking for a workaround using different video player, but... VLC a xine-lib don't support styled subtitles and for gstreamer there is a bug filled, with experimental plugin attached (using libass). So I've created packages for libass (it's built-in in mplayer, but I needed a standalone library) and for the gstreamer plugin (called assrender) and installed it. Yep, totem does not know about the new plugin for subtitles rendering, but after some googling and trying I sort-of learned how to make gst-launcher use it.

And well, though gst-launcher is good for debugging it's not good for playback, so I wanted to make a simple one-purpose media player using gstreamer and ruby (just for the fun of it :) But it seems, ruby bindings for gstreamer are not exactly in a state that makes it possible at the moment...

The assrender plugin, though highly experimental and not official, works pretty well and even support using fonts that are attached in matroska files (that's a really useful and pretty widely used), but because it supports only RGB colorspace output there are two colorspace conversions needed (YUV to RGB and back) which consumes cpu cycles and on my not so hi-end laptop it makes HD (1280p) videos, that are just about what my CPU is capable of, to fail miserably. So dear lazy web, if you want not-so-hard to do coding for gstreamer, you can help out with bringing libass support there. It would remove the last obstacle which prevents me from using gstreamer for everything ;-)

But at the same time, I am a bit concerned about mplayer as well - because on the same hardware, since Fedora Core 6 it's playback capabilities are getting steadilly worse. Not that it cannot play videos, but mplayer tends to fail resuming of paused video, then there's the issue with subtitles and lastly I've experienced a very weird bug (and workarounded it for my own purposes) - there's one assertion in ffmpeg code that prevents playing of certain matroska videos in mplayer.

Basically someone obviously expected that if you have chapters in a movie, than they will be strictly chronological, meaning next chapter (in list) won't start before current one. In matroska this is not always true and the failed assertion makes mplayer crash... For those who are interested, it's in libavformat/utils.c in compute_chapters_end () somewhere around line 1950 or 1960... It's this simple line:

assert(s->chapters[i]->start <= s->chapters[i+1]->start);

I'm still in doubts whether I should fill a bug or not about it... In my opinion there should be just if conditional, assert seems to me like an overkill, especially given that not aborting the code here (if the assertion fails) does not make mplayer doing things it is not expected to do.

Making High Quality Screencasts

Ok, I think I have finally figured a good way how to make high quality screencasts. So I'll write here about it here, both so that I have a place to look to when I forgot and for my readers ;-)

Needed Software (Package Names)

gtk-recordmydesktop
oggvideotools
mkvtoolnix-gui
subtitleeditor (or any other subtitle editor)

All the software is available in Fedora.

Recording the screencast

First, it's good to prepare what you are going to show so that you can record the screencast in one go. When you have decided what are you gonna show and how, open Applications -> Sound & Video -> gtk-recordMyDesktop. In the opened window you see a preview of your desktop in which you can select the area of the desktop you are going to record. If you are going to record just one application it might be useful to let it select the are for you by selecting a window you are going to record.

Set Video Quality to 100 (you want to record the screencast in highest possible quality, don't you) and if you don't want to record sound (which I do not) uncheck the check box to the left of Sound Quality. By clicking Save As you can select which the filename (and directory) to which the screencast will be saved.

Now, when you set everything, hide the window either by clicking ctrl+H or right-clicking on "record" icon in your notify are and selecting hide. Then you can start recording by left-clicking the record icon in your notify are. The icon will change to "stop". When recording, when you right-click on it the recording will pause (and the icon will change to "pause"), when you right-click on it once more it will resume. When you'll left-click on it the recording will stop and encoding (unless you encode on the fly, which can be set in settings) will start.

Once the encoding is finished you can close the gtk-recordMyDesktop and extract usefull data from it (since the .ogv file generated by it contains some skeleton track it renders it almost impossible to work with). We'll do that using oggSplit. Just open terminal, navigate to the directory you have the .ogv file in and run oggSplit <filename>.ogv. It will extract the video (and audio) track(s) into usable files. The file with video track will be named theora_*.ogg. Rename it to your liking.

Creating Subtitle Commentary

Now it's time to create subtitle commentary. You can do it with your favourite subtitle editor, the one I chose is subtitleeditor. I'll not delve into how can you create subtitles because it's pretty straightforward. Just bear in mind that you need the subtitles kept on the screen long enough so that the readers will be able to read them and they should better explain what is happening in the screencast.

Creating the subtitle commentary is good practice mostly because:

• Deaf people can read it


• It's easier to translate than spoken word


• Text does not have "strange whatever accent"

Combining Things Together

Now the most important thing is to put things together in one file. Open Applications -> Sound & Video -> mkvmerge GUI. You'll probably need only the Input tabs but if you are creating some more sophisticated/long screencast the other tabs might come handy too. First add all source files (the ogg file with video, the file(s) with subtitle(s), file with audio, ...) - just click on Add and add anything you want.

In General Track Options you can set for each track various settings. For audio and subtitle tracks it is good practice to select a language it is in. It is also not a bad idea to add Track name to each track.

Finaly set output filename (ending with .mkv) and hit Start muxing and you are done.

Adding Translation

The very same way you put the things together in the previous step, you can also add other tracks to already finished Matroska file. But, in case you'd like to add new subtitle translation you'll probably need one of the subtitle file in the screencast first so that you won't have to do it all from the start.

First open Applications -> Sound & Video -> mkvinfo. In it open the screencast Matroska file you'd like to translate and look for tracks. If the Matroska file has subtitle track, then sooner or later you'll find a track which will have Track type: subtitles at ... See whats Track number of that track. Suppose it's 2. Now open terminal and run mkvextract tracks <filename>.mkv 2:<subtitle-filename>. It might look like this:

$ mkvextract tracks echo-add-new-icon.mkv 2:echo-add-new-icon.ass
Extracting track 2 with the CodecID 'S_TEXT/ASS' to the file 'echo-add-new-icon.ass'. Container format: SSA/ASS text subtitles
progress: 100%


Now translate the subtitles using your favourite subtitle editor save them into some file and add it to the original screencast using mkvmerge GUI tool.

Some Screencasts Created This Way

Originally I researched how to do high quality screencasts because I needed to create screencasts accompanying various Echo tutorials.

Here are some:
echo-set-up.mkv
echo-update.mkv
echo-add-new-icon.mkv

Links

recordMyDesktop Homepage
oggvideotools Homepage
mkvtoolnix Homepage
Matroska Homepage
Theora Homepage
New high-quality totaly open (as in freedom) video compression format Dirac and its Schrodinger implementation (available in Fedora repositories)
Wikipedia article about Matroska container, OGG container, Theora video, Free Lossless Audio Codec (FLAC), Vorbis audio, Dirac/Schrodinger video.

oggvideotools - Thanks for That Package!

In my previous post I outlined some ways that could lead to good screen-cast. One of the biggest issues I have had have been that the gtk-recordMyDesktop output contained skeleton track which rendered that file nearly uneditable. I tried various magic with mencoder to be finally able to extract the theora video into something other, something that can be processed by mkvmerge. It could not be however played by totem. And the same was with the result mkv.

But not anymore. Recently, when checking mail, I noticed in fedora-extras-commits list interesting package - oggvideotools. And now, at the end of my wits, I remembered it, looked into the archives to find that it's actually a new package and have been built on Tuesday. So I downloaded the package from koji (I didn't checked whether it's already in repos), installed it and looked on the content. One of the binaries caught my attention: oggSplit. I wondered how it's used, but as traditional -help or --help or man didn't worked and the program homepage showed only what it does but not how it does, I tried feeding it one of the ogv files.

The result is rather pleasing. It generated two ogg files, one named theora-[some-probably-hex-number].ogg and named unknown-[another-number].ogg. And voila, the theora-*.ogg one works, mkvmerge with it works, ... COOOOL. The unknown-*.ogg probably contains the skeleton track and I'll delete it right away. It's also probably result of this oggSplit output:
unknown type ID 325334135

So, I can safely say that I can do a high-quality screen-cast in matroska file with theora video, now :-)

Making Good Screen-casts - How To?

I am gradually making some screen-casts for echo tutorials and I wonder what is the best way to make them. There is a lot of options and one needs to choose... What I've done so far is available on my Fedora People page.

First thing to choose is the application that will do the actual screencast. From rough investigation I discovered these three that are available in fedora/livna:

• instanbul - easy to use, however almost non-existant configuration, output compressed terribly much, which makes the video small but ugly
• xvidcap - configuration options are pretty good, you can customize the output as much as you want, however tend to freeze during captioning, also does not support encoding to theora
• gtk-recordMyDesktop - seems to support output only to ogv/theora/vorbis, not very well handled area selection, adds skeleton track to the output, usage not very intuitive

So after some thinking I dismissed instanbul, as the output does not have the desired quality (and I cannot do anything about that) and xvidcap due to frequent freezes (before that I managed to record one 3 min long screencast though). That leaves gtk-recordMyDesktop.

Next, we need to decide about the content - will we add commentary? If yes, will it be spoken or via subtitles? Subtitles are very good for translating, you just use the original one and use e.g. subtitleeditor (in fedora repos) to make a translation without changing the subtitles timing/styling. Also it's good for deaf people. On the other hand, you need to show the subtitles somewhere, which might render the part of screen on which they are shown unreadable. You can minimize the costs by using styled subs (e.g. ASS/SSA) to position the subtitles, but you'll then need special programs that can handle the styling (e.g. mplayer with -ass option).

Audio has the drawback that it can be hard to understand, for the person doing the screencast harder to create (not all of us are good English speakers), also it's harder to make a translation. So I guess subtitles are overally a better choice.

Now what format choose for video? If I had a choice, I'd go with MPEG-4/AVC (H264) which is excelent format with probably the best quality to compression ratio (and is pretty much used for most of HDTV content). It has however a drawback of being patent encumbered which renders it not an option for videos targeted on all Fedora users (even though you can play it in Fedora if you install additional codec packages from livna, but that's not an official Fedora repository). Software patents also put out of the game most of the other codecs, which leaves Theora. It has pretty good quality to compression ratio and is already the output of the gtk-recordMyDesktop, so if you are content with the compression, you don't need to recompress/re-encode the video.

Lastly we need to choose a container format. In opensource world the best choices are probably OGG/OGM/MKV (listed in no order). I don't know if OGG can handle subtitle tracks and OGM seems to be rather hack of OGG to better support videos. It's not much widely used either. On the contrary Matroska Video (mkv) is designed to be portable, fast, open, ... It can contain almost anything which renders it almost ideal container for shipping videos (it can contain multiple video, audio, subtitle tracks, fonts, various other files..., and yet it's generally very fast). It also comes with handy tools that makes adding new subtitle or audio track to existing video a piece of cake [mkvmerge(-gui)].

So what's my choice? It's obvious from the previous text:

• gtk-recordMyDesktop
• Theora
• ASS/SSA subtitles
• Matroska Video (mkv)
  
• (optionally Vorbis audio)

What do you think? Do I have even better choices?