FFmpeg, AppleTV and conversions

Last year, a few months before ending up in the hospital, I bought a LCD TV and an AppleTV device, to watch my animes before sleeping, in my bedroom, relaxing down. After the hospital I ended up not watching much anime anyway, but before that I noticed one nasty thing: my LCD TV “eats” away around 16-20 pixel around the borders with the result that the subtitles for the japanese anime were unreadable. Which was very nasty in my opinion.

After AppleTV Take 2 software was released I also didn’t have much time to play with it to modify it again to install extra codecs, nor I had time to watch much anime, even though I bought a PlayStation 3 to relax. Now, I also considered the idea of selling away the AppleTV, getting a bigger harddrive for the PlayStation 3 and live with that. But since I have already spent enough time in the hospital, having mobile access to my anime is still a feature I’d like to keep, and since I can easily make AppleTV and iPod use the same video files, while PS3 and PSP (which I don’t own, my sister does though) would require different copies, I finally decided to keep it.

I have already tried converting video files but FFmpeg failed, while VisualDub let me do some of the conversions; unfortunately I never had the time to finish the conversion before I ended up in the hospital again. Geez, you really have some trouble when you can count eras based on when you ended up in the hospital. Anyway, since now I got a new box, I decided to try doing the conversion once again with FFmpeg, with all the improvements that arrived up to now.

Before trying that, though, I wanted to be able to do one thing: try to remove data duplication between my workstation (Yamato) and the laptop (Intrepid). My easy way out of this is to make sure that they use the same partition for music and video, which is what they share. Linux and OSX can share mainly two filesystems: FAT32 (nasty) and HFS+. The problem is that HFS+, Apple’s filesystem, can have multiple variants: it may or may not be case-sensitive, and it might be journaled. Linux cannot write to journaled HFS+, and I dislike case insensitivity, so I set up a HFS+ case-sensitive non-journaled partition beside the Time Machine partition on the external hard drive, and connect the drive to Yamato.

Moving the music and video files off Yamato’s drives also helped me to reduce the amount of data I have on the internal drives, which in turn should reduce at least a little the stress on them until I can get a new setup. And Linux HFS+ support is not that bad after all, once you disable journaling. The problem is that I wanted to make sure I could use the same disk both connected to Yamato with network attached, and directly connected to Intrepid. A tried way to do this was to export the partitions via iSCSI, but that would have given exclusive access to the partition to the laptop, and thus caused me once again not to be able to share the data.

The other solution was to export the filesystem via NFS when connected to Yamato, which is what I tried, unfortunately, as I’ve written before, Linux does not support NFS export of HFS+ partitions; or at least it doesn’t right now. I love Free Software just for this reason: if a feature does not exist, I make it up; now I just have to hope that it gets applied upstream in a decent timeframe.

Now that the filesystem problem is mostly behind me (mostly because I still see some failures; I don’t know exactly what they relates to yet), I wanted to make sure I could get a way to convert my content with FFmpeg directly; Yamato is an 8-core system so it should be faster than the laptop to do the work. So I got the latest FFmpeg version out of FFmpeg, and tried.

So the first issue is getting the right container format, while ISO Media, QuickTime, iPod video files, and so on are all based on more or less the same idea, to the point that xine can easily use a single demuxer for all of them, as well as libavformat, there are some tricky issues with having those files working on QuickTime, iTunes and AppleTV. Luckily, FFmpeg has an ipod preset that does the job, mostly, which is very good. The problem is the “mostly”: these formats are based upon elements called atoms, atoms can have versions; the common version 0 that is supported by Apple’s software has 32-bit values for timescales, offsets and stuff like that; alternatively there exist a version 1 that uses 64-bit values instead; xine supports both, but Apple’s software does not. The intersting bit here is that FFmpeg produces version 1 atoms whenever the time scale does not fit as it is in the 32-bit values, without warning even when asking for the iPod container format. The trick here is to force a timescale conversion; most of the content I got is encoded with approximately 29.97 fps, but using a timescale in AVI files that does not fit into version 0 atoms; changing the timescale to basically the same through -r 29.97 fixes the problem for me.

Now, time to get correct video and audio codecs. If you have ever tried to do this you know you have to use H.264 for video and AAC for audio; I used 800kbit/s for video and 128kbit/s for audio; just make sure you pass the parameters before the output filename, otherwise they won’t be picked up. To make sure to enable the proper features for the x264 encoder, I used the presets that are in FFmpeg’s repository; I chosen to skip the -max version since that was giving me less than one frame per second, which was very very bad, I used -hq instead, which gave me good enough results. The problem is the default version enables 8×8 DCT and B-Pyramids, which QuickTime does not properly support. Files encoded with these two features enabled were synced properly to the AppleTV but crashed it down once played. Not so nice. To fix this, just replace the + symbol with a - in front of the two features dct8x8 and bpyramids. Problem solved, and the files play.

For some reason, FFmpeg’s MS-Mpeg4 decoder dies when enabling more than 8 threads, and with so many threads, only about four cores get used on Yamato; I’m not sure why the limit is imposed by the video decoder, since I would expect at least a thread to be able to handle audio, but anyway, since that’s the deal, I decided to convert two files at a time, and to do that, well we have GNU make don’t we?

I wrote this Makefile:

VPATH = $(ORIG)

SRCS = $(notdir $(wildcard $(ORIG)/*.avi $(ORIG)/*.mp4))
all: $(patsubst %.mp4,%.m4v,$(patsubst %.avi,%.m4v,$(SRCS)))

%.m4v: $(filter %.avi,$(SRCS)) $(filter %.mp4,$(SRCS))
    ffmpeg -threads 8 -i lt; -vpre libx264-appletv-hq.ffpreset -b 800k -ab 128k -acodec libfaac -padcolor 000000 -padtop 16 -padbottom 16 -padleft 16 -padright 16 -r 29.97 -f ipod "$@"