Some new notes about AppleTV

Another braindump so I can actually put in public what I’ve been doing in my spare time lately, given that most likely a lot of that won’t continue in the next months, as I’m trying to find more stable, solid jobs than what I’ve been doing as of lately.

If you follow me for a long time you might remember that a few years ago I bought an AppleTV (don’t laugh) for two main reasons: because I actually wanted something in my bedroom to watch Anime and listen to music and was curious about the implementation of it from a multimedia geek point of view. Now, a lot of what I have seen with the AppleTV is negative, and I’m pretty sure Apple noticed it just as well as I have. Indeed they learn from a lot of their previous mistakes with the release of the new AppleTV. Some of the “mistakes they learnt from” would probably not be shared by Free Software activists and hackers, as they were designed to keep people out of their platform, but that’s beside the point now.

The obvious problems (bulkiness, heat, power) were mostly fixed in hardware by moving from a mobile i686-class CPU to an ARM-based embedded system; the main way around their locks (the fact that the USB port is a standard host one, not a gadget one, and it only gets disabled by the lack of the Darwin kernel driver for USB) is also dropped, but only to be replaced with the same jailbreak situation they ahve on iPhone and other devices. So basically while they tried to make things lot more difficult, the result is simply that they hacked it in a different way. While it definitely looks sleeker to keep near your TV, I’m not sure I would have bought it if it was released the first time around this way.

At any rate, the one I have here is in its last months, and as soon as I can find something that fits into its space and on which I can run XBMC (fetching videos out of a Samba share on Yamato), I’ll probably simply get rid of it, or sell it to some poor fellow who can’t be bothered with getting something trickier but more useful. But while I want the device to actually accept the data as I have it already for what concerns Anime and TV series (assuming I can’t get them under decent terms legally), some months ago I decided that at least the music can bend over to the Apple formats — for the simple reason that they are quite reasonable, as long as I can play them just fine in Europe.

Beside a number of original music CDs (Metal music isn’t really flattered by the compression most online music stores apply!), I also have (fewer) music DVDs with videos and concerts; plus I sometime “procure” myself Japanese music videos that haven’t been published in the western world (I’m pretty much a lover of the genre, but they don’t make it too easy to get much of it here; I have almost all of Hikaru Utada’s discography in original forms though). For the formers, Handbrake (on OS X) did a pretty good job, but for the new music videos, which are usually in High Definition, it did a pretty bad job.

Let’s cue back FFmpeg, which, since last time I ranted actually gained a support for the mov/mp4 format that is finally able to keep up with Apple (I have reported some of the problems about it myself, so while I didn’t have a direct bearing in getting it to work, I can say that at least I feel more confident of what it does now). To be honest, I have tried doing the conversion with FFmpeg a few times already; main problem was to find a proper preset for x264 that didn’t enable features that AppleTV failed to work with (funnily enough, since Handbrake also uses x264, I know that sometimes even though iTunes does allow the files to be copied over the AppleTV, they don’t play straight). Well, this time I was able to find the correct preset on the AwkwardTV wiki so after saving it to ~/.ffmpeg/libx264-appletv.ffpreset the conversion process seemed almost immediate.

A few tests afterward, I can tell it’s neither immediate in procedure, nor in time required to complete. First of all, iTunes enforces a frame size limits on the file; while this is never a problem for content in standard definition, like the stuff I ripped from my DVDs, this can be a problem with High-Definition content. So I wrote a simple script (that I have pasted online earlier tonight but I’ll publish once polished a bit more) that through ffprobe, grep and awk could maintain the correct aspect ratio of the original file but resize it to a frame size that AppleTV is compatible with (720p maximum). This worked file for a few videoclips, but then it started to fail again.

Turns out, 1280×720 which is the 720p “HD Ready” resolution, is too much for AppleTV. Indeed, if you use those parameters to encode a video, iTunes will refuse to sync it over to the device. A quick check around pointed me at a possible reasoning/solution. Turns out that while all the video files have a Source Aspect-Ratio of 16:9, their Pixel Aspect-Ratio is sometimes 1:1, sometimes 4:3 (see Wikipedia’s Anamorphic Widescreen article for the details and links to the description of SAR and PAR). While Bluray and most other Full HD systems are designed to work fine with a 1:1 PAR (non-anamorphic), AppleTV doesn’t, probably because it’s just HD Ready.

So a quick way to get the AppleTV to accept the content is simply to scale it back to anamorphic widescreen, and you’re done. Unfortunately that doesn’t seem to cut it just yet; I have at least one video that doesn’t work even though the size is the same as before. Plus another has 5.1 (6-channels) audio, and FFmpeg seems to be unable to scale it back to stereo (from and to AAC).

I bought a software license

I finally decided to convert my video library to Mpeg4 containers, H.264 video and AAC audio, rather than mixing and matching what I had before that. This is due to the fact that I hardly use Enterprise to watch video anymore. Not only because my office is tremendously hot during the summer, but more because I have a 32” TV set in my bedroom. Nicer to use.

Attached to that TV set there is an Apple TV (running unmodified software 2.0.2 at the moment) and a PS3. If you add to that all the other hardware that can play video I own, the only common denominator is H.264/AAC in MP4 container. (I also said before that I like the MP4 format more than AVI or Ogg). It might be because I do have a few Apple products (iPod and AppleTV), but also Linux handle this format pretty well, so I don’t feel bad about the choice. Beside, new content I get from youtube (like videos from Beppe Grillo’s blog) are also in this format — you can get them with youtube-dl -b.

Unfortunately, as I discussed before with Joshua, and as I tried last year before the hospital already, converting video to this format with Linux is a bit of a mess. While mencoder has very good results for the audio/video stream conversions, producing a good MP4 container is a big issue. I tried fixing a few corner cases in FFmpeg before, but it’s a real mess to produce a file that QuickTime (thus iTunes, and thus the Apple TV) can accept.

After spending a couple more days on the issue I decided my time is worth more than what I’ve been doing, and finally gave up to buy a tool that I have been told does the job, VisualHub for OSX. It was less than €20, and that is usually what I’m paid by the hour for my boring jobs.

I got the software, tried it out, the result was nice. Video and audio quality on par with mencoder’s but a properly working MP4 container that QuickTime, iTunes, AppleTV, iPod and even more importantly xine can play nicely. But the log showed a reference to “libavutil”, which is FFmpeg. Did I just pay for Free Software?

I looked at the Bundle, it includes a COPYING.txt file which is, as you might have already suspected, the text of GPL version 2. Okay, so there is free software in here indeed. And I can see a lot of well-known command line utilities: lsdvd, mkisofs, and so on. One nice thing to see is, though, an FFmpeg SVN diff. A little hidden, but it’s there. Good.

The doubt then was if they were hiding the stuff or if it was shown and I did just miss it. Plus it has to have the sources of everything, not just a diff of FFmpeg’s. And indeed in the last page of the documentation provided there is a link to this that contains all the sources of the Free software used. Which is actually quite a lot. They didn’t limit themselves to take the software as it is though, I see at least some patches to taglib that I’d very much like to take a look to later — I’m not sharing confidential registered-users-only information by the way, the documentation is present in the downloadable package that acts as a demo too.

I thought about this a bit. They took a lot of Free Software, adapted it, written a frontend and sold licenses for it. Do I have a problem with this? My conclusion is that I don’t. While I would have preferred is they made it more clear on the webpage that they are selling a Free Software-based package, and that they would have made the frontend Free Software too, I think they are not doing anything evil with this. They are playing well by the rules, and they are providing a working software.

They are not trying to exploit Free Software without giving anything back (the sources are there) and they did something more than just package Free Software together, they tested and prepared presets to use for encoding for various targets, included Apple TV which is my main target. They are, to an extent, selling a service (their testing and presets choices), and their license is also quite acceptable to me (it’s like a family license, usable on all the household’s computers as well as a work computer in an eventual office).

At the end of the day, I’m happy of spending this money as I suppose it’s also going to further develop the Free Software part of the software too, although I would have been happier to chip in a bit more if it was fully Free Software.

And most importantly, it worked out of the tarball solving me a problem I was having for more than an year now. Which means, for me, a lot less time spent trying to get the whole thing working. Of course if one day I could just do everything with simply FFmpeg I’ll be very happy, and I’ll dedicate myself a bit more on MP4 container support, both in writing and parsing, in the future, but at least now I can just feed it the stuff I need converted and dedicate my time and energy toward more useful goals (for me, as in paid jobs, and for the users with Gentoo).

Audio conversion flies!

After a week spent closing envelopes and attaching labels, I was finally able to get the audio conversion branch of xine-lib 1.2 to play mp3s.

As libmad decodes the mp3s as 32-bit signed integer samples, the direct output cannot be sent directly to PulseAudio when playing it back (and PulseAudio is the only audio output I converted to the new plugin API), so I had to actually implement at least some basic conversion, in this case from almost every integer format to 32-bit floating point samples (which is the simplest conversion, although it’s CPU consuming.

Previously every mp3 file decoded with libmad was truncated to 16-bit signed integer samples, losing precision, and for what I can see amplifying the volume quite a bit, distorting the signal, but with my new code, there’s no precision loss, and the difference is audible.

You can see the difference in the following screenshot:

If you look at the screenshot you will see the stream info dialog windows of xine-ui: on the left there’s the window for xine-lib 1.1 while on the right there’s the one for xine-lib 1.2 audio_conversion branch. As you can see at the bottom of the window, the first one reports 16-bit for the stream, while the new one returns 32-bit as intended.

I can tell you, the difference is great, and this is just the first step! Hopefully it will be merged in xine-lib-1.2 and then it will be able to play hi-fidelity wave files just fine.

Unfortunately before I can actually merge this branch into main xine-lib-1.2, all the output plugins, the post plugins and most of the audio decoding plugins have to be ported, which is a HUGE amount of work for just one person. I’d also require to use some virtual machines to confirm most of the output plugins work afterward, so it will take me a lot of time alone. If you can handle converting and testing the code, that will certainly be appreciated.