Sometimes I think I’m wasting my time

Let’s start with a service communication: this morning I was able to order the V1075 phone on Vodafone’s e-Shop, it’s currently traveling to my home, yuppie 🙂

Now, on a more related note, this is half a rant, so I already say I’m sorry, but I feel this way tonight and I want to express it at my best.

As I told before, I was working for a while now on getting visibility support in FFmpeg, which allows to reduce the number of exported symbols, resulting in a faster loadtime for xine and similar.

Unfortunately, I hit a big obstacle in submitting these patches upstream, as many of the FFmpeg developers seems convinced that it’s just “GNU crap” that was invented to corrupt ELF format and so on. Well, a part the fact that ELF is designed to be extensible, so I think that everyone who tries to improve it is in the right, I don’t think visibility is crap (well, its implementation on GCC < 4.1 was crap, but not the theory itself).

Another “interesting” argument was that dynamic linking is slow by itself so it shouldn’t be improved and instead static linking should always be preferred. Let me discuss this on a security and simplicity standpoint: think of having to rebuild xine-lib, VLC, alsa-plugins, mt-daapd and any other program linking to FFmpeg every time that a security issue is discovered (that happens quite often lately) or you want to change an useflag of FFmpeg… now you can see where it’s useful to have dynamic linking. It’s slower, okay, but it’s more practical.

Anyway, the bottom line is that I’m asked to explain and defend my choice of working on those patches to see if they are worth merging or not, because they seems to be too invasive (noted, I said they were, but their maintainership is not invasive at all). As benchmarking on timing is quite pointless (xine startup is slow by its own, as I “already described”:, and bindings are done lazily which means that you need to count the time spent during playback, for instance), I’ve written a simple script that parses the output of LD_DEBUG=bindings and gives you statistics about the bindings themselves. It’s not yet complete, I’ll try to complete it soon and release it.

My mail was basically this:

So, as people seems to want some concrete results produced by the visibility patches I’m working on, I drafted up a parser for the LD_DEBUG=bindings
output, and counted the results (I’ll make the script available in the next
days, give me time to make it something les specific than what I’ve hacked up in 30 minutes of ruby hacking today).

The output comes out of a xine-ui playing an AVI with ISO MPEG-4 video stream, and MP3 audio stream (both decoded with FFmpeg plugin). xine-lib CVS HEAD with visibility enabled in that, too (and external FFmpeg from SVN of course).

Without visibility patch:

libavcodec required 625 symbols (591 in itself)
libavutil required 10 symbols (3 in itself)
libpostproc required 9 symbols (none in itself)

libavcodec resolved 702 symbols (591 from itself)
libavutil resolved 13 symbols (3 from itself)
libpostproc resolved 3 symbols (none from itself)

-rwxr-xr-x 1 root root 2605648 22 set 19:32 /usr/lib/libavcodec.so
-rwxr-xr-x 1 root root  438528 22 set 19:32 /usr/lib/libavformat.so
-rwxr-xr-x 1 root root   17576 22 set 19:32 /usr/lib/libavutil.so
-rwxr-xr-x 1 root root   32368 22 set 19:32 /usr/lib/libpostproc.so
Code language: JavaScript (javascript)

With visibility patch:

libavcodec required 105 symbols (71 in itself)
libavutil required 7 symbols (none in itself)
libpostproc required 10 symbols (none in itself)

libavcodec resolved 182 symbols (71 from itself)
libavutil resolved 10 symbols (none from itself)
libpostproc resolved 5 symbols (none from itself)

-rwxr-xr-x 1 root root 2558832 22 set 20:56 /usr/lib/libavcodec.so
-rwxr-xr-x 1 root root  426240 22 set 20:56 /usr/lib/libavformat.so
-rwxr-xr-x 1 root root   16984 22 set 20:56 /usr/lib/libavutil.so
-rwxr-xr-x 1 root root   32368 22 set 20:56 /usr/lib/libpostproc.so
Code language: JavaScript (javascript)

I’m not sure why libpostproc required different amount of symbols, different
runs gives me values between 3 and 6, probably depends by which postproc xine is trying to do depending on system load. The numbers for libavutil and libavcodec are otherwise constants.

It is a good improvement in the amount of bindings required, and please consider that xine does not use libavformat and I tried only with one video per time; the advantage increase when you play more videos of different formats. The size difference is minimal, but consider that I built it with -Os on itself, and that this isn’t C++ where the visibility improves drastically the size of DSOs.

It seems pretty straightforward to me. The decreased amount of bindings is significant, and the result is appreciable on the time need to start the first track in Amarok. Unfortunately, it seems like FFmpeg upstream cares nothing about this. I think the line I should read from the mailing list is “If it does not involve ffmpeg(1) proper or mplayer, patches are not welcome”… so I’ll probably follow the suggestion of Derk-Jan Hartman (of Videolan) and simply send and forget, if they apply the patch good, if they don’t, I couldn’t care less.

So, trying to preparing the sums of this experience: I’ve started four days ago with a whole night working on FFmpeg’s code, patching and fixing; I’ve spent the following days working on improving the patch together with Luca trying to make it less invasive, and last two days I worked on proving my point, but I’m asked to prove it in a way that’s just unfeasible (it’s almost impossible to get benchmark results of the time spent on bindings, because it’s spread during the file reproduction).

This bluntly results in 4 days totally wasted, 4 days of useless unpaid work, that I could have probably spent better on doing something else. I don’t think I’ll try to submit a patch to FFmpeg ever again now.