Privacy advocates: two weights, two measures

While I don’t want to say that all privacy advocates are the bad kind of crybabies that I described on my previous post there are certainly a lot I would call hypocrite when it gets to things like the loyalty schemes I already wrote about.

So as I said on that post, the main complain about loyalty scheme involve possible involvement with bad government (in which case we have a completely different problem), and basically have to do with hypothetical scenarios of a dystopian future. So what they are afraid of is not the proper use of the tool that is loyalty schemes, but of their abuse.

On the other hand, the same kind of persons advocate for tools like Tor, Bitcoin, Liberty Reserve or FreedomBox. These tools are supposed to help people fight repressive governments among others, but there are obvious drawbacks. Pirates use the same technologies. And so do cybercriminals (and other kind of criminals too).

Where I see a difference is that while even the Irish Times struggled to find evidence of the privacy invasion, or governmental abuse of loyalty schemes (as you probably noticed they had to resort complaining about a pregnant teenager who was found out through target advertising), it’s extremely easy to find evidence of the cyber organized crime relying on tools like Liberty Reserve. Using the trump card of paedophiles would probably be a bad idea, but I’d bet my life on many of them doing so.

Yes of course there are plenty of honest possible uses you could have for these technologies, but I’d also think that if you start with the assumption that your government is not completely corrupted or abusive (which, I know, could be considered a very fantastic assumption), and that you don’t just want to ignore anti-piracy laws because you don’t like them (while I still agree that many of those laws are completely idiotic, I have explained my standing already), then the remaining positive uses are marginal, compared to the criminal activities that they enable.

Am I arguing against Tor and FreedomBox? Not really. But I am arguing against things like MegaUpload, Liberty Reserve and Bitcoin — and I would say that most people who are defending Kim Dotcom and the likes of him are not my peers. I would push them together with the religious people I’m acquainted with, which is to say, I keep them at arm’s length.

modsecurity as final antispam solution? Not really

In my previous post about comments I’ve described some more notes about my antispam solution based on mod security, and then some people actually wondered about its validity as a final solution. So let me explain a bit further.

I’m not saying that modsecurity is perfect and that it does its job perfectly well; actually, I always have to disable some of its rules because are sometimes positively stupid or effectively braindamaged, like the “PHP source disclosure” rules in the latest release (2.5.10) that stops any request that would return “fread” to the user; so almost any half-interesting piece of source code.

In general, while modsecurity is doing its job, I recognize that it’s not a perfect solution to everybody. I still find it a nicer solution than captchas, but maybe the two things could be merged: instead of implementing this antispam measures in mod_security, at Apache level, they could be implemented at application level, with an extension library. At that point it’d be possible to choose whether the client is either of: trusted, untrusted, positively a spammer. First and last cases are obvious: the comment is either passed or killed right away; untrusted clients would be presented a bad bad captcha to solve.

This would tie in with not only user-agent based filtering, but also DNSBL and Honeypot’s httpBL and even email verification, and the blogspam webice.

One very interesting use of this would solve the current difficult-to-decide situation where anonymous clients, such as TOR, end up being abused by the spammers, and the decision between keep allowing them to comment, and that of just killing them all is not quite as easy; I can see the reasons for using TOR, but the amount of SPAM is just too much (I wonder if I should do something along the lines of making the blog available directly on the TOR network and then killing the IPs coming from lists like TornevallNET ).

And a word about email verification; some people seem to find it funny to use some common fake email addresses like anonymous@the.net when posting a comment. I seriously have a beef with that. The reason why this is, is very simple: I don’t care whether you enter an email address or not! I don’t require you to; if you do, you get gravatar integration and I can contact you, but that’s about it; you can simply avoid filing the field if you don’t want me to know your email address. If I could check the email address for validity with modsec, I’d be rejecting such comments to begin with.

I’m probably not going to work on this myself for now, but this is an idea if somebody wants to look at it.

And what about imported libraries?

Following the previous blog here also a list of projects that seem to like importing libraries, causing code duplication even for code that was designed to be shared.

  • cdrkit, again, contains a stripped down version of libdvdread, added, of course, by our beloved Jörg Schilling; bug #206939; additionally it contains a copy of cdparanoia code; bug #207029

  • ImageMagick comes with a copy of libltdl; bug #206937

  • not even KDE4 seems to have helped libkcal which even in its newest incarnation ships with an internal copy of libical, causing me to have three copies of it installed in my system;

  • libvncserver comes with a copy of liblzo2; actually there are two, one in libvncserver and one in libvncclient; even the source files are duplicated!; bug #206941

  • SDL_sound, Wine and LAME seem to share some mp3 decoding code, which seems to come originally from mpg123;

  • cmake couldn’t stay out of this, it comes with a copy of libform (which is part of ncurses); follow bug #206920

  • I’m not sure what it is, but DigiKam, Numeric (for Python) and numpy have a few functions in common; the latter seems to have even more than that in common; bug #206931 per Numeric and numpy, and bug #206934 for DigiKam.

  • ghostscript comes with internal copies of zlib, libpng, jpeg and jasper; unfortunately jasper is also modified, for the other three there’s bug #206893; by the way, the copies are present in both the gs command and in the libgs library;

  • OpenOffice comes with loads of duplicated libraries; in particular, it comes with its own copy of icu libraries; see on bug #206889

  • TiMidity++ comes with a copy of libmikmod; bug #206943

  • Korundum for KDE3 has a copy of qtruby embedded, somehow; I wonder if it isn’t a fluke of our buildsystem; bug #206936

  • gdb contains an internal copy of readline; –bug #206947

  • tork contains a copy of some functions coming from readline; bug #206953

  • KTorrent contains a copy of GeoIP (and to think I removed the one in TorK as soon as I’ve spotted it); bug 206957

  • both ruby and php use an internal copy of – I think – oniguruma; I haven’t looked if it’s possible to add that as a system library and then use it; bug #206963

  • MPlayer seems to carry a copy of libogg together with tremor support; bug #206965

  • pkg-config ships with an internal copy of glib; bug #206966

  • tor has an internal copy of libevent’s async dns support; funny, as it links to libevent; bug #206969

  • gettext fails to find the system copy of libxml2, falling back to use the internal copy; at least it has the decency of using a proper commodity library; bug #207018

  • both Perl and Ruby have a default extension based on SDBM, a NDBM workalike; there seems not to be a shared version of it, so they just build the single source file in their own extensions directly, without hiding the symbols; beside the code re-use not being available, if a process loads both libperl and libruby, and in turn they load their sdbm extension, stuff’s gonna hurt;

  • enchant has an internal copy of Hunspell; probably due to the fact that old Hunspell built only static non-PIC libraries, and enchant uses plugins; bug #207025; upstream fixed this in their Subversion repository already;

  • gnome-vfs contains an internal copy of neon; funny as it depends on neon already, in the ebuild; bug #207031

  • KOffice’s Karbon contains an internal copy of gdk-pixbuf; bug #209561;

  • kdegraphics’s KViewShell contains an internal copy of djvulibre; bug #209565;

  • doxygen contains internal copies of zlib and libpng; bug #210237 ; this time I used a different method to identify it as doxygen does not export the symbols;

  • rsync contains an internal copy of zlib; bug #210244 ;

Unfortunately making sure that what I’m reading is true data and not false positive, looking at the output of my script, becomes more difficult now for the presence of multiple Sun JDK versions; I have to add support for alternatives, so that different libraries implementing the same interface don’t show up as colliding (they are that way by design).

Today in Gentoo’s KDE land

And here I am, blogging about a a long day that made me feel so tired I’m knocked out on my bed, longing for reading some Peanuts strips. Today was almost entirely dedicated to KDE, so here I’ll try to point out the changes.

The first and surely biggest difference comes into KPDF handling. If you didn’t know, up to today we used a patch originally coming from KUbuntu and JRiddell, to make it use Poppler. The reason for this were that integrating xpdf code in KPDF meant having to bump it multiple time per release every time a new XPDF vulnerability was discovered (and this still happens, one was discovered just last week), and often we were already late on the patches train, plus the slowdown coming from arches taking time to mark stable, and at the time there was little or no downside to do this: no feature loss, and very little incompatibilities quickly fixed.

Today I was contacted by pinotree of the KDE team, KPDF developer, asking me again if I could drop the patches or at least add a vanilla useflag; I say again not because he asked me before, but KPDF’s main developer tsdgeos, criticised both us and KUbuntu a lot because of the patching, although at the time there were very little problems; this time I started thinking over this a bit more.

Many things changed since we started applying that patch: the focus on XPDF is now less than before, because Poppler already passed through most of the code, which means that the most stupid vulnerabilities were found already, there has been some little feature loss in the 3.5.7 version too, because the title of the postscript file queued for printing couldn’t be kept while using Poppler, and newer versions of Poppler also started behaving slightly differently from before, to the point that there are some major problems. Also I got a report of another PDF that is showing fine on KPDF’s code, but leaks memory when used on Poppler; outside that, Ioannis joined the KDE team, which is no more just me and Carsten, and I got kde-packager access, which means I can see the vulnerabilities as soon as they are discovered rather than when the advisory is released — You might not tell, but this is a great improvement. Pinotree also agreed to inform me, if possible, as soon as they discover something that has to be fixed, so that I can respond to it as soon as possible.

With the current situation, it was time to test an alternative. Right now there are two new revisions of KPDF in the tree, 3.5.5-r1 and 3.5.6-r1, they both are the same as their -r0 predecessor, but without the Poppler patch; in the case of the 3.5.5 version, also the security fix is applied. They should resolve quite a bit of problems, and they should be clean without further security issues up to now. Please test them, and report any possible issue you find. I haven’t applied the same to kdegraphics, because that’s way more massive to compile, so I left that one using the Poppler patch to have a single point of failure.

If you didn’t get the news, it’s no more true that split and monolithic ebuilds have the same exact features: as no developer seems to care enough for those, I’ve been merging to them only important changes and fixes, while I apply experimental or improvement patches to split ebuilds, or I remove them, like this time. I already did that with Lubos Lunak’s xinerama patches that are available on kwin and kdesktop but not on kdebase. Which means that if you want upstream support for KPDF, you need the split ebuilds.

Another drift between the two versions is with kdepim, that today, beside a revision bump to apply two hotfixes to KMail coming from its developers, lost the pda useflag for the monolithic version. The reason for this is that the new KPilot requires a pilot-link package that we don’t have, and thus I had to mask it entirely, until someone in PDA herd can get a hold of that. In the mean time if you need KPilot you need to use KPilot 3.5.5 and kdepim-meta.

Even more changes: KDVI got an emacs useflag, thanks to Sebastian Shubert, to enable search forwarding to emacs. I hope I got it right though, it’s the first time I add an emacs useflag to a package, I never used elisp.eclass before.

And again with something important, KDevelop 3.4 is in the tree, the new branch with about 500 closed bugs against the older 3.3 branch. This version not only features fixes by upstream, but also got the correct configuration for our ctags (named as exuberant-ctags), so that new installations will have it working out of the box (I maintain that packages that requires configuring tools’ paths should be pre-configured by ebuilds so that users don’t have to change them manually). Old installations will unfortunately have to fix the path from the configuration if it’s not already set correctly, sorry for that, I’ll be working on it maybe.

And for external software releases, Tom Albers released Mailody 0.4.0_rc2, that is also in the tree: his lightweight IMAP client improves with every release, with this series it also got signature support. I’ll probably use it as my main client on Intrepid’s installation, rather than KMail; after all right now I don’t have GPG support in either Operating system. Basically the only thing stopping me from using Mailody as my main mail client everywhere is GPG support… but I trust Tom to get it sooner or later :)

Another release, but this time not yet in the tree, has been TorK 0.13; unfortunately when I tried, it didn’t work; but Robert diagnosed the problem right away: this version requires the alpha development version of Tor, which means it wouldn’t work with what we had in Portage till now, but Gustavo (HumpBack) now committed under package.mask a new alpha version of Tor that I’ll depend upon tomorrow, when I’ll commit TorK 0.13, also under package.mask.

Another thing I’ll have to fix tomorrow even if it would be part of Gentoo KDE’s day, is the new handling of XDG environment variable on system level rather than KDE session level: Christoph Brill reported it and Pacho Ramos found the problem yet again :) One of the two variables is misplaced, and should return to be a session-only variable; I’ll fix that first time tomorrow.

This should cover all the work done for KDE today, and it’s not bad, I have to say. Of course, I also spent some time on Serielle Konsole: last night I was able to finally get it working almost perfectly and configurable: now by default it starts with a session opened on /dev/null – to avoid keeping a device that shouldn’t be opened – and you can start a session to a given device with given parameters by configuring a session, in the same way you configure the sessions for standard Konsole! The configuration of the schemas, the extra icons, the wallpapers, and the terminal profiles are also shared by the two, to the point that one of Serielle Konsole’s dependencies will be for sure Konsole.

I plan to add a few more things before the first release, like a command to send the Break signal (already present in the code, just need to be shown on the menu, on the popup, and get a shortcut assigned), support for resuming Serielle Konsole from a saved KDE session with the devices as were open before – like Konsole does for the directories – and a way to specify on command line the device to open (this was on request by Marco Gulino, who would like to use it for KMobileTools debugging :) ).

Now I could be talking about Amarok, about xine-lib not crashing anymore when you try to open a smb URL, about the new VLC snapshot in Portage from the bugfix branch of the guys at VideoLan, about my plans for ALSA in the next future and the new breakages introduced in 2.6.20 kernel and later, but I’m too tired and I’ll leave most of that for tomorrow. I’ll have a full day again.

Goodnight everybody, and enjoy today’s, tomorrow’s and next month’s fixes, so that I don’t apply them for nothing ;)

Porting, and porting, and porting….

So, seems like Gentoo/FreeBSD, that for some people is pointless and useless, is continuing through the road of improvements in general Free Software. But before explaining this, a little parenthesis.

As my ISP hasn’t resolved the routing problems yet, I had to find an alternative to be able to continue working on stuff, so now I’m slowed down, but able to browse the net, through Tor, a huge thanks to EFF for that software. Just so that you know, I’m controlling it through TorK, that you can install from my overlay for now. The problem is that you need to make /usr/sbin/privoxy executable by user by hand right now; I’ve already asked net-proxy to change privoxy’s ebuild; when that will be done, I’ll add TorK to portage. I’ve tested it on AMD64 and PowerPC Linux and x86 FreeBSD, as usual :)

Okay so what is this whole porting thing I’m talking about? First of all, Timothy Redaelli, a FreeSBIE developer, is currently helping me with Gentoo/FreeBSD. It’s a good thing to have a more “insider” look, as up to now the project was entirely in hands of nothing more than “advanced users”. He already ported netselect, the special ping-like program used by mirrorselect, to work with *BSD and OSX, and that allowed us to finally have mirrorselect itself, that was a missing thing respect Gentoo Linux; and he also provided (both to us and upstream) a patch to allow D-Bus to actually build and work on FreeBSD (finally), while now is working on creating an ebuild for the special FreeBSD branch of HAL, that will hopefully close the gap we have respect vanilla FreeBSD on that part.

Myself, instead, after making sure that upstream xine-ui builds on FreeBSD with the latest snapshot, I’ve been working on making PulseAudio working on FreeBSD. I currently have a patch that allows PA to build, and seems also to work. Unfortunately that box does not have a PCI soundcard (although I should have one stored somewhere) and I haven’t tried setting up the networked sound yet, so I’m not 100% sure it works. Of course on FreeBSD it can only use OSS output. I’m going to submit the patch upstream now, although Joe Markus Clarke (the GNOME FreeBSD guy) is listed as FreeBSD porter. FreshPorts only points to a 0.7 version of polypaudio, that is quite old….

I’ll be working more on PulseAudio support in Gentoo in the next days probably, starting from adding the support utilities, and hopefully trying to get it out of package.mask at the end of the next week, but I won’t promise anything. Support in software for PulseAudio 0.9 right now is likely to be little or none, xine-lib 1.1.2 does not support it and I’m not going to add 1.1.3 snapshots just yet.

Who said that Gentoo/FreeBSD was a way to waste precious time?