rbot and Gentoo

One nice thing about rbot is certainly, to me, the fact that installing it is mostly just an emerge away. For quite a while there has been a live ebuild of rbot that fetched first from Subversion and now from GIT.

The only problem was that the ebuild fetched the sources, then made a gem, installed it, and then removed some files. Not exactly the shortest way around.

Thanks to the guys in #rbot, this has been solved. Now rbot installs itself without using gems, but by simple setup.rb. This means not only that you won’t need rake and zip to install it, but also that the installed files have decent path, rather than being buried inside the rubygems directories.

An additional advantage comes from the size of the installed files; when you install a gem, a copy of the sources is left in the rubygems tree; this can be quite a waste. The size of the rbot binpkg has halved:

-rw-r--r-- 1 root root 959426 May 24 12:08 /usr/portage-packages/All/rbot-9999-r8.tbz2
-rw-r--r-- 1 root root 425034 Jun 23 20:18 /usr/portage-packages/All/rbot-9999-r9.tbz2

Add to this that the ruby-gettext dependency is gone too, now you got a nls USE flag that allows you to choose whether you want localised bots or not, and if you don’t want them, it won’t ask you for ruby-gettext at all. Nice, uh?

But it’s not finished here. I was hacking at the ebuild today for a different reason. rbot has a figlet plugin, that executes the figlet command and copy the output on IRC. The ebuild, though, didn’t have an option to pull in figlet, and I wanted to fix that.

So together with the removal of rubygems dependency, the new ebuild also has USE flags to enable the figlet, cal, host and fortune plugins. So you either get your dependencies pulled in, or the bot has the plugins disabled out of the ebuild. Cool!

And talking about the fortune plugin, listing the categories didn’t work on Gentoo, as we install the database in a different directory than rbot expected them to be. This is fixed in rbot upstream repository by yours truly.

Always a pleasure to fix rbot on Gentoo ;) Kudos welcome as well as bribes.

Please don’t build your examples by default

I’m in the middle of an emerge -e world I started to try getting as much glibc 2.8 failure have a bug already on Gentoo’s Bugzilla so that users hitting them don’t have to report them anew.

One thing I noticed during this rebuild is the amount of time spent with no good reason to build tests and examples.

I have the test feature disabled, I only enable it for the software I maintain, and I usually don’t have so much time to look at the test of all software. Still some packages, like dbus, hal, gtk, and so on, build their tests anyway. They don’t run them, but they build tests and example during the default make all call.

I talked with Lennart about that some time ago as also libdaemon does that. He pointed out to me that these examples are often a way for the developers to test that the code still links correctly, for instance, and thus should not be disabled by default during development. I agree it can be useful that way.

For this reason, I prepared a patch for libdaemon, that you can now find here, which I should have committed and I forgot about – I’ll see to do that soonish :P – which should make it possible to have the best of both options: developers still get their examples (and/or tests), while distributions can opt-out them.

For ebuilds, this means that the src_test function should do a make -C tests or make -C examples to build tests and examples, after such a patch is applied, to explicitly build the code that most users won’t need during standard run.

I know it’s just seconds, or minutes, of time to build the examples, but if you add all those up, you’re wasting lots of time. And we should always strive to optimise time usage, no?

Update: the conditional-examples patch was a dead link, so I now link directly to Lennart’s gitweb and the diff that he used; remember please that it works for examples, but tests might be slightly different.

Update (2017-04-29): That link is now dead so I removed the example, sorry.

About bootstrapping

Today I sent an email on gentoo-dev to ask developers not to use bootstrap scripts unless absolutely necessary.

Why do I ask this? Well the reason is actually quite simple. We have autotools.eclass and eautoreconf for very good reasons, using bootstrap scripts means bypassing that code.

First of all, with “bootstrap scripts” I’m referring to all those scripts, usually called autogen.sh or bootstrap.sh and variants thereof.

Most of these scripts only call autoconf, aclocal, automake, autoheader and libtool. Which is what eautoreconf does, just it does better. Some scripts do something more, like they call intltoolize or autopoint, or do stuff that is totally unrelated to autotools. In those cases you can either ask upstream to split the rest of the stuff in a different script, do the rest of the stuff by hand, or just give up and continue using the bootstrap script (although I very much dislike this last option).

But let’s limit ourselves to the script that are equivalent to eautoreconf, like the one in libwbxml that I fixed today. The first difference between the eautoreconf call and the bootstrap script can be seen by users too:

>>> Compiling source in /var/tmp/portage/dev-libs/libwbxml-0.9.0/work/wbxml2-0.9.0 ...
You should add the contents of `/usr/share/aclocal/libtool.m4' to `aclocal.m4'.
configure.in:3: installing `./install-sh'
configure.in:3: installing `./missing'
src/Makefile.am: installing `./depcomp'
src/Makefile.am:6: `CFLAGS' is a user variable, you should not override it;
src/Makefile.am:6: use `AM_CFLAGS' instead.
tools/Makefile.am:6: `CFLAGS' is a user variable, you should not override it;
tools/Makefile.am:6: use `AM_CFLAGS' instead.
./configure --prefix=/usr --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64 --build=x86_64-pc-linux-gnu

this, the bootstrap script output, becomes instead:

>>> Unpacking wbxml2-0.9.2.tar.gz to /var/tmp/portage/dev-libs/libwbxml-0.9.2/work
 * Applying wbxml2-0.9.2.make_install.patch ...                                                                                                                                [ ok ]
 * Running eautoreconf in '/var/tmp/portage/dev-libs/libwbxml-0.9.2/work/wbxml2-0.9.2' ...
 * Running aclocal ...                                                                                                                                                         [ ok ]
 * Running libtoolize --copy --force --automake ...                                                                                                                            [ ok ]
 * Running aclocal ...                                                                                                                                                         [ ok ]
 * Running autoconf ...                                                                                                                                                        [ ok ]
 * Running autoheader ...                                                                                                                                                      [ ok ]
 * Running automake --add-missing --copy ...                                                                                                                                   [ ok ]
 * Running elibtoolize in: wbxml2-0.9.2
 *   Applying portage-1.5.10.patch ...
 *   Applying sed-1.5.6.patch ...
>>> Source unpacked.

Nicer to see, to me, tells the user what’s happening, doesn’t show extra warnings that might confuse the user.

Another problem is that usually the bootstrap scripts are either an && concatenation of autotools commands, or don’t check if the previous steps failed, some checks if configure and/or other files got generated, and otherwise tell the user that the call failed. If eautoreconf fails, it tells the user which step failed, and instruct him to report a bug with the output of the command that failed. Nice.

The scripts are usually used to either add extra directory to search for M4 macro files or to pass options like --foreign to automake. In both cases the solution for upstream is to use the ACLOCAL_AMFLAGS variable in the toplevel Makefile.am that sets the default for automake calls.

For us, well, the latter is no problem at all: --foreign is used when some of the standard GNU documentation files are missing (because they are not relevant), autotools.eclass checks for those and appends it automatically. The first can be solved by passing the list of directories to append to the search through the AT_M4DIR variable.

Okay it’s all bell and whistles up to now, but why do I despise bootstrap scripts so much? Well a lot of bootstrap scripts tend to write their own checking code for autoconf and automake versions, as different distributions do it in different ways, and users need not to know the specifics of their distribution usually. The result is that even if you set WANT_AUTOCONF or WANT_AUTOMAKE in the ebuild, it might be ignored and the script might be calling a different version of autotools, even a version that is not available for the user.

Also, by using the script you bypass checks for direct autotools calls in the ebuild, which in turn might let you pass unwarned an usage of autotools without proper dependencies (autotools.eclass takes care of that for you). And it doesn’t warn you if you’re running autotools during the src_compile phase (which is not good, as we want the unpack phase to leave the source ready to be configured and compiled!).

Now of course, as I said some scripts do something more than just re-running autotools, some call autopoint (gettext) or intltoolize. These two cases aren’t, for now, handled by eautoreconf. I think I’ll add some support for those in the next weeks. I’m thinking something along the lines of EINTLTOOLIZE=yes before inheriting the ebuild.

Making it automatic on eautoreconf would be nice but it wouldn’t work that well with the automation, as you wouldn’t be getting the dependency added (eautoreconf runs way after dependency stage).

So at the end of the day the story is: unless you know exactly why you’re doing so, don’t use the bootstrap script! And if you do use it, leave a comment along the lines of # I know, but it's needed beside its call, so that my grepping through the tree wouldn’t flag it down, deal?

New year’s resolutions

So the new year is near, and this time I want to make some resolutions I’ll try to live up to.

The first is to start detaching from things, from objects. I have one big problem, I have difficulty detaching from objects because they remind me of something, even when the things are completely useless and just take up my space (and I don’t have much space). I attach myself too much even to shipment cardboard boxes for stuff I received or bought, and this becomes a problem on the long run. I also have to keep most boxes of the stuff I buy because of the warranty requiring me to keep the original boxes for at least two years (more if the warranty is extended).

I already tried putting an old soundcard on eBay, but the result was unsuccessful (not even a single offer), now I tried an old motherboard, who knows, maybe I can get rid of some stuff, and “stash away” some money. Unfortunately most of what I have here is probably crap for the average ebay user, so I doubt it would end up sold… I have a few old keyboards, some old video cards, and stuff like that. I’ll try to see if I can make something out of them, otherwise will probably either throw them away or put them up available for free for anybody (beside shipping costs).

The second resolution is the most important one, this year I have to take my driving license, and a car. I can’t be stuck in the nothingness where I live forever, I need a car, and I need to get a daily job, at least temporarily, to pay the car. I’m lucky with this, I have an electronic shop at ten minutes from my house (by car) and I know they change personell quite often, with some luck I should be able to push myself to do six months were just to pay for the car.

I’m finishing what I had left floating around me in Gentoo in the next few days so I can spend more time on working during the new year, as I really need to get some money saved, even if after my hospitalisation that is difficult. Trying to get away from 2007, I also spent basically all the money I have, buying stuff for me and gifts for my friends, this way I can start anew with the new year.

On a Gentooish note, today I created three new packages, one in the main tree and two in my overlay. The first is libasyncns by Lennart, I always put it back into my TODO list to add one day as it’s an optional dependency for PulseAudio; the arch teams now hate me because I’m asking them to re-keyword PulseAudio once again… Lennart you always use bleeding edge stuff, don’t you? :) First it was libatomic_ops (which I had to add to portage), then PolicyKit (which luckily enough steev added to portage about at the same time as I needed it), and now asyncns, which was there from the start, but I skipped up to now :P

The other two packages are Emacs modes; I updated nxhtml-mode (this time a different Lennart ;) ), now it actually works fine for me even on Gentoo, with indentation not throwing up annoying messages to me anymore. But it has a new dependency, app-emacs/cgi+ (new package), which in turn depends on app-emacs/cgi (also new) and app-emacs/httpd. Certainly nxhtml-mode is not one of the most self-contained packages in Gentoo (actually it’s completely self-contained if you download and install it manually, but as usual in Gentoo we prefer using separate packages if they are available).

And nxhtml-mode actually spawned its share of new packages as dependencies, which is not entirely bad as it allows Gentoo’s Emacs support to improve :D

My entry about the Oxygen cursors

After Harald’s entry about the Oxygen cursors, I wanted to give them a try too.

As the author is our very much Italian ruphy, I asked him about them directly :) And with some sinergy, there are now very interesting news for all you Gentoo users who want to try the cursors :)

First of all, I’ve added a live Git ebuild to my overlay for the oxygen cursors. As it’s a live Git ebuild, the cursors are generated when installing from it, this means that it takes some time to have them available, and also that it requires xcursorgen and InkScape (to do the SVG -> PNG rendering). Easily enough, the released copies will be distributed in pre-compiled tarballs, so there will be no dependency over either of them.

While looking at ruphy’s build scripts to build the cursor, I actually thought it would have been nicer to have them using a makefile, so that it would make proper use of dualcore systems (by using more than one job), and it would also make it easier to prepare the ebuild. So I took some time and.. rewrote the whole build system to use Makefiles instead. Thanks to Riccardo, the makefiles are now in the upstream repository, and the ebuild I committed was quite simplified from the one I wrote initially.

The nice thing about make is that you don’t use it only for C/C++ compiled code, you can use it to actually “make” almost anything. I use it for the nxml Gentoo schemas to download the schemas and clean up after itself too. Unfortunately knowing all the possible uses of GNU make is a bit difficult, I suppose I could read about it in the info pages, but as I hate reading a lot on the screen (and I don’t have any eInk-based reader yet – I’m waiting for something with the usability of Amazon’s Kindle and the openness of… nothing actually available), I just added this book to my wishlist… if anybody would like to help me help free software ;)

Anyway, I really do like the new Oxygen cursors, especially the black ones :) Thanks ruphy and the whole Oxygen team!

Yet again rbot

Yes, I know I’m boring when I start talking for a few days one after the other about a given topic. At the moment I’m boring with rbot, on which I worked a bit today too.

First of all, the news is that no more dependencies are needed or added to the ebuild, good. I also fixed the time plugin so that it’s disabled if timezone USE is not present. This should let the build stay quiet for a while.

Then I cleaned up the bugzilla plugin a bit, re-enabled it in ServoFlame, added the xine bugtracker and… prepared to release it.

I started versioning it on git, and I put a tarball on my site so that it’s more easily found. I also changed the license, it was MIT, but it’s now Affero GPL 3.

Why Affero? Well, plugins for a bot is something you often end up editing to improve; by forcing users to actually make available their changes it should make it more easy to improve the code on the long run. Also, I find it interesting to see if actually it would be used and the license properly applied (note that the easiest way would be to send the patches upstream so I actually make the modified plugin available to anyone).

If there is request, I’ll probably put back the old MIT-licensed version; actually at the moment even ServoFlame is using that, or rather a modified version of that, rather than the AGPL3 version (which is more advanced); this is because of the other thing I’ve done after preparing a tarball of the plugin: I wrote an ebuild for it.

So if you have my overlay, just emerging net-irc/rbot-bugzilla will give you a rbot with my bugzilla plugin enabled, the httpclient package installed (as it’s a dependency), and the default configuration already with the bugzillas ServoFlame access.

For now the ebuild is in my overlay, I’m still debating with myself if I should add it to portage (and then actually use portage to maintain the bugzilla plugin used by ServoFlame, too), or wait.

Translating ebuilds, a proof of concept

There was only one comment to my previous post, but I actually didn’t expect that one either, first because I probably lost most of my publicity as I’m no more on Planet Gentoo or Gentoo Universe, and second because I know it’s not an issue that concern most of the people who actually use Gentoo at the current stage.

Nonetheless, I wanted to give it a try to a proof of concept of ebuilds translation. If you’re interested in this, you probably want to fetch my overlay, and look at the media-sound/pulseaudio ebuild that is there. Right now there is only Italian translation for it, as that’s the only language I can translate it to, but it works as a proof of concept for me. To try it out, run LC_ALL=“it_IT” and then emerge –1 pulseaudio.

The trick is actually simple once you know it:

messages_locale() {
        locale | grep LC_MESSAGES | cut -d '=' -f 2 | tr -d '"' | cut -d '.' -f 1
}

This function is used to extract the locale currently set for LC_MESSAGE value. Why is this needed? Well, it’s simple: you might be using LC_ALL rather than LC_MESSAGE to set the locale, you also might be using just LANG rather than setting the LC_* variables, so at the end, using locale is the best shot to make sure we get the proper language for messages set up on the system. In the example I have you above, by rewriting LC_ALL we bypass all the other settings.

local msgfile="${FILESDIR}/${P}-postinst"
[[ -f "${msgfile}.$(messages_locale|cut -d '_' -f 1)" ]] && msgfile="${msgfile}.$(messages_locale|cut -d '_' -f 1)"
[[ -f "${msgfile}.$(messages_locale)" ]] && msgfile="${msgfile}.$(messages_locale)"

einfo ""
local save_IFS="${IFS}"
IFS=""
while read line; do
        elog "$line"
done < "${msgfile}"
einfo ""
IFS="${save_IFS}"

This is instead the code that actually handles the loading of the translated message that is then printed on screen for the user. It’s a very rough code as it is, I know already, so no need for pointing me at that: the tools’ chain shown above is ran at least two times up to four if the current language is in a country locale form (like pt_BR), rather than just a language name (it). The code is also prone to errors as it’s quite long by itself.

But as I said, this is a proof of concept rather than an actual implementation, this is just to demonstrate that it is possible to translate messages in ebuilds without filling the ebuilds with the messages in 20 different languages. Of course to avoid adding that big boilerplate code it should go either in portage itself in some way (but that makes adoption of translation a very long term idea, maybe EAPI=1 related) or in a more feasible i18n.eclass, that would handle all of it, included caching the value returned by $(messages_locale) so that it’s not called four times, but once only, and converting from UTF-8 (the usual encoding for in-tree files) to the local encoding, with iconv, if present.

This works well for the long log messages that are added at the postinst phase for instance, because they rarely change between one version and the next one and so have time to be translated. It doesn’t really fly for the short informative messages we have around, nor it works fine for eclasses messages.

For those, what I can think of on the fly is to try to standardise the strings as much as possible (for instance by letting the eclasses to the job), and then use gettext to translate those, with an “app-i18n/portage-i18n” package where the eclass can get their data from. I’ll try to see if I can get a proof of concept of that too.

My new year resolution, anticipated…

… is of starting thinking less of new stuff to do, and act more with the time I’m given.

First of all, I still have a backlog of things to do, and mail to write. I just seen today that I have a mail from Marius Morawski (the Gentoo/FreeBSD logo author) that I moved to my “TODO” folder now four months ago! Sorry Marius if I didn’t answer, I’ll try to as soon as possible; I can say to you already that your works are always fantastic :)

I’ve finally completed the keywording of kde-meta (with the exception of kdeaccessibility) for ~x86-fbsd, which means that KDE officially works on Gentoo/FreeBSD. I didn’t try XFCE in a while now, especially since it does not seem that well maintained in Gentoo lately; and GNOME won’t work, as usual. This is unfortunate, but I cannot really solve it in any other way than finding someone to take care of it.

I would really like some more help; someone who could take care of Amarok in my place for instance, someone that might take care of VLC, so that I can concentrate on xine. I tried to leave Konversation to Chris, but he slacked off when he needed to do the bump.

I really need more time, or more help so that I can drop maintainership of a few things leaving it to someone else; and still, I receive requests for new ebuilds. I’ll try to do my best, as usual, I hope it’s enough :) In case, bribes are accepted to change my priority listings ;)

Prepare yourself for new amaroK ebuilds

Continuing the I’m out of cool titles saga, this is mainly a changelog entry for myself that I want to put some lights on :)

I think I already wrote about amaroK’s team way to inform packagers of new versions of amaroK. And who idles around in #amarok @ freenode, probably already knows that before the actual release there are usually one or two (or more) release candidate releases. Last time I put them directly on portage, under p.mask, but I wasn’t happy enough of that solution because the SRC_URI was broken and I was really expecting bugs about that.
This time, as yesterday I made public my overlay, I pushed those ebuilds directly there so people can get them when they want. If you want to be updated when push something there, the changes appear in my CIA page (sorry yesterday the commits appeared on Gentoo project, now I changed them to flameeyes project anyway).

The current version available is 1.3.8_rc1 as the rc2 in #amarok’s topic was foobar and was code of 1.4 series instead ;) Well this mistake was actually useful to me as I could take a look at the new deps at least. Unfortunately, it’s probably going to be an headache, especially for me. The new (optional) deps are mostly missing in portage: libifp is completely missing, a bug is filed about that, but it’s in maintainer-wanted and the ebuild does not work as intended anyway; libgpod is handled by joem, and I’m not even sure if it’s the right version, the dependency over sys-apps/unieject is a problem for me as I’m using unieject here, I think it can be changed to virtual/eject but I want to hear from joem about that; exscalibar is not even reported on bugzilla.

What does that mean? Well it means mainly that amarok 1.4 on Portage won’t have the useflags to enable ifp and exscalibar support and might have libgpod support but that would be untested by me at least (I don’t have neither an iRiver nor an iPod). Gentoo developers that might want to add and manage them and test amaroK’s features related to thos are obviously welcome to step up; non developers wanting to join Gentoo are welcome, too. For the ones who are not developers and don’t want to become, and still wants amaroK ebuild to support libifp and libgpod and the features to be tested working.. find someone else to take care that has the hardware, or donate the hardware, that’s a solution too.

Okay and as I said “ebuilds” in the title, I actually have an interesting change to 1.3.x series ebuilds, too… from 1.3.8, the $LINGUAS variable will be respected, so it won’t install lots of locales and build documentation for all the languages it supports. In my case, it just built Italian language and documentation as I have LINGUAS=“it en” (I actually have that only to get Italian spell checker, as I use the entire system in English, but anyway).
That is probably going one of the main reasons to update to 1.3.8 as the changes from 1.3.7 are actually bugfixes, most of which we don’t really care about (NMM and HelixPlayer backends are disabled in Portage) and the important one (lyrc’s change) is already applied in 1.3.7-r1 (thanks Ian again :) ).