Unieject moves to GIT

It’s not like I love GIT unconditionally, I think Mike has quite a point about it. But it makes it way easier to handle repositories than Mercurial. So I am using it for almost all the projects I maintain alone.

Unieject up to now was still using Subversion on SourceForge.net; the problem was that git-svn didn’t grasp a rename that I made during the early life of the project when I imported the local Subversion repository to Berlios.

Today, after I couldn’t commit to Sourceforge because my password expired (is this something new?) I tried git-svn again and… it worked! It imported the repository correctly. After a bit of fiddling to replace the tags branches with actual tags, I was able to get my new repository online on the server.

I’ve now disabled SourceForge’s SVN for Unieject, the code can be found at https://www.flameeyes.eu/p/unieject.

I’m now debating with myself about either resuming to work on gitarella, or abandon it for cgit… the problem is that I’d have to prepare an ebuild for cgit at least, and I never tried to understand how to make an ebuild for a webapp. If somebody from the webapp team can give me some of his time to either teach me how to make an ebuild for cgit, or directly creating one, I’d be quite happy :)

Burglar alarms, libcdio and ZeroConf (again)

Again, I can’t sleep yet and it’s 7.16 in the morning… my insomnia starts pissing me off. Unfortunately it’s not just insomnia: lately there has been a huge amount of burglar alarms going off during the morning. Seems like people forget to disable them before opening the door, with two unpleasant results: the first is that they easily wake up people with light sleep, the second is that we grow used to hearing the alarms going off, so the only time it is going off because of a burglar entry, nobody will ever think of calling the police. A couple of days ago, there has been an alarm going off and then shutting up repeatedly for about two hours… at the body shop near to my house they were probably trying a burglar alarm, but couldn’t they find a decent way to handle it?

Anyway, since I’m stuck at the MacBook Pro nowadays, and I can’t work on Gentoo, I decided to take a look to complete one of the tasks I had in my TODO list for a long time: fixing a few issues with libcdio and OSX. When I started working on unieject I originally hoped to get it to work not only on Linux and FreeBSD, but on as many operating systems as libcdio allowed. One of the operating system I was going to try with was supposed to be OSX, but when I started looking into it, I found a very bad implementation of it; the truth is that the OSX driver in libcdio is mostly an hack and I wouldn’t suggest anybody to use it seriously, for now.

In particular, the eject function ends up executing through popen a call to hdiutil, a tool similar to FreeBSD’s cdcontrol, that already implements the eject command. Not really nice, eh? (In particular, FreeBSD’s cdcontrol is quite interesting, the main issue I have with it is that the interface is far from being similar to Linux’s eject: unieject aims not to implement things that are totally missing, but to implement them with the same interface on every supported operating system).

Anyway, thanks to Matt Messier, the xine contributor working on Mac OS X porting, I finally have the code to replace that abomination: starting from Mac OS X 10.4 (Tiger), there is a new framework, DiskArbitation, that handles unmount and eject of devices (it was private in 10.3, but it’s public nowadays). Matt already implemented the autoconf check for the presence of the framework, so importing that code into libcdio was trivial (the license of the two libraries is the same, GPL2+, it’s also worth nothing that xine-lib already uses libcdio for some features like vcd handling, but let’s skip over that, I’ll talk about it another time).

The main issue with doing this was that I had to install MacPorts on the laptop, as I didn’t have anything installed before, now I’m installing some parts of my usual environment: I’ll be stuck on this laptop for another month most likely, I can’t leave myself to forget how to develop ;) Unfortunately I don’t have much free space on the disk, I’ll either have to delete the Gentoo installation on it, or buy an external HDD to keep the photos my sister takes (she has a camera and a camcorder, both digital, but no computer).

There is another function that libcdio implements by calling an external program, and it’s the one to close the CD drive tray… unfortunately although I have a quite clean idea of what I should do to replace it, I can’t test it for sure as both the MacBook Pro and the iBook have slot-loading CD drives, the tray will never close ;)

I have to say, also, that libcdio would need a quite deep refactoring: there are functions wrapping MMC commands, functions wrapping partially MMC commands, functions that use the CdIo_t handle, and functions taking the path of the device to eject. Seems like functions were added when needed without caring too much for an interface. I know unieject also started this way, and I have in my TODO list to clean that up.

Talking about unieject, I’ll have to cleanup the partition unmount support for Linux, but to do that I’ll need Enterprise. The fact that you can change both the device name and the major/minor number assignment makes it impossible to find, just by the name of the device, the partitions that has to be unmounted. You can just hope that the device only contain one partition, but this is not always true when it comes down to USB mass storage devices.

The only way to actually get the information needed is to find the exact device in the /sys tree, and then analyse the children… in reality, the only way to handle this decently is to use HAL, as bad as that might sound for a simple utility as unieject. I think I might look into adding optional HAL support in the future.

For what concerns ZeroConf, after my rant about Knoppix not providing a stack, I’ve been thinking of how many ways ZeroConf can be used to make more transparent a network. The first thing that comes to my mind is the neat Bonjour support for CUPS that Apple developed and merged in CUPS codebase after acquiring it: I was at a support call last June, at a print shop where the network was totally fooled up (and technicians paid ten times I was paid weren’t able to fix it… it just needed a router!); after fixing the network, their main network printer changed IP address, rather than having one from the ISP’s MAN, it then had an internal local IP address. The two eMacs were still able to access the printer through AppleTalk, while Windows had to be reconfigured almost from scratch, there was no easy way to just tell it to change the IP address the printer was present at. Discoverable printers are a godsend sometimes.

Other uses that might be useful for local networks are of course the ability to see which boxes are up and running and providing SSH, SFTP, FTP or other services: opening iTerm from here tells me I have a box that allows all three of them: the AppleTV. I wonder if a feature like that is implemented in Konsole 4, I’d certainly make use of it. Then there is the ability to see the network shares: it would be cool to just boot a system, maybe even a LiveCD, and find on the desktop not only the local partitions to be accessed.. but also all the network shares for the fileserver.

Now, of course there are already methods to do this without going the MDNS/Bonjour/Avahi route: CUPS uses IPP and SLP discovery, Samba implements some features of the SMB protocol that allows discoverable shares, you can use SLP to broadcast presence of services, too. There was even an article on the GWN (if I recall correctly, but I’m sure I’ve seen one somewhere) on how to set up a discoverable rsync server for the local network by using SLP. But why should we have three different discovery methods (four if we count SSDP used by UPnP)?

It would be also a nice way to handle a trusted network segment where nodes are connected either for building (distcc) or for in-memory storage of temporary or cached data (memcache). Just think about a distcc that automatically finds the boxes that support the current architecture: you wouldn’t have to change configuration if you add, remove, shutdown or power on a new box…

An interesting exercise for someone who would like to experiment would be to rewrite the tutorial for discoverable rsync servers to use Avahi rather than OpenSLP; I have to reiterate I can’t do it as I’m stuck without Enterprise, and thus unable to do any Gentoo-related work, till about mid-september, when I will be able to buy the new hard disks.

Writing PAM documentation

So, as I promised to Tavis, I started working on writing some documentation for Gentoo developers on how to handle PAM in the ebuilds; I don’t have a final draft yet; just trying to write down what I think should be written in that guide took me the best part of two days, and now actually doing the documentation part is driving me so crazy that when I’m done I’ll probably need a good shrink.

You have no idea how convolute is to actually declare what needs to be done with respect to PAM support and PAM dependencies, how many special cases has to be documented, as although the PAM API is quite standard, just some modules are, the rest are specific of the implementation (that for Gentoo Linux is always Linux-PAM, while for Gentoo/FreeBSD it’s OpenPAM).

On the other hand, documenting this stuff is probably going to help on the long term, maybe someone will find time and will to work with me on maintaining the PAM packages, so that I won’t be the single point of failure for the default authentication system in Gentoo.

I also want to write some extensive documentation on the use of PAM on the user/admin side, and some guides on setting up extra modules (for instance I’d really like to take a look into adding some of the modules that are currently bitrotting on Bugzilla, but I just don’t have time right now); unfortunately this will require quite a lot of time, and I might just not have it at hand in the short term, so the priority of this is quite low. Bribes to change that priority, as usual, are welcome ;)

On a different note it’s interesting to know that the GNU projects are switching, one by one, to GPL-3 since the day after the release. I read it, I hope I understood it, I seem to like it. This is good because libcdio will probably switch to GPL-3 in the next future, which means that both unieject and xine will have to switch if they want to continue using newer versions of it (well, having a GPL-2+ license in Portage would be quite good already to at least pass a human test, but that’s another story).

Okay, now to decide whether I want to continue losing my life on PAM documentation (after all, I spent the whole Saturday night working on PAM documentation), or if I want to take a break for today…

Just a few little updates

I hate Summer and the heat; I’m still waiting the new PSUs to replace the old ones, and in the mean time I’m cooking myself.

I’ve prepared a new patch for OpenJDK to allow using -z defs on every platform, this way I can test more safely the future patches.

I have improved my elf parser to parse the .dynamic section, allowing me to get the sonames and the needed libraries for an ELF file; this is the base for the design of a new tool, that checks an elf file for really needed dependency, to be able to get the needed packages without needing to have the package built with --as-needed (the script will check the undefined symbols in the elf file and then check which of the dependencies stated in the NEEDED entries are actually needed); and to be a bit safer, it will also check if the ELF file uses symbols that might indicate runtime dependencies not otherwise identified (dlopen(), exec*(), system()), and give out a warning in case.

Also, I’ve resumed a bit of work on Gitarella and replaced the Log4r usage with Ruby’s Logger class, cutting down a dependency. I think I’ll do a similar thing with popt on Unieject, especially after what Emanuele told me about the way popt is distributed from upstream at the moment; I’ll probably work on a new release of Unieject with that and finally support for Mac OS X (thanks to Matt Messier who’s helping with xine-lib on OS X, I now have a pointer on how to implement eject with OS X).

Oh and one thing leading to another, I rejoined Gentoo today.

Ejecting the mobile phone

Okay, yesterday was my birthday, nothing to say about that as XMMS was removed already. Thanks to a friend of mine (Alberto), I gone shopping in the afternoon… strange but true, I also found the nullmodem cable!

What I was going to buy was a TransFlash/MicroSD memory card for my cell phone, so that I could use it as an mp3 player for those rare cases where I’d like to listen to music without a PC around.

The card works fine on the phone, and it all seems well, till I tried copying some music to it.. some of the music ended up fine, but most was lost, and the vfat corrupted (so much to require me to install dosfstools to run a dosfsck).

After some tries, I found the problem in a non-flushed buffer: when KDE was unmounting the device, it was calling “eject /dev/sdc1”, unfortunately unieject wasn’t smart enough to actually work on /dev/sdc, to the result that the device wasn’t being ejected at all.

I so decided to improve unieject so that it could take care of finding if the device it was asked to eject was a partition or an actual block device, and if it was a partition, to find its parent’s block device.

Easy you’d say, just use the final name of the device.. but no, udev allows you to change the names of the devices, so I ended up looking for the major/minor device numbers of the open device. But that just tells me if it’s a device or a partition, how do I find the device for a given partition?

I originally implemented it as an hack, by removing the digits at the end of the device name, and probing that for being the root device of a given partition, but of course it wouldn’t help if the devices were all renamed.

The best implementation I could think of used sysfs, and thus works only on Linux 2.6 (after all on 2.4 you expect the devices to have a sane naming scheme at least), but I admit myself it’s slow. It iterates over the content of /sys/block/*/dev and looks for a device with the correct major/minor devices.

The other problem was finding all the partitions for a given device; I couldn’t base myself over using the name here either, so I took all the directories in /sys/block/$namefoundbefore/ and removed the ones that are sysfs-related, and then hoped they were directly in /dev to find and unmount them. The solution might not be perfect, it might not cover all the possible cases for strange names of the devices, but it should cover most of it and thus allow a decent use of unieject.

Now I have this feeling that if I asked to someone in the known how I’m supposed to find the devices and the partitions, the answer would be “use HAL”.. and no I’m not interested in using HAL in a eject command! Why should I, in the first place, need HAL to look up simple data that would have been trivial to find with a stable naming scheme?

I also hoped that libsysfs would have helped me, but a part being one of the libraries where the documentation can be summarised with the following code:

int strange_var; // A strange variable

it also does not seem to allow for much more than looking up entries knowing their path already without using fopen to inspect the content.

Unfortunately even after fixing unieject to correctly unmount the partitions and eject the correct version, ejecting the mobile takes a lot of time to flush the buffers, and I’ve seen it losing data when loading too much data in a short time, and then ejecting. I wonder if there’s a way to tweak the size of the buffers on a per-device basis.

Off topic, last night I downloaded my nephew’s photos using DigiKam directly from my sister’s digital camera.. I think the current experience with gphoto2-based programs on Gentoo leave to desire, as it didn’t set the permissions on the USB device automatically (of course, I didn’t configure anything, but the point is that stuff shouldn’t require to be configured at all, many things already work this way).

BSD flags and wanting not to work….

In the past days I started thinking about the few problems that still afflicted Gentoo/FreeBSD. One of those was for sure the impossibility of stripping the files that got installed with the immutable (schg) flag from FreeBSD sources, that and the SetUID/SetGID files unable to drop world readability for the same reason.

Then, thanks also to Timothy’s suggestion a couple of days ago about using P7Zip and using mtree for restore permissions, I found a solution that I’ve already sent to Zac for integrating in the next versions of Portage. Basically I just need to use the mtree(1) utility to save the flags for the tree installed in the image directory, and then use it back to restore them on the merged tree in ${ROOT} (actually, the patch I’ve sent to Zac has a bug I just found, it does not use ${ROOT} :/ But I’m going to fix that soon enough). After that, most of the flags handling was finally moved from the Python side of Portage to the SH side, and it also seems to be faster..

In addition to that, I also removed the code, now redundant, that copied the flags between files using the chflags custom module installed by portage, and replaced the smaller remaining code with py-freebsd, so that Portage won’t have to rely on a non-standard Python module (and py-freebsd is actually maintained, so it’s a good thing).

This solves a couple of the minor annoyances that were still bugging me since I joined the project, so I consider it a good step forward, even if little. The other thing is the custom stage creation I used to use, but that is going away soon too, I just need to wait till Roy releases the alpha4 version of baselayout, then I’ll be all set to start working on the new stages, hopefully.

Unfortunately, I’ll be working all next week so I won’t have too much time for Gentoo, a part the night as usual, but that might not be as bright for me as it used to be. I just hope to be able to leave everything in order, so that I’m not strictly needed while I’m not available.

I should probably be writing some documentation about KDE eclasses soon, as that is really needed, but today I only found the will to update the Amarok maintainer’s guide and the autotools failures guide (the latter with the recent change to versions dependencies). The XMMS debacle stressed me quite enough, although I’m far from burning down, for sure, and I need some way to relax myself after the envelopes thing…

I also want to go down fixing the damn problem with FLAC files with ID3 tags and Xine, but that will probably end up requiring me a lot of time I don’t really have now. I’d be really pleased to have some help, even if it’s a proxy maintainer for stuff or something like that.. I need to start moving some of the stuff I’m taking care on to someone else…

I didn’t even find the time to review some changes for libcdio’s eject code, trying to merge some of the functions of libunieject into libcdio proper, and I feel sorry for that, but I didn’t really have the material time for that right now. I need longer days and more pleasant nights.

FSF/UNESCO Free Software Directory… debacle or simply unmaintained?

Today I was looking to the referrers, to see which sites links to mine. I’ve seen once again the entry for Gitarella in the Free Software directory by FSF/UNESCO (is the UNESCO partnership new? I didn’t remember it from some time ago).

Now, if you go looking at the page, you see it in a strange category, in my opinion (Web Authoring?), plus you can see by yourself that something isn’t right.

The URL of the site is correct (and that’s because I sent them an update to tell them of using that URL), but the one to the tarball points actually to the Gitarella browsing Gitarella itself, not the download page; the Source Information link points to nothing, Gitarella is not hosted by SourceForge.net.

Same for the Documentation link, it points to a good deal of nothing, just because they suppose most of the projects are on SourceForge. My contact instead is listed without full name, although I try to use “Diego Pettenò” to refer of myself almost everywhere, when there’s not a requirement for a nick (like in forums, irc and so on), so that I’m easily recognisable.

Now, you’d say, why do you blog about this instead of writing to them to improve the situation? Well, I did write to them, but it seems they didn’t update everything at all :/ Just added a couple more links, updated the version number, and forgot about the broken links and so on.

FSF, you should probably try to cleanup the entries you have there… another example is unieject entry that refers to CVS at SourceForge for source management, too bad I moved from BerliOs to SourceForge when the subversion service was already working, thus I always used Subversion there…

Sigh. Better use FreshMeat, it contains also non-Free software, it’s not sponsored by UNESCO, but as you can report your own projects, it’s usually better updated (although I admit I forgot to update stuff from time to time).

Finally, the release…

… of xine-lib 1.1.2 of course. Darren packaged the tarball and released it on SourceForge tonight, and now all distributions will have available the fix for FLAC files, hopefully..

I wish to thank Darren a lot, as this release really makes my work simpler, especially since, after this release, I’m going to merge the remaining (3) patches of Gentoo, the visibility one, the textrel one, the m4b.. actually the only remaining is the textrel one, but that I want a couple of tests more before.

I’m also trying to investigate some bugs, after I hunted down some issues that are now fixed in the 1.1.2 release in SourceForge.net’s bug tracker. Unfortunately I don’t have direct access to those bugs to close them down, and I had to ask Darren to do that, one by one, and it’s not that easy after all.

There are a lot of bugs that are caused by xine-ui and those should be fixed when we do another release of that, too, but it might take a while still.

I’m currently working on adding support for Darwin i386 to the configure.ac, that currently assumes Darwin == PowerPC (uncorrectly). This should solve two bugs currently reported in SourceForge.

Not that easy is proving the importing of unieject’s source code from SF.net’s SVN to a GIT repository (as I’m finding myself quite comfortable with GIT, I’m more than happy to move to that). I tried three times already to get the data, but SourceForge’s server is all but stable. I’ll be rsyncing it down and then converting locally later.

So, today not much commits to Gentoo, but a few to ruby-hunspell to prepare the release, a bit to gitarella as I wanted to improve it a bit, considering I’m using it more and more, and now xine. Not a bad mixture, is it?

Oh of course I won’t forget the Wavpack plugin for xine I talked about some time ago, I’ll be working on that as soon as the issue are taken care of, so that people can actually use it, but the problem is that to have it working correctly I also need a TagLib plugin for Amarok to use it, and thus it should be better if TagLib’s plugins would be taken out of Amarok as discussed some time ago.. If you can help somehow, it will be surely appreciated :)

Too hot

Today is too hot for me. And I mean it.

I hate summer and I hate heat, I hate when the temperature comes near the 30°C, and I hate the 90% humidity I have.

Today staying in my “home office” is difficult, and now I really can’t stay there anymore.
I ended up taking out my shirt and now I’m on my bad in my room, with the iBook.

Developing today is not possible, too hot to stay there, too hot to pass the whole day in front of the monitors because of the humidity. I worked on unieject this morning, and that is enough for today I suppose.

Actually, I also took some time to fix the licenses of my software, to update to latest GPL and LGPL texts available, and to update the address of FSF on all the files.

Now I will probably just find some time to hash out some documentation on gitarella, to release the first alpha alpha version, that at least might show some interest on that. It works basically, and that is what I wanted for it for now. Future development might make it use also SCGI, who knows? :)

My idea of adding WavPack support in xine seems to have changed a bit, Luca suggested me to rather write an ffmpeg plugin, so that the files would be played directly with it, but to that I should also add to my list the creation of a demuxer plugin for xine-lib (or it won’t be able to play wavpacks by themselves, but just wavpack streams) and a TagLib plugin inside Amarok to be able to get the metadata.

I’m afraid I won’t be able to take out much of useful by this pace, mainly because I never did anything like that in my life, and the time I can pour in that is strictly dependent on the free time I can get, and that is likely to be too few as I need a new job to be able to have enough money for the university.

Oh well, we’ll see.

When having the sources helps

So, tonight I didn’t sleep at all. This was planned and intended, but that is another story.

So, while I was passing the time, I decided it was the case to work a while more on unieject. First I decided to release a 5.3.1 version to fix a segmentation fault and to update translations.

Then after looking at the new eject -h output, I decided to implement the traytoggle option, so I started adding the support inside unieject for that, but I had no idea how to check if I had to eject or close the tray, so I looked at original eject’s code, as usual. The code there is pretty experimental, I’d say: it measures the time needed for the tray to close, and if it’s lower than a given value, it then ejects the tray.

Not sure what you think, but I don’t like such a method :)

There had to be a way to tell if the tray is open or closed, so I started looking into raw MMC commands. Unfortunately I still don’t have a proper documentation for MMC commands, and that sucks, I should probably look for one, but anyway.

Googling, I was able to find from Apple’s OpenSource pages one of the headers for the SCSI interface, and in that I found some command that had an interesting name: GET_EVENT_STATUS_NOTIFICATION. Googling againf or that, I found the IOSCSIMultimediaCommandsDevice::GetTrayState() function, that was exactly what I needed :)

After looking a bit also to libcdio sources, I was able to put that in a function that actually checks the status of the tray before deciding to open or close it, adding support for traytoggle to unieject in a saner way than original eject.

Now, who said that having access to the source, even under a restrictive license, is not a good thing? :)