My first experiences with with Amazon EC2

It really shouldn’t be a surprise for those reading this post that I’ve been tinkering with Amazon EC2 in the past few days, the reason for that is that you can find it out by either looking at my stream or at my commit feed and noticing how I ranted about EC2 and bumped the tools’ packages in Gentoo.

As a first experience with EC2, I have to say it does not really come out very nice… Now, the whole idea of EC2, doesn’t look half as bad. And on the whole I do think the implementation is also not too bad. What is a problem is the Gentoo EC2 guest support: while Amazon “boasts” support for Gentoo as a guest operating system, there is no real support for that out of the box.

*Incidentally, there is another problem: the AWS Web Console that Amazon make available killed my Firefox 3.6 (ground to a halt). I ended up installing Chromium even though it stinks (it stinks less than Opera, at least). It seems pretty much faster, but it’s lacking things like the Delicious sidebar, still. Seems like my previous post wasn’t totally far off. Sure there are extensions now, but as far as I can tell, the only one available for Delicious does not allow you to use Delicious as it was your bookmarks.*

The first problem you might have to affront is finding an image (AMI) to use Gentoo.. I could only find one (at least in the European availability zone), which is … a stage3. Yes a standard Gentoo stage3, without configured network, without SSH, without passwords, … Of course it won’t start. I spent the best part of three hours last night trying to get it to work, and at the end I was told that the only way to work that around is to install using Ubuntu as it was a live CD installing Gentoo on a real system. Fun.

So start up Ubuntu, create an EBS (stable storage) for the Gentoo install, install it as it was a normal chroot, create the snapshot and… here is one strange behaviour of Amazon: when you connect a new EBS volume to an instance, it is created as a block device (say, sdb). When you use a snapshot of that volume to register a machine (AMI), it becomes a partition (sda1). If, like me, you didn’t consider this when setting it up for install, and partitioned it normally, you’ll end up with an unbootable snapshot. Fun ensures.

By the way, to be able to register the machine, you have to do that through recent API tools, more recent than those that were available in Portage today. Hoping that Caleb won’t mind, I bumped them, and also made a couple of changes to the new API/AMI tools ebuilds: they now don’t require you to re-source the environment every time there’s an upgrade, and the avoid polluting /usr/bin full of symlinks.

So you finally complete the install and re-create the AMI, start an instance and… how the heck is it supposed to know your public key? That’s definitely a good question: right now there in Gentoo there is no way for the settings coming from Amazon to e picked up by Gentoo. It’s not difficult, and it seems to be documented as well, but as it is it’s not possible to do that. As I don’t currently need the ability to generate base images, I haven’t gone further pursuing that objective, on the other hand, I have some code that I might commit soonish.

Anyway, if you got interest in having better Gentoo experience on EC2, I might start looking into that more, in the future, as part of my general involvement, so let your voice be heard!