This Time Self-Hosted
dark mode light mode Search

Big filesystems

Very few of you probably remember that over two years ago, in October 2009, I did some investigative work on Portage Tree’s overhead to show just how much space was going to be wasted with small files on filesystems with too big block sizes.

It wasn’t the only time I noted that while for things like Portage, and likely your operating system’s file, it makes sense to have smaller-than-page-size blocks, it doesn’t seem as smart to do the same for bigger files such as music and video. At the time I noted that HFS+ somehow supported 64KiB blocks with the Linux driver – a driver that is very much unstable and often times unusable – while XFS refuses to play well with similarly-sized blocks, even though it is designed to support them.

I’ve read many people complaining that I didn’t know what I was talking about when I called for bigger block sizes for Linux’s filesystems. Many people insisted that the presence of extents in ext4 made it completely moot to have bigger block sizes. If that’s so, I wonder why ext4 now implements bigalloc which is basically a trick to allow bigger block cluster sizes.

I read about it, with the release announcement of kernel 3.2, while I was on vacation and I just couldn’t wait to try it out with some of my filesystems. Luckily I tried it with the least important one, though, as it’s far from being mature for using.

The current implementation does not support online resizing, so you’re supposed to use resize2fs with the unmounted filesystem … too bad that it fails to run entirely when using the latest version of e2fsprogs. Oh and don’t forget that the switch to turn on bigalloc is not documented anywhere yet.

So it is to be expected given that it’s a very new feature, but I wonder why half the fuss about 3.2 release was about a feature … that definitely is not ready for prime time even in testing ground. I just hope that work toward this kind of features will also mean that XFS will gain support for 64KiB blocks, which I would prefer to ext4’s 1MiB clusters in the first place.

Also I would like to point out one thing for those of you who wish to use this feature on volumes shared with Samba to OS X hosts: you’ll end up with tons of space wasted to .DS_Store files unless the inline data feature is also used, and the inode size is increased. On my filesystems, .DS_Store files weight between 741 bytes to 14KiB… I thought I configured Samba to use extended attributes to store the data instead of using external files, but for what I gathered on the Netatalk mailing list recently, this conflicts with the size limit applied to EAs on ext4… I guess this is another of those things that really need some tweaking to get right.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.