What does #shellshock mean for Gentoo?

Gentoo Penguins with chicks at Jougla Point, Antarctica
Photo credit: Liam Quinn

This is going to be interesting as Planet Gentoo is currently unavailable as I write this. I’ll try to send this out further so that people know about it.

By now we have all been doing our best to update our laptops and servers to the new bash version so that we are safe from the big scare of the quarter, shellshock. I say laptop because the way the vulnerability can be exploited limits the impact considerably if you have a desktop or otherwise connect only to trusted networks.

What remains to be done is to figure out how to avoid this repeats. And that’s a difficult topic, because a 25 years old bug is not easy to avoid, especially because there are probably plenty of siblings of it around, that we have not found yet, just like this last week. But there are things that we can do as a whole environment to reduce the chances of problems like this to either happen or at least avoid that they escalate so quickly.

In this post I want to look into some things that Gentoo and its developers can do to make things better.

The first obvious thing is to figure out why /bin/sh for Gentoo is not dash or any other very limited shell such as BusyBox. The main answer lies in the init scripts that still use bashisms; this is not news, as I’ve pushed for that four years ago, while Roy insisted on it even before that. Interestingly enough, though, this excuse is getting less and less relevant thanks to systemd. It is indeed, among all the reasons, one I find very much good in Lennart’s design: we want declarative init systems, not imperative ones. Unfortunately, even systemd is not as declarative as it was originally supposed to be, so the init script problem is half unsolved — on the other hand, it does make things much easier, as you have to start afresh anyway.

If either all your init scripts are non-bash-requiring or you’re using systemd (like me on the laptops), then it’s mostly safe to switch to use dash as the provider for /bin/sh:

# emerge eselect-sh
# eselect sh set dash

That will change your /bin/sh and make it much less likely that you’d be vulnerable to this particular problem. Unfortunately as I said it’s mostly safe. I even found that some of the init scripts I wrote, that I checked with checkbashisms did not work as intended with dash, fixes are on their way. I also found that the lsb_release command, while not requiring bash itself, uses non-POSIX features, resulting in garbage on the output — this breaks facter-2 but not facter-1, I found out when it broke my Puppet setup.

Interestingly it would be simpler for me to use zsh, as then both the init script and lsb_release would have worked. Unfortunately when I tried doing that, Emacs tramp-mode froze when trying to open files, both with sshx and sudo modes. The same was true for using BusyBox, so I decided to just install dash everywhere and use that.

Unfortunately it does not mean you’ll be perfectly safe or that you can remove bash from your system. Especially in Gentoo, we have too many dependencies on it, the first being Portage of course, but eselect also qualifies. Of the two I’m actually more concerned about eselect: I have been saying this from the start, but designing such a major piece of software – that does not change that often – in bash sounds like insanity. I still think that is the case.

I think this is the main problem: in Gentoo especially, bash has always been considered a programming language. That’s bad. Not only because it only has one reference implementation, but it also seem to convince other people, new to coding, that it’s a good engineering practice. It is not. If you need to build something like eselect, you do it in Python, or Perl, or C, but not bash!

Gentoo is currently stagnating, and that’s hard to deny. I’ve stopped being active since I finally accepted stable employment – I’m almost thirty, it was time to stop playing around, I needed to make a living, even if I don’t really make a life – and QA has obviously taken a step back (I still have a non-working dev-python/imaging on my laptop). So trying to push for getting rid of bash in Gentoo altogether is not a good deal. On the other hand, even though it’s going to be probably too late to be relevant, I’ll push for having a Summer of Code next year to convert eselect to Python or something along those lines.

Myself, I decided that the current bashisms in the init scripts I rely upon on my servers are simple enough that dash will work, so I pushed that through puppet to all my servers. It should be enough, for the moment. I expect more scrutiny to be spent on dash, zsh, ksh and the other shells in the next few months as people migrate around, or decide that a 25 years old bug is enough to think twice about all of them, o I’ll keep my options open.

This is actually why I like software biodiversity: it allows to have options to select different options when one components fail, and that is what worries me the most with systemd right now. I also hope that showing how bad bash has been all this time with its closed development will make it possible to have a better syntax-compatible shell with a proper parser, even better with a proper librarised implementation. But that’s probably hoping too much.

Autotools Mythbuster! Something you might not know about ./configure options

There are a lot of reasons to use autotools over custom buildsystems, most of them relate to the fact that autotools contain reusable generic code that allows to provide users with common ways of doing the same thing among different projects. Almost all of these features are available in a form or another on the majority of buildsystems, but sometimes they get either suboptimally documented, if at all, and they are different project from project which makes it very difficult to deal with them in a common abstract way.

One of these feature is options to enable and disable features and optional dependencies, to avoid automagic dependencies which are a problem for advanced users and from-source distributors like us. While BSD makefiles have their knobs, and most software provides access through make variables or preprocessor macros to enable or disable features and other things, the configure script generated by autoconf provides both a common interface to those options, and a documentation for them.

There are, though, a few interesting notes about this very useful common interface because a lot of projects either misuse it, or don’t know how deep the autoconf boilerplate is, and reinvent parts of it with no good reason. Let me try to show some of the interesting bits about it.

The first thing to know about the two AC_ARG_ENABLE and AC_ARG_WITH macros is that their arguments are: name, description, if present, if not present. The common mistake here is to consider the last two arguments as if enabled and if disabled; I’ve written about that mistake already a few years ago . This is not the case, and thus checking whether the option is enabled or disabled will have to be done outside of the option declaration for completeness.

Another interesting note is that the third parameter, if omitted, will by default generate a $enable_name or $with_name variable with the content of the specified option at configure (defaulting to yes and no for the positive and negative options when an explicit parameter is not passed through). It is thus possible to get a default-enabled feature variable using code like this:

AC_ARG_ENABLE([feature], [...], , [enable_feature=yes])

AS_IF([test "$enable_feature" = "yes"], [...])

Which is very handy since it avoids having to create custom variables and checking for them repeatedly (again, this is already written in my automagic guide ).

In the example above I explicitly skipped writing the documentation of the option itself, since that is another point where a lot of projects have confusion. If you look at the output of ./configure --help, the default options are all well aligned and make use of as much space as it ś available on the terminal window you’re running it into. On the other hand, some projects’ custom options are instead badly aligned, tightened down on a side of the screen, splitted among multiple lines, or going over the horizontal boundary of the terminal. This is because the upstream developers tried to fake the same alignment of autoconf, without knowing that what it does is usually adapting to the actual output of the system.

So for instance you got stuff like this, coming from the gnumeric configure script:

  --enable-compile-warnings=[no/minimum/yes/maximum/error]
                          Turn on compiler warnings
  --enable-iso-c          Try to warn if code is not ISO C
--disable-ssconvert     Do not build ssconvert (command line spreadsheet conversion tool)
--disable-ssindex       Do not build ssindex (spreadsheet indexer for beagle)
--disable-ssgrep        Do not build ssgrep (search for supplied strings in spreadsheet)
--disable-solver  Don't compile the solver
--enable-plugins="text html"  Compile only the listed plugins
  --enable-pdfdocs        Generate documentation in Portable Document Format

As you can see there are multiple lines that are aligned totally to the right, and that go long to the right, while some others try to align themselves, and keep some space to their left too. The reason can be found in the configure.in file:

AC_ARG_ENABLE(ssconvert,
  [--disable-ssconvert          Do not build ssconvert (command line spreadsheet conversion tool)],
  [], [enable_ssconvert=yes])
AM_CONDITIONAL(ENABLE_SSCONVERT, test x"$enable_ssconvert" = xyes)

While this time the third and fourth arguments are correct, there is something up with the second, that is the description of the option. It’s totally expanded in the configure file, spaces included. But since it would be silly to waste space and readability that way, autoconf already provides an easy way to deal with the problem, which is to use the AS_HELP_STRING macro (formerly AC_HELP_STRING):

AC_ARG_ENABLE(ssconvert,
  AS_HELP_STRING([--disable-ssconvert], [Do not build ssconvert (command line spreadsheet conversion tool)]),
  [], [enable_ssconvert=yes])
AM_CONDITIONAL(ENABLE_SSCONVERT, test x"$enable_ssconvert" = xyes)

which then produces:

  --enable-compile-warnings=[no/minimum/yes/maximum/error]
                          Turn on compiler warnings
  --enable-iso-c          Try to warn if code is not ISO C
  --disable-ssconvert     Do not build ssconvert (command line spreadsheet
                          conversion tool)
  --disable-ssindex       Do not build ssindex (spreadsheet indexer for
                          beagle)
  --disable-ssgrep        Do not build ssgrep (search for supplied strings in
                          spreadsheet)
  --disable-solver        Don't compile the solver
  --enable-plugins="text html"
                          Compile only the listed plugins
  --enable-pdfdocs        Generate documentation in Portable Document Format

It looks nicer, doesn’t it?

Hopefully, reminding people about this will allow projects to clean up their configure.ac (or configure.in for those still using the old naming convention), or for their users to submit patches, so that the output is decently formatted and usable, even by automatic systems like zsh’s tab completion of ./configure options.

P.S.: since I’ve changed gnumeric configure.in script, I’m going to submit it upstream now, so no need to get to fix that.