This post is an interlude between Gentoo-related posts. The reason is that I have one in drafts that requires me to produce some results that I have not yet, so it’ll have to wait for the weekend or so.
You might remember that my original IPMI plugin was written in POSIX sh and awk, rather than bash and gawk as the original one. Since then, the new plugin (that as it turns out might become part of the 2.1 series but not to replace both the old ones, since RHEL and Fedora don’t package a new enough version of Freeipmi) has been rewritten in Perl, so using neither sh nor awk. Similarly, I’ve written a new plugin for sensors which I also wrote in Perl (although in this case the original one also used it).
So why did I learn a new language (since I never programmed in Perl before six months ago) just to get these plugins running? Well, as I said in the other post, the problem was calling the same command so many times, which is why I wanted to go multigraph — but when dealing with variables, sticking to POSIX sh is a huge headache. One of the common ways to handle this is to save to a temporary directory the output of a command and parse that multiple times, but that’s quite a pain, as it might require I/O to disk, and it also means that you have to execute more and more commands. Doing the processing in Perl means that you can save things in variables, or even just parse it once and split it into multiple objects, to be later used for output, which is what I’ve been doing for parsing FreeIPMI’s output.
But why Perl? Well, Munin itself is written in Perl, so while my usual language of choice is Ruby, the plugins are much more usable if doing it in Perl. Yes, there are some alternative nodes written in C and shell, but in general it’s a safe bet that these plugins will be executed on a system that at least supports Perl — the only system I can think of that wouldn’t be able to do so would be OpenWRT, but that’s a whole different story.
There are a number of plugins written in Python and Ruby, some in the official package, but most in the contrib repository and they could use some rewriting. Especially those that use net-snmp or other SNMP libraries, instead of Munin’s Net::SNMP wrapper.
But while the language is of slight concern, some of the plugins could use some rewriting simply to improve their behaviour. As I’ve said, using multigraphs it’s possible to reduce the number of times that the plugin is executed, and thus the number of calls to the backend, whatever that is (a program, or access to /sys), so in many cases plugins that support multiple “modes” or targets through wildcarding can be improved by making them a single plugin. In some cases, it’s even possible to reduce multiple plugins into one, as I did to the various apache_* plugins shipping with Munin itself, replaced on my system with apache_status as provided by the contrib repository, that fetches the server status page only once and then parses it to produce the three graphs that were, before that, created by three different plugins with three different fetches.
Another important trick up our sleeves while working on Munin plugins is dirty config which basically means that (under indication from the node itself), you can make the plugin output the values as well as the configuration itself during the config execution — this saves you one full trip to the node (to fetch the data), and usually that also means it saves you from having to send one more call to the backend. In particular with these changes my IPMI plugin went from requiring six calls to ipmi-sensors per update, for the three graphs, to just one. And since it’s either IPMI on the local bus (which might require some time to access) or over LAN (which takes more time), the difference is definitely visible both in timing, and in traffic — in particular one of the servers at my day job is monitoring another seven servers (which can’t be monitored through the plugin locally), which means that we went from 42 to 7 calls per update cycle.
So if you use Munin, and either have had timeout issues in the past or recently, or you have some time at hand to improve some plugins, you might want to follow what I’ve been doing, and start improving or re-writing plugins to support multigraph or dirtyconfig, and thus improve its performance.