This Time Self-Hosted
dark mode light mode Search

Filtering compiler optimisation flags is not a solution

Strangely enough this post is not brought but something that happened recently (I usually write in response of stuff that happens), but it’s a generic indication that I’ve had to explain to too many people in the past. The most recent thing that might link to my writing this is my note about Pidgin crashing more on Fedora and a little discussion on the matter with a friend of mine.

So, we all know compiler optimisation flags in Gentoo, and most users trying some exotic ones probably know that a lot of ebuilds tend to filter, strip or otherwise reduce the number of flags actually used at build time. This is, in many cases, a violation of Gentoo policies, and Mark being both QA and Toolchain master usually get upset by them. Since this is often abused I’d like to explain here what the problem is.

First of all, not all compiler flags are the same: there are flags that change behaviour of the source code, and others that should not change that behaviour. For instance the -ffast-math flag enables some more loose mathematical rules, this change the behaviour of the math source code as it’s no longer perfect; on the other hand the -ftree-vectorize only changes the output code and not the meaning of the source code, and should then be counted in as a safe flag.

You can see already the gist here: -ftree-vectorize has been called for build and runtime errors in the past few years, so it’s often not considered safe at all and indeed it’s often considered one of the less safe flags. But there are a few catches here: the first is that yes, the implementation of the flag might be at fault, and in the past it caused quite a few internal compiler errors, or miscompilation of source code into something that fails at runtime. But both these issues has to be reported to the GCC developers to be fixed because they are bug in GCC to begin with, so if the issue is just ignored by disabling the flag, they won’t be fixed any time soon.

Sometimes, though, the issues are neither a problem of miscompilation nor a bug in GCC, yet the package fails to execute properly or fails to build entirely; the latter happened with mplayer not too long ago. In these cases there’s still a bug, and it’s in the software itself, and needs to be fixed. In the case of mplayer for instance it has shown that the inline assembler code was using global labels rather than local lables like it should have been in the first place. Fixing the code wasn’t that hard, compared with the flag’s filtering.

Now, don’t get me wrong, I know there are at least a few issues with the approach I just noted: the first is that as the FFmpeg developers found out, -ftree-vectorize is not often a good idea, and can actually produce slower code on most systems, at least for the common multimedia usage methods. The second problem is that, with the exception of the mplayer bug, most of the build and runtime failures aren’t straightforward to fix; and when the problem is in GCC, it might take quite a while before the issue is fixed; how should we work those situations out then, if not by filtering?

Well, filtering works fine as a temporary option, a workaround, a band-aid to hide the problem from users. So indeed we should use filtering; on the other hand, this is a problem akin to those related to parallel make or --as-needed: you should not let the user be bitten by the problem, but at the same time you should accept that you haven’t fixed the bug just yet. My indication is thus keep the bug open if you “solved” it by filtering flags!

I know lots of developers dislike having bugs open at all, but it’s not really fixed if you just applied a workaround. And if you close it, nobody will ever see it again, and this will result in a phantom bug that will take a much longer time to reproduce, verify, and fix properly. This is for instance the problem when I hit a package that, without any comment in either ebuild or change log, has a strip-flags call, which reduces the amount of flags passed to the compiler: finding whether the call is there because of a reported bug, or just because the Gentoo developer involved couldn’t be bothered by following the policy, requires time.

And finally, users please understand that the flags like -ffast-math or -fvisibility that do change the meaning of the source code should not be used by users but should rather be applied directly by upstream if they are safe!

Comments 7
  1. What about coming up with a list of flags that are generally safe to use together with a list of what they do and then point users at that list?

  2. Nah, stripping -fvisibility is not the right solution, you should send the cluebat squad to the user instead 😛

  3. If people add -fvisibility to CXXFLAGS they either need basic information or are beyond help.-fvisibility-inlines=hidden is another matter and should work for the vast majority of software. (Assuming a emerge -e world after major GCC updates.)

  4. Just as Reimar and David said; -fvisibility should not be stripped: the user should not be using it in the first place!If we started filtering all the flags that do change the ABI and the code generation, that would be a very long task.

  5. I agree with you about to NOT close bugreports which filters flags as a *temporary fix* and to leave it open until a proper fix is submitted upstream and/or in portage tree (ther former is preferable).Maybe it’s better to add these raccomendations to the Devel Handbook as at moment it’s quite useless to inject unwritten rules because every time someone have to deal with the Gentoo Devels they tend to apply *personal* opinions on the rules, creating different conventions and making impossible to have a decent dialog with them (half devels told you to do things in one manner, the other half told you to do the exact opposite; in the end, no matter what you do, a Gentoo Devel will bark always at you: “you are wrong!”).p.s.: maybe must be added also a new section on the Devel Handbook to list the gcc cflags that change the souce code and the one that do not do it.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.