Tag: Opera

Serving WebP images

A few months ago I tried experimenting with WebP so to reduce the traffic on my blog without losing quality. At the end the results were negative and I decided instead to drop the backgrounds on my blog and replace them with some CSS to provide gradients, which was not a lossless change, but was definitely easier to load.

After the VideoLan Dev Days 2013 (of which I have to write some report soon, I went to speak with Pascal again, and he was telling me that the new version of Chrome finally fixed the HTTP accept header, so that finally it will prefer WebP to other image formats if present. I confirmed this, as Chrome 30 reports Accept: image/webp,*/*;q=0.8. The q= parameter is not needed for Apache actually, but it’s a good idea to have it there anyway.

Thanks to this change, and mod_negotiation’s MultiViews it’s possible to auto-select the format, between JPEG, PNG and WebP, for Chrome users. Indeed if you’re visiting my blog with Chrome 30 (not sure about 29), you’re going to be served mostly WebP images (the CC license logos are still provided as PNG because the lossless compression was worse, and the lossy one was not saving enough bytes to be worth it).

I started working on enabling this while waiting at Terminal 1 at CDG airport (this is the last time I’m flying Aer Lingus to Paris), and I was able to finish the thing before my flight boarded. What I did realize just before that, though, is that Apache would still prefer serving WebP, I’d venture a guess that it’s because it’s smaller in size. This is okay for Opera, Firefox and (obviously) Chrome, but not for Safari or IE.

Of course if the other browsers were to actually report the formats they supported it would be all fine, but that’s not the case. In particular, Firefox actually prefers image/png to anything else (Apache will add a low q= value to any glob request, just to be on the safe side, which is why I said earlier that q= is not needed for it), so that even if I don’t make any more changes, Firefox will still prefer PNG to WebP (but won’t do anything for JPEG, so if the web server prefers WebP to JPEG, it’s going to be fine).

So how to provide WebP without breaking other browsers? One solution would be to use PageSpeed to compress the images on the fly to WebP when requested by a compatible browser, but that is a bit of overkill, and is hard to package right, and, most importantly, requires browser detection logic on the server, which is not very safe.

At the end I decided to go with a safer option: provide WebP only to Chrome users and not to users of other browsers, at least until they decide to fix their Accept headers. But how to do that? Well, I needed to check Apache’s source code, because documentation does not seem to explain that clearly and explicitly, but to decide which format to serve, Apache will multiply the q= parameter coming from the browser, or its implicit values (that make image/* and */* have a default value of less than 0.1) with the qs= parameter passed when declaring the type:

AddType image/jpeg .jpeg .jpg .jpe
AddType image/png .png
AddType image/webp;qs=0.9 .webp

By using the value 0.9 to webp, and leaving the default 1 to the other formats, I’m basically telling Apache that, all things equals (like if the browser is sending Accept: */*, Internet Explorer style), I prefer to provide PNG or JPEG to the users, rather than WebP. It will also prefer to serve JPEG to Firefox (which uses image/*). Chrome 30, on the other hand, explicitly prefers WebP over any other image format, and so Apache will calculate the preference as 1.0*0.9 for WebP and 0.8*1.0 for PNG and JPEG. I have not checked what Opera does, but it looks like all the browsers on my cellphone would support WebP but they don’t prefer it, so they won’t be served it either.

So right now WebP images for my blog are an exclusive of Chrome users; the win is relatively good, by halving the size of the Autotools Mythbuster cover on the right, and shaving off a few bytes from the top image for the links. There are definitely more interesting way to save bandwidth by re-compressing the images that I used around the blog (many of which, re-compressed, end up taking half the space), but that will have to wait for me to fix this bloody Typo as right now editing posts is failing.

Another thing that I will have to work on is a tool to handle the re-compression. Right now I’m doing so by hand, and it’s both a waste of time, and prone to errors. I’ll have to come up with a good way to quickly describe images so that a tool can re-compress them and evaluate whether to keep them in WebP or not, and at the same time I need to find a way to store the originals at the highest quality. But that’s a topic for a different time.

The WebP experiment

You might have noticed over the last few days that my blog underwent some surgery, and in particular that some even now, on some browsers, the home page does not really look all that well. In particular, I’ve removed all but one of the background images and replaced them with CSS3 linear gradients. Users browsing the site with the latest version of Chrome, or with Firefox, will have no problem and will see a “shinier” and faster website, others will see something “flatter”, I’m debating whether I want to provide them with a better-looking fallback or not; for now, not.

But this was also a plan B — the original plan I had in mind was to leverage HTTP content negotiation to provide WebP variants of the images of the website. This was a win-win situation because, ludicrous as it was when WebP was announced, it turns out that with its dual-mode, lossy and lossless, it can in one case or the other outperform both PNG and JPEG without a substantial loss of quality. In particular, lossless behaves like a charm with “art” images, such as the CC logos, or my diagrams, while lossy works great for logos, like the Autotools Mythbuster one you see on the sidebar, or the (previous) gradient images you’d see on backgrounds.

So my obvious instinct was to set up content negotiation — I’ve used it before for multiple-language websites, I expected it to work for multiple times as well, as it’s designed to… but after setting all up, it turns out that most modern web browsers still do not support WebP *at all*… and they don’t handle content negotiation as intended. For this to work we need either of two options.

The first, best option, would be for browsers only Accept the image formats they support, or at least prefer them — this is what Opera for Android does: Accept: text/html, application/xml;q=0.9, application/xhtml+xml, multipart/mixed, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 but that seems to be the only browser doing it properly. In particular, in this listing you’ll see that it supports PNG, WebP, JPEG, GIF and bimap — and then it accepts whatever else with a lower reference. If WebP was not in the list, even if it had an higher preference for the server, it would not be sent to the client. Unfortunately, this is not going to work, as most browsers send Accept: */* without explicitly providing the list of supported image formats. This includes Safari, Chrome, and MSIE.

Point of interest: Firefox does explicit one image format before others: PNG.

The other alternative is for the server to default to the “classic” image formats (PNG, JPEG, GIF) and then expect the browsers supporting WebP prioritizing it over the other image formats. Again this is not the case; as shown above, Opera lists it but does not prioritize, and again, Firefox prioritizes PNG over anything else, and makes no special exception for WebP.

Issues are open at Chrome and Mozilla to improve the support but they haven’t reached mainstream yet. Google’s own suggested solution is to use mod_pagespeed instead — but this module – which I already named in passing in my post about unfriendly projects – is doing something else. It’s on-the-fly changing the content that is provided, based on the reported User-Agent.

Given that I’ve spent some time on user agents, I would say I have the experiences to say that this is a huge pandora’s vase. If I have trouble with some low-development browsers reporting themselves as Chrome to fake their way in with sites that check the user agent field in JavaScript, you can guess how many of those are going to actually support the features that PageSpeed thinks they support.

I’m going to go back to PageSpeed in another post, for now I’ll stop to say that WebP has the numbers to become the next generation format out there, but unless browser developers, as well as web app developers start to get their act straight, we’re going to have hacks over hacks over hacks for the years to come… Currently, my blog is using a CSS3 feature with the standardized syntax — not all browsers understand it, and they’ll see a flat website without gradients; I don’t care and I won’t start adding workarounds for that just because (although I might use SCSS which will fix it for Safari)… new browsers will fix the problem, so just upgrade, or use a sane browser.

Browsers on the Kindle Fire

A few days ago I talked about Puffin Browser with the intent to discuss into more details the situation with the browsers on the Kindle Fire tablet I’m currently using.

You might remember that at the end of last year, I decided to replace Amazon’s firmware with a CyanogenMod ROM so to get something useful on it. Beside the lack of access to Google Play, one of the problems I had with Amazon’s original firmware was that the browser that it comes with is flakey to the point of uselessness.

While Amazon’s AppStore does include many of the apps I needed or wanted – including SwiftKey Tablet which is my favourite keyboard for Android – they made it impossible to install them on their own firmware. I’ve been tempted to install their AppStore on the flashed Kindle Fire and see if they would allow me to install the apps then, it would be quite a laugh.

Unfortunately, while the CM10 firmware actually allows me to make a very good use of the device, much more than I could ever have reached with the original firmware, the browsing experience still sucks big time. I’ve currently installed a number of browsers: Android’s stock browser – with its non-compliant requests – Google Chrome, Firefox, Opera and the aforementioned Puffin. There is no real winner on the lot.

The Android browser has a terrible network implementation and takes way too much time requesting and rendering pages. Google Chrome is terrible on the whole, probably because the Fire is too underpowered to run it properly, which makes it totally useless as an app. I only keep it around for testing purposes, until I get a better Android tablet.

Firefox has the best navigation support but every time I click on a field and SwiftKey has to be brought up, it takes a full minute. Whether this is a bug in SwiftKey or Firefox, I have no idea. If someone has an idea who to complain about it to, I’d love to report it and see it fixed.

Best option you get, beside Firefox, is Opera. While slightly slower than Firefox on rendering, it does not suffer from the SwiftKey bug. I’m honestly not sure at this point if the version of Opera I’m using right now renders with their own Presto engine or with WebKit which they announced they are moving to — if it’s the latter, it’s going to be a loss for me I guess, since the two surely WebKit based browsers are not behaving nicely for me here.

Now from what I said about Puffin, you’d expect it to behave properly enough. Unfortunately that is not the case. I don’t know if it’s a problem with my local bandwidth being too limited, but in general the responsiveness is worse than Opera, although not as bad as Puffin. The end result is that even the server-side rendering does not make it usable.

More reviews of software running on the Fire will follow, I suppose, unless I decide to get a newer tablet in the next weeks.

Inconsistent Scalable Vector Graphics

The one job I’m taking care of at the moment involves me drawing some stuff using SVG in C code, without using any support libraries. Without going into much detail, since I cannot because of an NDA, I can say the generated file has to be as small as possible since, as you might guess by now, it has to be done on an embedded system.

The task itself is not too difficult, but today I started the actual reduction of the code so that it fits in the software the I have to develop, and here starts the problems. The first issue has been I was tired of looking up the correct attributes for each SVG element, so I ended up doing the same I did for DocBook 5 and added a new ebuild to portage: app-emacs/nxml-svg-schemas:1.1 which installs the SVG 1.1 schemas so that Emacs’s nxml-mode can tab-complete the elements and attributes. I positively love Emacs and nXML since it allows me to have specific XML variants support by just adding its schemas to the system!

A little note now about nXML though: I’ll have to contact upstream because I found one nasty limitation of it: I cannot make it locate the correct shemas on a version basis, which means I won’t be able to provide SVG 1.2 schemas alongside as 1.1 with the code as it is; if I can get a new locating rules schemas that can detect the correct schema to use also through version, that’s going to solve not only SVG 1.2 but also future DocBook versions. So this enters my TODO list. Also, am I the only one using nXML in Gentoo? I’m maintaining all the three schemas ebuilds, it’s not like it’s a big hassle, but I wonder what would happen if I were to leave Gentoo — or more likely at this point if I were to end up in the hospital again; I hope I’m fine now but one is never sure, and my mindset is pretty pessimistic nowadays.

At any rate, I’ve been studying the SVG specifications to find a way to reduce the useless data in the generated file, without burdening the software with doing manual calculations. The easy way out is to use path and polyline elements to draw most of the lines in the file, which would be fine if it wasn’t they only accept coordinates in “pixels” (which are not actual pixel, but just the basic unit for the SVG file itself). This is not too bad since you can define a new viewport which can have an arbitrary size in “pixels”, and is stretched over the area. The problem is with supporting the extra viewports.

The target of the file to generate is to work on as many systems as possible, but it’s a requirement that it works on Windows with Internet Explorer, as well as Firefox. For SVG files under Internet Explorer there is the old, unmaintained and deprecated Adobe SVG plugin (which is still the default Internet Explorer will try to install) and the examotion Renesis Player which is still maintained. So I take out my test file, and try it.

I wrote the file testing it with eog which I’m not sure which SVG library uses for the rendering and with rsvg that uses librsvg obviously; with those, my test file was perfect, the problem has been with other software, since I got the following results:

  • Inkscape wouldn’t load it properly at all and just draw crazy stuff;
  • batik 1.6 worked;
  • Firefox, Safari and Opera shown me grey and red rectangles rather than the actual lines I wrote in the SVG;
  • Renesis Player shown me lines, but way too thick for what I wanted;
  • OpenOffice shown it with the right dimensions but didn’t translate it 2×2 cm down from the upper left corner like I instructed the svg to.

After reporting the issue on examotion’s tracker, since that is the most important failure in that list for my current requirements, I got a suggestion of switching the definition of font-size to direct attribute rather than through style so to change the actual svg measure unit. This made no difference for the three implementations that worked before, nor on examotion, but actually got me one step closer to the wished result:

  • inkscape still has problems, the white rectangle I draw to get a solid white background is positioned over the rest of the elements, rather than under like I’d expect since it’s the first element in the file; it also does not extend the grid like it should, so the viewBox attribute is not properly handled;
  • OpenOffice still has the problem with translation but for the rest seems fine;
  • Safari still has the same problems;
  • Opera 9.6 on Windows finally renders it perfectly, but fails under Ubuntu (?!);
  • Firefox official builds for Windows and OSX, as well as under Ubuntu, work fine; under Gentoo, it does not, and still show the rectangles;
  • Adobe SVG plugin work fine.

At this point I should have enough working implementations so that I can proceed with my job task, but this actually made me think about the whole thing about SVG, and it reminded me tremendously of the OASIS OpenDocument smoke which I had a fight with more than three years ago. I like very much XML-based technologies for sake of interoperation, but it’d be nice if the implementations actually had a way to produce a proper result.

Like in OpenDocument, where the specifications allow two different styles for lists, and different software implements just one of them, making themselves incompatible one with the other, SVG defines some particular features that are not really understood or used by some implementations, or can create compatibility issues between implementations.

In this case, it seems like my problem is the way I use SVG subdocuments to establish new viewports, and then use the viewBox feature to change their unit space. This is perfectly acceptable and well described by the specifics, but it seems to cause further issues down the line with the measure units inside and outside these subdocuments. But it seems like the problem is not just one-way, from this other bugreport on Inkscape you can also see that Inkscape does not generate so pure SVG as it should.

While XML has been properly designed to be extensible, thanks to things like namespaces and similar, one would expect that the feature provided by a given format would be used before creating your own extensions; in this case from that bug report you can see (and I indeed double checked that it still is the case) that Inkscape does not use SVG’s own features to establish a correspondence between “SVG pixels” and the size in real-world units of the image; indeed, it adds two new attributes to the main document: inkscape:export-xdpi and inkscape:export-ydpi, while SVG expects you to use the viewBox for providing that information.

Sigh, I just wished to get my graph working.