This Time Self-Hosted
dark mode light mode Search

IPv6 In Real Life

I have built quite the reputation as an IPv6 contrarian over the years, particularly as I yearly criticize FOSDEM for describing their dual-stack network as a “legacy” network. As I keep reminding people, I’m also a relatively early adopter of IPv6 at home (with related annoyances), and I have eagerly tested and debugged issues with services having (partial) IPv6 support, such as OVH back in the days. So if I’m denying that IPv6 “has won” and is the “current” protocol versus IPv4’s legacy, this is coming from the point of view of someone who would love to see more adoption of IPv6.

Last year, we crossed 10 years since the World IPv6 Launch, the organized “flag day” when many websites started offering their services over IPv6 without having to jump through many particular hoops — for many that meant starting to publish AAAA records for their hostnames, for others it meant stopping filtering those records when resolving hostnames. But in general, that was the day that we “all” roughly decided IPv6 was no longer an experiment.

Since that day, the amount of traffic received over IPv6 continued to increase steadily — which is definitely great news… or is it? This post is actually my way of suggesting that we may be measuring the wrong thing, when we keep talking about IPv6 in terms of relative traffic on the Internet — and that if we do want to realistically think of IPv4 as a legacy technology, there’s still quite a long way to go.

The reason why I’m still insisting that IPv4 is not a legacy technology, but rather the current technology is very simple: if you are on a dual-stack network that suddenly lost IPv6 connectivity, an average user may notice something odd (I say average because us IPv6 geeks would obviously notice — I wouldn’t be able to access my own home network service!), but most likely it would be a matter of latency, rather than completely unusable network. On the other hand, if the same network loses IPv4 connectivity, that will become very clear very quickly, as any service that depends on IPv4 would suddenly just stop working!

Of course, it’s technically possible to do what FOSDEM does, and provide an IPv6-only network with NAT64 on the way out to provide access to the rest of the Internet — except the fact that no (typical) ISP is willing to do that to their customers: too much hardware just won’t work on such a network, so even for those soho-class routers that support IPv6, there’s no way to treat IPv4 as legacy. Which in turns means that basically every single piece of equipment you would buy has likely only been tested with IPv4 present, and IPv6 present maybe out of luck — this is actually a great reason why opt-in IPv6-only network are actually a great thing to have, and indeed Google used to provide one like that for testing their consumer devices. I’ll get back to devices later, first I want to go back on why I believe that the traffic ratio is a good measure.

Five years ago I already described the world of IPv6 only (home) networks as Fantasyland — the main point I made for that most of the websites for services people use for banking, household management, shopping, and so on so forth, are not IPv6 compatible. Both that analysis and my more detailed (if a bit abandoned) IPv6 in Real Life website point two trends out to me: a number of websites will not get IPv6 support until their cloud providers will get IPv6 support (I’m told AWS finally has that available, compared to then), and on the other hand websites behind CDNs such as CloudFlare, Akamai, or Fastly are all happy to serve IPv6 traffic. Neither of these are surprising if you’re used to the complexity of cloud architectures or distributed services: your dependencies mandate what you can or cannot do to begin with.

The way the world is going, I can see that more and more websites are turning to CDNs to serve at least their public-facing websites, to the point that it sometimes feels like it would be really unwise not to use a CDN for a new corporate website in the first place. The reason for this are likely worth of academic research, but at the very least feel like a combination of increased audience (more people are online), a push from Google (primarily) for websites to improve their performance (or be down-listed in their results), a scary increase in ease to run DDoS (particularly after it having been used in protest), and generally a loss on the war on bots — I wanted to link to some of Troy Hunt’s reports about how CloudFlare protected his bank account by caching the content that would have served from Azure, but the only reference I found was on Twitter, and I don’t think it’s wise to count on it.

With more websites being served over CDNs, that increases the ratio of traffic going to the very same CDNs, which in turn (since they all pretty much do support IPv6) increases the traffic ratio of IPv6 — so why do I insist it’s the wrong measure? Well, at the same time as CDNs increased their ratio of traffic, the average user traffic patterns have changed — Google, Netflix, and Meta (full disclosure, I’m currently a Meta employee and a former Google employee) are big adopters of IPv6, and all of the traffic to their product is certainly part of the IPv6 traffic! An increase in IPv6 traffic (at a home network, or ISP level) could very well mean an increase in the amount of traffic caused by streaming YouTube, Netflix, or Instagram Reels rather than any change in adoption of IPv6.

To be fair, though, measuring the proportion of traffic that those services receive over IPv6 – which both Meta and Google do and report on – has value: it provides us with information about the availability of IPv6 networking — and it tells us that currently less than 40% of networks provide IPv6. And that’s considering that (as many would remind me) a number of Mobile Internet providers have been providing IPv6-only network for 4G/5G for years.

Another measurement that I have been hearing a lot about is the amount of websites in the Alexa Top 100 (or other similar rankings) that publish IPv6 records for their hosts. I find this is definitely more interesting, but it needs some interpretation — and as you can guess by the fact I set up a separate website tracking it, I don’t believe that the right answer for this is to track a top-100 list of domains.

The first problem here is the weighing — because most of these lists are global list of most visited websites, they skew hard for websites that target a global, or large country audience. Amazon sunset the Alexa list last year, and I honestly haven’t searched for what the current replacement for this is, but searching for the last snapshot of it, in the top-10 domains, you get four Chinese services and five “Big Tech” domains — the only odd one out there is Wikipedia.

What this means in practices is that even though there are countries with very good IPv6 infrastructure (Czechia being a clear winner on my site), they are unlikely to be recorded on these results — while the moment China could have a full deployment of IPv6 (according to my employer’s data, it seems to have jumped last year in August, but still not stable — although obviously with the Great Firewall existing, this is not going to be an easy measurement) it would cover a large number of most domain leaderboards.

But if the weighting could be solved by using per-country leaderboards, there is something else that cannot be solved as easily: the fact that nowadays a website is rarely a single hostname. I’m not talking about separated asset delivery hosts (which allow you to leverage CDNs even when implementing an in-house complex web app that cannot be fully outsourced), or worse yet the analytics hosts. I’m talking about the fact that (for very good reasons, in my opinion!) most companies would not let the their marketing copy and user-access account web apps sit on the same host, or even with the same authentication system!

This may sound odd, but it’s actually quite standard practice. I’ll take banks as example because this is something that everyone has to use, one way or another, but the same is true for most srvices out there. In both the UK and Italy,banks appear to have a confusing, unmemorable hostname for their online banking, versus the site with all of the commercial information. Somehow Chase US is once again the surprise outlier here. This separation is not just useful for security (a marketing person account being compromises won’t cause user account information to be easily compromised) but it also makes for a much easier integration with CDNs, as the “marketing” site does not need user customization.

In my implementation, I have explicitly tried to track the additional hostnames involved in making use of a service, not just being able to load its marketing page. So taking Medicinos Bankas in Lithuania: its marketing page is served by Cloudflare, but their online banking login page isn’t and it’s served by their own network. It’s reasonable, I’m not sure if I’d be comfortable if Cloudflare could theoretically see my banking information and possibly leak it.

Funnily enough I can totally see the opposite situation happening, as well — that is, marketing websites with no IPv6 while the actual service supports it. It’s not unreasonable to use WordPress to maintain a marketing copy website, but Automattic still to this day does not support IPv6. While other hosting provides exist, and you could host your own WordPress, it is not an unreasonable expectation to outsource a lower-sensitiveness website.

My methodology is obviously not perfect, it does not cover a lot of important cases, such as payment flows — very few online shops implement their own payment acquisition integration, as that would be a significant cost, with high requirements, that would distract from the core business of a shop or service provider. Whether it is PayPal, Stripe, Shopify (none of which appear to publish AAAA records for their main website!), or more local alternatives, these providers need to support IPv6 well before their customers, as demonstrated by the OVH case above. If payments were to fail, and customers couldn’t be acquired, that would be a significant cost to the business, making IPv6 a risk.

This is where the attitude of what I can only hope is a vocal minority of network engineers get in the way of their goal of IPv6 adoption: you cannot just say “our network is IPv6 ready, we’ll turn it on!” — you need to be able to at least talk with the folks working at “application level” and be able to test and verify that everything fits together. Otherwise you end up with Telegram-style screwups where the application is completely ignoring the real peer IP and logging (in the best case) a fake address. Sometimes, you may actually go further up than the application developers! Do you know whether all the dependent services your web application uses support IPv6? Do the users send requests to them directly, or do you proxy those requests? Are you sending the requestor’s IP address over, for the sake of rate limiting and user protection? If so, has that flow been tested with IPv6 requests?

I promise I would come back to hardware — but I don’t want to dig again in why I will insist that IoT customers are not stupid, so please read the previous blog post if that’s what you want to argue here.

As I said above, I have a dual-stack network at home, and while I have not tried to figure out the current ratio of traffic between the two stacks on my network, I found myself cursing IPv4 and DHCP some weeks ago. You see, I’m using UniFi network with a controller that I migrated between different systems over the past five years, and between various virtual machines, test SBCs, and various other devices, I ended up getting to the point where the subnet “autoscaling” triggered – or at least tried to because on the USG it doesn’t work right – since it would have started reusing addresses in the /24 IPv4 network it used to have.

On IPv6 things would have worked fine, as I have a full /56 delegated from Hyperoptic, which means I’m distributing “standard” /64 addresses within the network… except of course most of the devices need to be in the dual-stack world at best, and most of them need to stay in the “legacy” IPv4 world probably forever, because I doubt that hardware out there would care to update their stack. Even ESPHome, which is used by the majority of the devices I have at home, only works with IPv6 in very limited circumstances — despite the fact that it would gladly stay entirely local and would be a perfect application for IPv6 with stable EUI64 addressing in the first place!

The last thing hardware manufacturers would want to deal with is requiring features from a network that are not universally present — even Multicast DNS is at best a shortcut right now, and not the sole way to access most services, due to the way WiFi routers sometimes manage to mess it up entirely. How do you expect these devices to be developed with the ability to run on a network that has no IPv4 in the first place?

And let’s not even talk about Android and its reliability (or lack thereof) to obtain IPv6 over WiFi — with the well known stubborn engineer being discussed and deconstructed for years for his complete refusal to reach feature parity with basically every other operating system out there in the form of DHCPv6. While, yes, of course it would be lovely if this could be used as a forcing function for every network operator to ditch user-hostile practices, the reality is that this turns most into a question of how hard it is to even provide IPv6 support on WiFi networks that are not directly in control of their users, such as most public WiFi — it has to just work, for them, as they wouldn’t be able to debug every customer having a different problem!

Generally, my opinion has not changed (much) over the year. While IPv6 is a great technology in theory, from the point of view of businesses that need to deploy it, it is a risk — a risk that, for most small to medium businesses, has very little upsides, as I do not believe there is any one single “killer feature” in IPv6 that (non-geek) customers have been clamoring for. HTTPS was different, and even that is not fully deployed. While I believe an investment on the deployment of IPv6 before it becomes an effective requirement is a great idea, it is the type of R&D project that network people need to learn to “sell” to their management — while accepting that it might not be something a struggling business would be interested in. Legacy, but always reachable, is better than modern while risking to lose customers.

Now, don’t mistake my skepticism at the “Mission Accomplished” feeling with accepting failure. IPv6 has come a long way since 15 years ago, and the fact that I can indeed watch all kind of movies while my DHCP server decided to give up its soul (but, crucially, not receive email!) makes me fairly happy. More importantly, I can have a monitoring setup that can reach out from my hosting provider to my flat’s infrastructure without needing a VPN (not that Tailscale didn’t make that almost as easy), while not requiring to expose it to the wider Internet.

Indeed, for backend usage IPv6 is an absolute fantastic choice. I ran the Tinderbox with Linux Containers with their own (publicly routed) IPv6 with nearly no issue, and I could very easily jump into any of them by SSH without using jumphosts or static configurations — and this was over 10 years ago! Nowadays, if you do not serve the public, it is a great choice to have an IPv6 only server, and indeed most of the cloud providers nowadays charge you extra if you need a v4. (If I didn’t need to serve the redirector, I would probably have done away with IPv4 for my monitoring server!)

I do not know what’s going to tip the scale for consumer-facing services — I personally have a feeling that IPv4 will stick around for my lifetime, unless something big happens. Maybe it’s going to be regulation, maybe it’s going to be a black swan, or maybe it’s going to be the perfect router that becomes so ubiquitous that it is unreasonable for consumers to complain if something works there and not on their setup.

Does this mean that developers have excuses to ignore IPv6? I don’t think so! IPv6 support in software is most definitely needed — as noted above, IPv6-only servers are a thing nowadays, since they are cheaper. And unless you’re working for a consumer-facing service, the whole “legacy and safe” discussion doesn’t apply — for instance, I’m significantly miffed that Automattic does not support IPv6 at all, it means I cannot run my own WordPress instance on a v6 only server, while serving it to the public via a CDN (since to active Jetpack you need their servers to reach yours.) If I was a startup looking to be paid, I would also probably be annoyed at Stripe for not supporting IPv6, though that’s a bit more understandable anyway.

More importantly, we should be paying more attention to tutorials, because right now there’s too much content that just assumes “192.168.0.x” left and right — and without some IPv6-focused updates, it’s unlikely that newcomers would be able to avoid hardcoding these expectations by mistakes. I have ranted publicly about the terrible state of looking for documentation on using Docker with IPv6 — hopefully soon I’ll be able to get myself to document this in the official places, because it turned out to be a lot simpler, once I stopped reading old, out of date tutorials.

So, to close it off, let me repeat myself: I’m not a hater of IPv6 — I’m a realist, expecting IPv4 and dual-stack solutions to stay around for a very long time. I’m also skeptical of yelling “Mission Accomplished”, or saying that it is someone else’s problem if IPv6 adoption is not complete yet. I also am a bit… concerned about the amount of people who appear to have made IPv6 so much part of their persona that they refuse to hear any of the criticism related to its deployment or implementation. I wish them good luck.

If you are interested in keep track of IPv6 in real life, I welcome contributions to either the list of websites to check or to the methodology over GitHub (assuming you have IPv4 connectivity, since that is also still not publishing AAAA records) — I’m not wed to any of the current details of the implementation, and it would really be nicer to use an actual headless Chrome to see if the page renders at all with just v6, even though that would be a lot more complex and resource-hungry to generate.

Comments 5
  1. Many (large) ISPs (FTTH and mobile) are now providing IPv4 over IPv6. This trend is likely to continue as the IPv4 addresses are scarce. This may be hidden by the CPE (the box or the mobile phone) but if IPv6 fails, you won’t have IPv4 either. Content providers should worry about IPv4 being a second-class citizen as it needs to go through some additional boxes to reach its target. If these boxes fail or behave poorly, the major services are still available. This seems fair in this case to say that IPv4 is now a legacy (but not extinct) protocol.

    1. Not in the UK or Italy at least. Maybe Czechia and France?

      And if you can show me a home router that is using IPv6 only on the home network side, I’d be surprised. Mobile network? Sure, I noted that as an existing case — those cases exists but they’re definitely not as risky as some of the business cases I noted regarding payments.

      For home users, I’d be extremely surprised if any FTTH home router doesn’t provide local IPv4, which is definitely not the same case as “6 over 4” — because those routers would make it impossible to use any IoT device. Almost certainly those end up as CGNAT for the most part.

      I think that you’re expecting a momentum shift that will never happen by waiting for the IPv4 shortage. We’ve been hearing this for twenty years or more, and Telecom Italia has still not revoked the /29 allocation that they issued me when I had a business contract.

  2. I’ve recently gotten FTTH here in Germany, and it’s the first ISP that I’ve encountered in many years which has never heard of IPv6. I’ve asked them about it but got nothing. In all other regards, service quality is great though.

    You’re saying introducing IPv6 is a risk for businesses. Risks are taken if there is a big enough reward on the other side. Right now the visible reward is minimal… A couple of $/€/£ per year for IPv4 addresses saved – that’s it. Either there must be a much bigger visible reward, or prices of IPv4 addresses must go up dramatically for “ordinary people” to care about IPv6.

  3. If we admit that IPv6 will not fly this way, then we should think at more forcing methods especially on the server side. Fine the domain holders if they don’t have IPv6. If you look in history standardization won’t go if there isn’t some kind of enforcing. US is still not on metric. But those internet folks are stubborn.

    1. I’m hoping you’re parodying this, as otherwise you’re suggesting someone should be fined because they don’t adopt a technology that has failed to provide enough benefits to display the risks of adoption.
      IPv4 _is_ a standard — it might not be the standard you want, but it is a standard.

Leave a Reply to FlameeyesCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.