Running services on Virgin Media (Ireland)

Update 2020-09-16: lots of customers of Virgin Media Ireland appear to be finding this blog post trying to get IPv6 Prefix Delegation working. Unfortunately, the instructions are now completely out of date and provided only for a historical perspective. I haven’t lived in Ireland for three years now, so I don’t know what the current best practices are – if you do, please let the other readers know in the comments! – but the last setup I had for this involved using RA bridging, which is not the easiest thing to set up. Unfortunately I can’t even document how I did that, since movers stole my router when I moved.

Update: just a week after I wrote this down (and barely after I managed to post this), Virgin Media turned off IPv6-PD on the Hub 3.0. I’m now following up with them and currently without working IPv6 (and no public IPv4) at home, which sucks.

I have not spoken much about my network setup since I moved to Dublin, mostly because there isn’t much to speak, although admittedly there are a few interesting things that are quite different from before.

The major one is that my provider (Virgin Media Ireland, formerly known as UPC Ireland) supports native IPv6 connectivity through DS-Lite. For those who are not experts of IPv6 deployments, this means that the network has native IPv6 but loses the public IPv4 addressing: the modem/router gets instead a network-local IPv4 address (usually in the RFC1918 or RFC6598 ranges), and one or more IPv6 prefix delegations from which it provides connectivity to the local network.

This means you lose the ability to port-forward a public IPv4 address to a local host, which many P2P users would be unhappy about, as well as having to deal with one more level of NAT (and that almost always involves rate limiting by the provider on the number of ports that can be opened simultaneously.) On the other hand, it gives you direct, native access to the IPv6 network without taking away (outbound) access to the legacy, IPv4 network, in a much more user-friendly way than useless IPv6-only networks that rely on NAT64. But it also brings a few other challenges with it.

Myself, I actually asked to be opted in the DS-Lite trial when it was still not mandatory. The reason for me is that I don’t really use P2P that much (although a couple of times it was simpler to find a “pirate” copy of a DVD I already own, rather than trying to rip it to watch it now that I have effectively no DVD reader), and so I have very few reasons to need a public IPv4 address. On the other hand, I do have a number of backend-only servers that are only configured over the IPv6 network, and so having native access to the network is preferable. On the other hand I do have sometimes the need to SSH into a local box or HTTP over Transmission or similar software.

Anyway back on my home network, I have a Buffalo router running OpenWRT behind the so-called Virgin Media Hub (which is also the cable modem — and no it’s not more convenient to just get a modem-mode device, because this is EuroDOCSIS which is different from the US version, and Virgin Media does not support it.) And yes this means that IPv4 is actually triple-natted! This device is configured to get a IPv6 prefix delegation from the Hub, and uses that for the local network, as well as the IPv4 NAT.

Note: for this to work your Hub needs to be configured to have DHCPv6 enabled, which may or may not do so by default (mine was enabled, but then a “factory restore” disabled it!) To do so, go to the Hub admin page, login, and under Advanced, DHCP make sure that IPv6 is set to Stateful. That’s it.

There are two main problems that needs to be solved to be able to provide external access to a webapp running on the local server: dynamic addressing and firewalls. These two issues are more intertwined than I would like, making it difficult to explain the solution step by step, so let me first present the problems.

On the machine that needs to serve the web app, the first problem to solve is making sure that it gets at least one stable IPv6 address that can be reached from the outside. It used to be very simple, because except for IPv6 privacy extensions the IPv6 address was stable and calculated based on prefix and hardware (MAC) address. Unfortunately this is not the the case now; RFC7217 provides “Privacy stable addressing”, and NetworkManager implements it. In a relatively-normal situation, these addresses are by all means stable, and you could use them just fine. Except there is a second dynamic issue at hand, at least with my provider: the prefix is not stable, both for the one assigned to the Hub, and for that then delegated to the Buffalo router. Which means the network address that the device gets is closer to random than stable.

While this first part is relatively easy to fix by using one a service that allows you to dynamically update a host name, and indeed this is part of my setup too (I use Afraid.org), it does not solve the next problem, which is to open the firewall to let the connections in. Indeed, firewalls are particularly important on IPv6 networks, where every device would otherwise be connected and visible to the public. Unfortunately unless you connect directly to the Hub, there is no way to tell the device to only allow a given device, no matter what the prefix assigned is. So I started by disabling the IPv6 firewall (since no device is connected to the Hub directly beside the OpenWRT), and rely exclusively on the OpenWRT-provided firewall. This is the first level passed. There is one more.

Since the prefix that the OpenWRT receives as delegation keeps changing, it’s not possible to just state the IPv6 you want to allow access to in the firewall config, as it’ll change every time the prefix changes, even without the privacy mode enabled. But there is a solution: when using stable, not privacy enabled addresses, the suffix of the address is stable, and you can bet that someone already added support in ip6tables to match against a suffix. Unfortunately the OpenWRT UI does not let you set it up, but you can do that from the config file itself.

On the target host, which I’m assuming is using NetworkManager (because if not, you can just let it use the default address and not have to do anything), you have to set this one property:

# nmcli connection show
[take note of the UUID shown in the list]
# nmcli connection modify ${uuid} +ipv6.addr-gen-mode eui64

This re-enables EUI-64 based addressing for IPv6, which is based off the mac address of the card. It’ll change the address (and will require reconfiguration in OpenWRT, too) if you change the network card or its MAC address. But it does the job for me.

From the OpenWRT UI, as I said, there is no way to set the right rule. But you can configure it just fine in the firewall configuration file, /etc/config/firewall:

config rule
        option enabled '1'
        option target 'ACCEPT'
        option name 'My service'
        option family 'ipv6'
        option src 'wan'
        option dest 'lan'
        option dest_ip '::0123:45ff:fe67:89AB/::ffff:ffff:ffff:ffff'

You have to replace ::0123:45ff:fe67:89AB with the correct EUI64 suffix, which includes splicing in ff:fe and flipping one bit. I never remember how to calculate it so I just copy-paste it from the machine as I need it. This should give you a way to punch through all the firealls and get you remote access.

What remains to be solved at this point is having a stable way to contact the service. This is usually easy, as dynamic DNS hosts have existed for over twenty years by now, and indeed the now-notorious (for being at the receiving end of one of the biggest DDoS attacks just a few days ago) Dyn built up their fame. Unfortunately, they appear to have actually vastly dropped the ball when it comes to dynamic DNS hosting, as I couldn’t convince them (at least at the time) to let me update a host with only IPv6. This might be more of a problem with the clients than the service, but it’s still the same. So, as I noted earlier, I ended up using Afraid.org, although it took me a while where to find the right way to update a v6-only host: the default curl command you can find is actually for IPv4 hosts.

Oh yeah there was a last one remaining problem with this, at least when I started looking into fixing this all up: at the time Let’s Encrypt did not support IPv6-only hosts, when it came to validating domains with HTTP requests, so I spent a few weeks fighting and writing tools trying to find a decent way to have a hostname that is both dynamic and allows for DNS-based domain control validation for ACME. I will write about that separately, since it takes us on a tangent that has nothing to do with the actual Virgin Media side of things.

Silly projects: Gentoo on a WRT54GL

I have one silly project I should probably look forward to work on this weekend to vent off some steam: getting my WRT54GL router to run with Gentoo/MIPS. I know it’s probably going to fail because I know near to nothing about MIPS, I know nothing about Gentoo/MIPS, and I remember being told that the mipsel target that WRT54GL are is not well supported by Gentoo. Either way, I’m going to try.

You could probably be wondering why I would be trying something as silly as this, and the reasons are actually a bit of a series. The first problem is that I lack an IPMI agent for accessing Yamato remotely (for remote light out and other things) and I’m sure that’s going to be useful to me soonish. The second one is that I need to set up again the routing of my office with a single wireless client, instead of the current setup which I prepared last year with Yamato having a wireless card.

The problem with IPMI is that I plan on not being at home all day every day in the future; I actually hope to be able to get a driving license this year and make good use of it by finding a job out of home (especially important for my mental health lately!) and in that case I’d be likely to need a way to access Yamato remotely if it gets messed up. Having a low-power system like the WRT to work as a jump box is acceptable I’d say.

With routing, the issue is at the same time simpler and more complex. Simpler because I just need a router to route between the general wireless network (which is accessible by almost anybody) and the wired network that I use for the office and my bedroom. More complex because the original setup used the WRT54GL, then I decided to move to just a single network card, but now I’m in a setup that is quite messed up: Yamato routes all the traffic of PlayStation 3, AppleTV, and iMac, as well as the eventual computers I need to fix and other stuff like that which actually translates the whole thing into a real mess, especially because I ended up splitting the network to such extents that applying NFS ACLs simply by IP masks is impossible.

Of course I could just be using OpenWRT like I did before, but since every upgrade of OpenWRT has been a real mess to deal with (with all their changes into setup, interfaces, nvram and so on), I’m actually thinking that Gentoo would be easier to maintain for me, given I’d just have to update the image once in a blue moon hopefully. I could also just start the router through a TFTP-provide image and then leave it to be with that. At any rate, it would also be a nice experiment and a way to learn a bit more about embedded systems, so…

Right now I only found how to take it apart and I noticed that I have to solder in myself the pins for serial console access, unfortunately it’s almost 1am so soldering them in right now is out of question (I also have to find the pins, which I’m not sure I have at home, worse case scenario I’m going to desolder them from somewhere). I should probably go taking a look to whether the Linus kernel tree can boot on this thing. When I last used it, OpenWRT only supported 2.4 on it; while 2.6 was being worked on it didn’t work on this model, and the wireless network was supposed never to work on it, since it’s using the infamous Broadcom wireless chip (which nowadays might actually work out of the Linus tree via the b43 driver).

If anybody (especially the colleagues actually working on Gentoo/MIPS) have a clue about what I’m to expect out of this, I’d be quite happy to hear it, even if it’s “You’re crazy, it’s never going to work” or “Leave it alone, it’s too much hassle to bother with”.

Oh and yah, I know this thing is not powerful enough to build and it’ll have to go through cross-compiling, and cross-compiling with Portage is not nice, and all the stuff like that. I guess the point is that I intend to work a bit more on that matter, even if currently I’m not paid to do so (I was for a while some time ago). There are more than a few things that I’m interested in looking at to find a solution, actually. It’s very low priority (unless someone bribes me to pick it up) but maybe I can be of help to the broader picture somehow.

Atheros and drivers

Since the release of kernel 2.6.26 I’ve had some trouble with WiFi since madwifi-ng fails to load, and I’m left with using the ath5k driver that is provided by the kernel. As it turned out, that driver is pretty bad by itself, and I cannot use iSCSI over it.

Luckily, Atheros released the code for their HAL which should mean that ath5k is going to hugely improve over the course of the next few kernel releases, which is an extraordinary thing. I’m really looking forward for 2.6.28 which should improve the situation at least a bit, maybe allowing me finally to use iSCSI over wireless again.

Unfortunately it doesn’t seem like it’s going to take little time, 2.6.27 is not out yet and the e1000e driver is still broken; that would be nice at least to see how the webcam drivers will get once they are merged in the tree (is Skype going to work with them? When I tried them out of a git tree, the 64-to-32 bit IOCTL bridge failed on a few ioctls that caused Skype to fail accessing the webcam).

But who knows, maybe next year we’ll be having Atheros well supported by all the architectures Linux runs on, so one can easily have Atheros-based WiFi routers to work with OpenWRT, which would finally solve my wireless problems here…

I can dream, can’t I?

About wireless and routers

It’s not like my main interest in the IT field is networking hardware. Actually, albeit I like taking care of it on smaller environment, I don’t know much about large scale networking, and I feel “n00b” every time I hear some fellow devs talking about their work in that area.

Unfortunately it seems like lately I need to take care of a few networking issues for myself. Since the kernel 2.6.26 was released I was unable to use madwifi (as I wrote about); I admit I haven’t tested in the past weeks so it might be fixed now, I just didn’t have time/will to check again. In the mean time I’m using the ath5k driver, that seems to work but not as much as I’d need, so I cannot use iSCSI via wireless anymore, which sucks quite a bit when using the laptop.

There is then the problem with my router that created me a few problems. As I wrote, the table for mac address access control has only 32 entries. I thought it was just a limitation of the interface, but as it turns out, 3Com advertise all their routers as capable of supporting up to 32 clients. This mean that it’s quite unlikely that there’s a way to get around that limit in the table, and even less likely that the same class of routers would work for me.

At this point, I have to find a solution. The obvious one would be to use a Fonera to handle the Wireless connection; this way I could just leave wifi open and be done with it. The problem here is that I’d rather not use WPA2 (or, as someone also noted on my blog) the Nintendo DS won’t connect to WiFi, and that I’d like to have at least some sort of access control. An alternative would be to use a Linux-based access point. I do have the WRT54GL that I used to use when the office had not just Enterprise bug also Prakesh, Klothos and Farragut. The problem is that I’d have to update OpenWRT on it almost surely, and I have no clue how the support is nowadays.

The Linux support for Wireless networking in the past years has been quite in flux, but it started to become quite interesting in the past months. The b43 drivers that finally made possible to use at least a small subset of Broadcom wireless cards could probably allow OpenWRT to use a 2.6 kernel on the WRT54GL sooner or later, but I wonder how its access point mode support would be with that driver. For what I gathered, the ath5k is far from being an alterantive for the job.

Talking about wireless problems, it seems like there are problems even with identifying the correct region. Luckily the channel I’m using is available on all regions, so I’m fine for now. Hopefully, Atheros drivers improvements will follow soon, and ath5k will be able to work with iSCSI and all the rest, maybe even with AP mode so that Linux-based wireless routers would become much easier to manage. Of course the best would be if I could get one of them to run Gentoo directly.

See, the embedded part of the networking game is something I actually am interested about ;)

In the mean time I’m considering the option of resuming the WRT54GL with the same function it had before, and passing a shielded ethernet cable between my office and my bedroom, so I could get the PS3, the AppleTV and the laptop (while I’m there) to talk with enterprise through ethernet directly, and leaving the WRT54GL to connect them to the Wireless. It would probably also be healthier as it means less radio waves in general. The problem of this solution is that I’m not sure I have enough space in the cabling areas, and that I’d need two gigabit switches to make sure that the PlayStation3 and the laptop can communicate at the highest speed possible (I’d need one switch in the office, to join the “backbone” cable with Enterprise and the WRT54GL… the internal switch of the WRT is 10/100Mbit, so it would block Gigabit, and one in the bedroom to join PS3, AppleTV and the laptop).

It’s something I should consider, at any rate.

ADSL routers running open Linux in the next future?

While certainly there are a lot of home-class ADSL routers already running Linux, most of them are “closed” Linux, provided by the hardware manufacturer and that cannot be customised over a certain point.

I do have a router currently running OpenWRT, a Linksys WRT54GL, but I don’t use it as ADSL router, I use it as a wireless client that routes the wireless network on a wired network segment, the one where Farragut, Enterprise, Klothos and Prakesh are located. I needed to use this network setup because I can’t reach my office with suitably good phone cables that allows me to stay connected with a decent SNR, so I need to connect the wired segment to the wireless network that then gateways me to the WAN. And I had to use an OpenWRT-based WRT54GL because the cheapest router capable of understanding that the WLAN is where the default gateway is was at €300, which is quite too much for me.

OpenWRT is a nice piece of software, I tried three versions of it up to now, because I fucked up two times, once while I was trying the safe way to upgrade the whole version, and once when I tried to upgrade only the packages, but every time, reflashing and reconfiguring didn’t take more than a couple of hours, and every time I make the configuration cleaner and nicer.

I don’t use the firewall, for instance, and I map all the five ports of the switch to the LAN, as I don’t need to distinguish between WAN and LAN on there, the WAN is on the wireless network, but I do install extra services like CUPS and Avahi, both acting as reflectors for broadcasts, so that the services available on the wired and wireless networks can be seen on the respective other network. I also wrote a simple sh script to update DynDNS, although the current one I have here is designed to run on Whiterussian (and whatever the previous version was called, I forgot), while for the new Kamikaze I should update it to use the configuration files rather than the nvram. I’ll work on that in the next days and then upload the new version of the script.

Anyway, I was thinking lately about the ability to run OpenWRT on an ADSL router with onboard modem. I didn’t know how well supported ADSL modems are with OpenWRT, and I pretty soon discovered that even if there is some support, it is not yet possible to use most ADSL routers in their full blown capacities. Luckily this might change in the future, as I see that there is work in progress to support the Texas Instrument AR7 that at least D-Link uses for their latest routers models.

Why would I care of an OpenWRT-running ADSL router? Well, my first ADSL router was a D-Link DSL-500, running ATMos operating system, a decent piece of software, albeit closed source, you could access most features through the CLI interface either by telnet or by serial port, even when the web interface ruined them. The problem with it was that either the networking core or the hardware weren’t stable enough, and when running bittorrent or any other software sending and receiving lots of UDP packets, the router would crash down. This was unbearable on the long run, and thanks to Florian Steinel, I’m now using a 3Com router.

The 3Com router seems to have nicer hardware and handles the load way better. It also is an all-in-one router with WLAN, so I don’t need a standalone access point. The problem it has is with the software, or probably just with the web interface of it (there’s no CLI here, but I’m sure the underlying software is probably quite okay). For instance, the port forwarding table does not consider the network mask setting the router has: it only takes the first three octets of the router’s IP and allows to choose only the fouth for any forward rule. This means not only that if you set the route to 172.16.0.116 you can’t forward ports to devices in the same network with IPs like 172.16.1.34, but you can forward ports to 192.168.0.34 while using 192.168.0.3328 as IP address, so the limitation was not designed to limit you to a single subnetwork, it’s just that the forwarding page does not give a crap about the network mask at all; I tried to explain that to 3Com support, but they are quite worse compared to when I had an UsRobotics modem – then they were very good – and their level 2 support techs weren’t able to understand how I divided my network in two subnetworks… Also, the router does not provide any IPv6 feature.

While I’m almost totally satisfied with the current setup, and I’ll probably run this router for many years (minus failures), I find it somewhat restraining, and for instance I would welcome a firmware for it that would allow me to get IPv6 addressed for the internal network, NATing only the IPv4 requests, or at the very least a firmware which would allow me to make the subnetworks two /24: a /28 is pretty limited especially when you also use it for virtual machines.

So, I’ll certainly be looking forward for AR7-based routers in the next years, in the unfortunate case I need to switch router again (which I really hope won’t happen, I had enough hardware failing on me this year, and I still have to bring Klothos back from the dead with a new SATA controller; help is welcome :P).

PulseAudio bump will be late

At least a bit.

Yes, I know PulseAudio 0.9.7 was released and almost all the tools updated, I’ll be taking care of that tomorrow, after the meeting with the surgeon that will have to tell me whether I need to undergo surgery or not.

I already have an ebuild ready for PulseAudio 0.9.7, based on 0.9.6-r2, with the new init script. There are three new USE flags: dbus, samplerate and gnome. Obviously, even the old init script, available for baselayout 1 users, will also be supported.

So what’s the problem? The problem is that the —disallow-module-loading parameter we pass by default to the system-wide PulseAudio installation, for security (you don’t want random users in the pulse group to be able to load random modules into PulseAudio, do you?), disallows the whole PulseAudio from starting up.

In theory, the parameter should disallow just module loading after startup, but for some reasons now it disallows module loading even before startup. I’ll make sure to report to Lennart (who might be reading this already ;) ), but I’ll also investigate to see if I can fix it.

On the other hand, nxhtml is waiting for upstream to respond, although there are other problems that thanks to ulm are now being slowly fixed (I never tried packaging such a complex mode); and I’ll be working on a bump of sharutils, as you’ll see in the next days, as I’ve been busy trying to package it for OpenWRT (Kamikaze 7.09 brcm-2.4), it’s needed for tramp (Emacs) to connect to the remote host, although it doesn’t seem to work now either, more details tomorrow.

And I should take it easy.

The problem with _my_ cable…

So, a part the already told problem with the phone cable on the street, during this month I also had to fight with the house’s cabling.

I think I already wrote in the past about my problems with noise on the DSL and other funny stuff like that. I then decided, while I was waiting for the telcos to finally fix my line, to change the split cable I was using with a single long cable, and see if that helped. Unfortunately, I found that one of the cable pieces, the important one, being non-telephone cable, cannot be replaced: it’s stuck there.

For this reason, I ended up moving the ADSL router on the main floor, directly on the socket I was given by Telecom, and this actually improved the situation, a bit. I was planning to bridge the 8-port switch I have on this floor to the wireless network provided by the ADSL router with the old access point, a D-Link DWL-700AP.. too bad it doesn’t seem to like working as a bridge.

As a result, right now I’m connected with the iBook working as second router to join the two networks. I ordered a LinkSys WRT54GL-EU router but it will arrive probably only next Monday. Anyway, I also decided to split the two networks so that if I have guests they won’t be allowed to access the services of the wired LAN that are a bit more private.. but then, I had to forward the ports (80, 8080, 443 and so on) of the Internet IP to the addresses of the wired LAN, that are not on the same subnetwork. Usually it shouldn’t be an issue but…

3Com’s interface seemed to allow me to specify only the same network’s addresses when configuring virtual servers. The router had 192.168.1.124 as LAN address, and I always got 192.168.1.** to choose for the destination of the port forwarding. I then tried using a /16 address, trying to split the network later on, but didn’t work either, always the last octet was free to choose .. so I tried to use, instead of two /24 or a split /16, two subnetworks, /28, so that the only change was in the last octet and… miracle, it worked! :P The router (192.168.0.1728) is gladly accepting farragut’s address (192.168.0.3528) as destination of the port forwarding. Seems like whoever wrote the 3Com’s web interface really has little clue about how networks and subnetworks works, and even routing might be out of their understanding, if they require that the destination of the port forwarding is on the same network as the router …

Yes of course to work correctly with that setup, I have to set static routing of the 192.168.0.3228 network through the other router, and yes, I’m limiting a lot the number of clients that I can connect with wireless, but that is all I need after all, I never had more than two extraneous laptops here…

Oh well… I just hope I’ll be able to put OpenWRT on the LinkSys without big troubles, I’d like to use it to move out dnsmasq and ddclient from this box, so that farragut won’t need enterprise anymore for anything … I already tried using 3Com’s DDNS support before, but they lack support for hostmasks so I could only use flameeyes.is-a-geek.org without the VHosts :(

3Com produces good routers, they work just fine, never had a single performance problem since I use this one, but their interface, although better than, say, Belkin’s, still has a few rough points I’m afraid…