It’s 2014, why are domains so hard?

Last year I’ve moved to Ireland and as of last December, I closed the VAT ID that made me a self-employed consultant. The reason why it took me a while to close the ID was that I still had a few contracts going on, which I needed to have expire before. One of the side effect of closing the ID is also that I had/have to deal with all the accounts where said ID was used, including the OVH account where almost all of my domains, and one of my servers, were registered — notably, xine-project.org has been registered forever at a different provider, Register4less of UserFriendly fame.

For most accounts, removing a VAT ID is easy: you update your billing information and tell them that you no longer have a VAT ID. In some rare cases, you have to forgo the account, so you just change the email address to a different one, and register a new account. In the case of OVH, things are more interesting. Especially so if you want to actually be honest.

In this case, I wanted to move my domains out of my account, in Italy, with a VAT ID, to an account in Ireland, without said ID — one of the reasons is that the address used for registration is visible in most whois output, and while I don’t care for my address being public, the address is now my mother’s only, and it bothered me having it visible there. That was a mistake (from one point of view, and a godsend from another).

First problem is that you cannot change either your account or your VAT ID status out of an OVH account, which meant I needed to transfer the domains and server to a different account. There was (and possibly is, I don’t want to go look for it) some documentation on how to transfer resources across contacts (i.e. OVH accounts), but when I tried to follow it, the website gave me just a big “Error” page, unfortunately “Error” was the whole content of the page.

Contacted for help, OVH Italia suggested using their MoM management software to handle the transition. I tried, and the results have been just as bad, but at least it did error out with an explanation, about trying to cross countries with the transfer. I then contacted OVH Ireland as well as OVH Italia, with the former, after a long discussion where they suggested me to do … exactly what I did, “assured me” that the procedure works correctly — only for OVH Italia apologizing a couple of days later that indeed a month earlier they changed the procedures because of some contractual differences between countries. They suggested using the “external transfer” – going through your standard transfer procedure for domains – but it turns out their system fails when you try that, as the domain is already in their database, so they suggest using the “internal transfer” instead (which as I said does not work).

Since a couple of my domains were going to expire soon, this past weekend I decided to start moving them out of OVH, given that the two branches couldn’t decide how to handle my case. The result is that I started loading the domains onto Gandi — among the reasons, the VideoLan people and one of my colleague know them pretty well and suggested them warmly. This proved trickier, but it also provide one thing: not all French companies are equal!

I started by moving my .eu, .es and .info domains (I own among others automake.info, which redirects to my Autotools Mythbuster — the reason is that if you type the name of the info manuals on G+, it actually brings you there! I was actually planning to make them actually point to a copy of the respective info manuals, but I’ve not studied the GFDL enough yet to know whether I can). While the .info domains are still in limbo right now, as OVH has a five-days timeout before you transfer out, and the .es domains were transferred almost immediately (the Spanish NIC is extremely efficient in that regard, they basically just send you an email to confirm you want to change the registry, and if you accept, that’s it!), the .eu were a bit of a pain.

Turns out that EURid wants a full address to assign the domain to, including a post code; unfortunately Ireland has no post code, yet and even the usual ways to represent my area of Dublin (4, 0004, D4, etc) failed; even the “classical” 123456 that is used by many Irish failed. After complaining on Twitter, a very dedicated Gandi employee, Emerick, checked it out and found that the valid value, according to EURid (but not to Gandi’s own frontend app, ironically) is “Dublin 4”. He fixed that for me on their backend, and the .eu registration went through; this blog is now proudly served by Gandi and that makes it completely IPv6 compatible.

But the trial was not completed yet. One of the reasons why I wanted to move to Gandi now, was that Register4Less was requiring me sort-of-transfer the domain from Tucows (where they resold it before) to their new dedicated registry, to keep the privacy options on. The reason for that being that Tucows started charging more, and they would have had to charge me the extra if I wanted to keep it. On the other hand, they offered to transfer it, extend the expiration another year and keep the privacy option on. I did not like the option because I just had renewed the domain the past November for a bunch of years, so I did not want to extend it even further already — and if I had to, I would at that point try to reduce the number of services I need to keep my eyes on. Besides, unlike Register4Less and OVH, Gandi supports auto-renewal of domains, which is a good thing.

Unfortunately, for ICANN or whoever else manages the .org decided that “Dublin 4” is not a valid postal code, so I had to drop it again off the account, to be able to transfer xine-project.org. Fun, isn’t it? Interestingly both the .org and .it administrators handle the lack of a post code properly — the former as N/A and the latter as the direct translation N.D.. Gandi has been warned, they will probably handle it sometime soon. In the mean time it seems like the .eu domains are not available to Irish residents, as long as they don’t want to fake an address somewhere else.

And the cherries on top, now that I’m migrating everything to Gandi? Their frontend webapp is much better at handling multiple identically-configured domains, to begin with. And as they shown already their support is terrific especially when compared to the mirror-climbing of their other French competitors. But most importantly, have you read a couple of weeks ago, the story of @N? How an attacker got a hold of GoDaddy and caused trouble for the owner of the @N twitter account? Well, turns out that Gandi people are much more security conscious than GoDaddy (okay that was easy) and not only they provide an option to disable the “reset password by email” option, but also provide 2FA through HOTP, which means it’s compatible with Google Authenticator (as well as a bunch of other software).

End of story? I’m perfectly happy to finally having a good provider for my domains, one that is safe and secure and that I can trust. And I owe Emerick a drink next time I stop by Paris! Thanks Gandi, thanks Emerick!

Finding IDs to submit

I have written a lot about the hardware IDs but i haven’t said much about submitting new entries to the upstream databases. Indeed, the package just mirrors the data that is collected by the USB and PCI databases that are managed by Stephen, Martin and Michal.

As an example, I’ll show you how I’ve been submitting the so-called Subsystem IDs for PCI devices from computers I either own, or fix up for customers and friends.

First off, you have to find a system or device whose subsystem IDs have not been submitted yet. Unfortunately I don’t have any computer at hand that I haven’t submitted to the database already. But fear not — it so happens I had an interesting opening. I rented a server from OVH recently, as I’ve had some trouble with one of my production hosts lately, and I’m entertaining the idea of moving everything on a new server and service altogether. But the whole thing is a topic for a completely different time. In any case, let’s see what we can do about these IDs now that I have an interesting system at hand.

First of all, while I don’t have the server at hand to know what’s in it, OVH does tell me what hardware is on it — in particular they tell me it’s an Intel D425KT board (yes I got a Kimsufi Atom, I got the three months lease for now and I’ll see if it can perform decently enough), so that’s a start. Alternatively, I could have asked dmidecode — but I just don’t have it installed on that server right now.

First step is to look at what lspci -v says:

00:00.0 Host bridge: Intel Corporation Atom Processor D4xx/D5xx/N4xx/N5xx DMI Bridge
        Subsystem: Intel Corporation Device 544b
        Flags: bus master, fast devsel, latency 0
        Capabilities: [e0] Vendor Specific Information: Len=08 <?>

This is of course only the first entry in the list but it’s still something. You can see on the second line that it says “Subsystem: Intel Corporation Device 544b” — that means that it knows the subsystem vendor (ID 8086, I can tell you by heart — they have been funny at that), but it doesn’t know the subsystem device. So it’s what we’re looking for: an unknown system! Time to compare the output of lspci -vn — that one does not resolve the IDs, since we’ll need them to submit to the PCI database so if you’re not registered already, do register so that they can be submitted to begin with.

00:00.0 0600: 8086:a000
        Subsystem: 8086:544b
        Flags: bus master, fast devsel, latency 0
        Capabilities: [e0] Vendor Specific Information: Len=08 <?>

Okay so now we know that our first device is Intel’s (VID 8086) and has a000 as device ID — this brings us to https://pci-ids.ucw.cz/read/PC/8086/a000 easy, isn’t it? At the end of the page there’s a list of the known subsystem IDs; pending submissions does not show up the name, but they show up in the table with a darker gray background. All PCI ID entries are moderated by hand by the database’ s maintainers. When you’ll be reading this, the entry for my board will be in already, but right now it isn’t — if it wasn’t obvious, I’m looking for an entry that reads 8086 544b (which is under “Subsystem” above).

Now the form requires just a few words: the ID itself – which is 8086 544b with a space, not a colon – and a name. Note is for something that needs to be written on the pci.ids, so in most cases need to be empty. Discussion if when you wan tot comment on the certainly of your submission; for my laptop for instance we had some trouble with “Intel Corporation Device 0153” — which is now officially “3rd Gen Core Processor Thermal Subsystem”.

The name I’m going to submit is “Desktop Board D425KT” as that’s what the other entry in the database for that device uses as a format — okay it actually uses DeskTop but I’d rather not capitalize another T and see a kitten cry.

Now it’s time to go through all the other entries in the system — yes there are many of them, and most of the time the IDs are not set in the order of the PCI connections, so be careful. More interestingly, not all the subsystems are going to be listed in the same line. Indeed, the third entry that I have is this:

00:1c.0 0604: 8086:27d0 (rev 01) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00001fff
        Memory behind bridge: e0f00000-e12fffff
        Prefetchable memory behind bridge: 00000000e0000000-00000000e00fffff
        Capabilities: [40] Express Root Port (Slot+), MSI 00
        Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Capabilities: [90] Subsystem: 8086:544b
        Capabilities: [a0] Power Management version 2
        Capabilities: [100] Virtual Channel
        Capabilities: [180] Root Complex Link
        Kernel driver in use: pcieport

The subsystem ID is listed under “Capabilities” instead — but it’s always the same. This is actually critical: if the subsystem does not match, it means that it’s coming from a different component — for instance if you’re building your own computer, the subsystem of the internal CPU devices and those of the motherboard will not match, as they come from different vendors. And so would happen to add-on cards (PCI, PCI-E, AGP, …).

Sometimes, a different subsystem is also available on internal components that get different names from the motherboard itself — in this case, the Realtek network card on this motherboard reports a completely different ID and I really don’t know how to submit it:

01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller (rev 05)
        Subsystem: Intel Corporation Device d626
        Flags: bus master, fast devsel, latency 0, IRQ 44
        I/O ports at 1000 [size=256]
        Memory at e0004000 (64-bit, prefetchable) [size=4K]
        Memory at e0000000 (64-bit, prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 01
        Capabilities: [b0] MSI-X: Enable- Count=4 Masked-
        Capabilities: [d0] Vital Product Data
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [160] Device Serial Number 01-00-00-00-36-4c-e0-00
        Kernel driver in use: r8169

If for whatever reason you make a mistake, you can click on the “Discuss” link on the submitted content and edit the name that you want to submit. I did make such a mistake during submitting the IDs for this.

So here are the tricks.. happy submission!

A story of a Registry, an advertiser, and an unused domain

This is a post that relates to one of my dayjobs, and has nothing to do with Free Software, yet it is technical. If you’re not interested in non-Free Software related posts, you’re suggested to skip this altogether. If you still care about technical matters, read on!

Do you remember that customer of mine that almost never pays me in time, for which I work basically free of charge, and yet gives me huge headaches from time to time with requests that make little to no sense? Okay you probably remember by now, or you simply don’t care.

Two years or so ago, that customer calls me up one morning asking me to register a couple of second-level domains in as many TLDs as I thought it made sense to, so that they could set up a new web-end to the business. Said project still hasn’t delivered, mostly because the original estimate I sent the customer was considered unreasonably expensive, and taking “too much time” — like they haven’t spent about the same already, and my nine months estimate sounds positively short when you compare it with the over two years gestation the project is lingering on. At any rate, this is of no importance to what I want to focus on here.

Since that day, one set of domains was left to expire as it wasn’t as catchy as it sounded at first, and only the second set was kept registered. I have been paid for the registration of course, while the domains have been left parked for the time being (no they decided not to forward them to the main domain of the business where the address, email and phone number are).

The other day I was trying to find a way to recover a bit more money out of this customer and, incidentally, this blog, and I decided to register to AdSense again, this time with my VAT ID as I have to declare eventual profits coming from that venue. One of the nice features of AdSense allows to “monetize” (gosh how much I hate that word!) parked domains. Since these are by all means parked domains, I gave it a chance.

Four are the domains parked this way: .net, .com, .eu and .it. All registered with OVH – which incidentally has fixed its IPv6 troubles – and up to now all pointing to a blackhole redirect. How do you assign a parked domain to Google’s AdSense service? Well, it’s actually easy: you just have to point the nameservers for the domain to the four provided by Google, and you’re set. On three out of four of the TLDs I had to deal with.

After setting it up on Friday, as of Monday, Google still wouldn’t verify the .it domain; OVH was showing the task alternatively as “processing” and “completed” depending on whether I looked at the NS settings (they knew they had a request to change them) or at the task’s status page (as it’ll be apparent in a moment, it was indeed cloesd). I called them — reason I like OVH: I can get somebody on the phone to eat least listen to me.

What happens? Well, looks like Registro.it – already NIC-IT, the Italian Registration Authority – is once again quite strict in what it accepts. It was just two years ago that they stopped requiring you to fax an agreement to actually be able to register a .it domain, and as of last year you still had to do the same when transferring the domain. Luckily they stopped requiring both, and this year I was able to transfer a domain in the matter of a week or so. But what about this time?

Well, it turns out that the NIC validates the new nameservers when you want to change them, to make sure that the new servers list the domain, and configure it properly. This is common procedure, and both the OVH staff and me were aware of this. What we weren’t aware of (OVH staffers had no clue about this either, they had to call NIC-IT to see what the trouble was, they weren’t informed properly either) is the method they do that: using dig +ANY.

Okay, it’s nothing surprising actually, dig +ANY is the standard way to check for a domain’s zone at a name server… but turns out that ns1.googleghs.com and its brothers – the nameservers you need to point a domain to, for use with AdSense – do not support said queries, making them invalid in the eyes of NIC-IT. Ain’t that lovely? The OVH staffer I spoke with said they’ll inform NIC-IT about the situation, but they don’t count on them changing their ways and … I actually have to say that I can’t blame them. Indeed I don’t see the reason why Google’s DNS might ignore ANY queries.

For my part, I told them that I would try to open a support request with Google to see if they intend to rectify the situation. The issue here is that, as much as I spent trying to find that out, I can’t seem to find a place where to open a ticket for the Google AdSense staff to read. I tried tweeting to their account, but it seems like it didn’t make much sense.

Luckily there is an alternative when you can’t simply set up the domain to point to Google’s DNS, and that is to create a custom zone, which is what I’ve done now. It’s not much of a problem, but it’s still bothersome that one of Google’s most prominent services is incompatible with a first-world Registration Authority such as NIC-IT.

Oh well.

What do you mean it’s not IPv6-compatible?

For those who wonder where I disappeared, I’ve had a bit of an health scare, which is unfortunately common for me during summertime. This time the problem seems to be tied to anxiety, should probably pass once most of the stress related to taxes and work projects deadlines gives up.

Earlier this month we’ve had the IPv6 World Test Day, and things seems to have moved quite a bit since then. Even the Gentoo Infra team had a bit of work to be done to get ready for the test day and to set it up, and if you follow Apple’s software updates you probably know that they “improved IPv6 reliability” with their latest release.

I’m very interested in the IPv6 technology myself, and I’d very much like to rely more on it; unfortunately, as it happens, my hosting provider still hasn’t provided me with IPv6 addresses, nor it seems likely to happen soon. On the other hand, I’ve deployed it at home, even backing off from 6to4 which was my original hope to avoid tunnels (Hurricane Electric is much more reliable, and faster). While I can’t remember an IPv6 address by heart, I can set up proper, visible hostnames for my boxes so that I can compare the logs and not be forced to use NATed addresses all the time.

Now, given that IPv6 is fully deployed in my home network, if a website is set up to use IPv6, then it’ll be using IPv6. It could be a bit of a slow-down when you consider that I use a tunnel to get to the IPv6 network, but generally it seems to behave just as good, possibly because my home network is slow enough by itself. Of course, the website needs to be IPv6-compatible, and not just “IPv6 ready”.

What happens is that a number of websites have enabled IPv6 during the World Test Day, and when they saw that enough users were able to access them just fine, they kept the extra addresses on.. why doing twice the work to turn it off? But that kind of testing sometimes is not just good enough. While the main obstacle to IPv6 support is listening for and responding to IPv6-bound requests, there is code that deals with remote and local host addresses in most applications, including web applications. Validating addresses, applying ACLs, and all these things are due to require knowledge of the addresses it has to deal with, and so many times, they expect dotted-quad IPv4 addresses.

I’m still fighting with one real-world scenario as such. Most of my domains are registered through the French provider OVH who also started providing IPv6-access to their website after the World Day. All the management services work just fine (even though last I checked they didn’t provide a dynamic AAAA record, which is why I had to search for complex alternative approaches which, actually, I’m still keeping up with), as well as the pages detailing their products and services. But when I had to renew one of the domains, it stopped when I was supposed to be redirected to pay (via creditcard), with an internal server error (HTTP 500 Error).

After waiting over the weekend (and a bit, given I was swamped with work), I’ve decided to call to see if it was a known issue: it wasn’t, the system was supposedly working fine, and they suggested me to try a different computer. After testing with Firefox on Windows (no go), I’ve tried the infamous iPad and… it worked. A quick doubt I got was related to the connection protocol, and bingo: it works all fine with IPv4, but fails badly with v6.

This is a very plain example of how just listening for v6 connections is not enough: you need to ensure that the interaction between pieces of the software are still satisfied. In this instance, I can think of two possible reasons why it doesn’t work correctly with IPv6:

the system logs all the payment requests separately, and to do so, it assumes the remote host address to be a dotted-quad;
since the page redirects to their processing bank’s site, it probably signals it of the incoming request to avoid cross-site forgery, and again it assumes a dotted-quad address.

Whatever the problem, right now they fail to process payments properly, and when I reported it they shut me down twice (first on the phone “oh it’s not our problem, but the bank’s!”), then by mail (“everything should be fine now!” — no it isn’t).. and still they are publishing AAAA records for their website.

If even an European-wide ISP fails this badly at implementing IPv6 on their website (for one critical piece of infrastructure as payment processing is!), I’m afraid that we have very little hope for IPv6 to get deployed worldwide painlessly.

On IPv6 and Dynamic Hosts (and PowerDNS Express in particular)

So yesterday I wrote about my tests on bypassing an hostile NAT that left me with a public-accessible dynamic IPv6. This helps me a whole lot, but it ’s almost unusable for more than a couple of days as I cannot know the IP address (unless that is I mail it to myself each time it change). The idea of using Mobile IPv6 to get a stable address for the box left away because of the complexity, I came back on my steps to my original, possibly easy, option: dynamic hostnames.

Dynamic hostnames are a very old technology to work around the issue of dynamic IPs (which was much more common years ago), it seemed obvious to me that the solution was the easiest to implement: I get a stable address (in form of hostname) to the router, then I can get to the remaining hosts through a SSH jump (or some kind of limited-scope IPv6 routing).

Unfortunately the first obvious choice (DynDNS) is a failure: it does not support using IPv6 for dynamic hostnames as far as I can see, and that makes it useless for my aim here. The second option for me was using the OVH system for DynHOSTs — it’s a service I pay, so I was expecting it to have the needed features, unfortunately they also don’t allow using IPv6 for their hosts. There used to be a service that supported this kind of feature, called DNS6, but that seems to be now dead. Hurricane Electric is planning on supporting dynamic hosts at some point, but right now there is no support for it.

Then I started looking at some more complex solutions, including paid-for solutions and custom solutions. One of my first ideas was UltraDNS but no public pricing seems to be available and that is not something I’m very fond of; plus it’s based in the United States which gives me trouble with taxes and payments, an European solution would have been better for my requests.

After discarding this solution as well, I started down the most complex road for this kind of situations, at least that’s what I thought at that point: writing my own dynamic DNS system. Luckily, a job I took earlier this year gave me a bit of expertise with PowerDNS (the software) so I only had to write some CGI application in Ruby to modify a PostgreSQL database on this very server, and serve it from there. I started looking into the pieces of the puzzle for what concerned the CGI script, and found a number of other problems, mostly related to SSL and certificates (but that’s, again, something for another post), and then I looked at PowerDNS itself, starting from looking at the latest version available on the site.

When I looked at the homepage (which is the one I linked earlier), I noted two more interesting things: the first is that the developers offer a paid-for custom DNS service, the second that the company is in the Netherlands, so it’s within the European Union, which is good for my taxes. Also, the price for a single domain (which is what I’d be needing at first) is low enough that it looked acceptable ($2 per domain, $0.50 for the transaction, less than €1.90 total). Beside the usual user-friendly operation interface to set the DNS records, their service has the one thing that was important to me: API access based on the SOAP protocol (and a WSDL description), that allowed updating records via scripts.

While on paper the service is great – and cheap too! – there are way too many shortcomings in their approach:

Authentication shautentication: the SOAP interface is only available in simple HTTP, there is no proper authentication, but it’s all left to the single API key that is provider per user; this means that if you deploy this on a non-trusted network (and, well, do you really trust the rest of the Internet?), and somebody is able to get your API key, not only they can mess with the records you’re messing at a given time, but they can mess with any of your zones and domains.
API keys and WSDL: beside the fact that it’s the one and only authentication mechanism, the API key is not passed as part of the SOAP request; instead it is passed as a query parameter on the POST request, as part of the uri. Unfortunately, the WSDL that is reported by the interface is not fixed to use the API key. As their documentation only speaks about PHP and VB.NET, I assume that those two libraries ignore the endpoint URL provided by the WSDL response (as soap:address); Ruby on the other hand respects what the response tells it to use, and it turns out it does not include the API key back, resulting in receiving “Invalid User” messages back at any request.
No advanced editing: while the PowerDNS express user interface for assisted editing of zones is one of the best I’ve seen among various DNS services, including the ability to automatically add Google Mail records (which would have been nice given that I actually have them on a number of domains already), they don’t have an “advanced” mode, which would allow you to either edit the zone manually, or at least add any kind of records in a free-form way; the SOAP interface also doesn’t allow you to add all kind of records either, which is a bad thing. It gets even worse when you add the fact that you don’t have SSHFP as a record type that you can use to set the fingerprint of a SSH server — this was actually a half decent idea to provide some extra safety that the lack of real authentication didn’t give me.

I’m seriously disappointed by the bad quality of the service PowerDNS Express provides, even though their software (pdns) seems pretty good and their basic interface is one of the best, as I said. As it is, it’s definitely not an option.

Luckily, Robin (robbat2) saved me from writing and deploying my own custom solution, so I’ll be working on deploying that in the next few days, taking most definitely less time than it would require me to waste for writing all the code myself. In most recent Bind version, support for Dynamic DNS entries is supported via the nsupdate tool; this means that if I set up a standard bind instance on my server (which might be harder to configure, but requires less than half the dependencies of pdns, most importantly it doesn’t need boost, and does not require a database behind it), then I can simply use that (which provides a strong authentication system, and a complete authorization system) to provide dynamic host names, exactly like I intended to originally.

For now, I’ll like the two reference pages that Robin gave me, if I’ll encounter problems implementing it that way, I might post again about it: