Fishy Facebook Ads: Earthly Citizens, Shutter & Contrast, and many more

(If you prefer this in form of a Twitter thread, see this one.)

Let’s start with the usual disclaimer that despite me working for a company that sells advertisement, this post is my own personal opinion, not my employer’s. I have written about Internet ads for years, well before I joined the company, and so it’s nothing new. To the usual disclaimer I’m going to add a few words to point out that there will be a few company names used in this post — I’ll be very clear when I think they are involved in something fishy, and when I think they are not involved at all.

This all starts with me deciding to get myself a new camera. While I’m very happy about the photos that my usual camera produce, I wanted something lighter that I could go around town more often with. But I also have been having issues with my shoulder, and I’ve been looking out for a good “handy” backpack to keep my stuff in. This is all relevant information.

Indeed, if you follow me on Twitter you may have seen me asking around for suggestions on backpacks. And this is also relevant: since I’m actually not minding ads for relevant content for myself, I have not hidden my looking for a new bag, I spoke about it on social media, and I have searched for backpacks and bags on my normal Google session. This is, again, all relevant information.

Because of my Google searches, I have been seeing a lot of ads related to photography. Including the one for the chain of photography stores that convinced me to go and grab my new camera from them. Very few of those ads are useful to me, but that one in particular have been.

Then the other day, on Instagram, I saw the ads for a backpack from a never-heard-before company advertising as Earthly Citizens. I’m not going to link directly to their website, although I’m choosing to explicitly name them here so that people who may be looking for them on Google and other search engines have a landing page helping them. The backpack that they advertised is this one (archived link) and it actually looks very nice in theory, on offer at £87.75 compared to a RRP of £159.61. To compare, my trusty Think Tank Airport Essentials is £147.04, and that’s one hell of a good bag.

The amount of red flags on that advertisement was high: unknown brand, no branding on the actual bag, unrealistic “flash sale” with no dates on it, and so on. So I didn’t really pay much attention. Then of course, since I have looked at the ad, I started seeing the same bag on Facebook — together with nearly 900 positive comments. I decided to do a minimum amount of digging into it, and found out that the website that the ad points to is a standard Shopify instance, which means that digging into it with IP addresses or WhoIs information is useless. And since there’s no address provided for the company even on privacy pages, there’s not much to go by. I walked away.

A day later, another set of ads start appearing on my Facebook stream, and they are for a backpack that is stunningly similar, or rather identical. But from a different page that has a more “photography” feel to it, called “Shutter & Contrast”. And that piqued my interest a little bit, because it sounded like another one of those cloned bags that I have seen aplenty on Instagram, and I would actually like to find the source at that point.

Just like Earthly Citizens, Shutter & Contrast don’t seem to be very well reviewed. Searching the web for the name and combination of reviews, backpack and scam don’t bring up anything useful. They also have a Shopify site, although their page for the same backpack (archived, again) is a bit more somber and “professional-looking”.

Funnily enough, it looks like they have blocked copy-paste and right-click, so that you can’t quickly reverse-image-search their photos. It didn’t surprise me, as I remembered a BuzzFeed article on fake fashion stores outright stealing real designers’ photos, so stopping the quickest reverse image search option would obviously be high in their intentions. Of course it’s actually easy to work this around, with any of the browsers’ developer tools.

Another interesting part from the Shutter & Contrast shop page is that they actually have an address in their Privacy Page: 11923 NE Sumner St, STE 813872, Portland, Oregon, 97220, USA. Again I’m repeating it here for sake of those looking any information on this company, because if you look up the address, you’ll probably find a Yelp page for a closed location called My Trail Gear, although it has a different “STE” number. The reviews, calling this a scam and pointing out that there is at least two more companies using the address, called “Bear and Tees” and “Shark and Tees”.

Checking the address on StreetView shows a smallish warehouse. My best guess is that there’s a service at that address that is similar to Ireland’s Parcel Motel and Parcel Wizard: companies that allow you to receive and send goods from that address, and then forward it somewhere else. The different “STE” numbers are used to route the parcels to the right customer. This means that despite the bad reviews on Yelp, Shutter & Contrast might be legit.

So I decided to take a closer look at the first one again. Earthly Citizen has a fairly active Facebook page, and if you read their About section, it says:

Our goal is to source all the best travel related documents from all around the world and bring them directly to your doorstep

Earthly Citizens Facebook Page

They don’t seem to be doing anything like that. Instead they seem to mostly re-post Instagram pictures by other people. At least it appears they are crediting the photographers — but it’s clear that they are using someone else’s pictures for their own marketing (so that they get people to follow their account). This should be worrisome enough, but it doesn’t stop there.

If you look at what they sell, they appear to be selling a lot of random stuff that you would find in those trinkets/gadgets shop in big malls, without brands, rhyme, or reason. So it does not look like they are the “source” of that bag to begin with. But is Shutter & Contrast then?

Earthly Citizens say that there are “too many fake websites that steal content”. They would know since they seem to be one.

A very quick reverse image search finds the same exact image appears on AliExpress (not archived because they seem to defeat it), the Chinese shopping website. There are multiple sellers for it there as well, and most of them have the same images — the same images that both Earthly Citizens and Shutter & Contrast used on their website.

It might very well be that these are the bag equivalent of Gongkai, as there are a few stores that sell them, and the fact that they come from Guangdong does not mean they are not good. I have a lovely tripod I bought at the Shanghai Xing Guang Photography Market, it’s a Chinese brand, it’s proper carbon fiber, and I paid for it half the price that you would pay in store in Europe, taxes included. If that is the case, the markups that Earthly Citizens and Shutter & Contrast are applying are thievery: they price it at $110 and $83 respectively, while AliExpress’s most expensive seller has it at $52.

But there is one thing that I forgot about during my Twitter rant, and that my girlfriend pointed out: what about the pictures of people in the advertising? Neither AliExpress nor Earthly Citizens appear to have a picture of the backpack with a person. There are people with cameras, but nobody with the actual backpack that you can reverse image search for. There is a video on Earthly Citizens’s Facebook page, which is the same used by the Instagram ad, and that suggests that the bag physically exist, but it’s heavily watermarked that makes it hard to find the source on. Shutter &Contrast has a video unlisted on YouTube, on a white background with no logos shown, and just re-captioned to fit their marketing of it. It appears uploaded in February 2019.

More useful, Shutter & Contrast appear to also have a still picture of someone wearing what looks like the backpack they are selling, and that’s the first time in this adventure I managed to find that. Reverse image search brings us to yet another Shopify instance under the name ConnectedTechPacks (archived), which can also be found as BestGearPack. Their website is a bit more well made, and it appears to only sell that single backpack. Are they the source? I doubt so, since both websites were registered in April this year, and we know that the backpack existed in February. But they also have a couple of different people with the same backpack, and another angle of the same guy.

Another reverse image search later finds yet another Shopify instance with the same backpack, a set of GIF animations that are also heavily watermarked, but are the same as Earthly Citizens’s version.

So where did all this investigation bring us? Not really anywhere. I can’t find any trustworthy brand selling the backpack, and while I may be willing to risk my £40 on the AliExpress version – rather than twice as much with any of the other Shopify instances that I found – I don’t hold my breath for it to look at all like they show it, or have the build quality that I would trust my cameras with.

It does show just how easy it is to fool people nowadays. It’s easy to set up a “storefront” without needing an actual space anymore. It’s easy to “gain trust” by having people follow your page with no original content, just by re-posting content that professionals provided.

What about the 900 positive comments that the ad received? Well it’s possible that they are actual real satisfied customers who didn’t realize they got charged probably twice as much as they should have for the same bag you can get from AliExpress. Or they may be “bought engagement”. Or just a bunch of bots that have harvested someone else’s name and pictures to create fake profile to sell the stuff.

You know all the panic around politics and elections and fake profiles? It’s not just the elections. Fake profiles sell scams. And that can hurt people just as much as political elections. I remember when it was just the artists complaining about pages re-posting their content… we should have paid attention then. Now the same pages and the same techniques are used for more nefarious purposes and we all pay the price, sooner or later.

Ads, spying, and my personal opinion

In the past year or so, I have seen multiple articles, even by authors who I thought would have more rational sense to them, over the impression that people get about being spied upon by technology and technology companies. I never got particularly bothered to talk about them, among other things because the company I work for (Google) is one that is often at the receiving end of those articles, and it would be disingenuous for me to “defend” it, even though I work in Site Realiability, which gives me much less insight in how tracking is done than, say, my friends who work in media at other companies.

But something happened a few weeks ago gave me an insight on one of the possible reasons why people think this, and I thought I would share my opinion on this. Before I start let me make clear that what I’m going to write about is something that is pieced together with public information only. As you’ll see soon, the commentary is not even involving my company’s products, and because of that I had access to no private information whatsoever.

As I said in other previous posts, I have had one huge change in my personal life over the past few months: I’m in a committed relationship. This means that there’s one other person beside me that spends time in the apartment, using the same WiFi. This is going to be an important consideration as we move on later.

Some weeks ago, my girlfriend commented on a recent tourism advertisement campaign by Lithuania (her country) on Facebook. A few hours later, I received that very advertisement on my stream. Was Facebook spying on us? Did they figure out that we have been talking a lot more together and thus thought that I should visit her country?

I didn’t overthink it too much because I know it can be an absolute coincidence.

Then a few weeks later, we were sitting on the sofa watching Hanayamata on Crunchyroll. I took a bathroom break between episodes (because Cruncyroll’s binge mode doesn’t work on Chromecast), and as I came back she showed me that Instagram started showing her Crunchyroll ads — “Why?!” We were using my phone to watch the anime, as I have the account. She’s not particularly into anime, this was almost a first as the material interested her. So why the ads?

I had to think a moment to give her an answer. I had to make a hypothesis because obviously I don’t have access to either Crunchyroll or Instagram ads tracking, but I think I’m likely to have hit close to the bullseye and when I realized what I was thinking of, I considered the implications with the previous Facebook ads, and the whole lot of articles about spying.

One more important aspect that I have not revealed yet, is that I requested my ISP to give me a static, public IPv4 address instead of the default CGNAT one. I fell for the wet dream, despite not really having used the feature since. It’s handy, don’t get me wrong, if I was to use it. But the truth is that I probably could have not done so and I wouldn’t have noticed a difference.

Except for the ads of course. Because here’s how I can imagine these two cases to have happened.

My girlfriend reads Lithuanian news from her phone, which is connected to my WiFi when she’s here. And we both use Facebook on the same network. It’s not terribly far-fetched to expect that some of the trackers on the Lithuanian news sites she visits are causing the apartment’s stable, static, public IP address to be added to a list of people possibly interested in the country.

Similarly, when we were watching Crunchyroll, we were doing so from the same IP address she was checking Instagram. Connect the two dots and now you have the reason why Instagram thought she’d be a good candidate for seeing an advert for Crunchyroll. Which honestly would make more sense if they intended to exclude those who do have an account, in which case I would not have them trying to convince me to… give them the money I already give them.

Why do I expect this to be IP tracking? Because it’s the only thing that makes sense. We haven’t used Facebook or Messenger to chat in months, so they can’t get signal from that. She does not have the Assistant turned on on her phone, and while I do, I’m reasonably sure that even if it was used for advertisement (and as far as I know, it isn’t), it would not be for Facebook and Instagram.

IP-based tracking is the oldest trick in the book. I would argue that it’s the first tracking that was done, and probably one of the least effective. But at the same time it’s mostly a passive tracking system, which means it’s much easier to accomplish under the current limits and regulations, including but not limited to GDPR.

This obviously has side effects that are even more annoying. If the advertisers start to target IP address indiscriminately, it would be impossible for me or my girlfriend to search for surprises for each other. Just to be on the safe side, I ordered flowers for our half-year anniversary from the office, in the off-chance that the site would put me on a targeting list for flower ads and she could guess about it.

This is probably a lot less effective for people who have not set up static IP addresses, since there should be a daily or so rotation of IP addresses that confuses the tracking enough. But I can definitely see how this can also go very wrong when a household dynamic are pathological, if the previous holder of the address managed to get the IP on targeted lists for unexpected announces.

I have to say that in these cases I do prefer when ads are at least correctly targeted. You can check your Ads preferences for Google and Facebook if you want to actually figure out if they know anything about you that you don’t want them to. I have yet to find out how to stop the dozens of “{Buzzword} {Category} Crowdfunding Videos” pages that keep spamming me on Facebook though.

Updated “Social” contacts

Given the announcement of Google+ shutdown (for consumer accounts, which mine actually was not), I decided to take some time to clean up my own house and thought it would be good to provide an update of where and why you would find me somewhere.

First of all, you won’t find me on Google+ even during the next few months of transition: I fully deleted the account after using the Takeout interface that Google provides. I have not been using it except for a random rant here and there, or to reach some of my colleagues from the Dublin office.

If you want to follow my daily rants and figure out what I actually complain the most loudly about, you’re welcome to follow me on Twitter. Be warned that a good chunk of it might just be first-world London problems.

The Twitter feed also gets the auto-share of whatever I share on NewsBlur, which is, by the way, what I point everyone to when they keep complaining about Google Reader. Everybody: stop complaining and just feel how much better polished Samuel’s work is.

I have a Facebook account, but I have (particularly in the past couple of years), restricted it to the people I actually interact with heavily, so unless we know each other (online or in person) well enough, it’s unlikely I would accept a friend request. It’s not a matter of privacy, given that I have written about my “privacy policy”, it’s more about wanting to have a safe space I can talk with my family and friends without discussions veering towards nerd-rage.

Also, a few years ago I decided that most of my colleagues, awesome as they are, should rather stay at arms’ length. So with the exception of a handful of people who I do go out with outside the office, I do not add colleagues to Facebook. Former colleagues are more likely.

If you like receiving your news through Facebook (a negative idea for most of tech people I know, but something that the non-tech folks still widely prefer it seems), you can “like” my page, which is just a way for WordPress to be able to share the posts to Facebook (it can share to pages, but not to personal accounts, following what I already complained before about photos). The page also gets the same NewsBlur shared links as Twitter.

Talking about photos, when Facebook removed the APIs, I started focusing on posting only on Flickr. This turned out to be a bit annoying for a few of my friends, so I also set up a page for it. You’re welcome to follow it if you want to have random pictures from my trips, or squirrels, or bees.

One place where you won’t see me is Mastodon or other “distributed social networks” — the main reason for it is that I got already burnt by back in the days, and I’m not looking forward to have a repeat of the absolute filter bubble there, or the fact that, a few years later, all those “dents” got lost. As much as people complain how Twitter is ephemeral, I can still find my first tweet, while just disappeared, as I see it, in the middle of nowhere.

And please stop even considering following me on Keybase please.

Facebook, desktop apps, and photography

This is an interesting topic, particularly because I had not heard anything about it up to now, despite having many semi-pro and amateur photographer friends (I’m a wannabe). It appears that starting August 1st, Facebook will stop allowing desktop applications to upload photos to albums.

Since I have been uploading all of my Facebook albums through Lightroom, that’s quite a big deal for me. On Jeffrey Friedl’s website, there’s this note:

Warning: this plugin will likely cease to work as of August 1, 2018, because Facebook is revoking photo-upload privileges for all non-browser desktop apps like this.

As of June 2018, Adobe and I are in discussions with Facebook to see whether something might be worked out, but success is uncertain.

This is now less than a month before the deadline, and it appears there’s no update for this. Is it Facebook trying to convince people to just share all their photos as they were shot? Is it Adobe not paying attention trying to get people on their extremely-expensive Adobe CC Cloud products? (I have over 1TB of pictures shot, I can’t use their online service, it would cost me so much more in storage!) I don’t really know, but it clearly seems to be the case that my workflow is being deprecated.

Leaving aside the consideration of the impact of this on me alone, I would expect that most of the pro- and semi-pro-photographers would want to be able to upload their pictures without having to manually drag them with Facebook’s flaky interface. And it feels strange that Facebook wants to stop “owning” those photos altogether.

But there’s a bigger impact in my opinion, which should worry privacy-conscious users (as long as they don’t subscribe to the fantasy ideal of people giving up on sharing pictures): this moves erodes the strict access controls from picture publishing that defined social media up to now, for any of the users who have been relying on offline photo editing.

In my case, the vast majority of the pictures I take are actually landscapes, flowers, animals, or in general not private events. There’s the odd conference or con I bring my camera to (or should I say used to bring it to), or a birthday party or other celebration. Right now, I have been uploading all the non-people pictures as public (and copied to Flickr), and everything that involves people as friends-only (and only rarely uploaded to Flickr with “only me” access). Once the changes go into effect, I lose the ability to make simple access control decisions.

Indeed, if I was to upload the content to Flickr and use friends-only limited access, very few people would be able to see any of the pictures: Flickr has lost all of its pretension to be a social media platform once Yahoo stopped being relevant. And I doubt that the acquisition of SmugMug will change that part, as it would be just a matter of duplicating a social graph that Facebook already has. So I’m fairly sure a very common solution to that is going to be to make the photos public, and maybe the account not discoverable. After all who might be mining the Web for unlisted accounts of vulnerable people? (That’s sarcasm if it wasn’t clear.)

In my case it’s just going to be a matter of not bringing my camera to private events anymore. Not the end of the world, since I’m already not particularly good at portrait photography, and not my particular area of interest. But I do think that there’s going to be quite a bit of problems in the future.

And if you think this is not going to be a big deal at all, because most parties have pictures uploaded by people directly on their mobile phones… I disagree. Weddings, christenings, cons, sport matches, all these events usually have their share of professional photographers, and all these events need to have a way to share the output with not only the people who hired them, but also the friends of those, like the invitees at a wedding.

And I expect that for many professionals, it’s going to be a matter of finding a new service to upload the data to. Mark my words, as I expect we’ll find that there will be, in the future, leaks of wedding pictures used to dox notable people. And those will be due to insecure, or badly-secured, photo sharing websites, meant to replace Facebook after this change in terms.

Logging in offline

Over two years ago, I described some of the advantages of U2F over OTP when FIDO keys were extremely new and not common at all. I have since moved most of my login access that support it to U2F, but the amount of services support it is still measly, which is sad. On the other hand, just a couple of days ago Facebook added support for it which is definitely good news for adoption.

But there is one more problem with the 2-factor authentication or, as some services now more correctly call it, 2-step verification: the current trend of service-specific authentication apps, not following any standard.

Right now my phone has a number of separate apps that are either dedicated as authentication app, or has authentication features built into a bigger app. There are pros and cons with this approach of course, but at this point I have at least four dedicated authorization apps (Google Authenticator, LastPass Authenticator, Authenticator and Microsoft’s) plus a few other apps that simply include authentication features in the same service client application.

Speaking of Microsoft’s authenticator app, which I didn’t like above. As of today what I had installed, and configured, was an app called “Microsoft Account” – when I went to look for a link I found out that it’s just not there anymore. It looks like Microsoft simply de-listed the application from the app store. Instead a new Microsoft Authenticator is now available, taking over the same functionality. It appears this app comes from their Azure development team, from the Play Store ID, but more important it is the fourth app that appears just as Authenticator on my phone.

Spreading the second factor authentication across different applications kind of make sense: since the TOTP/HOTP (from now I will only call them TOTP, I mean both) system relies on a shared key generated when you enroll the app, concentrating all the keys into a single application is clearly a bit of a risk – if you could easily access the data of a single authentication app and fetch all of its keys, you don’t want it to bring you access to all the services.

On the other hand, having to install one app for the service and one for the authentication is … cumbersome. Even more so when said authentication app is not using a standard procedure, case in point being the Twitter OTP implementation.

I’m on a plane, I have a “new” laptop (one I have not logged on Twitter with). I try to login, and Twitter nicely asks me for the SMS they sent to my Irish phone number. Ooops, I can’t connect phone service from high up in the air! But fear not it tells me that I can give them a code from the backups (on a different laptop, encoded with a GPG key I have with me but not at hand in the air) or a code from the Twitter app, even if I’m offline.

Except, you need first to have it set up. And that you can’t do offline. But turns out if you just visit the same page while online it does initialize and then work offline from them on. Guess when you may want to look for the offline code generator for the first time? Compare with the Facebook app, that also includes a code generator: once you enable the 2-step verification for Facebook, each time you log in to the app, a copy of the shared key is provided to the app, so every app will generate the same code. And you don’t need to request the code manually on the apps. The first time you need to login, with the phone offline, you’ll just have to follow the new flow.

Of course, both Facebook and Twitter allows you to add the code generator to any authenticator TOTP app. But Facebook effectively set that up for you transparently, on as many devices as you want, without needing a dedicated authentication app, just have any logged in Facebook app generate the code for you.

LastPass and Microsoft authenticator apps are dedicated, but both of them also work as a generic OTP app. Except they have a more user-friendly push-approval notification for their own account. This is a feature I really liked in Duo, and one that, with Microsoft in particular, makes it actually possible to log in even where otherwise the app would fail to log you in (like the Skype app for Android that kept crashing on me when I tried to put in a code). But the lack of standardization (at least as far as I could find) requires you have separate app for each of these.

Two of the remaining apps I have installed (almost) only for authentication are the and Steam apps. It’s funny how the gaming communities and companies appears to have been those pushing the hardest at first, but I guess that’s not too difficult to imagine when you realize how much disposable money gamers tend to have, and how loud they can be when something’s not to their liking.

At least the Steam app tries to be something else beside an authenticator, although honestly I think it falls short: finding people with it is hard, and except for the chat (that I honestly very rarely use on desktop either) the remaining functions are so clunky that I only open the app to authenticate requests from the desktop.

Speaking of gaming community and authentication apps, Humble Bundle has added 2FA support some years ago. Unfortunately instead of implementing a standard TOTP they decided to use an alternative approach. You can choose between SMS and a service called Authy. The main difference between the Authy service and a TOTP app is that the service appears to keep a copy of your shared key. They also allow you to add other TOTP keys, and because of that I’m very unhappy with the idea of relying such a service: now all your key are not only concentrated on an app on your phone, but also on a remote server. And the whole point of using 2FA, for me, is that my passwords are stored in LastPass 1Password.

There is one more app in the list of mostly-authenticator apps: my Italian bank’s. I still have not written my follow up but let’s just say that my Italian bank used TOTP-token authentication before, and have since moved to an hybrid approach, one such authentication system is their mobile app, which I can use to authenticate my bank operations (as the TOTP-token expired over a year ago and have not replaced yet). It’s kind of okay, except I really find that bank app too bothersome to use and never bother using it right now.

The remaining authentication systems either send me SMS or are configured on Google Authenticator. In particular, for SMS, the most important services are configured to send me SMS to my real Irish phone number. The least important ones, such as Humble Bundle itself, and Kickstarter, which also insist on not even letting me read a single page without first logging in, send their authentication code to my Google Voice phone number, so they require an Internet connection, but that also means I can use them while on a flight.

Oh yes, and of course there are a couple of services for which the second factor can be an email address, in addition, or in place, of a mobile phone. This is actually handy for the same reason why I send them to Google Voice: the code is sent over the Internet, which means it can reach me when I’m out of mobile connectivity, but still online.

As for the OTP app, I’m still vastly using the Google Authenticator app, even though FreeOTP is available: the main reason is that the past couple of years finally made the app usable (no more blind key-overwrites when the username is the only information provided by the service, and the ability to change the sorting of the authenticator entries). But the killer feature in the app for me is the integration with Android Wear. Not having to figure out where I last used the phone to log in on Amazon, and just opening the app on the watch makes it much more user friendly – though it could be friendlier if Amazon supported U2F at this point.

I honestly wonder if a multiple-weeks battery device, whose only service would be to keep TOTP running, would be possible. I could theoretically use my old iPod Touch to just keep an Authenticator app on, but that’d be bothersome (lack of Android Wear) and probably just as unreliable (very shoddy battery). But a device that is usually disconnected from the network, only dedicated to keep TOTP running would definitely be an interesting security level.

What I can definitely suggest is making sure you get yourself a FIDO U2F device, whether it is the Yubico, the cheapest, or the latest BLE-enabled release. The user friendliness over using SMS or app codes makes up for the small price to pay, and the added security is clearly worth it.

It would take so little to be compatible

You might remember that I worked on my own static website template engine and that I made it Free some time ago. This template engine uses XSLT at its heart; just like the Gentoo XML format, and DocBook. This makes it slightly less powerful than something written in a “proper” programming language, but still powerful enough for most of my uses.

One of the features I implemented last year, was linking to Facebook so that my friend could share his projects directly there. At the time I implemented some basic OpenGraph support, and added some more links to help Google and other search engines to find the pages in the right place, including using a direct reference to Youtube’s video on those pages that almost entirely focus on the video.

Unfortunately that last feature was not used at all by Facebook, at the time, since it did mean loading a third party website within their page, which was at the very least frown upon. It didn’t really matter, at some later point I wanted to make use of HTML5 <video> elements, but doing so meant spending quite a bit of time trying to get it just right and supporting as many browsers as possible.. for the moment I gave up.

A couple of days ago, my friend published a new video on his site, and I went to link it on the facebook page… right away, instead of simply creating a link with the given thumbnail, it has found the actual YouTube embedding code and put it on the side of the provided link and title. Cool, it works!

So today I was spending some more time polishing FSWS, and I went to Facebook’s Open Graph page to see what else I could make use of. Interestingly enough, I note now that even the Open Graph page is now more complete as to which options are available. I don’t know why, but they decided to add support for indicating the width/height of the video in the page, which is all good and dandy. Until you note the name of the attributes: og:video:width.

What about that? Well, in my system, given I’m speaking about XSLT, I translate the OpenGraph meta tags into attributes to the fsws:page element, and related. This looks like the most RDF-compatible implementation as well, to me. Too bad that for this to work, the attributes need to have a valid QName and a valid QName only has one colon character. D’oh.

The end result of this is that I probably have to rethink most of what I did up to now, and avoid using attributes to represent Open Graph attributes, almost certainly breaking the few websites using FSWS already. But I wonder if they couldn’t just be a bit more compatible and avoid using the colon character as separator there…

I hate the Facebook API

So, my friend is resuming his movie-making projects (site in Italian) and I decided to spend some time to make the site better integrated with Facebook, since that is actually his main contact media (and for what he does, it’s a good tool). The basic integration was obviously adding the infamous “Like” button.

Now, adding a button should be easy, no? It’s almost immediate doing so with the equivalent Flattr button, so there should be no problem to add the Facebook version. Unfortunately the documentation tends to be self-referencing to the point of being mostly useless.

Reading around google, it seems like what you need is:

  • add OpenGraph semantic data to the page; while the OpenGraph protocol itself only mandates title, url, type and image, there is an obscure Facebook doc actually shows that to work it needs one further data type, fb:admins;
  • since we’re adding fb:-prefixed fields, we’re going to declare the namespace when using XHTML; this is done by adding xmlns:fb="; to the <html> field;
  • at this point we have to add some HTML code to actually load the Javascript SDK… there are a number of ways to do so asynchronously… unfortunately they don’t rely, like Flattr, on loading after the rest of the page has loaded, but need to be loaded first, following the declaration of a <div id="fb-root" /> that they use to store the Facebook data.
window.fbAsyncInit = function() {
  FB.init({ cookie: true, xfbml: true, status: true });

(function() {
  var e = document.createElement('script');
  e.src = document.location.protocol + '//';
  e.async = true;

* then you can add a simple like button adding <fb:like/> in your code! In theory.

In practise, the whole process didn’t work at all for me on the FSWS generated website, with neither Chromium nor Firefox; the documentation and forums don’t really give much help, most people seems to forget fields and so on, but for what I can tell, my page is written properly.

Trying to debug the official Javascript SDK is something I won’t wish for anybody to be forced to; especially since it’s minimised. Luckily, they have released the code under Apache License and is available on GitHub so I went on and checked out the code; the main problem seems to happen in this fragment of Javascript from connect-js/src/xfbml/xfbml.js:

      var xfbmlDoms = FB.XFBML._getDomElements(
      for (var i=0; i < xfbmlDoms.length; i++) {
        FB.XFBML._processElement(xfbmlDoms[i], tagInfo, onTagDone);

For completeness, dom is actually document.body, tagInfo.xmlns is "fb" and tagInfo.localName in this case should be "like". After the first call, xfbmlDoms is still empty. D’uh! So what is the code of the FB.XFBML._getDomElements function?

   * Get all the DOM elements present under a given node with a given tag name.
   * @access private
   * @param dom {DOMElement} the root DOM node
   * @param xmlns {String} the XML namespace
   * @param localName {String} the unqualified tag name
   * @return {DOMElementCollection}
  _getDomElements: function(dom, xmlns, localName) {
    // Different browsers behave slightly differently in handling tags
    // with custom namespace.
    var fullName = xmlns + ':' + localName;

    switch (FB.Dom.getBrowserType()) {
    case 'mozilla':
      // Use document.body.namespaceURI as first parameter per
      // suggestion by Firefox developers.
      // See
      return dom.getElementsByTagNameNS(document.body.namespaceURI, fullName);
    case 'ie':
      // accessing document.namespaces when the library is being loaded
      // asynchronously can cause an error if the document is not yet ready
      try {
        var docNamespaces = document.namespaces;
        if (docNamespaces && docNamespaces[xmlns]) {
          return dom.getElementsByTagName(localName);
      } catch(e) {
        // introspection doesn't yield any identifiable information to scope

      // It seems that developer tends to forget to declare the fb namespace
      // in the HTML tag (xmlns:fb="") IE
      // has a stricter implementation for custom tags. If namespace is
      // missing, custom DOM dom does not appears to be fully functional. For
      // example, setting innerHTML on it will fail.
      // If a namespace is not declared, we can still find the element using
      // GetElementssByTagName with namespace appended.
      return dom.getElementsByTagName(fullName);
      return dom.getElementsByTagName(fullName);

So it tries to workaround when the xmlns is missing by using the full name fb:like rather than looking for like for their own namespace! So to work it around at first, I tried adding this code before FB.init call:

FB.XFBML._getDomElements = function(a,b,c) {
  return a.getElementsByTagNameNS('', c);

This actually should work as intended, and finds the <fb:like /> elements just fine. If it wasn’t, that is, that the behaviour of the code then went ape, and the button worked intermittently.

At the end of the day, I decided to go with the <iframe> based version like most of the other websites out there do; it’s quite unfortunate, I have to say. I hope that at some point FBML is replaced with something more along the lines of what Flattr does with simple HTML anchors that get replaced, no extra namespaces to load…

The bright side is that I can actually use FSWS to replace the XFBML code with the proper <iframe> tags, and thus once it’s released it’ll allow to use at least part of the FBML markup to produce static websites that do not require the Facebook Javascript SDK to be loaded at all… now, if FSF were to answer me on whether I can use text of the autoconf exception as a model for my own licensing I could actually release the code.

The self-hosting dilemma

Today, the sources for the open-source alternative to Facebook, Diaspora were released. Similarly to what is often done by Google, the “old” Sun, and other big software companies when they release something Free or Open Source, the sources weren’t available incrementally, but were just made available with a big code drop, from where the actual (supposedly) open development started.

How does this work out? For now, there seem to be a lot of hype, as expected for something like this. On the other hand, Jürgen found already a problem with missing input sanitisation and expressed quite a few doubts about its distribution model. While I agree that it’s too soon to tell the future, I also have a lot of doubts.

While Jürgen actually criticises the technical side (and as you might guess, I agree with his points), I’m also very doubtful about the social part of the project. Why is that? Well, the main problem is that Diaspora for many looked like a way to “take control of their data” by hosting their own instances. This is something that, I fear, is going to hurt the collective soon enough. With a single word: spam.

While the instances should be more similar to (multi-user) than to WordPress (single-user) and the other blog engines, there is definitely a huge front of users who want not to upload their content at all, and rather get a huge number of instances around the globe. How is that bad, aren’t distributed systems better? Well, yes and no. While distributed, peer-to-peer networking is more resilient (because there usually isn’t just one “master” node), their load on the network can in theory become massive enough to be a problem; if each Diaspora user (or each family) were to host their own instance, and they actually reached and overtook Facebook, it would probably be a problem, but that’s quite unlikely to begin with.

Assuming that this actually became the case, or that at least enough “groups” of people actually set up instances (which would make much more sense I guess) what the result is going to be? Well, from one side you would have a situation similar more to Jabber than I guess; the (IMHO) big problem with Jabber is that S2S has never been perfect and a lot of people ended up being unable to contact others in a stable manner over it; nowadays, GTalk is likely the major provider of Jabber users, and that actually increased it quite a lot, to the point I don’t even use MSN any more (Facebook solved the other half of the problem, my non-techie friends). Servers that come and go, stale connections, slow connections, all that trouble is going to make Diaspora quite difficult to handle for most people, who nonetheless will still want to have their own installation of it; after all, isn’t that what it was designed for in the first place?

Once that problem is solved, the other problem is going to be access control and information push, which seems to be what most of the people having a problem with Facebook are concerned with. I haven’t looked at the actual architecture, so I might be totally wrong, but how is the distributed approach going to help people saving themselves? As I wrote before I don’t consider hands-on approaches to applications availability a bad thing in absolute, so in this I think that Facebook is most likely going to be much more effective for protecting users from rogue applications; if there is no central authority, it’s just in the hands of users to make sure that they don’t enable “rogue” applications. And again, I sincerely expect most people to just click “Enable” on enough of those, if it actually gains users on the “lower tech guys”; the same way that Facebook applications a few years ago kept spamming me with “John Doe knows something nasty about you” when notifications were shared — they dropped the feature from the API entirely as far as I can tell.

Okay, let’s assume that the Diaspora users will all be savvy enough to choose their applications well and the architecture is good enough to actually avoid spreading these applications like viruses over the Diaspora network. What is the next obvious step? Spam within the network. While to actually send content you have to have a dual-opt-in handshake (friend request, and accepted friend), the very request can be spam enough. Just think of the way the “followers” work on StatusNet right now: when you receive the notification or look at your followers’ list, there are high chances that the user is just providing a link to a spam site. Ooops. Now make this trivial to implement thanks to an open API to send requests and… consider that I did receive more than a couple of similar “spam requests” on Facebook, and there they had to go through the registration process and a more complex request API, and a centrally-managed firewall/antispam.

Then there is the sore problem that Jürgen noted: security. Not only the current demos seem to have a bit of concerns of XSS as they don’t sanitise input – at least partially – but there is the problem of using a bunch of “vendorized” (actually bundled) Rubygems, including Rails. I have nothing against Rails; heck this very blog is was running on Rails (edit: at the time of writing), but to actually keep it running properly I have configured Apache, tweaked it, installed ModSecurity and configured it with IDS and RBL — and this is for an application that has no webservice nature, so it should only allow actions through the limited comments interface or after login. While I’m no expert sysadmin, I’m pretty sure I have done a much better job that the average Diaspora user is ever going to think about; the nature of the “just fire it up and it’s done” is also going to be prone to ignoring updates and new releases for as long as possible. And you know what this bring us? A number of Internet-facing, private, around-the-globe servers with possibly-vulnerable code running cleanly. D’uh!

I know of a number of users who complained when ISPs started filtering SMTP peering, as they couldn’t run their own mail server; the cause? Spam, due mostly to a huge number of misconfigured mail servers. I wouldn’t be surprised if providers started filtering, or blocking users, based on Diaspora-caused spam.

Sincerely, I don’t think there is much difference, nothing that makes Diaspora particularly different from or special compared with things like phpBB (one of the worst security tracks I can think of), WordPress and the other commonly used web applications — on a technical level at least. What really is bad is the idea of self-hosting it for each person, or each family, or group. But that, seems to be what most people advertised the project for, as a “revolution” for social networking.

I’m skeptic, but I’ll be looking from the sidelines; for what I’m concerned Facebook is already enough; the best use for it I have is the sync with the Motorola Milestone so I can actually have the email addresses and the phone numbers of my acquaintances available when I’m on the go even if I didn’t copy them (yet) on my main addressbook. But even that is pretty vague. We’ll see.


I’m still not sure why I work with Rails, if it’s really for the quality of some projects I found, such as Typo, if it is because I hate most of the alternatives even more (PHP, Python/Turbogears/ToscaWidgets), or because I’m masochist.

After we all seemed to settle in with Rails 2.3.5 as the final iteration of the Rails 2 series, and were ready to face a huge absurd mess with Rails 3 once released, the project decided to drop a further bombshell on us in the form of Rails 2.3.6, and .7, and finally .8 that seemed to work more or less correctly. These updates weren’t simple bugfixes, because they actually went much further: they changed the supported version of Rack from 1.0.1 to 1.1.0 (it changes basically the whole internal engine of the framework!), the version if tmail and of i18n. It also changed the tzinfo version, but that’s almost pointless when considered in Gentoo since we actually use it unbundled and keep it up-to-date when new releases are made.

But most likely the biggest trouble with the new Rails version is implementing an anti-XSS interface compatible with Rails 3; this caused quite a stir because almost all Rails applications needed to be adapted for that to work. Typo for instance is still not compatible with 2.3.8, as far as I know! Rails-extensions and other Rails-tied libraries also had to be updated, and when we’ve been lucky enough, upstream kept them compatible with both the old and the new interfaces.

At any rate, my current job requires me to work with Rails and with some Rails extensions; and since I’m the kind of person who steps up to do something more even though it’s not paid for, I made sure I had ebuilds, that I could run test with, for all of them. This actually turned out more than once useful, and as it happens, today was another of those days when I’m glad I’m developing Gentoo.

The first problem appeared when it came time to update to the new (minor) version of oauth2 (required to implement proper Facebook-connected login with their changes last April); for ease of use with their Javascript framework, the new interface uses almost exclusively the JSON format; and in Ruby, there is no shortage of JSON interpreters. Indeed, beside the original implementation in ActiveSupport (part of Rails) there is the JSON gem, which provides both a “pure ruby” implementation and a compiled implementation (to be honest, the compiled, C-based implementation, was left broken in Gentoo for a while by myself, I’m sorry about that and as soon as I noticed I corrected it); then there is the one I already discussed briefly together with the problems related to Rails 2.3.8: yajl-ruby. Three is better than one, no? I beg you to differ!

To make it feasible to choose between the different implementations as wanted, the oauth2 developers created a new gem, multi_json, that allows to switch between the different implementations. No it doesn’t even try to give a single compatible exception interface, so don’t ask, please. It was time to pack the new gem then, but that caused a bit of trouble by itself: beside the (unfortunately usual) trouble with the Rakefile demanding RSpec to even just declare the targets (so also to build documentation) or the spec target having a dependency over the Jeweler-provided check_dependencies, the testsuite failed on both Ruby 1.9 and JRuby quite soon; the problem? It forced testing with all the supported JSON engines; but Ruby 1.9 lacks ActiveSupport (no, Rails 2.3.8 does not work with Ruby 1.9, stop asking), and JRuby lacks yajl-ruby since that’s a C-based extension. A few changes later and the testsuite reports pending tests when the tested engine is not found as it should have from the start. But a further problem appears in the form of oauth2 test failures: the JSON C-based extension gets identified and loaded but the wrong constant is used to load the engine, which results in multi_json to crap on itself. D’oh!

This was already reported on the GitHub page, on the other hand I resolved to fix it in a different way, possibly more complete; unfortunately I didn’t get it entirely right because the order in which the engines are tested is definitely important to upstream (and yes it seems to be more of a popularity contest than an actual technical behaviour contest, but nevermind that for now). Fixed that, another problem with oauth2 appeared, and that turned out to be caused by the ActiveSupport JSON parser; while the other parsers seems to validate the content they are provided with, AS follows the “Garbage-in, Garbage-out” idea: it does not give any exception if the content given is not JSON; this wouldn’t be so bad if it wasn’t that the oauth2 code actually relied on this to be able to choose between JSON and Form-Encoded parameters given. D’oh! One more fix.

Speaking about this, Headius if you’re reading me you should consider adding a pure-Java JSON parser to multi_json just for the sake of pissing off MRi/REE guys… and to provide a counter-test of a not-available engine.

At least, the problems between multi_json and oauth2 had the decency to happen only when non-standard setup were used (Gentoo is slightly non-standard because of auto_gem), so it’s not entirely upstream’s fault to not have noticed them. Besides, kudos to Michael (upstream) who already released the new gems while I was writing this!

Another gem that I should be using and was in need of a bump was facebooker (it’s in the ruby-overlay rather than in main tree); this one was already bumped recently for compatibility with Rails 2.3.8, but keeping itself compatible with the older release, luckily. Fixing the testsuite for the 1.0.70 version has been easy: it was forcing Rails 2.3.8 in the lack of multi_rails, but I wanted it to work with 2.3.5 as well, so I dropped that forcing. With the new release (1.0.71) the problem became worse because the tests started to fail. I was afraid the problem was in dropped 2.3.5 compatibility but that wasn’t the case.

First of all, I worked out properly the problem of forcing 2.3.8 version of Rails by making it abide to RAILS_VERSION during testing; this allows me not to edit the ebuild for each new version, as I just have to tinker with the environment variables to get them right. Then I proceeded with a (not too short) debugging session, which finally catapulted me to a change in the latest version. Since one function call now fails with Rack 1.1.0 for non-POST requests, the code was changed to ignore the other requests; the testsuite on the other hand tested with both POST and GET requests (which makes me assume that it should work in both cases). The funny part was that the (now failing) function above was only used to provide a parameter to another method (which then checked some further parameters)… but that particular argument was no longer used. So my fix was to remove the introduced exclusion for non-POST, remove the function call, and finally remove the argument. Voilà, the testsuite now passes all green.

All the fixes are in my github page and the various upstream developers are notified about them; hopefully soon new releases of all three will provide the same fixes to those of you who don’t use Gentoo for installing the gems (like, say, if you’re using Heroku to host your application). On the other hand, if you’re a Gentoo user and want to gloat about it, you can tell Rails developer working with OS X that if the Ruby libraries aren’t totally broken is also because we thoroughly test them, more than upstream does!

Facebook, usefulness of

Seems to me like the existence of Facebook is seen in either of two ways: either the coolest website in the world, or the most useless one; given I am subscribed, too one would expect me not to be in the latter category; but I really would take offence if you categorized me on the former category!

Indeed, I actually fall into a “third way” category: I find it useful to some point, but not very much. I do use it, and I have both friends, and a few work-related contacts (not many); I also have Gentoo users and developers, but I tend to select who I accept requests from (so if I spoke with you once or twice, it’s rare; if you’re somebody I happened to collaborate with for a while at least, then I’ll probably accept). I don’t feel like it’s an invasion of my privacy, to be honest, since my statuses are almost the same there as they are on and twitter; my notes are usually just my own blog, I might do some non-idiotic meme from time to time (more on that later), I don’t really do quizzes or use strange applications. I might have some not-so-public photos, but that’s really just nothing you’d have fun seeing me in, since they are usually nights out with friends; and if it was just for me they could also be public, I don’t care. I do have my full details: phone numbers, street address, email and IM users, but they are not really private details, given that my phone numbers and addresses correspond to my work details, and the rest, well, let’s just say I don’t really have much imagination and you can find me as Flameeyes almost everywhere.

So what’s the usefulness of Facebook at this point in time for me? Well, I do aggregate my blog, to show it to my friends, and let them share it with others so that others can read what I write (I hoped that my post about Venetian public offices ended up shared more but it doesn’t seem like my friends do seem to be interested in real politics outside that of parties); I reach more people, that don’t follow or twitter, and I do follow them too, so it really does not add much there either. When somebody I know I have as a contact on Facebook asks me for my details, well, my answer is just “Look at my Facebook profile”; it’s there for a reason). In general, it’s just another medium like this blog, like planets aggregators and so on. It does not really add much. It’s a bit more than an overhyped address book.

One note that is often made is that the idea of finding “people you haven’t seen in years” is pointless because… you haven’t seen them in years for some reason.Sometimes, though, it might just be a problem with losing contacts, going different ways but still interested in getting back in touch and hearing from, from time to time, so it works as a medium for that too.

And on a similar note, why do I find memes interesting, or even useful? Well sometimes you do know somebody, or at least met somebody but don’t know well enough to know some personal nitpicking details; memes might strengthen a bond between people by providing possibilities to compare and identify similar tastes and other stuff. In particular note-based memes (or blog based memes) don’t require you to use stupid third-party applications to do that. Yes I know it might sound silly, but I can use the example of an ex-classmate of mine who I haven’t seen in almost ten years for various reason, until facebook came and we actually found we now have common interests; people grow up and change.

Unfortunately, in all this I don’t see anything that can save Facebook from its financial problems: it really does not work for advertisement, most of the applications seems to be on the limit of fraud, and there is no fee to enter, nor there seems to be any particularly interesting or important paid services (as a counter-example, Flickr’s paid version, with no limit on photo upload and access to the original images, is a service for which even I pay!). For this reason, I really don’t relay (sorry for the words’ game) on Facebook to store important information (so yeah I do keep my address book updated outside of it), I wouldn’t be surprised if next month they start charging for service, or if in four they close down entirely. Nor I would miss them.

And to finish, why on earth am I writing about Facebook now? Well, I just want to warn my readers for why in the next few days they might find some Italian posts talking about Facebook; and in turn that is part of my plan to try instructing my friends and acquaintances on how to behave on the network, and with a computer. Which hopefully will allow me to write it once rather than re-explain it every other time I have to take over a PC to clean up from viruses and other issues.