Service Providers, Business Sustainability, And Society

I’m not sure how many people think consciously about the business plans of the service providers that they start using, or at least relying upon. I don’t do it terribly often, but I do sometimes, and I thought I would share some words on the topic, because I do think that the world would be a better place if we did think about the effect of our choices to the wide world.

I have for instance wondered aloud, over the years, on Twitter and even on this blog, about the fintech company Curve. Their services are actually interesting and valid… but they do not offer any interesting value at a premium — I would never pay for their “Curve Metal” tiers, because it just doesn’t add up. I couldn’t just figure out how they were expecting to keep running, given I expect the majority of their “customers” are consumer end-users that would follow the same procedure I followed: sign-up for the service, using the £5 offer, use it for the 90 days of free cashback, possibly use it a time or two while traveling, and otherwise just keeping it as a backup card. Before the pandemic I also wrote how they were giving out more free money, but more recently they decided to start crowdfunding. And they became very loud when they did. Which is what reminded me I had a half-drafted post on the topic (this post), which I should probably resurrect (which I did, since you’re reading this now).

Before going back to Curve and their crowdfunding, I want to point at two sayings that you’ll keep finding around you, when you discuss businesses, products, Internet, and privacy: «If it sounds too good to be true, it probably is» and «If you are not paying for it, you’re not the customer, you’re the product». These are good places to start a discussion, although I think there is a lot of nuance that is lost when trying to (over) simplify with these.

Full disclosure: I work for a big company that, primarily, offers services to end users free with ads, though the product I personally work on not only does not relate to ads, but does not even have ads in it. And while my previous employer was another big company that is part of the “AdTech” business (and in there I did work on ads systems for a long while), I have discussed ads in the past, and I even ran ads on this blog (back in the days before stable employment), so you can imagine that what I’m going to be writing about is my personal opinion and does not represent that of my current, past, or future employers.

So why do I think it’s important to figure out the sustainability of service providers? Well, because it becomes a problem for the whole of society when a fraudulent or scammy provider gets a certain about of market share, even if sometimes not evenly and not in a way that most people would be able to connect together. For instance you can take the examples of Enron, Bernie Madoff, Theranos, or Wirecard — organizations that promised too-good-to-be-true services and profits, and ended up bust, with different blast radiuses. For the last one we don’t even quite know yet what the blast radius will be once things settle down: the German financial services environment is likely going to be reshaped by last year’s scandal, and so it appears will be EY.

Though this is clearly not limited to financial services (VPN providers seem to be pretty much in the same position), it does look like the likes of Curve and Revolut are easily the most visible cases where a company apparently lacked a sustainable business plan, and decided to turn to a crowdfunding campaign — Curve just had one this past May, and it was so noisy that I know a couple of people who went on to find a way to delete their account (and the app) simply because they got tired of their pushy notifications (not a typo).

Now, that might sound not too bad — after all, crowdfunding for the most part just means someone is willingly going to pay to subsidise other people’s “free money”. But the next thing that Curve did after that was to increase referral bonuses for new users to £20, from the previous £5, and that smells to me even more fishy — because that sounds like trying to bring in a mass of users hoping that enough of them can’t figure out that the premium options of Curve are not worth the money.

On the other side of the tracks, Revolut has been pushing more and more for cryptocurrencies, which I’m not going to even pretend is a neutral thing. I care enough about the environment that their consumption alone makes me angry, but even more so, I find that the amount of scams related to cryptocurrencies at this point are wide enough to show that the whole concept is hostile to society. I do not support nor recommend Revolut to new users unless they live in countries like Ireland where there is no other option – in London, using Revolut feels like subsidizing scammers by lending respectability to cryptocurrencies.

But at this point, neither of those appear to have reached the full consumer scams, so it should be fine, right?

Well, let’s take a different example with the VPN market. I have complained over on Twitter a few times how I blame us geeks for the amount of VPN scams that are out there. Privacy maximalists tend to scare people with the idea that your ISP, the Starbucks, or the airport lounge you’re using can see everything you do — and while it is definitely the case that there’s a lot more data going around than you may think, ads such as those ExpressVPN pays the otherwise excellent No Such Thing As A Fish podcast to air, that suggests that your ISP would be able to tell what you’re Googling are not just falsehoods, but proper FUD.

But even accepting that ExpressVPN has no ulterior motive and are totally legit – I have no idea about that, I only used them before while in China – and leaving aside the fact that VPNs have huge targets painted on their backs, there’s still the matter that you need to trust your VPN provider. Which may or may not be more trustworthy than your home ISP — the two of of them having pretty much the same power. Most of the review websites seem to be talking more about commissions than trust, because the worst part is that there is nothing that allows you to verify their statement that they are not logging your traffic in the same way they keep insisting your ISP is doing.

So how do you trust a VPN provider? Well, for a start you may want to start considering who their founders are and how they get their money. And you’d be surprised how many dots you can connect this way. For instance, last year CNET wrote about Kape Technologies, a company that bought a Romanian VPN service called CyberGhots. In that article, they also noted that in addition to CyberGhost, the same parent company bought two more VPNs:

After buying CyberGhost, Kape then bought VPN ZenMate in 2018 and more recently Private Internet Access, a US-based VPN, in a move which Erlichman said in a press release would allow Kape to “aggressively expand our footprint in North America.”

Now, the problem is less about a single parent company owning multiple VPN services as they were different brands — this happen all the time in many other fields. Just look at the relationship between Tesco Mobile and O2, or banks such as Halifax and Lloyds. But the rest of the article does make for a good build up for why the whole situation is a bit suspect.

But more importantly, you may have heard of Private Internet Access before — they are the company that started heavily sponsoring Freenode a few years ago. And if you have been paying attention to Free Software projects’ communication in the past few months, you probably know by now that Freenode is a trash fire now. So given those connections, would you trust anything that has connections to these organizations and people? I clearly wouldn’t.

This same problem with trust and business sense applies to other businesses. With the exception of B Corporations, most companies out there are intended to make money. If nothing else, they need to make money so that they can pay the wages of the people working there. So I don’t generally trust companies that appear to be giving everything away — and rather prefer those that, if they are making money with ads, say so out right.

In the case of Fintech services — Wise (formerly known as TransferWise) is my example of choice for a company that is transparent when it comes to the cost associated with their services, and makes a good case for why they charge you, and how much so. I really wish more of them did the same because it would make it easier for people to choose how much trust to put in a company. Unfortunately it appears that the current trend in the market is to push as much grown as possible for companies to grab a captive audience before turning on the monetization screw.

Important note: this blog post was written before Wise announced they intend to go public (it was previously rumored, but I didn’t spot that). I guess I should now disclose that I will most likely consider buying some stock of the company, though probably not on the IPO day. We shall see. As I said, I do like their business sense.

Going back to a moment to that «if you’re not paying for it, you’re the product» as well — well, I don’t agree in full, but this is something that people do need to be keyed in to look out for. In particular, I don’t think that ad-supported businesses should disappear, and that everything should be hidden behind a paywall, because I do think that having wider access to information without making it costly is a good thing. But also I do think that there are services that are often crossing the line into being creepily interested in your data rather than “trade it” for useful information.

But I also think the scrutiny is often placed more on the big, established companies rather than the “scrappy” startups, or the more consulting-like companies. Heck, a few of you reading this are probably already ready to complain that both my current and past employers are seen as data hungry — but I can tell you that both companies, at least during my tenure, would never have someone state on a stage that collecting data from IoT sensors and just throwing it to a ML pipeline to gather unexpected insights, as it would go against every one of the privacy and data handling trainings and commitments…

And yet John Roese from Dell EMC stated that in his opening thoughts for LISA 16 (go to minute 44 in the open access video) in what sounds terribly like an advice to startups. To be honest, that’s not the only cringey thing in that opening talk — from a technical point of view, his insisting that persistent memory means you can’t just reboot a computer to reset the state of memory (as if re-loading the data in memory from scratch wouldn’t happen on request, whether this is persistent or not) is probably a worst phrase.

What I’m trying to say is that you need to be sure who your friends are, and it’s not as easy as to expect that all small players are ethical and all big ones are not. And asking yourself “how are they making money, if at all?” is not just allowed — it should sometimes be considered a necessity.

You can’t program the world to suit you

Last year, I was drafting more notes regarding the Free Software for SMB that I talked about before. While doing so I recognized that one of the biggest takeaway for myself is that successfully making a software project thrive takes a lot more than just good programmers, developers, designers. If you have customers you need people who know how to make a business work, you need people who can market your product, and you need people to remind you what the customers actually want as well as what they need.

It’s not an entirely new lesson — I wrote (in Italian) about helping Free Software without being a programmer fifteen years ago. I also wrote about the importance of teamwork two years ago. And I have spent a good chunk of my opensource and professional careers knee-deep in documentation matters.

I actually want to go back to the tweet that spawned the teamwork post:

Most things don’t work the way I think they work. That’s why I’m a programmer, so I can make them work the way I think they should work.

This is not meant to single out the author of the quoted phrase, but just to take it as an example of a feeling I get from many talks, and discussions, and in general just people out there. The idea that you can tech your way out of a problem. That by being a programmer you can change the way most things work.

And that’s not true, because the world is not running on servers, unless you found the Repository and I don’t know that. Indeed wielding “the power of programming”, thinking of changing the world just with that, sounds to me like a recipe for either failure or disaster.

I heard all kind of possible “solutions” to this — from insisting on teaching ethics in Software Engineering courses (with reasonable doubts about it), to regulating the heck out of any action businesses can take. I think the closest I have seen to something I would like (with all my biases of course) would be to make sure there is a mix of non-programming subjects in every university or high school that teaches programming as well. But even that has its own limitations, and I can totally say that I would probably have been frustrated by that and just ignored everything that’s not programming-related, when I was that age.

To make the example of Italy, that is under political turmoils most of the time, I could see a number of critiques of (in my opinion horrible) politicians based on where they went to school. In particular I saw some left-wing intellectuals criticising ministers (who have enough to be criticised about in deeds) based on the fact that they didn’t study in a lyceum but rather on a technical (or professional) school. Well, turns out I studied at a tech school, and I studied basic economics and (very basic) civic education for two years, and I found out the hard way that I know how VAT works much better than most of my local acquaintances who got an university degree after a lyceum: they never were introduced to the concept of VAT, the difference between types of taxes, and so on.

You could argue that there is no reason to know this particular tidbit, which is where I’m actually going to end up: there is no perfect education, the same way as there is no perfect solution. People need to learn to work with each other and they should know how to play each other’s strengths instead.

What I really would like to see proposed more often is focusing a lot more on teamwork. And not in the sense of “Here’s a topic for research, now work on it with your team”, which I had to do in high school — many of us have had the experience of being the only person working for a group assignment. What I would have loved to have would be cross-school year-long projects. Not competitions, but rather something that requires more than one type of expertise: trying to get three programming students in a room to work together, in my experience, turned to either two of them slacking off, because one of them actually enjoy doing the work, or if you’re lucky having someone with actual leadership skills telling them how to do their job… but still gives the impression that you just need programmers to do something like that.

In hindsight I would have loved instead if I had a project shared with some of my colleagues from electronics, mechanical and business tech-schools. Come up with a solution for a problem, that requires hardware and software, and a product plan that would include optimising the bill of material for small batch production and still make profits.

Sounds complicated? It is. Having had my own company, alone, for four years, made it very clear that there is a lot more than just being a programmer if you want to succeed. If you want to change the world, and in particular if you want to make the world a better place, then it takes even more energy, and a bigger group of people who can work together.

It also takes leadership. And that’s not something that I feel can be taught, and it’s the one that makes the most difference on whether the change is for good or not. I’m not good at leading people. I don’t have the right mindset most likely. I have trouble rallying people towards a common goal. I know that. I just hope that at some point, when I’ll be looking at more meaning in my work, I’ll find the right leader that can take what I can add to a good team, and let me shine through that.

I know it’s going to be repeating myself, but that is also what I mean with “there is no perfect solution”. If we decided that leadership is something that is important to score people, whether it is with school results, or with performance review at work, then we would be pretty much excluding a significant part of the population: not everyone wants to be a leader, are people who don’t want to be a leader worth less to society? Hint: this is not far from the question of how many multiples of a line worker a CEO should be worth.

And if you need a proper example of how “tech will not solve it”, just look at 2020 in general: tech is not really solving the Covid-19 world crisis. It does help, of course: videopresence, social network and chat services (including my employer’s), online “tabletop” games, shared documents infrastructure, online shopping, and so on… they all allowed people, isolating or not, to feel closer together. But it did not solve the problem. Even if we including medical sciences as “tech”, they still have not managed to find a way to deal with the crisis, because the crisis is not just medical.

People don’t ignore the lockdown requirements because they don’t have enough tech: it’s because there are other things in this world! It’s one thing to talk to my mother on the big screen of Portal, and another thing to spend a week at her house — including the fact that I can’t fix her house’s wiring while physically in another country. And then there is the big elephant in the room: the economy — tech can’t solve that problem, people working in industries that had to shut down because of the lockdown can’t just be “teched” into new roles; they can’t magically be vaccinated overnight; they need political leaders to make tough decisions around supporting them.

So no, you can’t program the world to suit your needs. Great for you if you have more tools in your toolbox – and there’s a lot more use for even basic programming literacy that has nothing to do with working as a programmer – but that doesn’t make you super-human, nor it allows you to ignore what’s going on in the world. If “being a programmer” is providing a superiority complex, I feel it’s more related to the fact that we’ve been well paid for a number of years now, and money makes the difference.

But that’s a topic for an entirely new rant, later on.

“Working From Home”

Despite having commented on my continuing lockdown, I have tried extremely hard not to comment too much on the whole WFH debate, at least on the blog. You might have seen me ranting about it a few times over on Twitter though.

First of all, I have to admit I was lucky — when the whole lockdown started, I didn’t have to scramble to find the space to work from home: I already set up a home office, I had a standing desk already, multiple monitors, proper home connection without relying on WiFi, and all the kind of ergonomic setup that many of my teammates had to scramble hard for. I had set this up when I got to London, because I remember how bad it was for me to have a work/life balance separation in Dublin, when I had a desk just sitting next to the sofa, and I would end up working till late instead of just sitting on the sofa to watch TV or play games.

And of course, I’m also counting myself lucky that neither me nor my wife fell ill, form Covid-19 or anything else, that our families – while struggling a bit – had been safe throughout this whole event. And also, since we’re not interested in having kids, that significantly reduced the amount of worry, and of work, needed to switch to the lockdown scenario. I can only imagine how much harder for families it still is and don’t envy them.

But at the same time, I do miss the office, and am hoping not to stay working from home forever. I spent many years working from home, alone and isolated, while I had my own company, back in Italy. And while I can do a significant amount of work individually, I do believe that teamwork brings better results. Thankfully, “telepresence” options such as Portal, Zoom, and Google Meet help significantly to coordinate the work, but they are not quite the same thing. I feel more relaxed working sitting at a desk next to my colleagues than I feel working at my desk with a camera pointed at me while I’m working — it makes me feel self-conscious.

I’m also painfully aware that even with the luck of being able to keep working from home, there’s a lot of things that are left to be desired. For instance, while lots of people are bringing up the fact that you don’t have to pay for commute anymore as a great positive, few go back to point out that you end up paying more for utilities such as electricity, water, and heating. We could see a good 20% increase in electricity usage since I started working from home, and while we’re (again) lucky that this is not a significant difference, I can see how the heating in the winter, for people leaving in cottages, would wipe out any commuting savings for the year.

And while I can definitely find an easier way to get my focus time from home, even taking turns preparing meals with my wife, the amount of time we spend for fixing up two extra meals a day (breakfast and lunch) is noticeable. The whole “free food” perk is not just about not paying for food: it’s about the time it takes to make the food, and the time it takes away from your workday.

There’s a lot more of course, on both side of the equation — and there’s the whole point that we’re in the middle of a pandemic that is literally reshaping the way we live. I’m just looking forward to go back to an office, and to have a commute — not because I want to spend an hour on a crowded Tube train, but because I want a little bit of time to mark the end of a workday, stop worrying about the issues of the day, and turn off my work phone, so I can join my wife at the end of the day in full without splitting my mind with work.

It’s tiring, and it’s getting to me, and I’m sure it’s getting to many. Be looking out to your friends and your colleagues. Cut them a break if they are snappier than usual, particularly if they have complicated home situations – kids, babies, sick family (even extended), risks, moving houses, … – as it’s likely they are not trying to tick you off, and it’s more than likely that you’ll need the same before this is all over.

Ode to the five litres tank

You may remember that last year I wrote about a “plastics free” store, selling spices, oil, and even laundry detergent. I have no idea how they are faring with the current pandemic, but let’s just say that unless they turned into a conventional store, there’s no way that I would be interested in going and buy spices, nuts and pasta from huge containers that are handled by dozens of customers per day — and particularly by kids sticking their grubby fingers into the nuts’ boxes to steal a macadamia.

Even if the concept would have been workable before, I doubt that after this whole experience it’s going to thrive — while I care about the planet, I care about not dying more, and I assume the same is going to be true for the vast majority of the public (but not everyone, I’m sure). So what are the alternative options to buying without plastic containers? I can only think of the idea of buying in bulk.

Back at the start of the lockdown, one of the things that was getting harder to find in the local supermarkets was soap — and if you have read the blog post linked above, you know that I’ve been using refills. In particular at home we have a very nice glass, 1L liquid soap dispenser bottle that came with some decent lavender liquid soap we bought in TkMaxx over a year ago, and we’ve been filling it with different brands’ soap, that are usually available around the £4/L mark. We had a couple of litres stashed away, but eventually they started running low.

So looking around we found a 5L tank of hand wash, targeted as commercial users, but easy to get a hold of in the pandemic. It’s a bit more expensive than what we found before, but we liked it better, particularly given the fact that it has not ruined my hands, despite us washing our hands a lot more than before. And that had me thinking that most most likely the 5L tank can be reused, rather than recycled, much more easily. For instance, you can use it to collect waste oil when deep frying, and then bring it to the correct recycling point for that. Or in any case you can throw it with the recycling.

But it’s not just the plastic involved that makes a difference. Just think of how often you would need to get these delivered in half a litre increment. The 5L tank is due to last us just about five months, so you get around two deliveries a year, instead of about two a months (or once a month if you can just order the refills in pairs). And because we liked the quality of the soap, we ended up ordering the shampoo from the same brand, and fill a plastic bottle instead; at least for my hair it works well, and I’m picky — and it costs nearly half per liter than my usual ones.

There’s more than shampoo and soap that can be bought in 5L tanks. Body wash, fabric softener, vegetable oil, … and liter-for-liter they clearly need less plastics, if that’s the main measure we use for pollution, and they require fewer trips to shops and fewer deliveries. They are a bit awkward to use sometimes (thus why we have a 1L bottle we pour the vegetable we use for cooking), but the main disadvantage is that they take space, and while we’re lucky to have enough space for them in our flat, I don’t think I’d have been able to make the space for them in Dublin (didn’t help that the closet had a ton of stuff left over from the landlord and the previous tenants, including umbrellas, 5cm square framed mirrors, and stuff like that).

And I’m taking the 5L tanks as an example, but they are a metonymy for a number of other bought-in-bulk items, many of which are hard to find here in London. Even toilet paper, another staple of lockdown hoarding: Dublin and London got me used to order it in 16- or 9-roll bags, while in Italy I was used to buying 48/64 rolls at a time. It’s non-perishable, and if you do have the space to just get it and stuff it somewhere until you need it, why increasing the number of times you need to order it?

Funny story here: when I moved to London, and found out that my local Sainsbury’s didn’t have anything over 4 rolls bags, I decided that it would be easier to order 60 rolls from Amazon and have it delivered. The cost was meaningfully lower, and at the time I was not setting up for groceries’ delivery, and rather going to the stores myself to pick up just the stuff I needed for the days — bringing toilet paper on the bus is bulky and uncomfortable. Unfortunately i forgot to check where I asked Amazon to deliver it, and I ended up receiving nearly a cubic meter of toilet paper to my office, and had to find a way to bring it home, considering it took me an hour to go from King’s Cross to home, between Piccadilly and bus. Thankfully, two trips with my Filson duffle bag at a late hour were enough to bring it home. I love that duffle bag.

What I’m suggesting is that city living needs to start adapting to the idea that people need storage space. When looking at apartments, you can’t but wonder what’s the chicken and what’s the egg, between the lack of cupboard storage and the just-in-time supply used by most grocery stores in the big cities. Maybe in five years we will all live in apartments that have enough cupboard storage that you only need to buy non-perishables once a month, and the local stores will be providing fresh food and only urgent needs.

There’s also another clear problem with getting people to use bulk-volume non-perishables: beside Amazon, very few sellers carry those as options, at least in the UK. Yes, there’s Costco here just like in the USA, but that’s not common, and you do need to make sure you account for the £15/yr options. In Italy if you have a VAT ID you often end up shopping at Metro, because that’s an option that opens up to you…

Again, this is the type of thing that needs to be adapted for, after this whole pandemic happened. Reducing the frequency of deliveries by buying in bulk should be favourable for both grocery stores and consumers, given how the panic buying broke most delivery systems. So maybe next year Morrisons will have more 5L tanks of stuff available for delivery, not just the vegetable oil.

The bakery is just someone else’s oven

Most of the readers of this blog are likely aware of the phrase “The Cloud is someone else’s computer” — sometimes with a “just” added to make it even more judgemental. Well, it should surprise nobody (for those who know me) that I’m not a particular fan of either the phrase, or the sentiment within it.

I think that the current Covid-19 caused quarantine is actually providing a good alternative take on it, which is the title of the blog: “The bakery is just someone else’s oven.”

A lot of people during this pandemic-caused lockdown decided that it’s a good time to make bread, whether they tried it before or not. And many had to deal with similar struggles, from being unable to find ingredients, to making mistakes while reading a recipe that lead to terrible results, to following recipes that are just giving wrong instructions because they assume something that is not spelled out explicitly (my favourite being the recipes that assume you have a static oven — turns out that the oven we have at home only has “fan assisted” and “grill” modes.)

Are we all coming out of the current crisis and deciding that home baking is the only true solution, and that using a bakery is fundamentally immoral? I don’t think so. I’m sure that there will be some extremists thinking that, there always are, but for the most part, the reasonable people will go and accept that baking bread is not easy and while freshly baked bread can taste awesome, for most people, making time to do it every day would cut heavily into time that is needed for other things, that may or may not be more important (work, childcaring, wellness, …), so once the option of just buying good (or at least acceptable) bread from someone else becomes practical, lots of us will go back to buying it most of the time, and just occasionally baking.

I think this matches very well the way Cloud and self-hosted solutions relate to each other. You can set up your own infrastructure to host websites, mail servers, containers, Dockers, apps and whatever else. But most of the time this detracts from the time you would spend on something else, or you may need resources, that might not be linear to procure, or you may make mistakes configuring them, or you may just not know what the right tools you need are. While some (most) of these points apply to using Cloud solutions from various providers, they are also of a different level — the same way you may have problems following a recipe that includes bread as an ingredient, rather than being for bread.

So for instance, if you’re a small business, you may be running your own mail server. It’s very possible that you even already have a system administrator, or retain a “MSP” (Managed Service provider), which is effectively a sysadmin-for-hire. But would it make sense for you to do that? You would have to have a server (virtual or physical) to host it, you would have to manage the security, you would have to manage connectivity, spam tracking, logging, … it’s a lot of work! If you’re not in the business of selling email servers, it’s not likely that you’d ever get a good return on that resource investment. Or you could just pay FastMail to run it for you (see my post on why I chose them at the end).

Of course saying something like this will get people to comment on all kind of corner cases, or risks connected with it. So let’s be sure that I’m not suggesting that this is the only solution. There’s cases in which, even though it is not you primary business, running your own email server has significant advantages, including security. There are threat models and threat models. I think this is once again matching the comparison with bakeries — there are people who can’t buy bread, because the risk of whichever bread they buy to have been contaminated with something they are allergic to is not worth it.

If you think about the resource requirements, there’s another similar situation there: getting all the ingredients in normal situation is very easy, so why bother buying bread? Well, it turns out that the way supply works is not obvious, and definitely not linear. At least in the UK, there was an acknowledgement that “only around 4% of UK flour is sold through shops and supermarket”, and that the problem is not just the increase in demand, but also the significant change in usage pattern.

So while it would be “easy” and “cheap” to host your app on your own servers in normal times, there’s situations in which this is either not possible, or significantly inconvenient, or expensive. What happens when your website is flooded by requests, and your mail server sharing the same bandwidth is unreachable? What happens when a common network zero-day is being exploited, and you need to defend against it as quickly as reasonably possible?

Pricing is another aspect that appears to match fairly well between the two concepts: I heard so many complains about Cloud being so expensive that it will bankrupt you — and they are not entirely wrong. It’s very easy to think of your company as so big as to need huge complex solutions, that will cost a lot more than a perfectly fine solution, either on Cloud or self-hosted. But the price difference shouldn’t be accounted for solely by comparing the price paid for the hosting, versus the price paid for the Cloud products — you need to account for a lot more costs, related to management and resources.

Managing servers take time and energy, and it increases risks, which is why a lot of business managers tend to prefer outsourcing that to a cloud provider. It’s pretty much always the case, anyway, that some of the services are outsourced. For instance, even the people insisting on self-hosting solutions don’t usually go as far as “go and rent your own cabinet at a co-location facility!” which would still be a step short of running your own fiber optic straight from the Internet exchange, and… well you get my point. In the same spirit, the price of bread does not only include the price of the ingredients but also the time that is spent to prepare it, knead it, bake it, and so on. And similarly, even the people who I have heard arguing that baking every day is better than buying store bread don’t seem to suggest you should go and mill your own flour.

What about the recipes? Well, I’m sure you can find a lot of recipes in old cookbooks with ingredients that are nowadays understood as unhealthy — and not in the “healthy cooking” kind of way only, either. The same way as you can definitely find old tutorials that provide bad advice, while a professional baker would know how to do this correctly.

At the end of the day, I’m sure I’m not going to change the mind of anyone, but I just wanted to have something to point people at the next time they try the “no cloud” talk on me. And a catchy phrase to answer them by.

Interns in SRE and FLOSS

In addition to the usual disclaimer, that what I’m posting here is my opinions and my opinions only, not those of my employers, teammates, or anyone else, I want to start with an additional disclaimer: I’m neither an intern, a hiring manager, or a business owner. This means that I’m talking from my limited personal experience that might not match someone else’s. I have no definite answers, I just happen to have opinions.

Also, the important acknowledgement: this post comes from a short chat on Twitter with Micah. If you don’t know her, and you’re reading my blog, what are you doing? Go and watcher her videos!

You might remember a long time ago I wrote (complaining) of how people were viewing Google Summer of Code as a way to get cash rather than a way to find and nurture new contributors for the project. As hindsight is 2020 (or at least 2019 soon), I can definitely see how my complaint sounded not just negative, but outright insulting for many. I would probably be more mellow about it nowadays, but from the point of view of an organisation I stand from my original idea.

If anything I have solidified my idea further with the past five and a half years working for a big company with interns around me almost all the time. I even hosted two trainees for the Summer Trainee Engineering Program a few years ago, and I was excitedly impressed with their skill — which admittedly is something they shared with nearly all the interns I’ve ever interacted with.

I have not hosted interns since, but not because of bad experiences. It had more to do with me changing team much more often than the average Google engineer — not always by my request. That’s a topic for another day. Most of the teams I have been in, including now, had at least an intern working for them. For some teams, I’ve been involved in brainstorming to find ideas for interns to work on the next year.

Due to my “team migration”, and the fact that I insist on not moving to the USA, I often end up in those brainstorms with new intern hosts. And because of that I have over time noticed a few trends and patterns.

The one that luckily appears to be actively suppressed by managers and previous hosts is that of thinking of interns as the go-to option to work on tasks that we would define “grungy” — that’s a terrible experience for interns, and it shouldn’t be ever encouraged. Indeed, my first manager made it clear that if you come up with a grungy task to be worked on, what you want is a new hire, not an intern.

Why? There are multiple reasons for that. Start with the limited time an intern has, to complete a project: even if the grungy task is useful to learn how a certain system works, does an intern really need to get comfortable with it that way? For a new hire, instead, time is much less limited, so giving them a bit more boring tasks while they go through whatever other training they need to go through is fine.

But that’s only part of the reason. The much more important part is understanding where the value of an intern is for the organisation. And that is not in their output!

As I said at the start, I’m not a hiring manager and I’m not a business person, but I used to have my own company, and have been working in a big org for long enough that I can tell a few patterns here and there. So for a start, it becomes obvious that an intern’s output (as in the code they write, the services they implement, the designs they write) are not their strongest value proposition, from the organisation point of view: while usually interns are paid less than the full-time engineers, hosting an intern takes a lot of time away from the intern host, which means the cost of the intern is not just how much they get paid, but also a part of what the host get paid (it’s not by chance that Google Summer of Code reimburses the hosting project and not just the student).

Also, given interns need to be trained, and they will likely have less experience in the environment they would be working, it’s usually the case that letting a full-time engineer provide the same output would take significantly less time (and thus, less money).

So no, the output is not the value of an intern. Instead an internship is an opportunity both for the organisation and for the interns themselves. For the organisation, it’s almost like an extended interview: they get to gauge the interns’ abilities over a period of time, and not just with nearly-trick questions that can be learnt by heart — it includes a lot more than just their coding skills, but also their “culture fit” (I don’t like this concept), and their ability to work in a team — and I can tell you that myself, at the age of most of the interns I worked with, I would have been a terrible team player!

And let’s not forget that if the intern is hired afterwards, it’s a streamlined training schedule, since they already know their way around the company.

For the intern, it’s the experience of working in a team, and figuring out if it’s what they want to do. I know of one brilliant intern (who I still miss having around, because they were quite the friendly company to sit behind, as well as a skilled engineer) who decided that Dublin was not for them, after all.

This has another side effect for the hosting teams, that I think really needs to be considered. An internship is a teaching opportunity, so whatever project is provided to an intern should be meaningful to them. It should be realistic, it shouldn’t be just a toy idea. At the same time, there’s usually the intention to have an intern work on something of value for the team. This is great in the general sense, but it goes down to two further problems.

The first is that if you really need something, assigning it as a task to an intern is a big risk: they may not deliver, or underdeliver. If you need something, you should really assign it to an engineer; as I said it would also be cheaper.

The second is that the intern is usually still learning. Their code quality is likely to not be at the level you want your production code to be. And that’s okay. Any improvement in the code quality of the intern over their internship is of value for them, so helping them to improve is good… but it might not be the primary target.

Because of that, my usual statement during the brainstorms is “Do you have two weeks to put the finishing polish on your intern’s work, after they are gone?” — because if not, the code is unlikely to be made into production. There are plenty of things that need to be done after a project is “complete” to make it long-lasting, whether they are integration testing and releasing, or “dotting the is and crossing the ts” on the code.

And when you don’t do those things, you end up with “mostly done” code, that feels unowned (because the original author left by that point), and that can’t be easily integrated into production. I have deleted those kind of projects from codebases (not just at Google) too many times already.

So yes, please, if you have a chance, take interns. Mentor them, teach them, show them around on what their opportunities could be. Make sure that they find a connection with the people as well as the code. Make sure that they learn things like “Asking your colleagues when you’re not sure is okay”. But don’t expect that getting an intern to work on something means that they’ll finish off a polished product or service that can be used without a further investment of time. And the same applies to GSoC students.

Ads, spying, and my personal opinion

In the past year or so, I have seen multiple articles, even by authors who I thought would have more rational sense to them, over the impression that people get about being spied upon by technology and technology companies. I never got particularly bothered to talk about them, among other things because the company I work for (Google) is one that is often at the receiving end of those articles, and it would be disingenuous for me to “defend” it, even though I work in Site Realiability, which gives me much less insight in how tracking is done than, say, my friends who work in media at other companies.

But something happened a few weeks ago gave me an insight on one of the possible reasons why people think this, and I thought I would share my opinion on this. Before I start let me make clear that what I’m going to write about is something that is pieced together with public information only. As you’ll see soon, the commentary is not even involving my company’s products, and because of that I had access to no private information whatsoever.

As I said in other previous posts, I have had one huge change in my personal life over the past few months: I’m in a committed relationship. This means that there’s one other person beside me that spends time in the apartment, using the same WiFi. This is going to be an important consideration as we move on later.

Some weeks ago, my girlfriend commented on a recent tourism advertisement campaign by Lithuania (her country) on Facebook. A few hours later, I received that very advertisement on my stream. Was Facebook spying on us? Did they figure out that we have been talking a lot more together and thus thought that I should visit her country?

I didn’t overthink it too much because I know it can be an absolute coincidence.

Then a few weeks later, we were sitting on the sofa watching Hanayamata on Crunchyroll. I took a bathroom break between episodes (because Cruncyroll’s binge mode doesn’t work on Chromecast), and as I came back she showed me that Instagram started showing her Crunchyroll ads — “Why?!” We were using my phone to watch the anime, as I have the account. She’s not particularly into anime, this was almost a first as the material interested her. So why the ads?

I had to think a moment to give her an answer. I had to make a hypothesis because obviously I don’t have access to either Crunchyroll or Instagram ads tracking, but I think I’m likely to have hit close to the bullseye and when I realized what I was thinking of, I considered the implications with the previous Facebook ads, and the whole lot of articles about spying.

One more important aspect that I have not revealed yet, is that I requested my ISP to give me a static, public IPv4 address instead of the default CGNAT one. I fell for the wet dream, despite not really having used the feature since. It’s handy, don’t get me wrong, if I was to use it. But the truth is that I probably could have not done so and I wouldn’t have noticed a difference.

Except for the ads of course. Because here’s how I can imagine these two cases to have happened.

My girlfriend reads Lithuanian news from her phone, which is connected to my WiFi when she’s here. And we both use Facebook on the same network. It’s not terribly far-fetched to expect that some of the trackers on the Lithuanian news sites she visits are causing the apartment’s stable, static, public IP address to be added to a list of people possibly interested in the country.

Similarly, when we were watching Crunchyroll, we were doing so from the same IP address she was checking Instagram. Connect the two dots and now you have the reason why Instagram thought she’d be a good candidate for seeing an advert for Crunchyroll. Which honestly would make more sense if they intended to exclude those who do have an account, in which case I would not have them trying to convince me to… give them the money I already give them.

Why do I expect this to be IP tracking? Because it’s the only thing that makes sense. We haven’t used Facebook or Messenger to chat in months, so they can’t get signal from that. She does not have the Assistant turned on on her phone, and while I do, I’m reasonably sure that even if it was used for advertisement (and as far as I know, it isn’t), it would not be for Facebook and Instagram.

IP-based tracking is the oldest trick in the book. I would argue that it’s the first tracking that was done, and probably one of the least effective. But at the same time it’s mostly a passive tracking system, which means it’s much easier to accomplish under the current limits and regulations, including but not limited to GDPR.

This obviously has side effects that are even more annoying. If the advertisers start to target IP address indiscriminately, it would be impossible for me or my girlfriend to search for surprises for each other. Just to be on the safe side, I ordered flowers for our half-year anniversary from the office, in the off-chance that the site would put me on a targeting list for flower ads and she could guess about it.

This is probably a lot less effective for people who have not set up static IP addresses, since there should be a daily or so rotation of IP addresses that confuses the tracking enough. But I can definitely see how this can also go very wrong when a household dynamic are pathological, if the previous holder of the address managed to get the IP on targeted lists for unexpected announces.

I have to say that in these cases I do prefer when ads are at least correctly targeted. You can check your Ads preferences for Google and Facebook if you want to actually figure out if they know anything about you that you don’t want them to. I have yet to find out how to stop the dozens of “{Buzzword} {Category} Crowdfunding Videos” pages that keep spamming me on Facebook though.

Software systems and institutional xenophobia

I don’t usually write about politics, because there are people with more sophisticated opinions and knowledge out there, compared to me, playing at the easiest level, to quote John Scalzi, and rarely having to fear for my future (except for when it comes to health problems). But today I need to point out something that worries me a lot.

We live in a society that, for good or bad (and I think it’s mostly for good), is more and more tied to computer systems. This makes it very easy for computer experts of one kind or another (like me!) to find a job, particularly a good paying job. But at the same time it should give us responsibilities for what we do with our jobs.

I complained on Twitter how most of the credit card application forms here in the UK are effectively saying «F**k you, immigrant scum» by not allowing you to complete the application process if you have less than three years’ addresses in the UK. In the case of a form I tried today, even though the form allows you to specify an “Overseas address” as previous address, which allows you to select Ireland as a country, it still verifies the provided post code to UK standards, and refuses you to continue the process without it.

This is not the first such form. Indeed, I ended up getting an American Express credit card because they were the only financial institution that could be convinced to take me on as a customer, with just two months living in this country, and a full history of addresses for the previous five years and more. And even for them, it was a bit of an issue to find an online form that did indeed allow me to type that in.

Yet another of the credit card companies rejected my request because “[my] file is too thin” — despite being able to prove to them I’m currently employed full time with a very well paying company, and not expecting to change any time soon. This is nearly as bad as the NatWest employee that wanted my employer’s HR representative to tell them how long they expected me to live in the UK.

But it’s not just financial institutions, it’s just at any place where you provide information, and you may end up putting up limitations that, though obviously fine for your information might not be for someone else. Sign-up forms where putting a space in a name or surname field is an error. Data processing that expects all names to only have 7-bit ASCII encoding. Electoral registries where names are read either as Latin 1 or Latin 2.

All of these might be considered smaller data issues of nearsighted developers, but they also show how these can easily turn into real discrimination.

When systems that have no reason to discard your request on the basis of the previous address have a mistake that causes the postcode validation to trigger on the wrong format, you’re causing a disservice and possible harm to someone who might really just need a credit card to be able to travel safely.

When you force people to discard part of their name, you’re going to cause them disservice and harm when they will need a full history of what they did — I had that problem in Ireland, applying for a driving learner permit, not realising that the bills for Bord Gáis Energy wrote down my name wrong (using Elio as my surname).

The fact that my council appears to think that they need to use Latin-2 to encode names, suggests they may expect that their residents are all either English or Eastern European, which in turn leads to the idea of some level of segregation of them away from Italian, French or Irish, all of which depend on Latin-1 encodings instead.

The “funnies” in Ireland was a certain bank allowing you to sign up online with no problems… as long as you had a PPS (tax ID) issued before 2013 — after that year, a new format for the number was in use, and their website didn’t consider it valid. Of course, it’s effectively only immigrants who, in 2014, would be trying to open a bank account with such codes.

Could all of these situation be considered problems with incompetence? Possibly yes. Lots of people are incompetents, in our field. But it also means that there was no coverage for these not-so-corner cases in the validation. So it’s not just an incompetent programmer, it’s an incompetent programmer paired with an incompetent QA engineer. And an incompetent product manager. And an incompetent UX designer… that’s a lot of incompetence put together for a product.

Or the alternative is that there is a level of institutional xenophobia when it comes to software development. In the UK just as in Ireland, Italy and in the United States. The idea that the only information that are being tested are those that are explicitly known to the person doing the development is so minimalist as to be useless. You may as well not validate anything.

Not having anyone from the stakeholders to the developers and testers consider “Should a person from a different culture with different naming, addressing, or {whatever else} norms be able to use this?” (or worse, consider it and answering themselves “no”), is something I consider xenophobia¹.

I keep hearing calls to pledge ethics in the field of machine learning (“AI”) and data collection. But I have a feeling that those fields have much less impact on the “median” part of the population. Which is not to say you shouldn’t have ethical consideration in them at all. But rather than we should start with teaching ethics in everyday’s data processing too.

And if you’re looking for some harsh laugh after this mood-killing post, I recommend this article from The Register.

¹ Yes I’m explicitly not using the word “racism” here, because then people will focus on that, rather than the problem. A form does not look at the colour of your skin, but does look at whether you comply with its creators idea of what’s “right”.

Two words about my personal policy on GitHub

I was not planning on posting on the blog until next week, trying to stick on a weekly schedule, but today’s announcement of Microsoft acquiring GitHub is forcing my hand a bit.

So, Microsoft is acquiring GitHub, and a number of Open Source developers are losing their mind, in all possible ways. A significant proportion of comments on this that I have seen on my social media is sounding doomsday, as if this spells the end of GitHub, because Microsoft is going to ruin it all for them.

Myself, I think that if it spells the end of anything, is the end of the one-stop-shop to work on any project out there, not because of anything Microsoft did or is going to do, but because a number of developers are now leaving the platform in protest (protest of what? One company buying another?)

Most likely, it’ll be the fundamentalists that will drop their projects away to GitHub. And depending on what they decide to do with their projects, it might even not show on anybody’s radar. A lot of people are pushing for GitLab, which is both an open-core self-hosted platform, and a PaaS offering.

That is not bad. Self-hosted GitLab instances already exist for VideoLAN and GNOME. Big, strong communities are in my opinion in the perfect position to dedicate people to support core infrastructure to make open source software development easier. In particular because it’s easier for a community of dozens, if not hundreds of people, to find dedicated people to work on it. For one-person projects, that’s overhead, distracting, and destructive as well, as fragmenting into micro-instances will cause pain to fork projects — and at the same time, allowing any user who just registered to fork the code in any instance is prone to abuse and a recipe for disaster…

But this is all going to be a topic for another time. Let me try to go back to my personal opinions on the matter (to be perfectly clear that these are not the opinions of my employer and yadda yadda).

As of today, what we know is that Microsoft acquired GitHub, and they are putting Nat Friedman of Xamarin fame (the company that stood behind the Mono project after Novell) in charge of it. This choice makes me particularly optimistic about the future, because Nat’s a good guy and I have the utmost respect for him.

This means I have no intention to move any of my public repositories away from GitHub, except if doing so would bring a substantial advantage. For instance, if there was a strong community built around medical devices software, I would consider moving glucometerutils. But this is not the case right now.

And because I still root most of my projects around my own domain, if I did move that, the canonical URL would still be valid. This is a scheme I devised after getting tired of fixing up where unieject ended up with.

Microsoft has not done anything wrong with GitHub yet. I will give them the benefit of the doubt, and not rush out of the door. It would and will be different if they were to change their policies.

Rob’s point is valid, and it would be a disgrace if various governments would push Microsoft to a corner requiring it to purge content that the smaller, independent GitHub would have left alone. But unless that happens, we’re debating hypothetical at the same level of “If I was elected supreme leader of Italy”.

So, as of today, 2018-06-04, I have no intention of moving any of my repositories to other services. I’ll also use a link to this blog with no accompanying comment to anyone who will suggest I should do so without any benefit for my projects.

The importance of teams, and teamwork

Today, on Twitter, I have received a reply with a phrase that, in its own sake and without connecting back with the original topic of the thread, I found significant of the dread I feel with working as a developer, particularly in many opensource communities nowadays.

Most things don’t work the way I think they work. That’s why I’m a programmer, so I can make them work the way I think they should work.

I’m not going to link back to the tweet, or name the author of the phrase. This is not about them in particular, and more about the feeling expressed in this phrase, which I would have agreed with many years ago, but now feels so much off key.

What I feel now is that programmers don’t make things work the way they think they should. And this is not intended as a nod to the various jokes about how bad programming actually is, given APIs and constraints. This is about something that becomes clear when you spend your time trying to change the world, or make a living alone (by running your own company): everybody needs help, in the form of a team.

A lone programmer may be able to write a whole operating system (cough Emacs), but that does not make it a success in and by itself. If you plan on changing the world, and possibly changing it for the better, you need a team that includes not only programmers, but experts in quite a lot of different things.

Whether it is a Free Software project, or a commercial product, if you want to have users, you need to know what they want — and a programmer is not always the most suitable person to go through user stories. Hands up all of us who have, at one point or another, facepalmed at an acquaintance taking a screenshot of a web page to paste it into Word, and tried to teach them how to print the page to PDF. While changing workflows so that they make sense may sound the easiest solution to most tech people, that’s not what people who are trying to just do their job care about. Particularly not if you’re trying to sell them (literally or figuratively) a new product.

And similarly to what users want to do, you need to know what the users need to do. While effectively all of Free Software comes with no warranty attached, even for it (and most definitely for commercial products), it’s important to consider the legal framework the software has to be used on. Except for the more anarchists of the developers out there, I don’t think anyone would feel particularly interested in breaching laws for the sake of breaching them, for instance by providing a ledger product that allows “black book accounting” as an encrypted parallel file. Or, to reprise my recent example, to provide a software solution that does not comply with GDPR.

This is not just about pure software products. You may remember, from last year, the teardown of Juicero. In this case the problems appeared to step by the lack of control over the BOM. While electronics is by far not my speciality, I have heard more expert friends and colleagues cringe at seeing the spec of projects that tried to actually become mainstream, with a BOM easily twice as expensive as the minimum.

Aside here, before someone starts shouting about that. Minimising the BOM for an electronic project may not always be the main target. If it’s a DIY project, making it easier to assemble could be an objective, so choosing more bulky, more expensive parts might be warranted. Similarly if it’s being done for prototyping, using more expensive but widely available components is generally a win too. I have worked on devices that used multi-GB SSDs for a firmware less than 64MB — but asking for on-board flash for the firmware would have costed more than the extremely overprovisioned SSDs.

And in my opinion, if you want to have your own company, and are in for the long run (i.e. not with startup mentality of getting VC capital and get acquired before even shipping), you definitely need someone to follow up the business plan and the accounting.

So no, I don’t think that any one programmer, or a group of sole programmers, can change the world. There’s a lot more than writing code, to build software. And a lot more than building software, to change society.

Consider this the reason why I will plonk-file any recruitment email that is looking for “rockstars” or “ninjas”. Not that I’m looking for a new gig as I type this, but I would at least give thought if someone was looking for a software mechanic (h/t @sysadmin1138).