Financing, Credit, and Technical Debt

Disclaimer: this post does not contain financial advice, but rather some personal perspective that fits my and my own financial situation in particular. Your mileage may vary, please don’t follow my footsteps unless you’re sure of what you’re doing.

Households: Credit Access Is Freedom

As I wrote in a recent post, I grew up in a family that fell in that often forgotten place of having enough money to get at the end of the month, but also not having enough disposable cash or savings to spend easily. It is the reason why, now that I have at least enough of a safety net, I don’t think I’ll volunteer to leave big tech, but it is also the reason why I found myself learning early on a number of lessons on buying things by instalment, and the risks of signing up for easy credit.

It’s from that particular point of view that I’ll start describing finance and credit, although there’s a lot more going on than that, and indeed I expect that students of economics would be facepalming at my simplifications. It is only one of the reasons why I’m telling you that this is not financial advice.

Financing options, growing up, have mostly been in the form of instalment plans that allowed you to buy something fairly expensive, despite not having the cash at hand to spend (or not being willing to part with said cash right away.) These made a number of purchases possible, including at least two (I forgot if even three) of my computers, which all have been bought on a 12 or 24 months plans. Together with important purchases, though, financing also made frivolous purchases a lot easier, including an uncountable number of encyclopaedias, and a Lord of The Rings inspired solid oak chess table, on which I never played chess.

It is this latter, the frivolous purchases, that tend to be more memorable to those criticizing easy credit and financing options — particularly those who got burned, or whose relatives got burned, by a mountain of debt from which is difficult to dig out of. Well, either that or if they are from the USA, it’s likely that they are thinking of student debt and that’s a whole other kettle of fish I have no experience with and honestly no personal interest to get deep into.

But at the same time, financing is probably the best tool to partly work around the Sam Vines “Boots” theory of socioeconomic unfairness — in short, that it is expensive to be poor. It allows families with not enough savings to replace the fridge that just decided to not be working one morning with a new, fairly expensive fridge that would last close to twenty years (true story!)

Financing and household appliances go hand in hand for families like the one I grew up in, because as I said we rarely had the savings (at least after my grandfather died), so it would have been difficult to get “high end” appliances (with high energy efficiency, high reliability, and so on) without the ability to spread the cost over a year or two. This became a lot clearer later on, after my parents separated and I couldn’t get any decent financing options as a freelancer (my income was too unpredictable month by month), and for a while we could only replace appliances with whatever we could have afforded at the moment — that meant replacing the previous dishwasher that lasted over ten years with a much cheaper one (think £200 vs £700), which required four repairs in the span of five years, only one in warranty, and ended up costing more than the £700 the more expensive one would have costed.

And, before an argument start about why would anyone need a dishwasher if they can’t afford it — dishwashers don’t only save time, they can also save money as they consume less water, reducing the amount of water that needs to be heated. Dishwashers are very much the embodiment of Vimes’s theory: if you can’t afford one, you’re likely spending more money in the long run washing dishes by hand, and also give you less time for other tasks, possibly forcing you to make a more expensive choice (e.g. train vs bus vs car.)

Easy instalment financing appears to have gone out of fashion in favour of Buy-Now-Pay-Later, which Patrick describes well on his newsletter, and thus I won’t be describing again (yet), but it does seem like some of the stigma around facilitating frivolous purchases is still there. Personally, I have a feeling that a lot of the criticism for these offers end up coming from people who never needed these options before — the same kind of uninformed complains I have heard levied against proprietary software users with specific needs, from Free Software Fundamentalists that never experienced those needs.

Businesses: Credit Access Is Lifeline

Up to here I have been talking about credit in a household settings, which I have experienced first hand. A similar, but not identical, situation is to be found in businesses, small and big as they come. I have only passing experience with this, so take this part with a pinch more salt.

If for a household, lack of access to credit means a greater expenditure of money and time, for a business, lack of access to credit might be the difference between existing and not: if your business cannot afford the equipment, materials, or service it needs to perform its duty, it is dead in the water. While cash (“capital”) is, if not a requirement, at least a significant advantage for starting a business, by experience you can’t rely on that cash alone to both pay for outflows (wages, rent, suppliers) and to acquire the necessary equipment — this parallels Patrick’s BNPL description of not being sensible for perishables (materials), but being more useful for more medium-term supplies (equipment).

This is, by the way, why running a business is a lot more complex than doing something of your own as a hobby: equipment you buy to run a business has a value that is subject to depreciation, and you do not (usually) recoup the cost of the equipment on a single customer or order. Having a significant upfront cost taken out from your capital, and having to wait for a long period to recoup it, means you have less cash available to pay employees and suppliers, and is thus frown upon, unless you have a lot of cash.

This is what I definitely have (negative) experience with. When I was running my consultancy in Italy, I had no capital to talk about, which is why I was a sole trader, rather than having set up a limited liability company (LLC), that in Italy required €30k capital injection at opening at the time — indeed, one of these days I should write about what I would totally do differently if I am ever desperate enough to go back to consulting. Not having an LLC didn’t just mean giving personal liability for any debt (including taxes), but also severely limited credit availability, including options such as equipment lease.

Equipment lease, for those who might not be aware of the concept (and only them, as those aware of the content will probably facepalm at the amount of shortcuts I’m taking explaining this, and the details I’m missing), is effectively financing-without-financing: instead of purchasing the equipment on credit and repaying it by monthly instalments, you receive equipment that is not “yours”, and pay the instalments the same way. The primary difference in how these two play is accounting, and bankruptcy: since the equipment is not owned by the business itself, but rather by the leasing company, it does not get sold at its effective valuation — while at the same time, there is no huge debt towards the financing of the purchase.

With this considered, for a business to have a certain amount of debt is not just considered normal, but I would say it’s even healthy. A business that could have taken a line of credit but passed on it is suspicious, a business that passed on the idea of leasing equipment in favour of purchasing is passing on the opportunity of spending that money hiring more people to expand the business. I’m simplifying here, but the essence of it is that running a business with zero debts is not a target most business owners aim for.

Software: Credit Access Is Controversial

Up to now, I’ve been talking about actual money. Before I start blabbering, I want to point out that this is not my idea – as most people before me, I’m writing on the back of those who came before me – and in this particular case, I need to credit my friend Caskey L. Dickson, who introduced me to the idea of reconsidering the parallel of the words “technical debt” during his LISA15 talk:

Now, while Caskey’s objective is to stop calling the kind of problems we find hiding in the shadows of a codebase or a design “technical debt” in favour of a closer corresponding economics term, I’m going to dissent, and instead explain why I think that technical debt is (often, but not always) the right term, and why senior engineers need to learn to manage said debt, rather than avoiding it at all costs (pun intended.)

So, what is technical debt? To the dismay of language prescriptivists, this is a term that is very subjective of people. In my field it usually relates to decisions that have been made in the past that are now causing you trouble, with no particular distinction beside that. I’ve seen it used to describe half-completed migrations, “temporary” hacks that persisted for over a decade, and even choice of languages, protocols or technology stacks.

For my argument to work, I need to define this a bit better, restricting it just enough that my argument holds true, and can be used by others to build useful arguments during, e.g. the design phase of a project. This, similarly to the previous jargon post, means that there’s space for disagreement and possible confusion for those using the term in a different way — but such is the risk when working with non-codified language, just like the lack of a schema can cause confusion when using a common format as well.

The definition of technical debt I’m going to use is is «the result of making a choice that eases work as it happens, but causes more work to be needed at a later date» — this additional work being the “technical (passive) interest” to make the similitude with “actual money debt” even closer.

I personally like using this “technical interest” concept because just like in the actual money world, it provides a way to distinguish and prioritize debt: a decision that requires very little work later on, but is just “not quite right” is like a 0% interest loan: as long as you do the work in time, it costs you nothing, but if you miss a payment (or forget to take the necessary action along the way), is still going to be expensive — while an expensive mortgage could cost you a pretty penny or even totally bankrupt you, if you keep delaying its repayment.

I can paint this as a specific example of something that most programmers and software engineers will consider bad, and hopefully technical debt: resource leakage. While resource management is a complex problem for which entire books have been written, it is not uncommon that very small resource leakage (memory, disk file usage, file descriptors, …) is not easily solvable, and becomes a “simple” TODO item.

Fixing resource leakage is tedious, and its importance is relative: if the leakage is correlated to some kind of event (such as requests) it is definitely a huge incident about to happen, but if it is correlated to a timer, it might just be acceptable: a little waste of resource every hour “just” puts a time limit to how often your service needs to be restarted — which might be perfectly fine if you have continuous deployment set up, and the service gets updated daily, but will indeed be a problem if your organization introduces code freeze calendars. The technical passive interest of these situations are all different, and it’s up to the more senior engineers to decide whether a particular debt is a technical bankruptcy risk or not.

In this particular case, the problem can also be considered a rake, since it affects the ability to other contributors to a project to do the right thing, and it is unlikely to be properly marked around — if the service changes owners, who are not aware of this restart requirement, it is possible that code freezes will be introduced without considering it. I think I would personally go and collect the rake the first time it caused an outage, but at the same time, you can argue that a stateless service, properly provisioned would not be affected by a problem like this, which is why I think it works well as an example of technical debt accounting and management.

Technical Mortgages and Leveraged Work

What I described just now sounds more like a technical credit card than a technical mortgage, so let me add another example, inspired by the situation I found myself in at a past customer. Say you’re building a new application or service from scratch, and your options are to invest heavily in a team that can pick up “the right language” to build it from the ground up, or build something in a badly-suitable language with the engineers you already have on board. The choice of language is not just a matter of preference or trend, as choosing one language over another can restrict which other architectural decisions you can make.

For the particular case of the client I was working with, they chose the same proprietary language that they used for one of the previous services — a language that predated any of the web frameworks that were in common use when the new service was being designed, that was no longer supported by the original inventors and maintainers, and with a very limited pool of available developers, but a language for which the staff of the company at the time still had experience. This choice of language, even just for one part of the service, mandated a particular database (because no drivers for other database were available in that language), which in turn restricted the choice of languages to rewrite components in, as that database only provided a JDBC driver in addition to the language’s own driver.

Choosing to use an old, proprietary, unsupported language is the perfect example of technical debt: it saved a significant amount of time in the early stage development phase, because the people who wrote the first alpha of the service didn’t need to get up to speed with new languages, frameworks, and design choices — but it also had a significant amount of “technical interest” not just in the forced design choices, but also to keep the application and database servers working on modern Linux installations (did I say the whole software set was proprietary and unmaintained?)

How do you get out of a similar situation? Well, I don’t know for sure — I had an idea back then, but I wasn’t allowed to implement it, but I found vindication in Will Larson’s An Elegant Puzzle, that not only describes similar project, but provides me with terminology to refer to them: high leverage work. This is again an economics term, which to oversimplify means you are investing a certain amount to hope for a gain that is multiple times more than the money you put in.

What I thought we should have done at that point, was take a “consolidating loan” — since the main cost was in supporting the web application written in the proprietary and abandoned language, while the database wasn’t (at that point) costing us much to maintain, I suggested we could have hired some three/four junior developers to re-implement the web application in Ruby on Rails – this was 2012, they were going for a dime a dozen – and run it on JRuby to allow it to reuse the database without having to make further changes. We could have replaced the database with just about anything else afterwards, since once you have both components (web and non-web applications) using JDBC, nothing stops you from lifting the whole schema and data into PostgreSQL or something.

This wouldn’t have been the state of the art — in 2012 Cloud Computing was already showing itself as a much safer bet, and this type of design was not really going to scale to that point, but the focus was not on building the most elegant solution, but a solution that would be workable enough to provide value to enough customers to pay for the engineers working on it, the bills, and the required equipment to expand to the next level of product. In this aspect, all the decisions were more akin to commercial debt than personal.

Personal Rules Of Thumb

As often when it comes to personal recommendations, your mileage will vary. Personally, while I don’t like introducing technical debt, I find that sometimes it is the right choice, which is why I think managing technical debt is a better task than avoiding technical debt. And the way I decide whether it is okay to take the technical loan or not usually depends on a number of factors that are hard to just list and choose from.

For once, nothing is more permanent than a temporary fix is pretty much a true reminder. You shouldn’t just introduce a temporary fix, and leave it be without plans and follow ups, in my experience. This does not mean that you should aggressively address the temporary fixes all the time (it might not make business sense), but in the same way I “notch a rake”, I try to keep track of these temporary solutions. For instance, you can track the fix with a bug or task in an issue tracker, and point out “This condition needs to be satisfied for this fix not to be needed” — as long as your organization does not suffer from being trigger-happy on bug bankruptcies, this can be enough of a reminder when the current list of open bugs is triaged.

The worst “temporary fix” I ever introduced was in a piece of Python automation, where I discovered an annoyingly undocumented corner case: a dictionary was used to store object by their name, but nothing was preventing to add two different objects with the same name — due to the nature of the framework, the first of the objects that was added could still be reached and executed… but it wouldn’t be checkpointed, making it ephemeral to restarts. This wasn’t causing a lot of pain, as it required a certain set of conditions to be true at the same time (code written to trigger this corner case, automation held by those objects not being idempotent, and the automation task restarting at a specific point in time), but it was the kind of rake that would take a lot of time to debug if it did trigger, given the difficulty to reproduce it.

I couldn’t at that point just make adding a second object with the same name an error — turned out a number of teams wrote code that unwittingly exercised that code path, and not all of them were trivially fixed; thankfully all of those that were susceptible to the actual corner case were fixable. But just leaving it in place would have meant increasing the interest as it wouldn’t prevent new code to be written with the mistake present. The solution was disgusting from point of view of a software engineer, but it served its purpose: I changed the code so that if a second object was added with a name that already existed in the dictionary, it would take a traceback, and look if the path to the main entrypoint of the binary included one of a set of strings (the teams that were unable to trivially fix their code), and only if not, raise an exception.

This solution stank, but meant two things: the first that no new code could be written using the wrong interface, and the second that the teams that didn’t fix their code yet would be present in a list of “naughty” teams. The stop-gap solution was technical debt under basically any definition, but it turned out to be repayable within a fixed timeframe, and less than a year later it went away. The last team team needed a bit of convincing though, because…

No need to fix this, the system is deprecated and going away is the flimsiest of excuses, and is often used to reject the idea of introducing “technical debt” in the form of actually removing pain or mistakes from a system that is officially on its way out. In the case above, the automation system was officially deprecated in favour of a newer, more designed and less congealed one, but the old system was still in use. Too often I heard fixes to that system described as technical debt because it’s effort that is going to be thrown away at some point in the future.

And this is the other thing that in my opinion you need to face when deciding whether to actually allow technical debt or not. If a system is indeed slated to be decommissioned within a clear timeframe, fixing it “properly” may be a waste of time, indeed. In such a situation, accepting a solution that papers over a problem might be appropriate. Yes, it’s technical debt, but for an entity that is about to be spun off in its own badco, which you may never need to repay! Hopefully. Introducing technical debt to cover a problem in a system that is officially deprecated, but doesn’t have a timeline to be decommissioned is risky — sometimes even more so than leaving the old system broken, since a broken old system would at least provide an incentive to switch to a newer, non-broken system.

Let’s go back to the resource leakage example: if you know that your resource leakage is time-based, and you know that the only way for it to trigger an outage is if no rollout is completed for more than a week, you know that some type of fix need to be applied before a two-weeks freeze such as it is common for the end of year holidays. If the system is planned to be decommissioned by September, and it is March, it might be perfectly fine to just not do anything! If the system is just “deprecated” but won’t go away until everyone of its internal customers have migrated, now you need to provide some solution — hiding the problem by forcing the system to restart every 24 hours, for a system that is expected to stay in use, is often sneered at, but it could be the technical loan necessary to avoid spending time on a deprecated system, while working on making it easier for the customers to migrate.

So to conclude, I think the only recommendation I can give to folks is to not feel too tied down to the idea of avoiding technical debt all the time — while it takes quite a bit of time to build the understanding of a system that allows choosing the right type of debt to take on, it’s useful to at least keep an open mind that debt is not always a toxic asset, particularly not when it allows you to “hit the market” — whether it is a public launch of a feature that is held up with duct tape, or completing the migration from an old system to a newer, more maintainable one.