To Effectively Address Technical Debt, Stop Calling It Technical Debt
Technical debt or “tech debt” is defined by Wikipedia as “the implied cost of future reworking required when choosing an easy but limited solution instead of a better approach that could take more time”. It’s an important concept. Technical debt is something that should be paid attention to, paid off early, and prevented from occurring.
One of the most common questions software engineering leaders struggle with is “what percentage of my team’s time should be focused on paying down technical debt?” The problem is, there is no right answer to that question. In fact, even asking the question in that way causes more harm than good.
The engineering leader may decide that a number like 20% sounds about right. They think, this way we are addressing technical debt, but not so much that it’s getting too much in the way of new features.
What happens when we look at technical debt this way
Technical debt becomes a catch-all for any engineering driven project that has an unidentified or unclear customer facing output.
Product Managers and CEOs start rolling their eyes when they hear the word “tech debt”. They wonder: What are these mystical important projects that our CTO keeps insisting we spend 20% of engineering’s time on? The CTO keeps saying these “tech debt” projects are so important and that the benefits are so hard to convery, so we negotiate or acquiesce to a percentage based on arbitrary anchors and what we’ve heard other companies do. We don’t even want to talk about it anymore because we’re so sick of arguing about it…just give them their 20%.
At the same time, some engineers complain that we’re not spending enough time on technical debt and that the percentage should be raised to some other arbitrary number like 30%. The trouble is, there is always more technical debt that can ever be repaid since codebases are so complex and are constantly evolving. When the only standard set regarding technical debt is that we should be spending 20% of our time on it, engineers advocate for projects they personally want to work on instead of being focused on what would be most impactful for the company.
If that’s the wrong way to look at technical debt, what’s the right way?
Treat technical debt like every other project
To most effectively address technical debt, it’s necessary to treat technical debt like every other project. Every new customer facing feature or enhancement that gets built has (or should have) a goal or some intended positive effect for the company, ideally as measured in a metric. Technical debt shouldn’t be any different. In fact, the term technical debt is so overloaded and has so much baggage and misuse, it’s often best to steer away from using the term at all.
Instead of gathering up all the possible technical debt projects that engineers happen to propose, you can require that the benefit of every project be described and quantified as clearly as possible. Better yet, you can set a bold goal (e.g. reduce the time it takes a new hire to provision their development environment from 4 hours to 15 minutes) and ask your team to think critically and creatively about what can make the most progress towards that goal.
Focus on the benefits
A project that would commonly get bucked under the “technical debt” umbrella can have one or more end benefits. Some of the most common are engineering velocity, quality, app/site speed, and security.
Re-architecting a confusing core part of the codebase can improve the ability for engineers to onboard to the company more quickly and for the engineering engineering team to release features faster. Making the codebase more modular can make it less likely for engineers to introduce bugs or security issues. Reducing a cumbersome set of redirects can make the page the user wants to access load faster.
Engineering velocity is notoriously hard to measure in one number, which could be one reason why technical debt projects are so frequently not quantified. However, it’s crucial to always have the benefit articulated (ideally in a numerical way). Otherwise there’s no way to determine whether it’s worth investing in one project or another one. It’s worth noting that it’s also notoriously hard to measure user experience in one number, but we shouldn’t use that as an excuse not to quantify and measure benefits of projects intended to improve user experience.
So instead of just referring to a project as “refactoring our test case infrastructure”, you can add the detail that “this project is estimated to take 30 engineering person-weeks and is expected to reduce the code merge-to-deploy time from 20 minutes to 10 minutes”.
But how can I compare those benefits to business metrics like revenue? Isn’t it like comparing apples to oranges?
Companies of every size deal with comparing apples to oranges on their product roadmap. Typically companies have multiple metrics they are trying to improve. For example: revenue, number of users, and net promoter score (NPS). Even if a company is totally aligned around one long-term metric (for example, profit) they still must trade off short term profit vs. long term profit or trade-off a smaller amount of certain profit vs. an unknown amount of potentially larger profit. Companies frequently trade off how much engineering time to invest in starting a new business line (which could eventually become bigger than the core business or it could fail) vs. making immediate improvements to the core product that will increase profit today. Evaluating trade-offs is as much intuition as it is science.
For a good trade-off conversation, each project or initiative needs to convey an expected benefit and not just occupy some arbitrary percentage of the roadmap. It wouldn’t be acceptable to say that every quarter that 25% of the projects must be ones meant to increase revenue, 25% must be meant to improve customer experience, and so on. Technical debt projects should be no different.
Contrasting the approaches
Here are two comparisons of the approaches to illustrate the difference. One example is to the Engineering, Product, and Design (EPD) team and the other is to the Board of Directors.
To the EPD team:
Reducing technical debt is important. Every quarter we spend 25% of our time on technical debt. Please let your manager know what technical debt projects you think we should take on.
vs.
We have a goal this quarter to decrease our high severity bugs per quarter from an average of 16 to 5 and medium severity bugs from 47 to 20. This will greatly improve the experience our customers have with our product. We’d love you all to think of ways to build systems, re-architect code, or create processes that will help achieve this quality goal. If you have an idea, please use this brief template to describe the project and how this will help reduce bugs.
To the Board of Directors:
As usual, we spend 20% of our engineering time on technical debt projects to help keep our codebase clean. This percentage is in line with industry benchmarks (i.e. what we used at my last company and what I’ve seen in blogs).
vs.
This quarter, a big focus for our engineering team is to increase the speed at which we can release new customer facing features. We’ll be taking on several projects this quarter totaling 130 engineering person weeks (10% of total engineering time), which we estimate will increase the speed at which we develop and release features in the following quarter by 15-20%. The details of how we came up with this estimate are in this one-pager that we can dive into if you’d like.
You may notice that the second approach for each of the cases doesn’t mention the word “technical debt”.
Tracking time spent on technical debt
This article explains the benefits of not treating technical debt projects as any different from any other project. It can still be beneficial to understand what percentage of engineering time in any given quarter or sprint is focused on what types of projects. This allows you to monitor each sprint or quarter and notice if anything seems unbalanced. The key is to not set goals around how much percentage of time is spent on each type of project, but to use the buckets as a way to alert you that you might be over-investing or under-investing in a bucket.
You could create a “tech debt” bucket for this, but I challenge you to have more meaningful and descriptive bucket names for engineering-driven projects. The buckets you choose should vary somewhat by company. For example, you may have buckets of: new features on the core product, new features on new business lines, enhancing old features, bug fixes, engineering velocity, and security. If you find that in the last quarter you spent only 5% of time on engineering velocity projects it may be a sign that you should consider investing more next quarter, but only if you can find projects that would generate enough value to the company.
Case studies on measurement
Sometimes you have to get creative to set goals and/or make them measurable.
When I led Airbnb’s engineering infrastructure organization, we were experiencing issues where a large percentage of everyone’s time was going towards operational toil which was getting in the way of us building things intended to reduce downtime, increase productivity of all engineers at Airbnb, and make the company more data informed. The organization had 150 engineers on subteams ranging from Site Reliability Engineering to Machine Learning Infrastructure to Core Storage to Data Tools and there was no one metric that measured operational toil across all the teams. So for one quarter, I asked every team to make one of their goals to reduce operational toil by 50% by a metric or set of metrics that each team would define. This gave them the permission and space to make big changes in their own efficiency in a measurable way when there was no one metric that could be handed down to them.
As I alluded to earlier, measuring developer velocity is really hard to measure in one number. So at Airbnb, we created a Developer Experience scorecard (the idea borrowed from our Security team’s scorecard) which listed key attributes about the developer experience and assigned a letter grade to values of each attribute (merge to deploy time, test run time, etc.). Then we did a weighted sum of all the attributes to give an overall grade and set goals on upping our grade each quarter. We all agreed on what a B-,B,B+,A-, etc. would look like across all attributes of the developer experience. I must note that this sort of weighted sum letter grade goes against everything advised in the development of OKRs. Yes, it’s important to have your key results be easily measurable and understandable. But when you can’t find one that works, this letter grade system can sometimes be the next best thing.
Summary
Technical debt has become a bloated and ill-defined term in practice, often resulting in nearly any engineering-driven project being classified as a “tech debt” project. Instead of setting percentage targets on how much time engineering will spend paying down technical debt, treat technical debt projects like any other project that is meant to drive a goal. Throw away the term “technical debt” altogether, focus on setting goals around developer velocity and related areas, and quantify the benefits of every engineering-driven project so it can be properly traded off against other projects.
If you'd like to get 1-2 articles like this per month directly to your inbox, you can subscribe on Substack or LinkedIn.
P.S. I help founders and technical leaders scale their companies and teams through hypergrowth. I also coach successful individuals in their late 30’s and early 40’s who have reached a plateau and want to feel more alive again in work and life. If one of those sounds like you, please feel free to get in touch at kevin@kevinricecoaching.com.