Pragmatic Ways for Your IT Team to Deal with Technical Debt
ByOn February 21, 2013 At 3:27 pm
Responses : Comments are off for this post
Welcome to the OnTechnicalDebt Expert Interview Series.
This interview will focus on the ground-level steps that should be taken in order to help IT teams deal with their Technical Debt in a pragmatic and efficient way, while working in a fast-paced Agile environment.
Today I have the pleasure of being with Johanna Rothman – she speaks and writes a lot about managing high-technology product development. She has assisted many managers, teams, IT organizations to become more effective by applying some pragmatic approaches to the issues of project management. She has written several posts on Technical Debt and on how IT teams can effectively and successfully release products faster. So we’re really excited to have her here today.
We are also here with Carolyn Seaman, who will be leading the interview – she is an associate professor at UMBC where she is the leader for Technical Debt research studies (http://www.technicaldebt.umbc.edu/). She is also a Research Fellow at the Fraunhofer Center in Maryland.
1. What metrics can be watched over to see if we have Technical Debt that needs to be dealt with?
Carolyn Seaman: Obviously when IT organizations come to face Technical Debt, there are different kinds of challenges that they face depending on what they want to do with it. And people tend to understand the concept of Technical Debt – that’s why it has become so popular (it is easy to talk about and people understand it at an abstract level). But they get into trouble when they actually try to apply some sort of techniques, procedures or mechanisms to deal with it. So that is what we are going to focus on today: some ground level things that organizations can do to help them deal with Technical Debt. And the first thing that I want to focus on is visualizing Technical Debt in a way that is meaningful to a software team. Some IT teams really don’t even realize that they are building up debt in their systems or don’t understand what types of debts they have or where it is.
Johanna, my first question for you is: Do you know any specific signs or indicators that a project manager in an Agile environment can look out for that will help them realize when they have Technical Debt in their systems that need to be dealt with and where they need to look for it?
Johanna Rothman: Yes, and this is a great thing. This is why you want to monitor your velocity. Every team’s velocity is personal. Velocity is your way of saying: What did we do in the last iteration? What are we doing in the next iteration? And you use that to say: Are we pretty much on the same track? And If you are using Kanban, then that’s fine. You look at your cumulative flow and look at if you are roughly spending the same amount of time as before on your throughput. Because if you have a lot of Technical Debt and you’re growing your Technical Debt, then your velocity is going to go down and your throughput is going to go down. If you don’t look at it and you do not look at it over time (trends are everything!) you will lose the big picture. And while I want to say: What is our velocity so we can do a little bit of planning for the next iteration? I also want to look at velocity to say: Are we losing ground from iteration to iteration? Not to punish ourselves… but to see if we are growing Technical Debt, because this will determine if we are taking too many shortcuts.
We have this problem in our industry, product owners with what I call ‘feature-itis’: Give me features! Give me features! (You’ve probably seen that post on my blog). When velocity goes down over time and it continues to go down… and we have more and more trouble getting the features out… now we have to take action! If you have proof in the form of velocity over time (i.e. trend lines), and you have proof that your throughput if you’re using Kanban flow, then here you have data. And data trumps guts every single time.
2. How big of a role can Static Analysis Tools play?
Carolyn Seaman: Some people have started talking a bit about automated static analysis that looks at the source code itself to find different types of anomaly-looking problems. And people have started playing with that in terms of finding particular kinds of Technical Debt. Do you have any experience with that or any thoughts on what role automated static analysis tools can play when trying to manage Technical Debt?
Johanna Rothman: I think it’s useful to do this. But I wonder… Are you using it in code that you’re changing? Or just using it to look at all of the code? Especially in Agile, I don’t want to look at all of the code; I only want to look at the code I am going to change now… I don’t want to look at code that I don’t want to change.
I’m subscribed to YNGTNI – You’re Not Going To Need It – even though it is useful to understand where your code is structured and how it is structured. I am a big fan of saying: Do we have a mess? And do we want to go in and clean it up? And when do we want to go in and clean it up? In my opinion, developers have to go in and look at every single defect. I don’t care if it’s a typo or a huge mess. Developers have to go in and clean it up as they encounter it. If you look at it and it irritates you… then you should go in and fix it. That’s refactoring as you go.
How much analysis do we really have to do as we proceed? I have very mixed feelings about this. We don’t tend to have to fix the entire system in an iteration. So I think it is very useful if the output of one iteration is to say: Let’s analyze what is going on here so we have an idea of what we are going to do next. But I wonder sometimes about these big plans… Because every time we spend a lot of time re-designing or re-architecting, which is what a static analysis tool is going to allow us to do if we’re looking at it for the entire system, that’s time we are not going to put up features. So can we do it on a really small piece of code? That would be great! But every time I do this with people, they don’t want to look at a really small piece of the code; they want to do it on the entire system, and then they want to look at code coverage, and it’s like, you give the mouse a cookie! This is a children’s book – if you give a mouse a cookie, he wants a glass of milk, and then he wants a napkin and then he has to cut his hair, and it can go on, and on, and on. So this is the opposite of a cascading effect. You started off with one little thing and now you don’t just have Technical Debt you have this entire new project.
So I am not opposed to static analysis tools, I just want to know: When does it end? I like tools… I just don’t want to do tools for the sake of tools. It’s a problem I have! I want to get the features out as much as the product owner says. And I want to make sure that we are not falling into the trap of re-designing when we should be re-factoring.
Carolyn Seaman: Does that come up against the problem then of knowing what you’re going to touch next? It seems like a lot of time Technical Debt comes up, or causes a bigger problem, is when you find unexpectedly you have to touch a piece of code that you hadn’t expected to and you haven’t touched in a long time. And you open it up and you realize it has been degrading while no one was looking. So, if you are going to use automated static analysis tools in a targeted way like you were describing, how can you be sure that you were targeting it at the right places?
Johanna Rothman: Right, so no-one has a crystal ball. When you are surprised by something, and you’re saying, “Oh my goodness, this turned out to be bigger than I thought!” then you have several options
- You decide to say at the stand-up, I have an impediment, I ran into a bunch of Technical Debt and I want to have a problem-solving session with the rest of the team. So after the stand-up, I want 10 minutes or an hour – this is not something that you necessarily consult by yourself – and then the rest of the team can say, yes, we need to swarm about this.
- You can decide to do a spike.
- You can decide to do the bare minimum to implement this feature. You write another story, and you close the door on the debt. Nothing says you have to fix this now. If you worry that you’ve been accumulating debt, what’s a few more cents or a few more dollars so that you can plan when to deal with this. I really think that there is almost no problem in software that doesn’t get better with the application of a few more minutes of thinking… and sometimes that thinking is best done as a group.
Again, if you’ve already said: Oh my gosh! We need to swarm about this as a team and we have to apply static analysis tools. And in fact we are going to take this next iteration to clean up a bunch of stuff and deal with technical debt… Then I have no problem with this! And I think that’s the best way to use static analysis tools. But if you encounter this and you are off by yourself and you thought you were going to do a feature and now, holy moly, we have just encountered this big moose sitting on the table… I don’t think that’s the right time to go off on your own. So I think you have to say:
- What is the best use of my time?
- How do I deal with this?
- And who else do I inform?
3. What questions should we be asking ourselves before jumping into dealing with Technical Debt?
Carolyn Seaman: You have started touching on my next question actually, so let me jump on that. Say that there is a team that has become aware of the big moose on the table. They’ve either run static analysis tools or they have looked at their velocity or somebody has run into something that has surprised them – and say, “We’ve got this big mountain of debt, what are we going to do about it?” Before they jump in and start fixing stuff, what questions should they ask themselves? Like you said, everything benefits from a few more minutes of thinking. During those few extra minutes before they go jump in and try to fix all the debt, what should they be asking themselves
Johanna Rothman: What is the value of fixing things versus letting it sit? And I am a big fan of fixing. So I am not saying to let it sit. But I do think we have to look at the value. All too often, we have people saying, “Give me features! Give me features!” And the problems usually are: the system has too much Technical Debt; or we can’t do continuous integration; or we are so stuck with our own tool that we have no ability to provide features to our product owners; or we are so stuck in the old way of thinking (front-end and back-end), that we cannot provide features end-to-end through the system.
So when we think about Technical Debt, we have to think about how we re-trigger our thinking to move to this more Agile approach so we can provide a thin-GUI API (Application Programming Interface) and provide end-to-end features through the system in order to make features that are testable by the testers, unit tests for the developers, and to keep our velocity high. That allows our managers to make good portfolio management decisions. Because I am a huge fan of Management Technical Debt: if the managers don’t make good management decisions, they have management debt! If they are always picking people off this project, trying to put them on that project, then that doesn’t do anybody any good.
Let me rewind, let me pop back up the stack. I really like it when people encounter one of these big mooses on the table, or the after effects of the big moose on the table. You should make a card or you do a spike or bring it to the attention of the team, or somehow should do problem-solving around it. Don’t necessarily be the hero and take it on by yourself. This is why I really like short iterations. 1-week iterations: you know that there is no way you can take this on by yourself; 2-week iterations: you are less tempted to take this on by yourself; 3-week iterations; now the hero stuff comes to the floor, you say well I might be able to do this; 4-week iterations: you’re going to try to do this yourself.
I like saying:
- What do I have to do now to finish the story I am working on?
- What do I have to do now to help my team members finish the stories that they are working on?
- What do I have to do now to identify impediments?
- What do I have to do now to explain to the product owner or the customer what is going on?
- What do I have to do now to identify risks?
- What do I have to do now to help us get through this iteration so that we can make our plans for the next iteration? Because anything I discover in this iteration is fodder for the next iteration.
4. What’s the easiest way for us to define our Technical Debt?
Carolyn Seaman: Let me switch gears just a little bit and jump back to Technical Debt as a concept. The metaphor itself can sometimes seem large; and it is a growing definition too. It sometimes seems like more and more stuff is getting pushed under the umbrella of Technical Debt, and for some people it’s very confusing. It seems like anything that can go wrong in the code or in the project can be called Technical Debt, and the closer you get to that, the more meaningless the definition becomes. So let me ask you this: What is your working definition of Technical Debt?
Johanna Rothman: So for the ‘technical’ part of Technical Debt: this is what you owe the product and the things that you didn’t do. It can be architecture decisions that you postponed; it can be tests that you didn’t write; it can be design decisions that you didn’t make… And for the tests that you didn’t write, it can be unit tests or system tests. It’s what you owe the code. It’s incompleteness. That’s what I mean by the Technical Debt for the ‘technical’ stuff.
I referred earlier to Management Debt: that is decisions that managers did not make; decisions that management owes the teams. It’s all this owing stuff. And so, the more postponement of decisions, the more you patch over things. And if those are decisions that you postpone for a short-cut then you have to make those decisions later.
Now sometimes, especially in Agile, I think that a lot of these things are good. When you decide, “I am not going to make a decision on Architecture on the very first couple of iterations. I am going to implement 3–5 features and see where that leads me to evolve the architecture”, you could say that I have Architecture Debt, and I would agree with you. But I believe that this Architecture Debt is useful because I believe that if you evolve the architecture from those 3–5 features and see where that leads you, you will have a better architecture than if you tried to plan it from the beginning. And my experience is that this actually works. Now I’m not saying that you should not use patterns, because I think that you should. I’m not saying that you should not refactor as you go, because I think that you should. But my experience says: if you try to plan this stuff out from the beginning, you will be wrong. I mean, every time I try to plan it out from the beginning, we would get most of the way through the project, something would happen, and boom! We would fall flat on our faces, and oh my goodness, we were wrong, now how do we recover? So I would rather have debt at the very beginning, where we can then refactor to patterns, then to have the debt in the end. Call me silly, but this seems to work.
So when you consciously take on debt, you should say: I am going to take on debt from the very beginning and then pay it off while it is small. Here we have a plan to pay off the debt, and I don’t see that as a big deal. But I have been part of start-ups where they said we are hell bent on speed: We are doing everything we can to get something that looks like a demo where we have a happy path, and all we are doing is getting a happy path that works. Do not step off the happy path because anything other than a happy path is no guarantee of anything else that is going to work. OK fine, but we know that if you do step off the happy path, bad things are going to happen. We will fix this after we get to the demo, after we get our funding after, after, and after… Well, that means we are taking on significant Technical Debt! Now that’s a decision right? It’s a management and technical decision… For a product, I don’t think that that’s necessarily a wise decision. If you want a product you can build on release after release, then that in my opinion is just crazy because you just don’t have a basis on what to build.
5. Can we categorize our Technical Debt to help build strategies around them?
Carolyn Seaman: You’ve given a lot of examples of different types of debt. So let me ask you: The idea of Technical Debt is so big – it covers so much – is there a way to break Technical Debt down into different categories so that teams can understand the concept better and can identify which types of debt they are really concerned about and build their strategies around them?
Johanna Rothman: Chris Sterling actually has a good book; I believe it’s called ‘Managing Software Debt, Building for Inevitable Change’. I really like his taxonomy. He’s got Architectural Debt and Technical Debt. It’s a very nice way to talk about Technical Debt – there are 7 or 8 categories of it. I would go with his categories.
Carolyn Seaman: Categorizing it this way, do you think it helps them to get a little more practical and more concrete about dealing with that, if they pick a particular kind. Are there any strategies or guidelines on these types of Technical Debt that you could recommend?
Johanna Rothman: For me, when I work with teams, they are not thinking about the kind of debts that they have when they encounter it. They often are over-whelmed. I am thinking of a particular client who has debt in all of the categories. They call them tickets – and they have thousands of tickets! So they are not going to categorize their tickets. What I have said to them is for their product owner to take 30 tickets at a time… Why 30? Because it’s a manageable number. And take the top 30 for each of the iterations (they have 3 teams), and say: Can each team take 10 of the top 30 tickets for each of these iterations? The answer might be ‘No’ but you have your top 30. You know what your features are. Now you have a way to give them out piece by piece. The teams don’t have to go out and look for work – they know what they have to do! And you know the debt; you’ve discussed the debt with the teams while you were working on the road map (you have to have a road map for features and for debt). You can’t ignore the debt and say, “We just want features”. You have to put the debt into the roadmap so that there is a negotiation on the debt and the features. So this was a new idea for the product owner and the teams are pretty happy because it made the debt explicit. And with this, everybody is getting a little bit of what they want. When the team got to say, “If we fix this piece of debt, it will make these features go faster”, that is what got the product owner’s attention! So, it didn’t get that piece of debt fixed as soon as they wanted, but it got it fixed sooner. It got on the queue and the backlog faster.
6. How do we communicate to product owners to get them to allow Technical Debt at the expense of features?
Carolyn Seaman: Actually that leads me to the next and last question for you today. And it is about this issue that you’ve brought up a bunch of times when product owners push for features. You just told a little story about using Technical Debt, particularly data and information on it to convince a product owner that paying off the Technical Debt is worthwhile. In general, does that work? Does using the Technical Debt metaphor help to convince management and product owners to get them to allow Technical Debt to be paid off at the expense of features? Does that work, and what are the best strategies in terms of communication?
Johanna Rothman: This is where I really think that having your trends of velocity or trends of your throughput really works. Having your cumulative flow diagrams really helps. If it takes you longer and longer to push out features that are the same estimated size, I don’t see how it can hurt. If you explain to people: We thought it was going to take this long, but it’s now taking us this long. This is the root cause of many of our problems. We are not sitting around eating bonbons. We are working our little tooshies off. But, we can’t work any faster because this thing is hitting us in the face every single day, we have to address this! The more features you keep pushing on us, the longer it takes us to address it. We are working around this problem. If you just let us fix this… now we can make all this happen faster.
And what I have seen with product owners is that it helps to show this to them and explain it to them with data – and paper is fine! Or snapshots of your board are fine! You don’t have to keep an excel spreadsheet; you don’t have to keep it in a tool. But you need to keep some data so that they can see that your trend line is not going up. They need to see that the trend line is going down. This is why a burn-up chart is very helpful and the burn-down chart at times is useless. I talk about this in ‘Manage It!: Your Guide to Modern Pragmatic Project Management’, and in ‘Manage Your Project Portfolio. Increase Your Capacity and Finish More Projects’. I have charts in there and I also have arguments on my blog.
Carolyn Seaman: Do you have any last words or comments for our listeners today?
Johanna Rothman: I think that the most effective way of dealing with this is trying to explain to people that you are working hard and that you need data to communicate this with management. So burn-up charts of stories, using your velocity charts, is really helpful; showing product owners and customers that you are on their side. It’s not an either or, it’s us.