Technical Debt Folklore – “All technical debt is intentional”
ByOn December 5, 2012 At 1:50 pm
Responses : Comments are off for this post
Some months ago we presented three webinars to Lockheed Martin, IEEE, and Boeing on current research in automated identification of Technical Debt through code analysis tools. The topics covered a basic introduction of the Technical Debt concepts and terminology, as well as first research results describing how current tools (code smell tools, ASA issue tools, and other tools) can point to areas of debt in a code base. After presenting to more than 600 engineers, we also were interested in common beliefs (i.e. folklore) about technical debt in this community. For this, we invited webinar participants to fill in a short survey. The following presents some of the preliminary highlights of the results.
Survey Goals and Questions
The goal of the survey is to validate a set of statements about Technical Debt. I and my colleagues, Rodrigo Spínola, Forrest Shull, and Carolyn Seaman, searched a number of online websites and blogs, as well as some published papers, for potential folklore statements, and then refined the list into the items in the table below. These statements might have been expressed by individuals or by groups, but were not scientifically based initially. The rationale of our survey method is that, if these statements are confirmed (e.g. commonly agreed or disagreed to) by a large group of practitioners, then they are more likely to be good candidates for future research. On the contrary, mixed answers to a statement can indicate that it is not commonly believed, depends on other factors, or that the statement itself is not yet formulated as concisely as needed. Commonly believed folklore can help researchers to gather ideas for theories, hypotheses, research questions, and follow-up experiments.
The following statements were included in the survey. Participants indicated their level of agreement for each of them using a 5-point Likert scale:“1: strongly disagree”, “2: disagree”, “3: neither agree or disagree”, “4: agree”, or “5: strongly agree”.
- Accruing technical debt is unavoidable on any non-trivial software project.
- Technical debt usually comes from short-term optimizations of time without regard to the long-term effects of the change.
- It is very difficult for software developers to see the true effect of the technical debt they are incurring.
- “Working off debt” can be motivational and good for team morale.
- The root cause of most technical debt is pressure from the customer.
- Unintentional debt is much more problematic than intentional debt.
- The individuals choosing to incur technical debt are usually different from those responsible for servicing the debt.
- If technical debt is not managed effectively, maintenance costs will increase at a rate that will eventually outrun the value it delivers to customers.
- No matter what, the cost of fixing technical debt increases the longer it remains in the system.
- Paying off technical debt doesn’t result in anything the customers or users will see.
- The biggest problem with technical debt is not its impact on value or earnings, but its impact on predictability.
- Technical debt should not be avoided, but managed.
- Not all technical debt is bad.
- All technical debt is intentional.
Participants and Threats to Validity
We received a rather low number of responses (17 participants), which results in limitations and the need for care when interpreting these numbers. The results are likely to point us in the right direction, but are statistically not significant when evaluated with scientific rigor. Furthermore, this small subset of responses originates from a potentially biased group of participants, i.e. subjects who have had unpleasant experiences with Technical Debt (and so were motivated to attend our webinar).
Results are presented below by rank of agreement and tendency. The former metric orders the statements by their agreement as computed by a statistical measure (standard deviation). This is a rough indicator for the degree to which subjects gave the same answer for a statement, versus statements where answers were mixed, i.e. more spread out across the Likert scale. The latter metric, tendency, is computed as the median of answers on the 5 point Likert scale, indicating if ,on average, subjects agreed or disagreed with the statement.
Interpretation and Conclusions
The first three items in the table offer the strongest evidence of agreement on some Technical Debt folklore. High disagreement with the statement that all Technical Debt is intentional would indicate that many practitioners have been surprised to find Technical Debt that was not incurred intentionally. This supports the ongoing line of research into tools that analyze source code for “hidden” debt. The second item, which was strongly agreed with by our survey participants, indicates that Technical Debt is a very real problem, and can be dangerous if it gets out of hand. This motivates continued research in general, as well as adoption in practice of techniques to manage Technical Debt. In light of the third item, indicating agreement that not all Technical Debt is bad, this also motivates investigation into that “sweet spot” between an acceptable and healthy level of debt, and a level that is approaching dangerous.