Pragmatic Agile Development: How to develop fast & contain Technical Debt
ByOn January 6, 2012 At 1:24 pm
Responses : Comments are off for this post
Mr Brown and Mr Black lived next door to each other. They both needed to build new houses for themselves to give more space for their families and a better living environment.
Mr Brown hired a group of waterfall builders. After a year he asked how they were getting on. They assured him that they had completed the family’s needs assessment and produced a detailed design. It would only take another three months of checking and validating before they could start work. Three months later Mr Brown got a job in another part of the country and his wife was pregnant with twins, so they needed a bigger house in another part of the country.
Mr Black hired some agile builders. The foundations were in within two weeks for his approval and every two weeks he reviewed progress and agreed what would be built in the next two weeks. The house was quickly completed and he moved in. Unfortunately the house sank on its foundations with the weight of the furniture and collapsed. “You screwed up by not digging deep enough foundations,” Mr Black cried. “No we didn’t,” they retorted. “It is entirely your fault because you signed off the foundations at the end of week two!”
A group of senior IT executives met over dinner in London to discuss a pragmatic approach to agile. One that can rapidly deliver working software while managing technical debt. These were their recommendations:
Technical debt will always exist and it is unrealistic to expect to reduce your legacy debt all at once. However, it is not inevitable for agile development to add to technical debt, rather it is a result of ignoring operational requirements. By itself, it is not a problem, as long as the organisation knows it is creating technical debt and understands the consequence of it on the customer experience.
Very few organisations measure it, or if they do, they measure it in the wrong area, such as stable legacy systems that haven’t changed for years and are not causing problems. They should be measuring the .NET or Java developments, which tend to accrue massive amounts of technical debt.
Service providers often know more about an organisation’s technical debt than they do. If technical debt is measured on a continuous basis, it becomes a quality improvement mechanism, as developers learn to make better choices.
In order to provide performance and resilience, a hybrid approach is needed. A project must start by drawing up an overall blueprint of the final system. This to define the operational requirements of an effective underlying platform. It covers the overall architecture, main component capabilities, scaling factors and interoperability standards. Some components may already exist and involvement of architects is absolutely critical. There is no shame in this waterfall-type initial phase, as long as it is not detailed.
An agile development approach is then used to deliver rules, interfaces and functional outcomes, including existing components. Infrastructure is delivered by providers, but in line with the sprint ‘heartbeat’ to make it work effectively. Every sprint must review progress on acceptance of the operational requirements.
Short sprints are good
Short sprints, typically two weeks, act like a large metronome or drumbeat for the IT organisation. Working against immovable deadlines encourages everybody to by hyper-successful, not chaotic. The idea is to ‘fail fast and recover quickly’. Errors in approach can only last two weeks and must be fixed in two weeks. The level of rework from a sprint is a lot less than that which is massively under-reported on large waterfall developments.
Success depends on changing the culture of the organisation. It starts with being really honest about the overall company culture and how agile the development can really be. Getting top-level support at the outset is essential.
Developers must become much more open and find a way to communicate that problems exist, so that they can be reworked. They must understand the risk of technical debt and be encouraged and incentivised to disclose it.
They must also understand that the single most important user or customer function is that the functionality is actually available and works efficiently when they want to use it. Use the phrase ‘operational requirements’ or ‘operational imperatives’, not ‘non-functional’, which automatically makes them lowest priority. They are not directly customer-facing, but they must be done for the functionality to work.
Similarly, the latest update in scrum, the first in ten years, has replaced the word ‘prioritisation’ with ‘ordered backlog’. Prioritisation focused the mind on the ‘sexy’ functionality, like spinning logos.
Switch something off
Software development projects offer an opportunity to replace outdated systems and remove their associated technical debt. This also gets rid of the hidebound thinking that caused the debt. It is generally faster, cheaper and easier than trying to retrofit new requirements onto old system that the organisation doesn’t fully understand.
People are deterred from replacing old systems by fear and risk. They also cite the cost of changing, such as tens of millions for a system that only costs £50,000 a year to run. However, they should consider the risk to the business, which could cost it £60 million.
Functionality can be migrated out of monolithic systems into common services on a phased basis. Replacing the eventual residue should be straightforward.
There is a steep learning curve in agile development and a high failure rate is to be expected. The number of developers who really understand agile development is quite low. Hire the best people or consultants you can afford who really know what it is about.
– By Rod Newing