On Technical Debt http://www.ontechnicaldebt.com Online Community Fri, 05 Feb 2016 13:00:43 +0000 en-US hourly 1 http://wordpress.org/?v=4.2.7 Technical Debt and Breaking Down “Tribal Speak” http://www.ontechnicaldebt.com/uncategorized/technical-debt-and-breaking-down-tribal-speak/ http://www.ontechnicaldebt.com/uncategorized/technical-debt-and-breaking-down-tribal-speak/#comments Fri, 05 Feb 2016 13:00:43 +0000 http://www.ontechnicaldebt.com/?p=3688 read more]]> In this great podcast from .Net Rocks! the discussion on how to handle technical debt takes an interesting turn towards the discrepancy in communication between different stakeholders on a software project. Most posts on our blog, as well as most other posts out there on technical debt, access the developer mindset. This means that when we talk about technical debt we are thinking about it from a perspective that does not necessarily help convince those managers that are focused on meeting their performance goals and have a hard time picturing the benefit of allowing a developer team to spend 20% of their time cleaning up old code.

To many managers this paying back of technical debt is a hindrance to meeting company goals because they are not understanding what is behind the term technical debt. What these managers need to understand is that reducing technical debt is a big picture goal. However, much of the weight on communicating this big picture importance of technical debt rests on the shoulders of developers – and as mentioned in this podcast, developers are not widely known for their “command of soft skills” (aka speaking persuasively to business stakeholders). Perhaps then the best way to get across the importance of technical debt reduction would be to approach it from the mindset of a business stakeholder concerned with meeting his/her performance goals?

DevOps can fit nicely into a scenario like this, as technical debt’s ramifications can begin to show themselves on the operations end and therefore make its costs visible to managers. This way you don’t have to persuasive when talking about technical debt, but simply show the numbers. The issue that managers mostly have with paying down technical debt is that they don’t see that it is not just an expense, but that it has a return on investment.

Ultimately, communication is the main inhibitor when trying to manage technical debt. It helps to think about what technical debt really means. Inherently, the assumption is that it is a purely technical term. But technical debt is not something that comes to fruition due to a lack of skill but due to a lack of time. Time is a concept that isn’t specific from team to team and therefore be used to create awareness on the issue of technical debt.

The other issue that comes about when you actually want to back and refactor code and you aren’t even sure of how much debt you have accumulated. If you don’t know how much debt you have how are you supposed to demonstrate the value in paying it back? If you don’t address this black box complexity, debt simply continues to grow and it becomes more difficult to add code or fix small problems. Fixing this decline in velocity becomes paramount and defining the reasons for the decline in velocity is central to fixing the problem. Here is where the technical debt metaphor comes into play.

There may be developers out there who enjoy debugging complex code and finding these developers to join your team maybe the solution. However, these types of developers are few and far between. So the true solution is improving communication on technical debt between domain experts, user interface people, and developers. Improving these dormant lines of communication means developing a shared language between all stakeholders in a project.

This means breaking down “tribal speak” (developer language vs. test language vs. domain language). How do you go about doing this? Working through business cases together by going into details and questions from all perspectives can be effective. Also iterating discussions throughout the lifetime of an application is necessary; you can’t just sit down once and expect everyone to come out of a meeting understanding every aspect of a project perfectly. Therefore, when meeting with all stakeholders a common glossary, to ensure that everyone agrees on what terms really mean, is imperative.

Developers are the most most adept at dealing with the ambiguities that come about from certain terms in a project because they are the ones that most often work between teams and work directly in fixing the glitches that arise in code. Just as in natural, colloquial language, software language has ambiguities.

Ex. “Cut trees down and then cut them up.”

The assumption that arises when talking about technical debt is that you or someone else has done something wrong. But it is again important to communicate that technical debt doesn’t necessarily have anything to do with skill level. So instrumenting and measuring technical debt (you can use tools like CAST AIP) can begin to demonstrate the cost of technical debt and help to trace out the differences between accidental and essential technical debt. Once you can measure technical debt and its cost, you can begin to see where unrealistic deadlines are causing unnecessary technical debt and what baseline of technical debt you are willing to accept in a product.

Just make sure that throughout this whole process communication does not break down and devolve into “tribal speak”.

To listen to the full original podcast, visit here.


http://www.ontechnicaldebt.com/uncategorized/technical-debt-and-breaking-down-tribal-speak/feed/ 0
How Innovation Debt Is Just As Damaging as Technical Debt http://www.ontechnicaldebt.com/uncategorized/how-innovation-debt-is-just-as-damaging-as-technical-debt/ http://www.ontechnicaldebt.com/uncategorized/how-innovation-debt-is-just-as-damaging-as-technical-debt/#comments Mon, 01 Feb 2016 13:00:03 +0000 http://www.ontechnicaldebt.com/?p=3684 read more]]> Often times in the development process large amounts of technical debt result in stalled innovation from a given team. However, what happens when what is stalling innovation is not only poor development practices but insufficient investment in developers themselves? This would be what is called, in this article, innovation debt. This sort of debt is comparable to technical debt in that a lack of attention to maintaining clean code results in long term deficiencies, the same way that a team which is too busy preparing new features for an app can’t learn about changes in languages, frameworks, libraries, or tools.

Over time a poor codebase will become extremely costly for an organization, and the same can be said for a lack of training and free space as it limits a team’s productivity and will make it so a team no longer has the skills to properly maintain in-house legacy systems and applications.

This innovation debt reveals its costs in the following terms:

  • Your best developers will start to leave
  • It will then be more difficult to fill these vacancies
  • Productivity will begin to fall
  • And your software will become brittle

So how do you avoid building up innovation debt?

Ultimately, the components that are necessary for continued training have to be incorporated into your corporate structure. This is just how organizations should be dealing with technical debt: baking in good coding practices into their business processes. For dealing with innovation debt these continued training practices could manifest themselves as monthly meetings where team members present an interesting issue area they have worked on or as a company directive to send developers to conferences in order to keep up with the latest trends.

Prioritizing knowledge sharing whether from external or internal sources is integral to raising the threshold for collective team knowledge and skill. However, this new focus on experimentation and learning means that an acceptance and expectation for failure also has to be taken into account by the organization. Trying out new methods and technologies comes with inherent risk; therefore, creating an environment where failure is not immediately reprimanded is conducive to innovation. Invaluable learning can take place from what seem like purely failures – your team can begin to form its own framework on what technologies work in a given circumstance, and which do not.

To read the full post innovation debt and how to avoid and manage it, visit here.

http://www.ontechnicaldebt.com/uncategorized/how-innovation-debt-is-just-as-damaging-as-technical-debt/feed/ 0
Maintaining Technical Debt and Team Morale in a Large System http://www.ontechnicaldebt.com/uncategorized/maintaining-technical-debt-and-team-morale-in-a-large-system/ http://www.ontechnicaldebt.com/uncategorized/maintaining-technical-debt-and-team-morale-in-a-large-system/#comments Thu, 28 Jan 2016 13:00:50 +0000 http://www.ontechnicaldebt.com/?p=3678 read more]]> In this post from InfoQ, Thomas Bradford explains his experience on working with a monolith java-based system that had improper test coverage and huge technical debt.

When asked what the biggest issue would be when maintaining a large java system, Bradford responded that the prevalence of legacy code riddled with quality issues. This leads to a situation where developers can’t modify the system or fix bugs because they might end up simply introducing more due to code complexity and previous development practices (heaps of technical debt).

The technical debt that was present in the legacy code that Bradford was working with was due to the system having been built quickly and as a monolith. This resulted in little to no test coverage (depending on the section of code you were looking at) and the monolithic style of the system created a mess of code with duplicated sections and long methods. A scenario like this makes maintenance extremely difficult and demonstrates an environment where little to no effort had been spent getting the system under control.

One of the biggest motivators to change the way thing were being done and to improve the system came when Bradford heard from various technical leads that often times a product would be rewritten twice, each time with little variance in the result. There was clearly a problem in the code base that kept developers from delivering features efficiently. Bradford said that asides from changing the strategy of working on the java system there also had to be a philosophical change in how developers worked.

Strategically it is impossible to simply ignore technical debt that has built up in a codebase, and that is true in any system. If you allow the technical debt build up to continue you will eventually reach a point where developers won’t be able to release any new functionality due to bricked-in code or because developers are so frustrated by the sluggish pace in which they are able to work. Philosophically, Bradford suggested that developers distance themselves from the practices they had become used to over the years (which had caused the slew of technical debt issues in the system) and begin to approach problems with a new mindset.

In order to deal with the insufficient test coverage that had become problematic in the monolithic java system first there had to be an internal effort to get the system under control. Following that effort, unit test coverage was built up but only in the areas that were particularly difficult to work on. But ultimately expanding test coverage in this manner is not the be-all and end-all of problems in a tech debt burdened system. Radical architectural refactoring that takes the monolithic system and start decoupling it has to occur.

Another important aspect of Bradford’s interview has to do with sinking team morale that often is the result of a system burdened by technical debt and quick-and-dirty practices. Bradford mentions how developers were afraid to change anything in the code base because in the monolithic system one change could have major unforeseen consequences in another part of the system. By enacting the changes mentioned above and giving developers more autonomy and trust in managing the system they work with daily, morale can only get better.

The idea that work can left until a later moment is a dangerous one, because more often than not it simply goes undone. This is what leads to incurring technical debt. If you find yourself in a situation similar to the one Bradford outlines in the interview, you will have to decide whether a short-term tightening in functionality output is worth avoiding the long-term failure of your organization. The answer to this should be clear: yes it is.

Professional developers have to responsible for the software quality they are building. This means not giving into external pressures to consistently compromising quality because ultimately these compromises will affect everyone in the organization negatively. Its the developers job to make the case for software quality above anything else.

To read the full interview visit here.

http://www.ontechnicaldebt.com/uncategorized/maintaining-technical-debt-and-team-morale-in-a-large-system/feed/ 0
When You Should Start Paying Off Your Technical Debt http://www.ontechnicaldebt.com/uncategorized/when-you-should-start-paying-off-your-technical-debt/ http://www.ontechnicaldebt.com/uncategorized/when-you-should-start-paying-off-your-technical-debt/#comments Mon, 25 Jan 2016 13:08:30 +0000 http://www.ontechnicaldebt.com/?p=3676 read more]]> Much of what comes with being an entrepreneurial leader is knowing when to accept certain tradeoffs. When you have a whole organization under your control it is impossible to constantly uphold a perfect balance; the result is often times compromises and temporary fixes. These compromises and short-term solutions are what is called technical debt. While technical debt is not an inherently negative concept, if left unchecked it become an additional risk in any organization.

At any software company technical debt invariably becomes an important topic of conversation. The focus of these conversations should be on when to let technical debt persist and when to start paying it off. In order to do this though, you need to understand what technical debt entails.

Technical debt is the result of fulfilling an immediate need (with a compromise on the quality of your code) that comes with some serious long-term consequences. And like credit card debt, technical debt has high interest rates that compound over time – making it more difficult to pay off the longer you let it remain in your code.

The CEO of BodeTree discusses how at the inception of any company, in order to find the adequate market fit for their product there is a inherent process of experimentation, adjustment, and iterations of code that lends itself to the creation of technical debt. But when should this technical debt be accepted, and when should you start to plan on paying it back?

If you are looking at a company in its early stages of development then there is likely to be quit a bit of accrued technical debt – this is just part of doing business. So you should be willing to accept technical debt in the following three situations.

  1. When you are dealing with your first big customer: quick iterative development and adjustments lend itself well to this scenario when you are looking to refine the product quickly to customer feedback. Therefore, technical debt becomes the means to an end.
  2. When you are building a minimum viable product due to a lack in secure funding: development work is costly so building quick and dirty may be your only option in order to get the product out on the market.
  3. When you are testing new features in an established product: sometimes a new feature may not even be the right fit for your product or market, so it makes sense to cut corners in development in order to see if the feature is even worth your while.

These are the times when you should let technical debt be; but when should technical debt be seen as an untenable part of your code base?

The time to pay it back would be when it begins to show itself by affecting the performance of your product and the efficiency of the team.

What are the signs of reaching such a point?

You will know that you’ve reached this point when making changes to your product becomes an arduous task to the amount of features that are interconnected. Your code will become brittle and any change could have some serious ramifications in other areas of your code that could result in breakages and software outages.

For BodeTree the signal was clear when a sudden influx of customers revealed what features were not being used and which features weren’t working. The BodeTree team was able to identify what could be cut out and therefore focus on the quality and performance of the code in their system – in the areas that mattered most. A strategy like this saves capital because it allows you to surgically correct the areas where technical debt is the most costly and prioritize your development work accordingly.

The key, as mentioned at the start of this post, is knowing what the right balance is of acceptable and not acceptable technical debt is in your organization.

To read the full post visit here.

http://www.ontechnicaldebt.com/uncategorized/when-you-should-start-paying-off-your-technical-debt/feed/ 0
The Risks of Measuring Technical Debt http://www.ontechnicaldebt.com/uncategorized/the-risks-of-measuring-technical-debt/ http://www.ontechnicaldebt.com/uncategorized/the-risks-of-measuring-technical-debt/#comments Thu, 21 Jan 2016 13:15:37 +0000 http://www.ontechnicaldebt.com/?p=3669 read more]]> It has become a recent practice in organizations to measure technical debt in their software – but how often do you think about why you are measuring technical debt? What will you and your team do once you have information on how much technical debt you have in your software? What risks have you uncovered in the process? Simply because you have the ability to measure technical debt, doesn’t mean you necessarily should.

Measuring technical debt can be extremely valuable but only in the right context. Your organization first needs to have a working definition of what it means for a software component to be complete; in other words, your organization needs to have a measurable definition of “done”. What teams need to keep in mind is that the gap revealed by measuring technical debt (the gap between the work being put to release and what your working definition of ‘done’ is) has far reaching consequences.

The scenario drawn out in this post presents an organization that went through the process of measuring technical debt. However, their software at one point fails causing physical and financial damages to the organization. Here is where the measurement of technical debt can become an issue, especially if your processes are not well equipped to deal with such a failure.

After this failure let’s say you are being questioned by jury on the software failure that occurred and you mention that technical debt was present in the version of the software that was released and then failed; it could then be argued that you deliberately released a product that was flawed and faulty. Even if the technical debt that was present came from poor practices unrelated to functionality (like poorly documented code) these nuances are likely to be lost in a case on business’s liability for the failure.

As software functions today, it is easy to consistently update it thus creating an environment where it is common for organizations to release imperfect or incomplete software. Therefore, if your development team is deliberately measuring and documenting code you could be building a case up against yourself in case of technical failure.


If you have a well moderated risk management policy in place and processes that ease the identification and measurement of technical debt, as well as contingency plans to deal with the risk of building software with technical debt, you are in a better place than without.

Another downside of measuring technical debt is that it impedes innovation. If technical debt measures are demonstrated as persistently increasing it could then be used as another excuse by developers for not delivering features by their deadline. Ultimately, if a business does not consider work that is left undone as a pressing business concern than it should not be labeled technical debt. Instead it should simply be stored in the backlog, along with documented efforts through robust testing, and automation.

So what then is the difference between measuring technical debt and recording defects or publishing known issues? The word debt inherently carries negative connotations; it sounds deliberate and risky (technical debt by definition is both of these things but if you are aware of the technical nuances of inducing technical debt than you know that this is not necessarily a bad thing).

When business and development teams are in synch with their understanding of ensuring that software is meeting predefined standards of software quality that is the time that technical debt measurement is best utilized. If you plan to release software with quantified technical debt present it is necessary that all aspects of business are aware of the risks present of putting speed before quality.

Ensuring that your organization has a set of best practices and follows them closely is imperative to using and managing technical debt to the benefit of your product.

To read the full post visit here.

http://www.ontechnicaldebt.com/uncategorized/the-risks-of-measuring-technical-debt/feed/ 0
Technical Debt & Risk: One and the Same http://www.ontechnicaldebt.com/blog/technical-debt-risk-one-and-the-same/ http://www.ontechnicaldebt.com/blog/technical-debt-risk-one-and-the-same/#comments Wed, 06 Jan 2016 13:00:00 +0000 http://www.ontechnicaldebt.com/?p=3666 read more]]> Technical debt is a very important concept to developers that is often lost on the management end. Developers use the concept to describe the consequences of a pressure to meet deadlines; to release features on time developers will write quick and dirty code, compromise on standards, and leave code quality out to dry. Technical debt is the choice between making sustainable software and making short-term decisions that are detrimental to code in the long run.

For management, debt wouldn’t be the term they’d use to describe the concept – but they would rather see it as the risk of increasing costs or missed deployment schedules due to the current development process. Any code written has inherent risk associated with it. Therefore, technical debt is not seen as something ineffectual, but as, sometimes, a necessary risk. To be able to decide whether the risk is acceptable or not technical debt has to be quantified.

There are several ways to measure the presence of technical debt in a code base:

  • Estimate Accuracy: without technical debt the estimations of time needed to develop a certain feature would become more accurate overtime. As developer knowledge in a certain area increases (knowledge on tools and frameworks, team performance, etc) estimations by developers would become more accurate. If this is not happening (and estimations become less accurate and more difficult to pinpoint) it is likely that the system is plagued with technical debt. The decreasing accuracy of estimates can be measured and, therefore, be used to measure technical debt.
  • Time to First Feature: the time for adaption by new hires is a useful metric for technical debt. This encompasses the comprehensibility of the code and the developer’s own ability to understand the code. Both of these components are important, but if it has become typical that new hires need more time to bring components into production “it is likely that that the learning curve has become too flat”. Technical debt is often the cause of this.
  • Internal Metrics Failures: developers and devops often use tools and metrics to see if code is meeting certain standards. It is likely that your development team already has an internal tool in place to track that status of software; it can be the case that this tool indicates poor behaviour most of the time, and that can be normal. But also keep in mind, that if you have a component that has been marked for 30 days as performing below satisfactory, there’s probably technical debt present. The duration of failure to meet standards is a key indicator technical debt.

So once you’ve measured the presence of technical debt in your system, how do you manage it?

Technical debt is often seen as solely a negative thing, but just as we mentioned that risk is inherent to code – technical debt is also inherent to a system. Many teams immediately fight against technical debt, even before it has actually shown itself to be present. Other teams entirely ignore technical debt, mounting pressure on their developers and digging themselves into a deeper debt.

Ultimately, it is necessary to know how much debt you are incurring. This means either documenting debt in the moment, or when it starts showing itself in the system. The latter option is perhaps the better one, as it automatically helps prioritize those debts that need to be addressed first.

Once you know where and how much debt you have, repaying it from time to time is important. It is best to repay it sooner rather than later, because the longer they are in a system the harder it is to identify. Refactoring is the best way to go about this. This means that there will probably always be certain levels of technical debt present in your system, which means certain levels of risk, so preparing for possible failures with a contingency plan is also a good idea.

Dealing with technical debt is dealing with risk. In a business you can’t be risk-averse, but deal with it smartly. If you deal with technical debt in the ways mentioned above, you system should be safe from any from any major problems and enable business growth.

To read the full post visit here.

http://www.ontechnicaldebt.com/blog/technical-debt-risk-one-and-the-same/feed/ 0
Technical Debt & Software Quality Tools http://www.ontechnicaldebt.com/uncategorized/technical-debt-software-quality-tools/ http://www.ontechnicaldebt.com/uncategorized/technical-debt-software-quality-tools/#comments Mon, 04 Jan 2016 13:00:13 +0000 http://www.ontechnicaldebt.com/?p=3659 read more]]> It’s estimated that the federal government spends about $80 billion a year on IT; over 70% of this goes to maintaining legacy systems with the remaining 30% going to new investment in next-generation systems. This keeps old systems running but it creates a problem by exposing them to risk as the costs of maintenance grows and money for new systems is further limited. There’s an emerging conflict between new and older systems, resulting in a dangerous status-quo.

Mark Ryland, Amazon Web Services’ chief architect, spoke to House Committee on Oversight and Government Reform’s subcommittee on IT in September and explained the danger of continual funding of older systems. This balance, between funding old systems vs. new systems is just part of the conversation that policy makers are having with regards to dealing with technical debt.

Bill Curtis, Executive Director at the Consortium for IT Software Quality (CISQ) states that this debt is the result of building software that has some errors in it, and waiting to fix these errors. The time between when these errors were made and when they are fixed is what amounts to debt. As technical debt mounts customer’s will begin to complain about slow delivery, which in turn will result in more quick and dirty practices and more technical debt. This becomes a vicious cycle that needs to be managed.

There are now new tactics to address this problem: employing a uniform set of standards for software quality. This can be done by including standards contracts and service agreements.  Software standards makers, like CISQ, want to use their measurements like performance, efficiency, ,maintainability, reliability, and security as the base for estimating technical debt in a system.

If you can analyze your code to determine the amount of effort needed to fix the aforementioned measurements, you can also measure technical debt in a system. CAST provides a product that is designed for analyzing code while under development. This includes levels of complexity in programming projects – Booz Allen Hamilton, which often has more than 300 software development projects underway in one day, uses CAST in order to measure coding practices in their systems.

While Booz Allen uses the CAST tool to analyze its internal projects, government agencies have also inquired the agency on the tool to see how it would impact of new users to their legacy systems. The tool can also be used for gauging certain service level agreements; certain request for proposals can be based on CAST service levels.

Dr. Bill Curtis also said that CISQ intends on delivering a standard for technical debt, at a future meeting of the Object Management Group standards consortium, that concerns structural quality of code, or in other words, how well a system is built. Technical debt has huge costs associated with it, in order to improve the efficiency of a system it needs to be addressed; software quality tools may just be the answer.

To read the full post visit here.


http://www.ontechnicaldebt.com/uncategorized/technical-debt-software-quality-tools/feed/ 0
Technical Debt and Cybersecurity: A Bad Relationship http://www.ontechnicaldebt.com/uncategorized/technical-debt-and-cybersecurity-a-bad-relationship/ http://www.ontechnicaldebt.com/uncategorized/technical-debt-and-cybersecurity-a-bad-relationship/#comments Tue, 29 Dec 2015 13:30:39 +0000 http://www.ontechnicaldebt.com/?p=3656 read more]]> The costs associated with maintaining technical content, not advancing its capabilities, is often referred to as technical debt. As we have mentioned before, technical debt can be a huge hindrance to innovation – but it also presents an obstacle for cybersecurity readiness. Reducing technical debt is a critical component of having a a successful cybersecurity program.

We know that technical debt has become an endemic part of any software technology – it is inherent to the process of balancing deadlines with software maintainability – but it affects more than just an organization’s competitive advantage (by having first-to-market software functionality). Security personnel, if dealing with software systems plagued with technical debt, will have to deal with slower deployment schedules because of refactoring efforts rather than looking to address the prospect of new security threats to existing systems.

The worst case scenario would be if an organization continues investment in platforms that are no longer meet necessary efficiency standards. In this scenario new capability takes a backseat to simply delivering the least-necessary value possible. Security debt would be the term to describe this: application vulnerabilities that come as a result of laggardly behaviour.

Driven by the evolution of information technology by cloud computing, there has been a reduction in the life cycle of effective software solutions. This has pushed software providers to speed up delivery of capability through continuous delivery models. It has become expected that vendors deliver and maintain continuous software practices.

For example, security identity and access management (IAM) products, which were once deployed on premises, are now accessible in multiple ways:

  • Infrastructure as a Service (IaaS) – provides the ability to script and automate deployment and management, which makes it attractive comparative to other solutions
  • Software as a Service (SaaS) – presents accessible IAM products with no mandatory on-premise components
  • Platform as a Service (PaaS) – grants the ability to integrate IAM products into development platforms and allows engineers to add identity management operations into SaaS applications.

The adoption of cloud-based services has provided organizations the capacity to reduce technical debt by balancing continuous delivery solutions and controls for security. This is the case because as cloud adoption becomes more prevalent, it frees technical teams from the management of software and allows them to focus on assessing and adopting new technology to stay ahead of changing security threats.

Other advantages of cloud practices is the security teams will become familiar with devops and continuous delivery models. This will increase the speed of the adoption process and will separate technical debt that are often tied to legacy solutions. Reducing security vulnerabilities is strongly related to automation and integration.

Organizations that continually evaluate whether their current investments are going into legacy platforms that are worth the money and whether they should migrate to new platforms will have the advantage over adversaries looking to exploit security vulnerabilities.

Ultimately, ignoring technical debt and not instilling the best practices to avoid it – not only hurts the sustainability of your product, but the security of your enterprise.

To read the full post visit here.

http://www.ontechnicaldebt.com/uncategorized/technical-debt-and-cybersecurity-a-bad-relationship/feed/ 0
At a Glance: Machine Learning and Technical Debt http://www.ontechnicaldebt.com/uncategorized/at-a-glance-machine-learning-and-technical-debt/ http://www.ontechnicaldebt.com/uncategorized/at-a-glance-machine-learning-and-technical-debt/#comments Tue, 15 Dec 2015 13:05:20 +0000 http://www.ontechnicaldebt.com/?p=3648 read more]]> In this podcast from Software Engineering Daily, D. Sculley discusses the problem of technical debt with respect to machine learning systems.

Below is some of the key learning from the podcast:

Machine learning, as defined by Sculley, is the intersection of statistics, computational power, and engineering. Ultimately, it is the science of learning from large amounts of data and developing systems that exhibit behaviour that would not have been able to have been defined with simple rules brought about by ‘if and then’ questions. These rules, in a machine learning system, are learned from large amounts of data.

“Machine learning offers a fantastical powerful toolkit for building systems quickly”

Machine learning by deriving information from large quantities of data allows for complex systems to be built quickly. For example, if you want to develop a spam classifier to determine if an email is spam or not the least efficient way to go about this would be to think of all the possible methods a spammer would enlist to get their message opened . Not only does this take a lot of time, but it leads to brittle code which spammers will be able to find a way to get around in due time. By using data to create rules (this is assuming large amounts of data are being used), the development process will be shorter and more efficient to making less brittle software.

However, like any system, machine learning systems are susceptible to incurring technical debt. Technical debt is the service costs of moving quickly vs. moving well. In developing software it is often easy to add on a feature or using cut and paste methodology to save time, but not making the most maintainable software. This leads to accumulated technical debt. The debt metaphor is of interest because it explains the idea of taking on a cost now for work not done, and being able to pay it back later.

So how does technical debt compound like financial debt?

Once you acquire debt once, it becomes easier to acquire more and more debt overtime. Once a project has technical debt, which is most likely to be an API that was widened too much or includes cut and paste pieces, it can then be tempting for developers to not add on to the code in a clean way because there is already debt present. As this practice continues it becomes more and more difficult to avoid.

What makes machine learning systems so much more susceptible to technical debt?

Technical debt was not widely recognized by the broader development community a few years ago, although those working on machine learning systems found the technical debt trap to be present. Machine learning systems often have the same problems regarding technical debt as more traditional software, since they are both built off of code. However, there are a number of areas that are unique to machine learning. They are fixed in the fact that machine learning solves problems that cannot be formally defined, thus data does the defining for developers and guides system behaviour. Data then becomes the direction from which technical debt becomes tricky for machine learning systems.

Machine learning can result in hidden technical debt, this is debt that one is not aware of and hasn’t accounted for. An example, would be if you get a credit card and are aware that in its terms there is an 18% interest rate vs. if a college student who didn’t read the fine print and goes on to buy a new couch. There are different consequences to each scenario.

The reason for which machine learning often results in this type of debt is that complex models have the effect of eroding boundaries. In traditional software development a frequently used method of managing system complexity its through encapsulation (making a piece of code only to do one thing and be separate from those that do other things). This is done in data structures, abstractions, and APIs. Machine learning systems take many sources of information and combines them in a way that is not defined. This leads to an erosion of boundaries.

It is not possible to take any particular part of the machine learning system and abstract it out in a meticulously defined way. This leads to a scenario where changing any one part of the system leads to possibly changing another part of the system behaviour or potential changes across the entire system.

For example, going back to the spam classifier: it is possible that the classifier does a bad job of identifying spam in Iceland. This could be due to the fact that for whatever reason, Icelandic spammers have an entirely different way of approaching the space. You might want to correct this lapse by adding features that are specific to Iceland or by tweaking regularization parameters or other settings that change system behaviour. This may lead to improved behaviour with respect to Icelandic spam. However, spam in Sweden becomes more difficult to detect because some of the credit that was assigned to  Icelandic spam was actually from Sweden. By trying to separate the process for Sweden from the rest of the model, there were unintended consequences.

The lack of defined abstraction boundaries in the system leads to this issue, as one of the key reasons for using machine learning is that the behaviour exhibited by a model is created by the data. The natural world creates this data, and thus clearly defined boundaries are not present. Machine learning is the union between hard-coded software logic and evolving external data – the date often times becomes more important that the hard-coded software logic and leads to blurry boundaries and data driven changes.

One of the aspects of machine learning that leads to debt creation are Sculley refers to high debt design patterns. This occurs from the use of glue code (code written in some scripting language like python or perl) that allows for data to get from one system into another. This glue code stems from the fact that often times it is more cost effective to buy a machine learning package and use it to your specific needs. However, within the package there maybe different data formatting and therefore code needs to developed to link data within the package to external code. As there is not a standard API set up, if a change needs to be made to the central package all the glue code has to be undone. This is a costly process and demonstrates the type of debt that plagues machine learning systems.

This is just a taste of what this podcast has to offer – visit here to get a further in depth look at the discussion on machine learning and technical debt.


http://www.ontechnicaldebt.com/uncategorized/at-a-glance-machine-learning-and-technical-debt/feed/ 0
The Brethren of Technical Debt: Cloud Functionality Debt http://www.ontechnicaldebt.com/uncategorized/the-brethren-of-technical-debt-cloud-functionality-debt/ http://www.ontechnicaldebt.com/uncategorized/the-brethren-of-technical-debt-cloud-functionality-debt/#comments Tue, 08 Dec 2015 13:00:12 +0000 http://www.ontechnicaldebt.com/?p=3644 read more]]> In this post, the author cites overhearing cloud vendors and implementors bemoaning how cloud application software customers fail to activate new functionality found in quarterly releases of multi-tenet cloud application solutions. This is the type of new functionality which would help efficiency processes and alleviate possible bottlenecks. But, customers aren’t turning the new functionality on.

Why is this the case?

In older on premise applications, customers often deferred releases as long as possible. Often times the costs of implementing the upgrades were greater than the functionality that the upgrade would deliver. These upgrades, no matter how minor vendors thought the impact would be to customers, were treated like nightmares to avoid until regulation or a mandate forced them to take it on. However, as more and more of these updates were deferred, customers began to incur technical debt. This debt represents the economic and time costs associated with bringing an application to its current form. This issue was mainly seen as prevalent only in on-premise solutions.

One of the benefits of multi-tenet application software is supposed to be that lower tech stack upgrades – like database or security upgrades – are the responsibility of the vendor. Application upgrades are also meant to be their responsibility. In theory, this should appease the problem of technical debt from deferred customer upgrades – but that doesn’t seem to hold true. It seems that most multi-tenet cloud application vendors deliver newly upgraded software with the new functionality turned off and customers don’t know they need to activate it. This creates, what is called in this article, cloud functionality debt.

Cloud functionality debt occurs when customers don’t take the time to asses how new functionality will affect business processes. It also occurs when customers don’t understand new functionality and fall back on old practices that don’t improve processes. They find the upgrade to be an inconvenience to business-as-usual. But, for whatever the reason, cloud functionality debt becomes a problem to customers and vendors.

From the vendor perspective, the new features help them reel in new customers and perhaps expand what their customers need from their firm as new functionality is desired. Just as dealing with traditional technical debt, cloud functionality debt requires a plan from customers in weighing the costs and benefits of implementing new functionality. It may seem like a lot of work to implement upgrades at any given moment, but in the long run it can keep systems running more sustainability and at a higher level of quality. Similar to incurring large amounts of technical debt, building on systems that have cloud functionality debt keeps systems from working as they should. In the fast-paced environment of the tech industry, sticking to past practices just isn’t good business.

It’s time to account for all types of debt that could be plaguing your systems and addressing it accordingly.

To read the full post visit here.

http://www.ontechnicaldebt.com/uncategorized/the-brethren-of-technical-debt-cloud-functionality-debt/feed/ 0