Clamberry Blog

All projects strive to provide the most ‘value’ for the least ‘cost’. Value is the set of features that provide tangible benefit to a customer, i.e. for which a customer is willing to pay. Cost is everything invested or incurred in order to deliver that value, but which adds no value of its own. Agile teams strive to deliver customer value as soon as possible; therefore the goal is create value with minimal cost. Architecture has no intrinsic customer value. So, the agile approach, typically, is to minimize focus on architecture; deliver value in the form of features; get feedback; and add features (value) and evolve the design through repeated, short iterations. As features are added and the design evolves, a coherent architecture will emerge.

The question is: Does this approach work?

I recently attended an interesting presentation (7/7/11) at Agile New England titled, Software architecture and agility: What colour is your backlog?, by Dr. Philippe Kruchten (http://www.kruchten.com/). The talk addressed the question “Is ‘Agile architecture’ an oxymoron?” as well as a discussion of cost vs. value and the nature of technical debt. The slides for the presentation are available at: http://www.agilenewengland.org/images/stories/event_assets/2011/20110707/110707_Kruchten_What_colours_is_your_backlog.pdf. This blog entry shares ideas presented in this talk, with my own perspective tossed in.

Before continuing, there are a few interesting ideas that were presented that would be useful additions to our typical agile/scrum lexicon.

Stories fall into four categories:

1. Features – offer positive value

2. Defects – cause negative value

3. Architecture/Infrastructure – incurs up-front cost but offer potential longer term net savings.

4. Technical Debt – incurs ongoing cost as overhead

All must be exposed and managed. All should be part of the backlog.

• Utils (Kruchten’s term) – A way to refer to the relative value of a feature to a user. Just as we use “points” as a term to assess cost to implement a story, “utils” refer to their relative value – regardless of how it gets implemented.

• Worth (my term) – A way to refer to the relative savings or cost that a facet of the system continues to effect in the ongoing development. Where points are the initial cost to develop something, worth is its ongoing savings/costs. It is its positive or negative value to the system, but not to the user.

• Architecture Owner – Analogous to the product owner’s role of defining and prioritizing stories based on their “utils”, the architecture owner’s role is to recognize and define “architecture” stories and prioritize them based on their potential to save costs – to raise the net worth.

• Architecture – “The stuff that will be hard to change later”.
  Architecture decisions impact decisions down the road. Architecture has no inherent value but potentially enables the team to produce the “feature” stories at lower cost.

• Technical debt – “Issues found in the code that will negatively affect future development but not those dealing with feature completeness”
  – Chris Sterling
  Left unpaid, technical debt will always have an accrued cost (my 2cents).

All stories have points (costs) to implement them. Features and Defects have positive and negative “utils” as their ongoing value. Architecture and technical debt raise or lower the net “worth” as their ongoing benefit/cost.

In agile development, as we iterate though our sprints, we evolve our design to fit the current stories. Since we address the needs as we go, we adapt the design only for the current need. We do not bother to anticipate what might be needed. After all, who knows if we will anticipate correctly? And cost spent on mistakenly anticipated need is, at best, a lost cost that cannot be recouped. With this approach, all costs are at least directly justified.

This argument appears to make good sense. Given the goal that agile software projects have, to produce value quickly, it seems prudent to do work in a way that gets value quickest. Why spend more time (cost) to produce a set of features (value) than is needed?

As strategic planning is minimized, there are often tactical decisions made that create problems in evolving the code down the road. These decisions and their cost are a project’s “technical debt”. It is the price paid later for the immediate savings. The analogy is that of financial interest as cost incurred later in order to enjoy something now that is not immediately affordable. Architecture is like savings. It takes longer to get what you want but you avoid paying the interest. Sometimes a loan is a deliberate and strategic decision, like taking out a mortgage. But sometimes it is reckless, like arbitrarily running up a large credit card bill. So it is with software. And, to beat the analogy to death, sometimes a recklessly high mortgage or loan can be “refinanced” to reduce the incurred higher up-front costs as times allow, like software refactoring. But we don’t always have time to take that effort; we just keep up with the high monthly payments. We complain, but we keep writing the checks.

So how do you know how much to invest in architecture and whether the right investment is being made? How do we increase our systems’ net worth?

The right answer, not surprisingly, is that it depends. The tension between agile and architecture can be characterized as one of adaptation vs. anticipation. The key is to strike the right balance. Iterative refinement is the only effective way to develop complex architecture. But you need features to validate the architecture.

There are some pragmatic guidelines (paraphrasing Kruchten):

Because agile teams are, correctly so, focused on delivering value, there needs to be some check that architecture is not being short-changed in the process. Agile teams like to hope that architecture will emerge as the design evolves. But the truth is that, left unchecked, agile teams have a tendency to run up technical debt faster.

• If no one is driving the emergence, there will not be an emerging architecture.

• Technical debt is often the accumulation of too many YAGNI (Ya Aint Gonna Need It) decisions.

In addressing the unknowns/uncertainties in a project:

• Defer features till later (lower risk, fewer dependencies)

• Take on architecture early (high risk, more dependencies)

You need architecture to create a sound framework for developing features. But, as stated earlier, you need features to validate the architecture.

• Monitor your architectural investments – address problems as soon as possible

• Development of features and architecture needs to be woven together

• Product and Architecture owners need to work together and trust each other

• Keep in mind that correct architectural decisions offer accrued saving, but incorrect architectural decisions can also become technical debt that creates ongoing costs

Pay down your technical debt before it bogs you down and bankrupts you.

Be aware that technical debt can be broken down into four categories (Fowler, 2009, 2010):

Deliberate

Reckless: “We don’t have time for design”

Prudent: “We must ship now and deal with the consequences”

Inadvertent

Reckless: “What’s layering?”

Prudent: “Now we know how we should have done it”

It’s ok to take on technical debt – if you are prudent about it.

There’s a lot to think about as we figure out how to effectively meet our upcoming challenges.