No bugs journey episode 2: It’s a matter of values…

February 21, 2015 at 4:54 pm

Read episode 1 first.

Here we are at episode 2, and time for another question. Of the three, which one do you value the most?

  1. Shipping on schedule
  2. Shipping with a given set of features
  3. Shipping with high quality

Write down your answer.

Ha ha! It was a trick question. Pretty much everybody is going to say, “it depends”, because it’s a tradeoff between these three. Let’s try an alternate expression:

  1. Build it fast
  2. Build a great product
  3. Build it right

All of these are obviously important. Hmm. Let’s try a different tact…

If you get in a situation where you need to choose between schedule, features, and quality, which one would you choose?

  1. Shipping on schedule with the right set of features but adequate quality
  2. Shipping late with the right set of features and high quality.
  3. Shipping on schedule and with high quality but with fewer features than we had hoped

Write down your answer.

Let’s talk about option #1. First off, it doesn’t exist. The reason it doesn’t exist is that there is a minimum quality level at which you can survive and grow as a company. If you are trying to hold features and schedule constant, as you develop features you are forcing the quality level down – it *has to* go down because something has to give if you get behind (short estimates, etc.). That means you start with a product that you just shipped, and then it degrades in quality as you do features, and then at some point you want to ship, so you realize you need quality, so you focus on getting back to *adequate*. Unfortunately, fixing bugs is the activity with the most schedule uncertainty, so there is no way you are shipping on time.

It’s actually worse than this. Typically, you’ve reached a point where your quality is poor and you know that there is no way that you are going to get the features done and reach acceptable quality, so you cut features.

And you end up with a subset of features that ships late with barely adequate quality. There’s an old saying in software that goes, “schedule/features/quality – pick 2”, but what few people realize is that it’s very easy to pick zero.

I hope I’ve convinced you that the first approach doesn’t work, and given that I’m saying this series is about “no bugs”, you probably aren’t surprised. Let’s examine the other options.

I’ve see #2 work for teams; they had a feature target in their head, built their software to high quality standards, and then shipped when they were done. This was common in the olden days when box products were common, when the only way to get your product out there was produce a lot of CDs (or even diskettes) and ship them out to people. It worked, in the sense that companies that used the approach were successful.

It is, however, pretty disruptive on the business as a whole; it’s hard to run a business where you don’t know:

  • If new revenue is going to show up 1 year or 3 years from now
  • When you’ll be spending money on sales and marketing
  • Whether the market has any interest in buying what you build

Not to mention making it tough on customers who don’t know when they’ll get that bug fix and enhancement they requested.

Which leaves option #3, which through an amazing coincidence, is the one that is the most aligned with agile philosophy. Here’s one way to express it:

Given that:

  • It is important to the rest of the business and our customers that we be predictable in our shipping schedule
  • Bugs slow us down, make customers unhappy, and pose significant schedule risk
  • The accuracy at which we can make estimates of the time to develop features is poor
  • Work always expands (people leave the team or get sick, we need to spend time troubleshooting customer issues, engineering systems break), forcing us to flex somewhere

The only rational option that we have is to flex on features. If we are willing to accept that features don’t show up as quickly as we would like (or had originally forecast), it is possible to ship on time with high quality.

I chose the phrase “it is possible” carefully; it is possible to build such a system but not get the desired result.

Flexing on features effectively

Time for another exercise. Studies have shown that you will get a much better review of a plan if you spend some focused time thinking about how it might go wrong.

Thinking about the world I just described, what could go wrong? What might keep us from shipping on time with high quality?

Write down your answers.

Here’s my list:

  1. At the time we need to decide to cut features, we might have 10 features that are 80% done. If we move out all 10 of them to the next iteration, we have nothing to ship.
  2. We might have a hard time tracking where we are early enough to make decisions; most people have seen a case where all the features were on schedule until a week before shipping and suddenly 25% were behind by a week or more. If this happens, it may be too late to adapt.
  3. We might have teams that are dependent on each other; the only way to make my schedule is to cut work from team A, but that means that teams B & C can’t finish their work as planned, and they will have to adjust, using time we don’t have.
  4. This release was feature A and a few bugfixes, and feature A isn’t on schedule. We’ll look silly if we just shipped bugfixes.

(At this point I’m really hoping that you don’t have something important on your list that I’ve missed. If so, that’s what comments are for…)

How can we mitigate? Well, for the first one, we can focus on getting one feature done before we move on to the next one. That means that we would have 8 features 100% done instead of 10 features 80% done. This is one of the main drivers for the agile “work together as a team” approach.

This mitigation works for the second one as well. If we make our tracking “is the feature fully complete and ready to ship”, we can tell where we are (3/10 features current done and ready to ship (“done done” in agile terminology)) and we have a better chance of predicting where we are going. This is another driver for “work together as a team”. Note that for both the first and the second one, the more granular our features are, the easier it is to make work; it works great if the team has 5-10 items per iteration but poorly if it only has two. This is the driver for “small self-contained stories” and “velocity measurement” in agile.

I have a few thoughts on the third one. You can mitigate by using short cycles and having teams B and C wait until A is done with their work. You can try to break the work A does into parts so the part the other teams need can be delivered further.  Or you can go with a “vertical team” approach, which works great. 

For the fourth one, the real problem is that we put all of our eggs in one basket. Chopping feature A up will give us some granularity and the chance to get part way there. I also think that a shorter cycle will be our friend; if we are giving our customers updates every month, they will probably be fine with a message that says, “this month’s update only contains bugfixes”.

To summarize, if we have small stories (a few days or less) and we work on them sequentially (limiting how many we are working on at one time), our problems about tracking and having something good enough to ship become much more tractable. We can predict early what features are not going to make it, and simply shift them to the next cycle (iteration). That is our pressure-relief valve, the way that we can make sure we have enough time to get features completed on the shipping schedule with great quality.

The world isn’t quite as simple as I’ve described it here, but I’ve also omitted a number of advanced topics that help out with that, so I think it’s a pretty fair overview.

Before I go on, I’d like to address one comment I’ve heard in relation to this approach.

If we flex on features, then I’m not going to be able to forecast where we will be in 6 months

The reality is that nobody has ever been able to do that, you were just pretending. And when you tried, you were often spending time on things that weren’t the most important ones because of new priorities.

Instead, how about always working on whatever is the highest priority for the business, and having the ability to adjust that on a periodic basis? How does that sound?

The culture of commitment

Time for another question:

What are the biggest barriers to making this work in your organization?

Write down your answers.

I have a list, but I’m only going to talk about one, because it is so much more important than the rest. It’s about a culture of commitment.

Does your organization ask development teams to *commit* to being done in the time that they estimated? Does it push back when developer estimates are too large? Do developers get emails or visits from managers telling them they need to be done on time?

If so, you are encouraging them to write crappy code. You have trained them that being on the “not done” list is something to be avoided, and they will do their best not to avoid it. They can either work harder/longer – which has limited effectiveness and is crappy for the company in other ways – or they can cut corners. That’s all they can do.

Pressure here can be pretty subtle. If my team tracks “days done” and I have to update my estimate to account for the fact that things were harder than I thought, that puts pressure on me to instead cut corners.

This is one reason agile uses story points for estimation; it decouples the estimation process from the work process.

Changing the culture

Here are my suggestions:

  1. Get rid of any mention the words “committed” or “scheduled” WRT work. The team is *planning* what work they will attempt in the next iteration.
  2. Change the “are you going to be done?” interaction to ask, “is there anything you would like to move out to the next iteration?” Expect that initially, people aren’t going to want to take you up on this, and you may have to have some personal interactions to convince somebody that it’s really okay to do this.
  3. When you have somebody decide to move something out, make sure that you take public note of it.

    “To adjust the amount of work to the capacity of the team and maintain shippable quality, the paypal feature has been moved out to the next iteration.”

  4. When teams/individuals check in unfinished features (buggy/not complete) instead of letting those features make it into the common build, force them to remove them or disable them. Make this part of the team-wide status report for the iteration (“the ‘email invoice’ feature did not meet quality standards and has been disabled for this iteration”).
  5. I highly recommend switching to a team-ownership approach. Teams are much more likely to change how they work and improve over time than individuals.

You will need to be patient; people are used to working in the old ways.

This cultural change is the most important thing you can do to reduce the number of bugs that you have.

No Bugs Journey Episode 1: Inventory

February 6, 2015 at 8:06 am

Over the past few years I had the opportunity to work in an environment in which we achieved a significant reduction in bugs and an associated increase in quality and developer satisfaction.

This series will be about how to start the journey from wherever you currently are to a place with fewer bugs, and a bunch of things to think about and perhaps try on your team. Some will be applicable for the individual developer, some will be about an engineering team, some will be for the entire group.

As with all things agile, some techniques will work great for your team, some may require some modification, and some may not work at all. Becoming agile is all about adopting an experimental mindset.

Whether you are on a team awash in bugs or you are already working towards fewer bugs, I hope you will find something of value.


We’re going to start by stepping back, taking stock of the situation, and practicing the “Inspect” part of the agile “Inspect and adapt” cycle. We’ll be using a feedback technique that is often used in agile groups.


The agile software community is very effective at finding useful techniques outside of the field and adopting them, which is great. They are not great at giving credit for where those techniques were created, so there are a lot of “agile” practices that really came from somewhere else. Which is why I wrote “is often used in agile groups” rather than “agile technique”…

The technique we’re going to use is known as “affinity mapping”, which has been around for a long time.


(Note: the following exercise works great as a team/group activity as well. If you want to do it as a group exercise, go find a few videos on affinity mapping and watch them first).

Find some free time and a place where you won’t be disturbed. Get a stack of sticky notes and a pen.

Your task is to write down the issues that are contributing to having lots of bugs, one per sticky note. Write down whatever comes into your head, and don’t spend a lot of time getting it perfect. You can phrase it either as a problem (“lots of build breaks”) or a solution (“set up a continuous integration server”).

Start with what is in the forefront of your mind, and write them down until you run out of things to write. There is no correct number to end up with; some people end up with 4 notes, some people end up with 20.

Now that you have done that, look at this list below, and see if thinking about those areas leads you to get any more ideas. Write them down as well.

  1. Planning
  2. Process tracking and management
  3. Reacting to changes
  4. Interactions between roles (dev, test, management, design, customer)
  5. Recognition – what leads to positive recognition, what leads to negative recognition
  6. Who is respected on the team, and why?
  7. Promotion – what sort of performance leads to promotion
  8. Developer techniques
  9. How is training handled
  10. Who owns getting something done

This second part of the exercise is about thinking different (aka “out of the box”), in looking for non-obvious causes. I’m explicitly having you do it to see if you come up with some of the topics that I’m going to cover in the future.


We now have a big pile of sticky notes. Our next task is to review the notes to see if there is any obvious clustering in them. Put all of the notes up on a wall/whiteboard, and look at them. If you find two notes that are closely related, put them next to each other. If you are doing this as an individuals, you will probably only find one or two clusters; if doing it as a group, you will find more.

Note that we are not doing categorization, where each sticky is in a group. At the end I expect to see a lot of notes that aren’t in clusters. That is expected.


Take a few minutes and look at what you have. My guess is that you have a decent list of issues, and my hope is that you thought of things that you weren’t thinking about before.

At this point, you might be saying, “If I know what the issues are, why shouldn’t I just go off and start working to address them? Why do I need to read what Eric says?”

If you said that, I am heartily in support of the first statement. Determine which sticky/cluster you (or your team) wants to tackle, figure out a small/cheap/low-risk experiment that you can try, and start making things better.

As for my value-add, my plan is for it to add value in two ways:

First, I’ve been through this a few times, and my guess is that you missed a few issues during the exercise. I know that I missed a ton of stuff the first time I tried this.

Second, I’m hoping to give some useful advice around what techniques might be used for a specific issue, common pitfalls, and ordering constraints (you may have to address issue B to have a decent chance of addressing issue A).

Next time

Next time I’m to talk about the incentives that are present in a lot of teams and how they relate to bugs.

No bugs journey episode 2: Stop encouraging your developers to write bugs

It’s about to get real.