I got some great comments on the post, and I answered a few in comments but one started to get very long-winded so I decided to convert my response into a post.

Integration tests before refactoring

The first question is around whether it would be a good idea to write an integration test around code before refactoring.

I hate integration tests. It may be the kinds of teams that I’ve been on, but in the majority of cases, they:

1. Were very expensive to write and maintain
2. Took a long time to run
3. Broke often for hard-to-determine reasons (sometimes randomly)
4. Didn’t provide useful coverage for the underlying features.

Typically, the number of issues they found was not worth the amount of time we spent waiting for them to run, much less the cost of creating and maintaining them.

There are a few cases where I think integration tests are justified:

1. If you are doing something like ATDD or BDD, you are probably writing integration tests. I generally like those, though it’s possible they could get out of hand as well.
2. You have a need to maintain conformance to a specification or to previous behavior. You probably need integration tests for this, and you’re just going to have to pay the tax to create and maintain them.
3. You are working in code that is scary.

"Scary" means something very specific to me. It’s not about the risk of breaking something during modification, it’s about the risk of breaking something in a way that isn’t immediately obvious.

There are times when the risk is significant and I do write some sort of pinning tests, but in most cases the risk does not justify the investment. I am willing to put up with a few hiccups along the way if it avoids a whole lot of time spent writing tests.

I’ll also note that to get the benefit out of these tests, I have to cover all the test cases that are important. The kinds of things that I might break during refactoring are the same kind of things I might forget to test. Doing well at this makes a test even more expensive.

In the case in the post, the code is pretty simple and it seemed unlikely that we could break it in non-obvious way, so I didn’t invest the time in an integration test, which in this case would have been really expensive to write.  And the majority of the changes were done automatically using Resharper refactorings that I trust to be correct.

Preserving the interface while making it testable

This is a very interesting question. Is it important to preserve the class interface when making a class testable, or should you feel free to change it? In this case, the question is whether I should pull the creation of the LegacyService instance out of the method and pass it in through the constructor, or instead use another technique that would allow me to create either a production or test instance as necessary.

Let me relate a story…

A few years ago, I led a team that was responsible for taking an authoring tool and extending it. The initial part had been done fairly quickly and wasn’t very well designed, and it had only a handful of tests.

One day, I was looking at a class, trying to figure out how it worked, because the constructor parameters didn’t seem sufficient to do what it needed to do. So, I started digging and exploring, and I found that it was using a global reference to an uber-singleton that give it access to 5 other global singletons, and it was using these singletons to get its work done. Think of it as hand-coded DI.

I felt betrayed and outraged. The constructor had *lied* to me, it wasn’t honest about its dependencies.

And that started a time-consuming refactoring where I pulled out all the references to the singletons and converted them to parameters. Once I got there, I could now see how the classes really worked and figure out how to simplify them.

I prefer my code to be honest. In fact, I *need* it to be honest. Remember that my premise is that in TDD, the difficulty of writing tests exerts design pressure and that in response to that design pressure, I will refactor the code to be easier to test and that aligns well with "better overall". So I am hugely in preference of code that makes dependencies explicit, both because it is more honest (and therefore easier to reason about), and because it’s messy and ugly and that means I’m more likely to convert it to something that is less messy and ugly.

Or, to put it another way, preserving interfaces is a non-goal for me. I prefer honest messiness over fake tidiness.

Note: I am focusing only on the design impact of TDD. To better understand the overall impact, see this series of posts by Jay Bazuzi.

My first experience with TDD was back in 2002 or so, and it was in C++, so there weren’t any mocking libraries available. That meant that I had to use hand-written mocking classes.

When hand-mocking, you need to create separate classes, write each of the methods that you need, etc. If the scenario is complex, you may have to write several classes that coordinate with each other to accomplish the mock. It’s more than a little pain at times, and creating a new class always seems like a bit of an interrupt in my train of thought.

That is as it should be. One of my foundational principles of running development teams is that pain – which, in this context, means, "tedious things I have to do instead of writing new code" – is a great incentive. That which is tedious is an automatic target for reduction and elimination. So, you have developers fix their own bugs because it will cause them pain to do so.

(This is, of course, not a panacea; there are plenty of organizations where this will not result in better quality because the incentive towards writing bugs is so strong, but I digress…)

Then mocking libraries showed up the scene. No need to write new classes, just write some mocking code and you’re done. That reduced the pain and reduced the design pressure that ‘the test is hard to write" was exerting, which reduces the improvement.

Then a few mocking libraries showed up that let you mock statics, which allows you to test things that were totally untestable before, and that further reduced the design pressure. You can do some pretty ugly things with those libraries…

Refactoring gets harder

Many of the mocking libraries have another problem – they use a string-based approach for defining their mocks. This means that they are not refactoring-friendly; after you refactor your code, you find out that your tests won’t even run; you have to go and hand-modify them so that they now match your new refactoring. This makes refactoring more painful, and makes it more likely that you will just skip the refactoring

Discussion

As you have probably gathered, I am not a fan of mocking libraries. They make it too easy to do things that I think you shouldn’t, and they short circuit the feedback that you would otherwise be feeling. Embrace the pain of writing your own mocks, and use that to motivate you towards better solutions. I’ll be talking more about better solutions in future posts.

There is one situation where mocking libraries are great; if I need to bring an existing codebase under test so that I won’t break it as I work on it. In that case, I need their power, and I will plan to get rid of them in the longer term.

I’ve been paying attention to TDD for the past few years – doing it myself, watching others doing it, reading about it, etc. – and I’ve been seeing a lot of variation in the level of success people are having with it. As is my usual approach, I wrote a long and tedious post about it, which I have mercifully decided not to inflict on you.

Instead, I’m going to do a series of posts about the things I’ve seen getting in the way of TDD success. And, in case it isn’t obvious, I’ve engaged in the majority of the things that I’m going to be writing about, so, in the past, I sucked at TDD, and I’m sure I haven’t totally fixed things, so I still suck at it now.

Welcome to "You suck at TDD"…

Rewrite the steps

The whole point of TDD is that following the process exerts design pressure on your code so that you will refactor to make it better (1). More specifically, it uses the difficulty in writing simple test code as a proxy for the design quality of the code that is being tested.

Let’s walk through the TDD steps:

1. Write a test that fails

2. Make the test pass

3. Refactor

How does this usually play out? Typically, we dive directly into writing the test, partly because we want to skip the silly test part and get onto the real work of writing the product code, and partly because TDD tells us to do the simplest thing that could possible work. Writing the test is a formality, and we don’t put a lot of thought into it.

The only time this is not true is when it’s not apparent how we can actually write the test. If, for example, a dependency is created inside a class, we need to do something to be able to inject that dependency, and that usually means some refactoring in the product code.

Now that we have the test written, we make the test pass, and then it’s time to refactor, so we look at the code, make some improvements, and then repeat the process.

And we’re doing TDD, right?

Well…. Not really. As I said, you suck at TDD…

Let’s go back to what I wrote at the beginning of the section. I said that the point of TDD was that the state of our test code (difficult to write/ugly/etc) forced us to improve our product code. To succeed in that, that means that our test code has to either be drop-dead-simple (setup/test/assert in three lines) or it needs to be evolving to be simpler as we go. With the exception of the cases where we can’t write a test, our tests typically are static. I see this all the time. 

Let’s try a thought experiment. I want you to channel your mindset when you are doing TDD. You have just finished making the test pass, and you are starting the refactor set. What are you thinking about? What are you looking at?

Well, for me, I am focused on the product code that I just wrote, and I have the source staring me in the face. So, when I think of refactoring, I think about things that I might do to the product code. But that doesn’t help my goal, which is to focus on what the test code is telling me, because it is the proxy for whether my product code is any good.

This is where the three-step process of TDD falls down; it’s really easy to miss the fact that you should be focusing on the test code and looking for refactorings *there*. I’m not going to say that you should ignore product code refactorings, but I am saying that the test ones are much more important.

How can we change things? Well, I tried a couple of rewrites of the steps. The first is:

1. Write a test that fails

2. Make the test pass

3. Refactor code

4. Refactor test

Making the code/test split explicit is a good thing as it can remind us to focus on the tests. You can also rotate this around so that "refactor tests" is step #1 if you like. This was an improvement for me, but I was still in "product mindset" for step 4 and it didn’t work that great. So, I tried something different:

1. Write a test that fails

2. Refactor tests

3. Make the test pass

4. Refactor code

Now, we’re writing the test that fails, and then immediately stopping to evaluate what that test is telling us. We are looking at the test code and explicitly thinking about whether it needs to improve. That is a "good thing".

But… There’s a problem with this flow. The problem is that we’re going to be doing our test refactoring while we have a failing test in our test suite, which makes the refactoring a bit harder as the endpoint isn’t "all green", it’s "all green except for the new test".

How about this:

1. Write a test that fails

2. Disable the newly failed assertion

3. Refactor tests

4. Re-enable the previously failing assertion

5. Make the test pass

6. Refactor code

That is better, as we now know when we finish our test refactoring that we didn’t break any existing tests.

My experience is that if you think of TDD in terms of these steps, it will help put the focus where it belongs – on the tests. Though I will admit that for simple refactorings, I often skip disabling the failing test, since it’s a bit quicker and it’s a tiny bit easier to remember where I was after the refactoring.