We’ve been doing some planning ’round these parts – planning that I unfortunately can’t talk about – but it’s led to a fair amount of discussion about architecture, both inside the team and outside the team.

Which has got me thinking about reuse.

Reuse has been one of the Holy Grails of software development for a long time, along with… Well, work with me, I’m sure there are others. True AI!. That’s another.

Anyway, reuse has been discussed since time immemorial (October 13th, 1953), for some pretty sound reasons:

• Software is hard and expensive to develop


• We already break things apart into “components” to simplify development


• We’ve all written subroutines that are used in multiple places.

It seems that if we did a little more planning, paid a little more attention, were just a little smarter, we could build our components in a more general way, and others could benefit from them.

And yet, people have been trying to do this for a long time, and have mostly failed at it. There are successes – widely-used successes – but they’re fairly small in number. Surprisingly, people are still optimistic about going down the reuse path, and since they are likely to fail anyway, I therefore present some rules that can help them get there faster.

Eric’s N rules for failing at reuse

Authoring reusable components:

• It really helps to have delusions of grandeur and importance. You are going to be the ones who succeed at doing what others have failed at.
  


• Pick a wide/diverse scope. You’re not building a good UI framework for client applications, your framework is going to work for both client and web applications.
  


• Plenty of technical challenge. That’s what makes it fun and keeps you interested.
  


• No immediate clients. You are building a *Component*, and when you are done, clients can use it.

In my experience, that’s more than enough by itself, but it helps if you can throw in some obscure algorithms and quirky coding styles. I’m already assuming that you don’t have any real tests.

Consuming other people’s components:

• You absolutely *must* sign up to work with something as it is being developed. There is simply no better way to waste vast amounts of time. Unfinished implementations block you, regressions set you back, and even if you don’t get those, you at least get hours of refactoring to switch to the new version. It’s best if you do this on a “milestone” basis, roughly every 6 weeks or so. That’s short enough that you’re randomized often, but long enough that waiting for bug fixes is really painful.
  


• Commit to using the other component early and deeply. If somebody asks, “what will you do if <x> is too buggy, too late, or doesn’t do what you need?”, you can’t have an answer.
  


• Make sure that your use of the component is a variant of what the component is really being used for. In other words, the creator of the component is building the thing to do <x>, and you want to do <y>, which is kindof like <x>.
  


• A quick prototype should be enough to ensure that you can do it.
  


• If you can’t get the authoring group to commit to producing what you want, take a snapshot of their code and work on it yourself.
  


• Don’t plan any schedule time or impact to deal with issues that come up.
  


• If possible, buy the component you’re using. Because if you paid for it, the quality will be higher, and they’ll have to give you good support.

I hope these tips help you.

If you’re a bit leery of reuse, then good for you. I have only a few thoughts to offer:

If you’re thinking about doing something, it’s always a build vs buy decision. Even the best general-purpose framework out there is just that – a general-purpose framework. It’s not designed to do exactly what you want to do.

In the abstract, there are three phases of using a component in your code:

Phase 1 is great. The component is doing what you want, and it’s quick and easy to do it. Let’s say for sake of argument that this gets you to the 80% point in your project, and it gets you there quick.

Phase 2 is a harder. You’re starting to reach the limits of the component, and it’s tough to get it to do what you want. Tough enough that it’s taking more time, and you’re using up the time that you saved in phase 1. But you still feel like it was the right decision.

Phase 3 is much harder. It’s taken you as long to get here as a custom-written solution would have taken, and making further progress is considerably slower than if you had written everything. Worse, you can see the point where you’ll reach a wall where you can’t do anything more, and it’s close.

Different projects obviously reach different phases. Some never venture out of phase 1, and others are deep in phase 3. It’s hard to tell where you’ll end up, but if a given component is central to what you do, you are much more likely to end up in phase 3.  

The obvious problem is that prototyping is always done in phase 1, and the rapid progress you make there is oh-so-tempting. The whole application UI is laid out in a week of work using Avalon. I got this demo with moving pictures done in 3 days using XNA. We all want to believe that it’s really going to be that easy.

Stay strong against the lure of the siren song.

As somebody who is interested in Scrum but hasn’t yet had a chance to try it, I’ve been paying attention to the various experiences people are having with it.

I’ve been noticing something for a while, but I didn’t really realize that there was something bigger going on.

I call that phenomena “Scrumbut”. It shows up in the following way:

“We’re doing Scrum but…“

• our sprints are 12 weeks long…


• we do two normal sprints and one bugfix sprint…


• we do all our planning up front…


• we skip the daily meeting…


• our managers decide what’s in each sprint…


• we haven’t read the books yet…


• our team has 30 people…

I’m not a strict methodologist – a specific methodology may need to be adapted to a specific situation. But most of these are anti-scrum rather than modified-scrum.

That this phenomena exists may not be news to you, and it wasn’t to me. But what I realized this last week is that scrumbut has led to another phenomena…

Namely, it has led to scrum being a naughty word. Managers are working with groups that say they are doing scrum, and then when scrumbut doesn’t work, they decide that scrum doesn’t work.

How to approach this? Well, I think you need to advocate specific principles rather than advocating scrum. If you tell your management that you are going to be “ready to release” on a monthly basis and that they get to give feedback on what has been done at what to do next every month, I think you will likely get a better response.

NewsGator – which I use to fetch blog posts – today decided to fetch an August third post for me. At least it’s from this year.

In it, Larry talks about the volume control of a TV going from 0 to 63. And he discusses a possible reason for it, as do many of his readers.

Which brings me to:

Eric’s Law of Arbitrary Limits

If you are designing a piece of tech gear – be it hardware or software – and you need to choose an arbitrary limit or range for something, make sure the limit is either 2^n or 2^n – 1.

The majority of the developers who use your product will convince themselves that you’ve chosen that limit for some well-thought-out reason, such as the internal data structure of your product, rather than just choosing a number because. They will speculate on why you chose that particular number, and you won’t spend your time justifying the limit.

And they will be happy that they understand “the secret” behind why you chose such a strange number…

A while back I came across this post about agile.

I originally was going to write a comment about that post, but I think the commenters have done a good job at that. I will point out that the environment described in the post is a lot like many of the internet boom startups, few of which are around right now.

But in writing some thoughts down, I realized that I was really writing about why I found agile to be interesting. So that’s what this post is about.

I’ve worked on a ton of projects, some short, some long, some small teams, some big teams. But fundamentally, writing software is an exercise in:

• Making progress in the face of imperfect information

and

• Managing change

These shouldn’t be a surprise to anybody. When we start coding, we always have imperfect information. We don’t know if our design will really meet the customer’s needs, we don’t know how long it’s going to take us to write, we don’t know if our architecture is feasible to build, we don’t know if it will perform well enough,  etc. And things will change along the way. Market conditions will change. Customers will modify their priorities. New technology will become available. People will join or leave the team.

There’s a name for trying do something in that sort of environment, and it’s not “engineering”.

It’s research.

Applied research, to be specific.

And doing research requires a different approach than engineering.

Way back in 1961, President Kennedy made his famous “moon speech“.  At that time, the only US astronaut was Alan Shepard, who had flown a 15 minute sub-orbital flight. The ability to send people to the moon seemed out of reach. But the leaders at NASA came up with a plan – they decided on three different programs: Mercury, Gemini, and Apollo. Each program had a theme and a set of goals, and each flight had a set of smaller goals.

Sometimes things worked, sometimes they didn’t. After each flight, they looked at what had happened, decided how to adapt, and went on to the next flight. And over the span of 8 years, they accomplished their goal. All by following the “design a little, build a little, fly a little” approach.

After Apollo, NASA decided to build the shuttle. They did a lot of design up front, made a lot of promises (both schedule and capabilities) and then started building. They ran into lots of difficulties with the main engines (ultimately solved), and the thermal protection system (not really solved). Ultimately, they finished the vehicle, but it took years longer than they expected and ultimately didn’t do what they had designed it to do or what they needed it to do. And it has very high maintenance costs.

The analogy to software development projects should be obvious. Shuttle is like most big software projects – lots of planning up front, lots of promises, followed by delays as problems are addressed, and then the ultimate release of something that doesn’t measure up to the original idea. We convince ourselves that “this time it will be different”, despite the fact that the same thing has happened every time we take this approach.

Incremental development accepts the realities of the environment, and takes a different approach. You know that you don’t know enough to finish the project and you know that things are going to change, so you bite off a manageable portion – say a month’s worth of work – do enough design, write some code, and then see how well it works.

And after that month, you modify your plan based on what you learned.

Design a little
Code a little
Fly a little

That’s what agile means to me.

Yesterday, I had the chance to go on the Kitsap Kolor Klassic (really, the Kitsap Color Classic), but I didn’t.

Excuses were proffered and accepted. I was tired from all the rides I’d done. I needed to work on my deck. I had family committments. All were good.

But in truth, I just didn’t feel like a long ride. So, I went out on a short ride (about 30 miles), and 12 miles into it, my bike converted itself from a 30-speed to a 2-speed through a break in my rear shifter cable (right at the shift lever). And since you can’t run a chain from the small chainwheel to the small sprocket in the back without a lot of mechanical pain (and risk of chain break), neither of the gears I had were of much use.

A quick look around showed that Richard Dean Anderson was nowhere in sight, so I jury-rigged the shift cable. Here’s what you do:

1. Shift the bike onto the largest sprocket by pushing the derailuer by hand (watch out for the point parts)
2. Bend the cable in half sharply in the middle of the exposed section on the downtube.
3. Tie the bent cable into a single knot. This gives you a loop tied into the cable.
4. Loop the remaining cable around the front attachment for the cable (where the cable comes out of the sheath from the shifter), back through the loop, and pull it tight.
5. Wrap the cable around itself, put it back through the loop, and arrange it so it won’t perform minor surgery on your leg as you pedal.
6. Ride. You may have to readjust it as you go.

Happy was I that this didn’t happen in the middle of a long supported ride. My laziness saved the day!

Today, I rode the Mountain Populaire 100K, put on by the Seattle International Randonneurs.

Randonneuring is a long-distance cycling discipline that originated in France (hence the name) way back in the 1800s. It’s organized around a series of rides known as “brevets”, which are pronounced exactly the way you would expect if you speak French. The goal is to finish a ride of a specific distance within a specific time limit. For example, the 200 km brevet typically has a an overall time to finish of 13:30, the 300 km a time limit of 20:00, and so on – all the way up to 75 hours for a 1000 km ride.

Given the length of most of the brevets, some clubs host “populaires”, which are shorter events for new riders for “introducing new riders to the ways of randonneuring”.

These rides are different from most organized rides in the following ways:

• They are not supported. There are a few people who follow the group around, but there is no sag wagon, no mechanics, no food.
• There are no dan henry’s – you need to follow the route sheet.
• There are controls along the route. Some are manned, where somebody will sign your sheet. Some are unmanned, where you’ll need to answer a question to prove that you were there.
• They’re typically much smaller.

Instead of doing a typical introduction, the folks at SIR decided to host a “Mountain Populaire”. Instead of doing a typical course, it’s a course with as much climbing as possible. (note that I’m assuming that SIR is different in this regard – it may be that all Populaires are like this).

In this case, the course packs 5480 feet of climbing into 110 km. The climbs are:

• Zoo hill (short version)
• 164th
• Tiger Mountain (from the South)
• Highland Drive
• Tolt Hill
• Duthie Hill
• Mountain Park Blvd

There is a claimed 8th hill, but I don’t recall exactly where.

So, how does this compare to the Summits of Bothell or 7 Hills? Well, 7 Hills has a lot of climbing, but only seminary hill and winery hill are really challenging. Summits of Bothell has a lot of steep climbs, but most of them aren’t very long. And they’re both in the 40ish mile range.

This ride is 69 miles, and while it does have a couple of fairly easy climbs – 164th and Tiger mountain – it starts out with a 1000′ climb, and then finishes with a 700′ climb, both of which have slopes in excess of 15%. My legs were certainly tired when I got to Mountain Park, but I had to tack back and forth to make it to the top (I was not the only one).

Definitely the hardest ride I’ve been on, and a nice way to end the season. Beautiful day, and a nice group to ride with.

Recommended

Yellow Sticky Exercise

Take one pack yellow stickies (aka “Post It” brand sticky paper notes). Place them strategically on your hands and arms, and wave them around for 10 minutes.

Wait… That’s the wrong version.

The yellow sticky exercise is a tool that is used to collect feedback from a group in a manner that encourages everybody to give feedback, doesn’t waste time, and lets people give the feedback (mostly) anonymously.

Microsoft has a tradition of doing “Post Mortems” on our projects, which are designed to figure out what went wrong, what should be done about it, and assign an owner. What typically happen is the group complains for an hour, three people dominate the conversation, several rat holes are encountered, a few things get written down as action items, and nothing ever happens with the results.

The yellow sticky exercise is an alternative. It works well whenever you want to figure out the combined opinion of the group. It was taught to me be a very sharp usability engineer.

Get everybody together in a room. Each person gets a pad of stickies and a pen. Both pens and stickies should be the same color, so things are anonymous.

In the first segment, each person writes down as many issues/problems as they can, one per sticky note. It’s useful to tell people ahead of time so they can come with lists ahead of time, but that’s not necessary. This generally takes about 10 minutes, and you continue until most of the people have run out of things to write down.

Ground rules for issues:

1. They can’t be personal (ie no names on them)


2. They should be specific


3. They should have enough detail so that anybody in the room could figure out what they meant.

If you’re doing this to collect the problems you have, it’s good to ask people not to put down anything personal, and to include enough data so that

At this point you have a lot of diverse feedback items, and you need to get them placed in groups. That is done by the group. You ask the group to put the stickies up on your wall/whiteboard in meaningful groups, and just let them go at it. When there’s a group of 5 or more stickies, ask somebody in the group to come up with a label for the group and write it on the whiteboard or on a sticky. You should also ask people to move stickies around if they belong in a different group.

When everything is on the wall and people are happy with the groups, you’re done with the group. Somebody will need to stick the stickies on paper and own typing them all up.

If you do this, I’m confident that the breadth and the depth of the feedback will be much better than other methods.

Back in 1996, I was a QA lead for the C++ compiler, and our group wanted to incent people to fix and close bugs.

One of the other QA leads had the brilliant insight that lego blocks make excellent currency amongst development teams, and I – because of my demonstrated aptitude in generating reports from our bug-tracking system – became the “Lego Sheriff” for the group, handing out blocks. I believe the going rate was three blocks per bug.

Not surprisingly, some people started to game the system, to increase the number of blocks. Those of you who are surprised that somebody would go to extra effort to get blocks that retail at about a penny per block have never seen a millionaire fight for to get a free $10 T Shirt.

But I digress.

That there was a system to game was due to a very simple fact. Our goal wasn’t really to get people to fix and close bugs, our goal was to get the product closer to shipping. But we didn’t have a good way to measure the individual contribution to that, so we choose active and resolved bug counts as a surrogate measure – a measure that (we hoped) was well correlated with the actual measure.

This was a pretty harmless example, but I’ve seen lots of them in my time at Microsoft.

The first one I encountered was “bugs per tester per week”. A lead in charge of testing part of the UI of visual studio ranked his reports on the number of bugs they entered per week, and if you didn’t have at least <n> (where <n> was something like 3 or 5), you were told that you had to do better.

You’ve probably figured out what happened. Nobody ever dropped below the level of <x> bugs per week, and the lead was happy that his team was working well.

The reality of the situation was that the testers were spending time looking for trivial bugs to keep their counts high, rather than digging for the harder-to-find but more important bugs that were in there. They were also keeping a few bugs “in the queue” by writing them down but not entering them, so they could make sure they hit their limit.

Both of those behaviors had a negative impact, but the lead liked the system, so it stayed.

Another time I hit this was when we were starting the community effort in devdiv. We were tracked for a couple of months for things like “newsgroup post age”, “number of unanswered posts”, or “number of posts replied to by person <x>”.

Those are horrible measures. Some newsgroups have tons of off-topic messages that you wouldn’t want to answer. Some have great MVPs working them that answer so fast you can’t a lot to say. Some have low traffic so there really aren’t that many issues to address.

Luckily, sharper heads prevailed, and we stopped collecting that data. The sad part is that this is one situation where you *can* measure the real measure directly – if you have a customer interaction, you can *ask* the customer at the end of the interaction how it went. You don’t *need* a surrogate.

I’ve also seen this applied to blogging. Things like number of hits, number of comments, things like that. Just today somebody on our internal bloggers alias was asking for ways to measure “the goodness” of blogs.

But there aren’t any. Good blogs are good blogs because people like to read them – they find utility in them.

After this most recent incident of this phenomena presented itself, I was musing over why this is such a common problem at Microsoft. And I rememberd SMART.

SMART is the acronym that you use to remember the measures that tell you that you’ve come up with a good measure. M means measurable (at least for the purposes of this post. I might be wrong, and in fact I’ve forgotten what all the other letters mean, though I think T might mean Timely. Or perhaps Terrible…).

So, if you’re going to have a “SMART goal”, it needs to be *measurable*, regardless of whether what you’re trying to do is measurable.

So, what happens is you pick a surrogate, and that’s what you measure. And, in a lot of cases, you forget that it’s a surrogate and people start managing to the surrogate, and you get the result that you deserve rather than the one you want.

If you can measure something for real, that’s great. If you have to use a surrogate, try to be very up-front about it, track how well it’s working, don’t compare people with it, and please, please, please, don’t base their review on it.

Yesterday I rode the Summits of Bothell. Like its more popular cousin to the south, S (I’m not a big fan of “cute” acronyms so I’m not going to use it…) is a ride focused around hills, originally 7 but now 8 in a bid to outdo the competition.

Their other bid to outdo the competition is in steepness. The advertisement sheet says two very troubling things. The first is an offhand comment about equipment:

Triple cranks and <24″ low gear highly recommended as well as beefy brakes

24″ refers to “gear inches”, a measure of how far your bike moves forward for each pedal revolution. An average “riding on the flat” gearing for me is something like a 42-17 (teeth on the front/teeth on the rear), which is about 67 gear inches. My lowest gear (with my 12-27 rear cassette), is 30 gear inches. 24 gear inches is very low.

The second comment is a list of the hills:

Ascents – 14% (2), 16% (3), 18% (1).

Though I run a site for bicycle climbs, I am not a climbing specialist. But I have done some steep stuff. 10% is steep. 15% is painfully steep, and above that is just crazy.

But I’ve survived the Zoo multiple times, so I know what it takes to get up those kinds of hills. And I’m doing the Mountain 100 km Populaire in September, which includes the zoo (and other such climbs), so I need the practice at the steeps.

In other words, my strategy is sound.

My tactical decisions, however, are fairly suspect. I’m riding with a few friends, but what I’ve forgotten is that rides like this are self-selecting – the people that show up are the ones that can *climb*. I have at least 25 pounds on all of these guys, and at 170 pounds on my 6’2″ frame, I’m not carrying a lot of weight.

Franklin doesn’t ride that much more than me, but is strong on the flats and fast up the hills. He “went on a long run” yesterday.

Steve is riding his steel Colnago. Steve is scary fit – on our group rides, he’ll just be talking to you at the bottom of a climb near the back of the group, and then he’ll just ride past everybody on the ascent. He “rode 120 miles” yesterday. That is not a good sign, it’s a sign that I’m out of my league. Anybody who rides 120 miles on Saturday and then shows up for a pain-fest on Sunday is to be watched.

And finally, we have Joe. Joe has focused in on the ride guarantee – “If not completely satisfied, you can ride the course a second time for FREE!”

He is planning on taking them up on that.

The start is very low key. A sign-in table, where you get a map (this is the 2005 map, and doesn’t show the 8th hill up Hollyhills drive) and a T-shirt (included in the $25 fee). Some water and sports drink, and that’s about it. And there are only about 20 people milling around. A nice change from the 3000 people that do flying wheels and the 9000 that do STP.

We head out to the first hill (Hollyhills), and ride up it. I start slow at the bottom, but stay about 10 seconds behind the group. It’s fairly easy – 7%-8%, and I can compete on those grades. The group crests the top, circles once as I crest, and we descend down. One down.

The second hill (Bloomberg) is a different story. It starts at around 10%, the group gaps me, and then it kicks up to about 13%, and the gap grows. And then it gets steep – 16% (ish) for the last half block.

At this point, I part ways with the group. This is a very good thing – I killed myself on the southern cousin by trying to stay with this same group.

The next three hills – Beckstrom, Nike, and Westhill – blend together. They’re steep, I ride up them trying to keep my HR in the low 150s. The day is perfect, and I talk to the very few riders I run into as we ride between hills.

Which takes us to Finn Hill. I’ve climbed this hill a few times, but never in this direction. This hills is probably the steepest one of the ride. I ride slow (around 4MPH), but manage to get up it in one piece without having to tack back and forth across the hill. This is the only place I see people walking.

Norway hill is next, but it’s Norway from the South side, the easy way up, then Brickyard Road, and back to the finish, my finish-line ice cream, and my Costco cookie.

I don’t have my full stats, but I do know my average speed was 13.4 MPH. There are no real flats on this ride, and there are a lot of stop signs and traffic lights, so don’t expect to be pace-lining it.

Discounting the considerable amount of pain involved in the climbs, this was a very enjoyable ride – I liked it more than its smaller cousin. Very low key – the rest stops were very tiny but with great volunteers, cold water and the right food. And they had some cookies at the end of the ride, when you really want something solid. The course markings were all great.

I do have one small complaint with the signage. When I’m climbing, it’s nice to know how to mashall my effort, and to do that I need to know when the pain will end. On several of the climbs, you’ll finish the really steep section, and then ride at least half a mile on gentle slopes or rolling terrain to get to the “summit”. Delaying the signage until that point diminishes the feeling of accomplishment at finishing the steep section, and makes you think there’s more steep. I think Norway is the only exception, where the climb finishes right at the top.

It also makes the stats seem weird. Bloomberg hill is 440′ high at the true summit, but the climb distance is perhaps 7000 feet, giving it a gradient of only about 6%. But the section on 240th gains 230 feet in 1770 feet, putting it right at 13%, and that’s the average for that section, not the max.

It would be nice to have an indication of the steeps on the map (perhaps with some beads of sweat on the route), and a sign that says, “Pain lessens” at the end of each steep section.

I was going to suggest that they get a real website, but that would encourage more riders to participate…

Dare wrote a post talking about the advisability of making developers do operations.

Which is really part of a philosophical question…

When you’re setting up a software organization, how much specialization should you have, and where you should you draw the lines around the responsibilities of the various groups?

Some orgs take a very generalized view of what people own, and others take a very specialized view. I’ve worked in both sorts of environments.

I’ve worked for a startup where, as a developer, I wrote the code, tested the code, built the code, made tapes to ship out to customers, and answered customer support calls.

And I’ve worked in other organizations where the job of developer was to implement what was written down in the spec and pass it off to the QA org. Those orgs typically had structures and policies designed to insulate the developers, so they wouldn’t be distracted.

That eliminated a bunch of the outside noise that they would otherwise have to deal with, and make them more efficient at getting their development work done.

And how did those efficient organizations fare in their products?

Not very well.

They were reasonably good at shipping software, but their software didn’t turn out to be very good for users. New updates didn’t address issues that users had been hitting. New features were hard to use and/or didn’t hit the sweet spot. They answered questions that users didn’t ask.

All of this was because the developers were out of touch with people who had to deal with their software. They didn’t feel the pain that the users were experiencing setting up their software. They didn’t feel the pain when a bug in the software meant that the user’s business was loosing money. And they didn’t understand why users were having trouble using features that seemed obvious to them.

All that happened in DevDiv, and the issues showed up in our customer satisfaction numbers. So, it was decided to let developers (and the testers, and PMs…) talk directly with customers.

There was a fair amount of angst around this decision. It would take up too much dev time. Developers would insult customers. Customers didn’t know enough to give good feedback.

But it turned out that all of those things were wrong. The developers liked to solve problems, and they also liked to help people. They remotely debugged customer issues on other continents. And they listened very closely to the detailed feedback customers gave about how the current software didn’t meet business needs and what was good and bad about future plans.

And the organization adapted what they were planning, so that it addressed the areas that needed addressing.

Distraction is not the enemy. Pain is not the enemy. Pain is to be embraced, because only through feeling pain are you motivated to make it go away.