Adding an overdue link.

I don’t think I’ve linked to Chris’ blog before. I worked with Chris in my early VC++ days when we were both on the VC++ QA team.

Chris is a developer now, and has been writing about his experiences with SCRUM (not scrum), and integrating it into the larger process in his division.

A question related to performance came up today on a mailing list that I’m on, and I stated that the right way to do things was to do performance work based on measurements (ie don’t engage in speculative optimization).

Another list member wrote back and said, “I understand what you’re saying, but isn’t that at odds with what many books say about designing for performance?”

I wrote an answer, and decided that it would be a good thing to share. This is what I think about writing performant code -what do you think?

*****

Yes, those two perspectives (design up front vs measure and tune) are somewhat at odds with each.

To do good performance optimization requires that you understand the performance bottlenecks of the system. There are two ways to understand that: intuition, or measurement.

A considerable amount of performance work is done on intuition. The problem with intuition is that it turns out to be a poor predictor – the performance bottleneck in most systems ends up being somewhere different than where you expect it to be. There may be exceptions to this – consultants who repeatedly implement a similar design for different customers, perhaps – but for most software, you just don’t have a very good idea where the bottleneck is going to be.

While performance work is often enjoyable, it tends to be fairly time consuming. If you optimize an area that didn’t need to be optimized (ie your image shows up in 0.05 seconds instead of 0.07 seconds), you are spending time that you could be using to implement/polish other features, work on the performance of areas that do matter, or finish early. And you’re generally creating code that is harder to read and maintain.

So, that leaves measurement. The important dictum here is “measure early, measure often”. “Measure early” may even mean writing some prototype code to validate some overall assumptions about the system – is it possible to pull data from the database quickly enough over the existing network to support what you need to do? How fast can DirectX render a frame?

Many software projects have time devoted to “performance tuning”. This is a “good thing” if you spend the time on an ongoing basis, measuring and improving as necessary. It’s a “bad thing” if you finish your implementation and then start looking at performance, as the kinds of changes that improving performance requires are generally the kinds that you don’t want to be making in the endgame. I’ve seen lots of examples where you get to a point where you a) understand the perf issue and b) understand how to fix it, but can’t because of where you are in the development cycle.

Using agile methods can help. If you have good unit tests, performance refactorings are less risky, and if you run on a short cycle, it’s harder to put things off. But you need to develop a “performance culture” in the team so that they care about performance all the time.

Hope that makes sense.

Rory wrote a post entitled “Whole Brain Coding” a couple of days ago, in which he asserts that coding requires both the left and right halves of the brain, the left brain working on the sequential and analytical parts of the task, and the right brain working on the intuitive and holistic parts (reverse these if you live in the southern hemisphere…)

When things are going well and you’re in the “flow“, my guess is that you’re seeing involvement of both sides of the brain, but I’m not sure that that’s all there is to it (I’m not asserting that Rory said that). I did a few searches to try to see what research had been done into the “flow”, but didn’t come up with much. There is:

In the Zone: A Bio-Behavioristic Analysis of Csikszentmihalyi’s Flow Experience

but I have a hard time parsing sentences like:

Primarily, the decision making process behind such behaviors as disparate as creative thinking, problem solving, or walking to the store are all dependent upon and influenced by somatic or neural activation variables that are mediated by abstract environmental contingencies.

I think that’s saying, “The way we make decisions is dependent on what’s going on around us”, which makes me happy that I’m not a psychologist who has to read and write papers like that.

There’s also

Understanding the Psychology of Programming

which is a light intro to the topic.

On the whole math vs. coding thing, though I have a math minor and enjoyed my math classes up through linear algebra and multivariable classes, I ended up in software for two reasons:

1. There’s more opportunity in it
2. Coding is way easier than math for me

Cyrus takes on Hungarian Notation

I’ve written a fair bit of code that both uses Hungarian, and code that doesn’t use Hungarian. I think Hungarian works okay for C code, but when you get into the object oriented world, you can’t really come up with prefixes that are both meaningful and short, so I currently prefer the .NET style of naming.

I had a discussion at lunch yesterday about the right way to do design.

Waaaaaay back when I was in school – when Van Halen’s “Jump” was at the top of the charts – we were introduced to the Waterfall Model of software development. (aside – it Royce’s model was iterative, but it was rarely discussed that way – the typical discussion broke the whole project down into phases).

Anyway, the waterfall model (and some other models as well) have distinct phases. In waterfall, you first collect requirements, then you design, then you implement, etc.

I’ve never found that to be very workable. As Field Marshal Helmuth von Moltke said, “No plan survives contact with the enemy” (actually, what he said was, “No operation plan extends with any certainty beyond the first encounter with the main body of the enemy” (though he likely said something like “Kein Betrieb Plan verlängert mit jeder möglicher Sicherheit über dem ersten Treffen mit dem Hauptkörper des Feindes hinaus” (or at least that’s what he would have said if he spoke English and used Babelfish…)))

That doesn’t mean that you should have a design, it just means that you should expect your design to change along the way as you learn more. But without spending time on design, you are likely to make some errors of architecture that will be difficult or costly to fix.

And just going off and starting coding would not have been tolerated by Tony Jongejan, my high school programming teacher.

At least that’s what I used to think. These days, I’m not so sure, for most applications (see caveats later).

The problem is that, unless you’ve already built such a program (some of which is codified in the prototyping and “build one to throw away” schools of thought), you rarely have enough information to make informed choices, and you won’t have that data until you actually write the code.

I think that you’re far better off if you focus on building clean and well-architected (at the class level) code that has good unit tests, so that you’ll have the ability to adapt as you learn more. It’s all about the resiliency of the source to the change that is going to come, and letting the code evolve to do what it needs to do.

Caveats? Well, there are a few.

If you’re in the library business, you will at some point have to put a stake in the ground and ship the thing, which will severely constrain further modifications. Your library may also be used in ways you didn’t imagine.

I think the right way to solve that is with a little design up front, a lot of prototype use, and then a good review process before you ship it.

So what do you think? What amount of design is right before coding?

I’ve been looking through a few of my “Improve your Dev Skills” books:

• Refactoring
• The Pragmatic Programmer
• Test Driven Development in .NET

What are your favorite books in this genre? Why?

Jay wrote a post entitled Properties? Not my bag, baby.

When I first started writing C# code, I used properties for everything. But recently, I’ve felt that I was wasting a lot of time writing trivial properties. Yes, I know that in Whidbey I’ll be able to use expansions to write them easily, but that still means that I have to deal with the property bodies cluttering up my code.

So, that got me thinking about whether it makes sense to be writing properties in the first place. After a bit of thought, here’s my current position:

Properties are a great thing for component libraries. There are certainly cases where you would want the future-proofing and decoupling that properties gives you.

But when you’re working on a single project that gets built all at once, I don’t think you’re getting any future-proofing benefits, and you have to pay the “property tax” the whole time.

This may be heretical, since “use properties” has been the common guideline.

What do you think?

Josh writes:

I’ve been working professionally with C# for about 2 years now and am just blown away by the things I can do with it. Lately though, I’ve felt like I’ve plateaud at this beginner to intermediate level. Every book I look at is either “Hello, World” or beyond my level of expertise, I feel like I’m always one step behind the game and my code is becoming stale.

That’s a very interesting question. I’m not longer a professional programmer (though I was for a little over a decade), so you’ll also want to listen to the comments that others are sure to write.

If you haven’t read them, there are a couple of books that I’d definitely recommend.

Code Complete
The Pragmatic Programmer

Both of these are currently on my bookshelf (actually, they would be if somebody hadn’t walked off with my copy of Code Complete).

To expose yourself to some new ideas, I’d also suggest:

Refactoring: Improving the Design of Existing Code
Extreme Programming Adventures in C#
Test-Driven Development in Microsoft .NET

Whether you agree with these approaches or not, they are certainly thought-provoking.

The best programmers that I’ve known always have a few side projects going on, where they’re exploring new areas, new environments, etc. If you’ve never done network programming, try writing some socket code, or play around with remoting, or DirectX. Play around with tools, like a profiler, or measure the performance of different options. Get a book on .NET IL and spend some time under the covers (Richter’s book is a good guide here).

Finally – and this may seem like weird advice from a C# guy – spend some time writing some code in a language such as Perl. Scripting languages lead to a different mindset that can be useful in language.

I’ve just finished doing a security review of my MSDN columns, and as part of that,
I needed to add some security to an add-in architecture, so the add-in architecture
won’t run antisocial code. The approach taken by .NET security is reminiscent of ACLs,
which means (at least to me) that it’s confusing and I need to read the code 5 or
6 times before I understand it. Along the way, I got a chunk of code from somebody
on the .NET security team, which I’ll reproduce here:

private NamedPermissionSet FindNamedPermissionSet(string name)
{
	IEnumerator policyEnumerator = SecurityManager.PolicyHierarchy();

	while (policyEnumerator.MoveNext())
	{
		PolicyLevel currentLevel = (PolicyLevel)policyEnumerator.Current;

		if (currentLevel.Label == "Machine")
		{
			IList namedPermissions = currentLevel.NamedPermissionSets;
			IEnumerator namedPermission = namedPermissions.GetEnumerator();

			while (namedPermission.MoveNext())
			{
				if (((NamedPermissionSet)namedPermission.Current).Name == name)
				{
					return ((NamedPermissionSet)namedPermission.Current);
				}
			}
		}
	}
	return null;
}
    

Ugly, isn’t it?

The root of the problem is that PolicyHierarchy returns an IEnumerator instead of
an object that supports IENumerable.  This means that I can’t use foreach on
it, and have to write the traversal myself. This decision also means that, without
a collection returned, there’s no place to hang a useful method like “LookupLabel()”
on, so all users are stuck writing the search code themselves (or encapsulating it
off into a separate method).

PolicyLevel.NamedPermissionSets uses a different approach, that of returning an IList,
which is better than an enumerator, as I could use foreach on the IList, but the person
who wrote this sample didn’t because they didn’t know about foreach. If they had,
they would have done the right thing.  

We need to not do this.