Michael's Technical Blog

Dependency Injection: The Testability Enabler

2010-07-12T13:18:00.007-05:00

Whenever I talk to a team that wants to do unit testing, the inevitable counterarguments always revolve around testability of existing legacy code. Michael Feathers' excellent book on the topic of testing existing code gives a good argument to any counterarguments to unit testing, but I can sum it all up in one post: you can test just about anything with one simple pattern: Dependency Injection.

When people aren't designing for testability, they often create designs like this:


public class Program
{
  public void UpdateSalaries()
  {
    Database d = new Database("connection-string");
    d.ExecuteSql("UPDATE Employee SET Salary = 1000000 where LastName = 'Hedgpeth');
  }
}

So they come to my meeting talking about testability and say to themselves, "There's no way that can be tested." That's not true with Dependency Injection.

With dependency injection, you simply take the responsibility of creating the database object away from the object you are testing. Then, in the test environment, you pass in a fake that you control. So our test becomes:


[Test]
public void UpdateSalaries_ShouldPayMeAMillionDollars()
{
  var database = new Mock();
  Program program = new Program();
  program.UpdateSalaries(database.Object);

  database.Verify(d => d.ExecuteSql(Program.MakeMySalaryAMilionDollarsSqlText);
}

And the code to test:


public class Program
{
  public const string MakeMySalaryAMillionDollarsSqlText = "UPDATE Employee SET Salary = 1000000 where LastName = 'Hedgpeth'";

  public void UpdateSalaries(IDatabase database)
  {
    database.ExecuteSql(MakeMySalaryAMillionDollarsSqlText);
  }
}

You really have little excuse to not test when you know the dependency injection pattern. While there are more complicated patterns for creating a testable design, I've found that this pattern is all you need a vast majority of the time.

Another attempt at being cool

2009-09-10T06:31:00.002-05:00

Today I'm going to try out a black theme in Visual Studio. Maybe I can be cool like those hipster Ruby people.

NHibernate collection element has wrong number of columns

2009-09-09T06:11:00.005-05:00

I recently got this error with NHibernate:


NHibernate.MappingException: collection element mapping has wrong number of columns: ClassName.PropertyName type: component[PropertyName,Expected,Actual]

I was trying to map an existing structure I had matured without persistence. The class is:


public class Difference {
  public string PropertyName { get; set; }
  public object Expected { get; set; }
  public object Actual { get; set; }
}

The problem is NHibernate doesn't know what to do with the objects and so it throws that error. My solution was to change the class to string values, so NHibernate would know how to persist them:


public class Difference {
  public string PropertyName { get; set; }
  public string Expected { get; set; }
  public string Actual { get; set; }
}

NHibernate instance not of expected entity type error

2009-09-09T06:04:00.003-05:00

When you get this message in NHibernate:


NHibernate.HibernateException: instance not of expected entity type: DerivedClass is not a: IMyInterface
at NHibernate.Persister.Entity.AbstractEntityPersister.GetSubclassEntityPersister(Object instance, ISessionFactoryImplementor factory, EntityMode entityMode) in c:\CSharp\NH\nhibernate\src\NHibernate\Persister\Entity\AbstractEntityPersister.cs: line 3864

It means that you're trying to relate to a class that is not mapped in the NHibernate mapping. This happened to me because I was using a test fake when testing my NHibernate mappings with SQLLite. The solution is to only use real, mapped classes when testing with NHibernate persistence.

StaleStateException in Nhibernate with Unexpected Row Count

2009-06-30T12:06:00.003-05:00

If you get this exception:


NHibernate.StaleStateException: Unexpected row count: 0; expected: 1
at NHibernate.AdoNet.Expectations.BasicExpectation.VerifyOutcomeNonBatched(Int32 rowCount, IDbCommand statement) in c:\CSharp\NH\nhibernate\src\NHibernate\AdoNet\Expectations.cs: line 29
at NHibernate.AdoNet.NonBatchingBatcher.AddToBatch(IExpectation expectation) in c:\CSharp\NH\nhibernate\src\NHibernate\AdoNet\NonBatchingBatcher.cs: line 40
at NHibernate.Persister.Entity.AbstractEntityPersister.Update(Object id, Object[] fields, Object[] oldFields, Object rowId, Boolean[] includeProperty, Int32 j, Object oldVersion, Object obj, SqlCommandInfo sql, ISessionImplementor session) in c:\CSharp\NH\nhibernate\src\NHibernate\Persister\Entity\AbstractEntityPersister.cs: line 2754
at NHibernate.Persister.Entity.AbstractEntityPersister.UpdateOrInsert(Object id, Object[] fields, Object[] oldFields, Object rowId, Boolean[] includeProperty, Int32 j, Object oldVersion, Object obj, SqlCommandInfo sql, ISessionImplementor session) in c:\CSharp\NH\nhibernate\src\NHibernate\Persister\Entity\AbstractEntityPersister.cs: line 2666
at NHibernate.Persister.Entity.AbstractEntityPersister.Update(Object id, Object[] fields, Int32[] dirtyFields, Boolean hasDirtyCollection, Object[] oldFields, Object oldVersion, Object obj, Object rowId, ISessionImplementor session) in c:\CSharp\NH\nhibernate\src\NHibernate\Persister\Entity\AbstractEntityPersister.cs: line 2940
at NHibernate.Action.EntityUpdateAction.Execute() in c:\CSharp\NH\nhibernate\src\NHibernate\Action\EntityUpdateAction.cs: line 78
at NHibernate.Engine.ActionQueue.Execute(IExecutable executable) in c:\CSharp\NH\nhibernate\src\NHibernate\Engine\ActionQueue.cs: line 130
at NHibernate.Engine.ActionQueue.ExecuteActions(IList list) in c:\CSharp\NH\nhibernate\src\NHibernate\Engine\ActionQueue.cs: line 113
at NHibernate.Engine.ActionQueue.ExecuteActions() in c:\CSharp\NH\nhibernate\src\NHibernate\Engine\ActionQueue.cs: line 147
at NHibernate.Event.Default.AbstractFlushingEventListener.PerformExecutions(IEventSource session) in c:\CSharp\NH\nhibernate\src\NHibernate\Event\Default\AbstractFlushingEventListener.cs: line 249
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event) in c:\CSharp\NH\nhibernate\src\NHibernate\Event\Default\DefaultFlushEventListener.cs: line 19
at NHibernate.Impl.SessionImpl.Flush() in c:\CSharp\NH\nhibernate\src\NHibernate\Impl\SessionImpl.cs: line 1477

Then you should look for an Id Guid that is instantiated by your object as Guid.NewGuid(). Instead, you need to use Guid.Empty (that's how NHibernate looks at an object as new).

A Great Overview of where I'm at

2009-05-26T09:16:00.002-05:00

I went to an agile user group a few weeks ago and was left scratching my head a bit. The people in the group (including the leaders) seemed to be at a place where I was a couple of years ago. I got very little from attending. I wondered if I'm all that smart or if I'm just delusional. I'm sure the answer lies somewhere in the middle.

Whenever I listen to what Jeremy Miller has to say, I don't get that same feeling. I see a person who is right there with where I'm at with software, except he's ahead of me and can give me good direction. Unfortunately, there isn't much out there to direct people who have adopted my alt.net, tdd style. This video is a great investment of an hour to listen to the numerous nuggets of wisdom Jeremy has to offer.

By the way, I've started using StructureMap and love it. I think I'll post a few blog posts on what I've done to help those who think IoC might help them.

StackOverflow, my new friend

2009-04-15T07:07:00.001-05:00

I've been really impressed with how StackOverflow is laid out. I've had a few questions answered very quickly with it. If a short google search doesn't pan out, ask your question on SO and you'll get a specific, good answer within 10 minutes.

As much as I love Resharper,

2009-04-09T23:18:00.001-05:00

It probably wasn't a good idea to use 4.5 on the first day it was released. I've had a few crashes, a lot of slowdowns. Reminds me of the early days.

They'll fix it in a week or so...I'm debating whether I should go back to 4.1 or stay where I'm at and suffer...

On the positive note, the new rename is nice with the auto-complete.

A Matter of Perspective

2009-03-26T11:40:00.003-05:00

Three people walk up to me. The first is a developer starting on my project. He asks, "So how do we do things around here?" I say, we do XP.

Next, walks up a new project manager assigned to my project. He asks, "So what is our process?" I say, we do Scrum without time-boxed iterations (or, in another way, with 2-3 day iterations).

Finally, an executive walks up and asks, "How does your team work?" I say, we follow Lean Principles that were inspired by how Toyota manufactures their vehicles.

It's all the same process. It's just a matter of answering to the person in a way that would communicate to their expertise the best way. I still think XP is the best way to communicate to a developer. And lean is the best way I've found to communicate to an executive. As for Scrum, I'm secretly hoping it goes away.

Quality versus Cost

2009-03-23T09:06:00.002-05:00

I was listening to The Toyota Way on my ipod on the way to work today and listened to something interesting today. The author said, roughly, If you want to focus on lowering costs, don't focus on lowering costs. Instead, work to increase quality and your costs will go down.

Sounds like a good strategy to me. In organization, much of the inefficiencies exist as a result of lower quality, so purposefully lowering costs (laying off people, not hiring, not buying the right tools for the job) would have the effect of lowering quality, which would make the problem worse. At best, you would have a net 0 effect.

But instead, if you look at waste in the process (or value stream as the lean people say), then you will find that much of the waste is due to poor quality. When you address that issue, the costs go down. Then you can either let attrition even out your workflow, or do more work and make more moeny. I would think that most companies given that choice would take the latter. Thus, a strategy that is good for the company's bottom line and employee's sense of security.

Evangelism or Pragmatism?

2009-03-20T15:35:00.003-05:00

I recently listened to an Alt.NET podcast with Scott Bellware and had some thoughts.

First, I thought Scott was dead-on when he described the self-indulgence and intellectual snobbery involved in the agile community. It's bothersome, not productive, and needs to stop.

Second, I was troubled by Scott's insistence that those who hold alternative views within the .NET community need to evangelize more. We need to go to companies on nights and early mornings and show them how great this stuff is. We need to sacrifice.

Does he have a family?

How about this. How about I make a product that is obviously superior, makes my company a lot of money, and transforms my company's way of doing things? If I do that along with everyone else in this community, don't you think sooner or later that the market will notice, and the vast majority of Microsoftians will jump on board?

This is capitalism. Wouldn't the pragmatic approach be better than trying to "evangelize"? I would think so. But sadly I don't see this line of reasoning used very often...which smells like everyone is full of crap and caught up in their own intellectual thought experiment. That's not a thought I'd like my managers to have. And evangelism does nothing to dispel that; only solid, reliable, consistent, and game-changing results will.

Would you be excited about this?

2009-02-10T21:53:00.003-06:00

I was talking today with my colleague his desire to want to add a lot more features to our solution so people can be excited about what we're doing. I'm all with him, I'd like that too, but it reminds me about how you shouldn't let your desire to really wow your customers get in the way of delivering something that will be useful to a subset of them. You can then use the feedback they give you and the extra capital to take it even further.

I'll be honest with you, I wouldn't have bought this piece of crap to my left. But someone did. And they got excited about it. Lessons were learned. Market momentum was created, and I finally did reach me. I've bought two ipod products since. I'm really glad Apple was smart enough not to keep people waiting for this ugly duckling, so I could have a refined ipod.

The lesson: find in your customer base the user that's really gunning for what you have to offer, then release something to them early and often. You'll be surprised with how different their values are from yours, and how totally different features excite them than you.

Refactoring First

2009-02-10T07:54:00.003-06:00

The word "refactoring" means different things for different projects, depending on how much technical debt one has. On a project with loads of technical debt, no unit tests, and very little object-orientation, refactoring can mean "get rid of everything here and replace it with something better". On a typical agile project with good testing but little support for refactoring (i.e. viewing it as a liability), refactoring oftentimes means "the design would be better if we did X", to which the business almost always says "OK, we'll put that on the roadmap/release plan" and then does nothing.

I have been on such projects in my career and have been frustrated when I could not refactor but felt it was necessary, but it has only been until recently that I've understood my role in creating the situation where technical debt was tolerated. In the past, I would follow the TDD pattern, but I would test first, fix the test, and...hmmm....no refactoring. I would get to the end, and I'd be done....no refactoring. I was right there, I could have spent another day or two making the design sustainable, but I didn't. And that falls on me.

But I've learned it isn't even the refactorign within the TDD process that's the most important thing. It's most important that when I do a feature, I refactor the design first (yes, that means changing the design without changing the behavior) to make it seem like I planned for the feature in advance. Let me give an example (disclaimer: I wouldn't actually have done this refactoring, I'm just using a simple example to illustrate my point). I have this code:

public void Copy(string sourceDirectory, string temporaryDirectory)
{
 // do some stuff
}

So now I need to copy to a second temporary directory on a network as well. Most of the time, even in a TDD world, we would do this:

public void Copy(string sourceDirectory, string temporaryDirectory, string networkTemporaryDirectory) { }

But in this overly simplistic example, we just added to the method signature. It's like we said, "Oh crap, I didn't think about the network temporary directory, I'll just add it to the end."

So why not do an Introduce Parameter Object refactoring first:

public void Copy(string sourceDirectory, TemporaryDirectories temporaryDirectories) { }

public class TemporaryDirectories {
 public string TemporaryDirectory { }
}

and then add the other one. When you add it this time, it's like you had planned it all along. And this is what real emergent design looks like. It doesn't look like bolting a bunch of stuff on as you fly by the seat of your pants (mixed metaphors, sorry). It means changing your design to fit the requirements in front of you, and only those requirements.

Testing is Bunk and other arguments

2009-02-09T15:04:00.003-06:00

I'm very pro-test-driven development. In fact, I can't see doing software any other way; it's so effective for me that even if I were on a project that does no unit testing, I'll still unit test to make sure my stuff works.

It's good though, when you're into something, to read people who disagree with your approach. It helps you think. If I find myself getting immediately offended by opposition to my ideas, that's a huge red flag; I've stopped thinking and have descended into a cultic worship of some idea without regard for reason.

This is why I enjoyed Joel Spolsky and Jeff Atwood podcast transcript. Joel and Jeff went on a huge rant about how dumb TDDers can be with writing so many tests and how clean code and design doesn't really matter. In short, they blasted many ideas I've been into lately.

It's also good to hear from the people I respect in response to these kinds of rants, because when disagreements flare up they force you to have to make the argument. Bob Martin, who was specifically named in the podcast, had a good rant here (but maybe a little too emotional). His best response was in response to a great post by Ron Jeffries about going fast.

Agile's ability to make you go much faster creates a bit of a dillemma for each and every feature you implement. The guy I'm working with and mentoring has commented that he could have gone faster on this particular feature if he didn't have to write the tests. Bob's comments are invaluable:

How do you really get things done quickly? You listen to your grandparents. Remember what they told you? “Slow and steady wins the race.” and “Anything worth doing is worth doing well.” How does a professional craftsman get things done quickly? He/she adopts an attitude of calm, focuses on the problem to be solved, and then step by step solves that problem without rushing, without yielding to the need for speed, without surrendering to the desire for a quick conquest.

That's exactly correct. My colleague and I were at lunch the other day and he commented, "You know what I'm learning here is like the difference between being an amateur and a professional." Bob does a great job of describing what a true craftsman/professional programmer aspires to be.

What's wrong with Agile Part 3: People just don't "get" it

2009-01-29T10:12:00.002-06:00

One of the frustrations I've had over the years trying to "evangelize" agile is that it doesn't seem very intuitive to most people, at least at first. Let's take unit testing for example.

Why would you want developers to do the work of qa? I reply, well, it isn't like that exactly, they're more making sure their stuff works and has a good testable design. The skeptic replies, yeah, in a perfect world, but I have deadlines. I'll sit on the sidelines while you ruin your project with that unrealistic, pie in the sky crap.

This is frustrating because I've seen how well unit testing works and it seems like such a slam dunk to me, but for whatever reason people just don't "get" it. If I take this further, I end up throwing my hands up in the air, and wondering why the world isn't as smart as me.

Well, maybe there is a better way.

How about this: I unit test on my project and demonstrate marketable features faster than the customer can think them up and make the company a ton of money. People "get" money, customer satisfaction. But especially money. And hey, if I can't translate unit testing into cash for my company then I have no business unit testing.

So now we're off of the ideological ivory tower and even with the rest of the projects out there. It feels great not having to argue the ideas as much, and just tell people, "I'll deliver defect-free features faster than you know how to ask for them." Now it's time to deliver...

Actions, Funcs, and Predicates, oh my!

2009-01-29T08:27:00.005-06:00

For better or worse, C# is going down a path where you will need to become very familiar with Actions, Funcs, and Predicates. Not only that, but you'll need to understand them expressed as lambdas. This is a good post to bring you up to speed (H/T: The Morning Brew). Seriously, if you don't get it the first time read it again. If you still don't get it, fire up Visual Studio 2008 SP1 and try it out yourself. Much of this stuff (minus the lambdas) is available in .NET 2.0, if you're willing to write anonymous delegates.

Here's a sample:


// code that takes an action
public void PerformActionOnAllElements(IEnumerable elements, Action action)
{
 foreach (T item in elements)
 {
   action(T);
 }
}

// the code as lambda
List pricesOnCheck = new List();
decimal subtotal = 0.0m;
PerformActionOnAllElements(pricesOnCheck, price => subtotal += price);

// or if you want to be really cool, just use the .NET built in syntax
pricesOnCheck.ForEach(price => subtotal += price);

I first learned about this in More Effective C#. It's here to stay. So even if you hate lambdas and all that they imply, you really owe it to yourself learn it enough to be able to read code created this way. You might even find yourself writing this code after not too long...

What's Wrong with Agile, Part 2: Management Impedance Mismatch

2009-01-28T07:05:00.002-06:00

In part two, I discuss another problem I see with the traditional agile model, and that's how its basic philosophy is incompatible and at odds with most managers I've met. When you read the agile manifesto, one of the phrases is this:

Individuals and interactions over processes and tools

I think at some level the manifesto is correct. It's fundamentally more important that individuals interact correctly and that they and their interactions are valued than a particular process or tool be in place. The problem here is that most managers I've ever met view their job as putting the right people in place within the right process using the right tools. So in essence they're trying to create the right environment for others to succeed.

Then the agile hotshot comes along and tells them that the processes and tools aren't important. Only the people are. The manager is left feeling that the agile hotshot is full of crap because she knows that processes and tools are important. Without them you would have leaderless chaos. And I'm sure many agile projects out there use the manifesto to this end, and to their own detriment. And so you have a fundamentally different mindset from a pure agile person to a pure competent manager.

How have I learned to deal with it? Well, in my new lean world, it turns out that processes and tools are much more important than I thought. It's not that individuals and interactions are not important, it's that they are all important.

If I were starting a project with a competent manager, I would begin by suggesting a development process and how it eliminates waste within the product development system. The process itself would value individuals and interactions (using pair programming, colocated teams if possible, etc.), but to the manager that would not be the focus. To the manager, we are eliminating waste and going from start to usable software as quickly as possible. We are also decreasing, and even eliminating many of the tradeoffs that keeps him up all night, like whether the product should be high quality or on time (both!) or whether to release this quarter or next (next week!).

This approach values individuals and interactions in a special way–with management, who are individuals just like the rest of us and have their own set of values and philosophies.

What's wrong with Agile, Part 1: Long Feedback Loop

2009-01-26T23:21:00.003-06:00

For my entire career, I have been very pro-agile. Agile has done a lot for me; without its principles I wouldn't know much of what I know today. A couple of years ago, I finally got a chance to experience a project that was "fully" agile. The results were definately mixed. Some good, some bad. I'd like to write a few posts on some of the bad parts of agile. I'll end each post with a short description of what I've since learned about how to deal with that problem.

So, in no particular order, the first problem with agile that comes to mind is that it still maintains a long feedback loop within a system that does not maintain enough focus. In English, this means you start out an iteration with the greatest of intentions, somewhere in the middle things change and everything goes to hell, and then you try to scrape by with a good enough solution at the end, after working way too many hours. Then you have a retrospective where you think of one or two things to change, and maybe you change, but somehow you relive that hellish existence again. It's like being trapped in the movie Groundhog Day. It never stops.

Then you go to the agile gurus and they tell you that you're not doing it correctly. Implement the process they say. All of it. Hire a coach if you still don't know what they're talking about. Is it just me, or is there something wrong with a system if you're the problem every time there is a problem? That kind of reminds me of a cult. I don't want to be in a cult, so I'm going to reject that. Can it be that maybe something is fundamentally wrong with how we're doing things in the normal agile environment?

I think the fundamental change that is needed is to start focusing on keeping less work partially done and on how quickly you can start a feature to it being used by a customer. This is the basis of lean. Three week iterations are great, but they only go so far. By the time you're at the end of them, it's too late and you're ready to go on to the next thing. Instead, focus on one thing at a time, limit your group to the smallest amount of work possible, and optimize that system to make it the most effective it can be. Kanban is a great tool for doing this.

It kind of reminds me of the difference between Pair Programming and Code Reviews. With the PP way, you're constantly focusing on right now and how you can get there the fastest with the best design. With Code Reviews, you're trying to release the software. Both have their place, but all other things being equal, I'll go with the continuous improvement and focus on one thing at once.

Convention over Configuration

2009-01-26T08:35:00.002-06:00

One of the tenets of Ruby on Rails is "Convention over Configuration", which on its face seems backwards but in practice gives software a lot of power. Jeremy Miller has written an article in MSDN magazine if you want to read more about it, but I'll offer my views here as well.

The basic idea with "Convention over Configuration" is that software is much simpler, and much less wasteful if you do the same thing throughout it instead of providing configuration for anything that can possibly be configured. This seems backwards because you think that you need to make software flexible so it can easily change. What one fails to realize here is that the configuration creates waste (in the form of complexity), and that configuration many times does not create business value.

So how does this translate in the real world? Recently I released a feature that read some information from a file on the C: drive. Originally, I wanted to create a user-defined properties file, but I realized that all of my users already had this file on their C: drive and imposing some other way on them would increase the barriers to my feature creating business value. So I released it reading the file from the C: drive. You can go right to the code and see "C:\file.txt". It's in a constant. It's baked into the software. No configuration. Just convention.

A few hours after I released the feature, one of the users asked why she couldn't use the file in her user directory. She was doing things better than the other users. But again, everyone else was doing it the C: drive way. And she didn't have any problem changing. So rather than jump on the configuration bandwagon and make crazy xml files to solve all of these problems, I asked her to move the file from her user folder to the C: drive. Convention over configuration. And it works, and I'm off creating features that add value to their work instead of solve a problem for one user that could easily be solved by copying a file with simple file IO.

You would think that the users would be upset by the inflexible nature of my software at this point. In reality it's quite the opposite. I believe this is because (1) I'm delivering them tested, quality features that they want every few days, and (2) Since there's not a lot of configuration, it just works. Nothing to set up, worry about. No complexity. No technical speak. Just press the button.

Hyperproductivity on Distributed Teams

2008-09-09T06:42:00.003-05:00

This is an excellent presentation (notes here) by the cofounder of Scrum (Jeff Sutherland) on how to create a hyperproductive state on a distributed team. He begins by talking about some of the fundamentals of agile development, that XP contains some of the engineering aspects of how to create a hyperproductive state, and how Scrum contains the process aspects.

The process aspects come from the way Toyota runs its business, as described on the book The Toyota Way. Apparently, when Toyota wants to create a new car, it gives people 90 days to create a working prototype, then it gives them a long list of things to fix on that car. In another 90 days, when those things are fixed, it comes up with a new list. And on and on. Take it from me, you end up with a well designed car at the end of that process.

Sutherland also talks about ScrumButt, when a company says they're doing Scrum, but they leave something out for one reason or another. He mentions the Nokia Test as a good way to determine if you're doing Scrum or not. He has some interesting metrics on how leaving certain parts of scrum out will affect your productivity (he says it increases it 30% over the waterfall method, but compared to 400% that people can have in the hyperproductive state). I've grown in my appreciation for how much every aspect of the Scrum process depends on one another. If you but anything, the synergy that could be created is lost, and you leave productivity on the table.

Finally, Sutherland gets to the distributed teams part of the presentation. He quickly goes through how much distributed teams cost, which was way too fast for me, but his end result is that on a project he tracked they spent $6M to support spending a $2M to India...not good business sense. He did note some situations where distributed teams worked (allbeit, in the extreme hyperproductive state, but in one that would make good business sense in some situations).

He advocates not separating the distributed teams. The obvious thing to do when having a lot of distributed people is to put them all on one team, so that they're at least not distributed to them, and have two teams: the distributed team and the nondistributed team. You'll have to watch the video for all of his reasoning there, but he makes a compelling case. This also involved establishing a "local velocity" with all resources locally and then tracking that velocity when work switched to a distributed environment. This even won praise by senior management because it appears that the local team spent more time understanding the customer because they had to explain it to someone in another culture and country....Interesting!!!

Programming's Bottleneck

2008-08-18T08:24:00.005-05:00

I put together a couple of dressers we bought at World Market last night which got me thinking of pair programming. Pair programming is when two programmers work on the same problem at the same computer. It does have an element of being counter-intuitive, because why would it save time to have to people do the same thing rather than one? As with most things, it depends. There are three types of projects where pair programming has different degrees of helpfulness. Fortunately, these types of projects map very nicely to the dressers that I put together yesterday, so I will use them as a metaphor.

1: Discovery then Repetition

Our project was to build two identical dressers. We opened the first box and read the directions. We didn't have the first clue as to what to do, but we had good directions. It was still difficult to figure out what to do at key points in the process where the directions were ambiguous. When these came up, Annie and I worked together to think out the solution. Once we figured out what to do, we worked independently. Total time to build: 3 hours.

2. Repetition

The second dresser that we built was exactly like the first. We hardly looked at the directions and developed more of an assembly line mentality. We managed each other and worked independently. Total time to build: 1.5 hours.

3. Initial Prototype

We obviously didn't do this, but this is a situation where someone wants to create a dresser but it hasn't been created yet. It gets designed and the first prototype and directions are created. Total time to build: days

What is the bottleneck?

The three types of projects have differing bottlenecks. In the first project, the major bottleneck was our ignorance. That's what took us twice the amount of time. We didn't know what we were doing. Also, we made a couple of bad mistakes that made us have to unscrew the whole thing. At one point, we were almost done but had but in a key component in backwards. We spent the next 45 minutes recovering from that mistake.

In the second project, we didn't work together. We didn't need to. The bottleneck here was our ability to screw in screws. Pairing in this situation would not be helpful.

In the third situation, working together would be essential. If you've never built a dresser before but had good, solid carpentry skills, you would still need another person there. The bottleneck is not cutting the wood and screwing the screws in. If you make a mistake, you have to start over at this stage.

Tying it to Pair Programming

Most managers I meet seem to think that programming is a Type 2 project (repetition). If there is a productivity problem, add more people because you need more hands on the keyboard. In fact, most if not all projects I've been on are Type 1 projects. Yeah, there is some repetitive work, but much of the work done is recovering from stupid mistakes brought about by not solving the problem before.

As a project becomes more agile, however, the project becomes more like a Type 3 project. You don't have directions. You have a business case. You need to solve it. And you need it to work. Now. So, as a project moves more in the agile direction, pair programming becomes even more of a necessity, because the bottleneck becomes less and less about how fast you can type and more and more about how fast you can accurately solve the problem. In the latter situation, pairing up makes the most sense.

Threading Discoveries

2008-08-11T07:18:00.004-05:00

I haven't been much of a threading expert throughout my career, but that is starting to be a requirement for what I'm working on, so I thought I would post some aspects of threading as I learned them.

I had a situation where a worker thread needed to wait for input from the UI thread before it continued working. I've seen this many times:


while (true)
{
if (m_Form.IsReadyToContinue)
{
  break;
}
Thread.Current.Sleep(1000);
}

While this does work, it seems wrong, even if you don't know why. Fortunately, .NET provides the AutoResetEvent. In the form, you simply create the event as a member variable:


private readonly AutoResetEvent m_ReadyToContinue = new AutoResetEvent(false);

The false is the current state of the event. An AutoResetEvent will reset itself back to "false" every time a Wait is asked for (see below). So, when you're ready in the form, all you have to do is call the Set method:


m_ReadyToContinue.Set();

Now for the worker thread. You need to have an instance of the same AutoResetEvent instance as well. Here, you simply call the WaitOne method:


// inside worker thread
m_ReadyToContinue.WaitOne();

I wish this interface was a little more fluent, but what you're saying here is "Wait until one set happens." If a Set() already happened, it will not wait at all. This turns the hacky forloop above into a simple and elegant method call.

One other warning: you should never call WaitOne on the UI thread; it's only for worker threads. Calling it on the UI thread would cause the UI thread to freeze up. So if you have interaction between the UI thread and a worker thread, try to make the UI thread the notifier (via the Set() method) and the worker the waiter (via the WaitOne() method).

Setting up Continuous Integration in 5 minutes

2008-07-28T07:31:00.006-05:00

I recently had to add a project to TeamCity that had no MSBuild script. At first, I thought about using the MSBuild script on my project as a starting point and editing it to match what was needed. That would have taken me a couple of hours. I didn't have a couple of hours.

So, I decided to play with the solution-based building with TeamCity and was very impressed. When you look at that feature, you think "It's going to compile it for me, but that's it." Then you see this dialog:

(Click on the image for a larger version)

So in five minutes, you can have continuous integration set up with compilation, code analysis, and unit tests running! You can even add artifacts if you know the path to the artifacts.

Your needs will increase, but for a starter project that has no build process, this is a pretty nice and cost-effective introduction.

By the way, I'm using TeamCity 4.0 EAP.

TeamCity Personal Build Behaviors, Part 3

2008-07-28T07:21:00.001-05:00

I have absolutely loved getting to know the personal build feature because using it has saved me so much time this week. I've found that when you can trust what it's doing, and know when it doesn't do what you expect, you'll use it a lot, even when you know that there are unit tests that are failing. You'll do this just to give you the information on the ones that you haven't run, or to give you code analysis errors. It's truly addicting. After all, why tie up your machine when another machine will do all of the work for you!

That being said, there are some boundary cases I wanted to share to help you learn what's going on with these features. Hopefully you'll be able to use the feature more with the gained understanding.

Scenario 1: Pass
Edit File A (don't save)
Run a personal Build

I would expect it to save the file then run the personal build on that saved file. It does do that; you don't have to worry about if you've saved your files.

Scenario 2: Fail
Edit File A (save)
Run a personal build
Close visual studio
Wait until build finishes

I would expect it to check in the files even though you closed visual studio. It does NOT do that. The personal build succeeds, and TeamCity says that you have committed the build, but when you look at source control, it did not commit your changeset. This is because Visual Studio will check in the files on your machine. If it's not open, it won't even do that.

Scenario 3: Fail
Edit File A (save)
Run a personal build
Close visual studio
Open visual studio
Wait until build finishes

I would expect it to check in the files when the personal build passes, but it does not do this. It will only check in the files on your behalf if you keep the session of visual studio open that initiated the personal build.

This means no reboots while personal builds are running!

TeamCity Personal Build Behaviors, Part 2

2008-07-28T07:10:00.003-05:00

In my previous post, I outlined how TeamCity keeps files checked out when running a personal build, and checks them in automatically.

Which leads to another question: what happens when you need to go back to the changeset you are running a personal build against and fix some problems from that build? That risk may keep you from charging forward and making drastic changes to your code because you wanted to commit at a certain point and don't want your new changes committed yet.

Fortunately, TeamCity helps you out here. When you run a personal build, they'll actually shelve that code on Team Foundation Server:

The "Delayed commit" is what TeamCity adds to your shelveset.

If you run into problems, you can simply

(1) shelve what you're working on,
(2) unshelve the personal build,
(3) fix the problem,
(4) run a new personal build against that code, and
(5) unshelve the code from (1) above and keep working.

This should be rare and is relatively straightforward, so it's the workflow I have in the back of my mind to keep me from being afraid to run personal builds.

TeamCity Personal Build Behaviors, Part 1

2008-07-28T06:59:00.003-05:00

One of the things that keeps me from running more personal builds more often on TeamCity is that it keeps my files checked out on Team Foundation Server while running the personal build. So (the thought went) I had to wait for the personal build to succeed before making any additional changes because it didn't check my files in yet. Usually a company like Jetbrains is a little smarter than that, so I decided to do a little test. Here's what I did:

(1) Make an edit to File A
(2) Make an edit to File B
(3) Run a personal build with both files checked and "Commit if successful" checked.
(4) Make an additional edit to File B
(5) Make an edit to File C

So, when the personal build happened, if TeamCity is doing the right thing, I would expect:
File A to be checked in
Only the original change to File B to be checked in
File B still checked out with changes made in (4)
File C still checked out

This is exactly what happens. TeamCity is smart enough to check the file in for you but keep the files checked out on your machine that you have changed since you ran your personal build. So you're free to keep working with a personal build running. That saves a lot of time!

Thus, this

2008-07-28T05:15:00.003-05:00

Over the years, I have had quite a bit to say about technical issues. I'll often search for an issue I'm working with and find it in a blog and (to myself) thank the person who bothered to post that tidbit of information that ended up saving me quite a bit of time. Sometimes, I'll realize that I should blog that tidbit of information that I didn't find with a search, for any other reason than to make the world a better place.

I've thought about putting those types of posts on my regular blog before, but it didn't seem right because when I tell most people what I do, the first thing they ask is whether I can fix their printer. So I've realized that blogging about TeamCity or test-driven development patterns can only serve to alienate my already depleted readership.

I've also in the past sent an email out to the colleagues I think would be interested in that information, but that has the drawbacks of (1) not being searchable for the rest of the world, (2) cluttering up their email inbox, and (3) having to guess who would want to read what I say. This way, they can subscribe to the RSS with Google Reader and read it when they want to.

Thus, this.