Back at the beginning of September Kevin Rutherford tweeted:

‘Anyone know the Japanese for “red, green, refactor” ?’

Now as it happens I know a Japanese speaker very well. Red was easy “akai”. Green was a little tricky because the Japanese actually have two words for Green, “aoi” and “midori.” We went with midori.

Refactor was a little bit tricky. After all, there was no such word in the English language until Martin Fowler published Refactoring. Indeed, “refactor” still haven’t made it into my English dictionary.

My translator suggested the Japanese would probably stick with the English term so we got “akai, midori, Refactor.”

At which point Kevin asked: “akai, midori, kaizen?”

Its taken me a few weeks to have an in-depth conversation about this with my translator but it seems Kevlin is right.

Kaizen is actually two Japanese words:
Kai: change
Zen: good

So the net effect is: “to change [improve] for the good”

Which seems to sum up refactoring perfectly.

Now, any other Japanese speakers out there like to refactor?

And just a gentle reminder that Changing Software Development is now available in Japanese. My personal Japanese translator didn’t do this translation, it was handled by the Japanese publisher.

I’ve blogged before about “The Alignment Trap” (Originally in December 2007 and after my piece in The Agile Journal in February 2009). This is the counter-intuitive evidence that sometimes doing things right is more beneficial than doing the right think. I often discuss this in my presentations and there are two standard reactions: for some this research rings true while others question it.

Those who question it often ask: is there any other evidence?

Well, there just might be.

Delivering Value from Information Systems and Technology Investments: Learning from success is a survey report from Cranfield School of Management in the UK. I only heard about this report a few months ago and its is now four years old but I see little reason to suspect things have changed since 2006.

The researchers define four levels of maturity in organisations developing systems. The first level is project delivery with the focus on the supply side. In order to reach the higher levels organisations must first be able to successfully deliver the IT projects they undertake.

In other words: doing things right is a prerequisite to success.

(I should point out that this report has nothing to do with Agile, Waterfall or any other method. Its about the ability to realise benefits from IS/IT projects whatever the approach.)

The second level of maturity concerns realising the expected benefits from projects. Here attention expands from the supply side to look more at the demand side and whether the business cases proposed are realistic. It is here that “doing the right thing” starts to be the issue.

Levels 3 and 4 concern realising benefits consistently across the whole portfolio.

The report contains some other nuggets of information which are with reporting:

• 73% of survey respondents said significant improvements were needed to get satisfactory value from IS/IT. (This 73% is curiously close to the 71% of companies in the “Maintenance zone” in the Alignment trap report.)
• 38% of businesses openly admit benefits are overstates in business cases in order to obtain project funding
• 80% report that review of work against the original objectives is inadequate

Finally, the other statement is worth analysing from this report is:

“The data also indicates that organisations have undue faith in business cases and that the deployment of formal methodologies gives managers a false sense of security, perhaps an excuse for not becoming sufficiently involved.”

If we take that statement apart it supports the Agile approach:

• “Undue faith in business cases”: Implies more tools are needed to tackle value. Agile would suggest a try, experiment, fail-fast, fail-cheap approach
• “Formal methodologies … false sense of security”: That could be PRINCE2, Scrum or something else. The methodology doesn’t have the answers. You need to be pro-active and think for yourself.
• “Not becoming sufficiently involved”: Yet again we hear that lack of customer/user/manager involvement is a cause for failure. We have a few tools for working around these problems but nothing beats having actual involvement.

In my Agile Business Conference present (“How much quality can we afford?”) I talked about the Reuse Myth, this is something always touch on when I deliver a training course but I’ve never taken time to write it down. Until now.

Lets take as our starting point Kevlin Henney’s observation that “there is no such thing as reusable code, only code that is reused.” Kevlin (given the opportunity!) goes on to examine what constitutes “reuse” over simple “use.” A good discussion itself but right now the I want to suggest that an awful lot of code which is “designed for reuse” is never actually re-used.

In effect that design effort is over engineering, waste in other words. One of the reasons developers want to “design for reuse” is not so much because the code will be reused but rather because they desire a set of properties (modularity, high cohesion, low coupling, etc.) which are desirable engineering properties but sound a bit abstract.

In other words, striving for “re-usability” is a developers way of striving for well engineered code. Unfortunately in striving for re-usability we loose focus which brings us to the second consideration…. cost of re-usability.

In Mythical Man Month (1974) Fred Brooks suggests that re-usable code costs three times as much to develop as single use code. I haven’t seen any better estimates so I tend to go with this one. (If anyone has any better estimates please send them over.)

Think about this. This means that you have to use your “reusable” code three times before you break even. And it means you only see a profit (saving) on the fourth reuse.

How much code which is built for reuse is reused four times?

I would suggest the answer to this question is: very little.

Consequently development teams which write a lot of reusable code are costing their organisations a lot of time and money. Waste.

Reusable code is not the solution to any of our IT problems. It is a supply side only solution and not a very good one at that. While it may reduce the amount of code that is written it reduces it by artificially constructing supply rather.

Thus, reuse, as a solution to software supply problems, is a myth.

This leave two questions we need to answer.

First: how are we to get those desirably engineering properties if we can’t/don’t push reuse?

The good news here is that Test Driven Development, i.e. writing code that can be automatically tested with other code we write, also has the effect of promoting modularity, high cohesion, low coupling, etc. Thus the answer to this question is: Make your code testable.

This approach will retain focus and deliver worthwhile benefits.

The second question: How should we manage reuse? After all, there are some genuine situations were reuse is the right answer.

Here there are two answers, a micro and a macro answer.

The micro answer, when your working in a development team is: look for emergent reuse.

Don’t plan for reuse but look for opportunities to reuse something that has gone before. When you see the opportunity take the previous work and enhance it just enough to reuse it in your case. This way you only pay the price for what you actually need when you need it.

If you later find the same code is useful again then repeat the process. Improve it just enough to use it a third time.

Remember: you have the tests from the earlier answer to make it safe to do this. Without the tests things get difficult.

Now you’ve reused your code three times, you’ve pay the price a bit every time, and you have the tests to show it still works. By now the code is going to be getting pretty close to generically reusable.

Still, maybe you go round this look a third or fourth time. It doesn’t matter is making one piece of code reusable costs more in the long run than it would have done if you did it the first time because… You haven’t spent money making a lot more code reusable that never needed it.

Telling the future is hard. It is difficult to know whether code will be reused or not, so default to Not and save the money. Occasionally you will be wrong and it will cost more, but overall you will be right and you will save money.

The macro answer: if this code really is widely reusable by many people then, go into business. Market this as a library, a plug in, an application what ever. Let the market decide the economics of reusability.

My presentation from the Agile Business Conference (ABC) 2010 is now online: Quality – How much can we afford?

For the last few months I’ve been wading through Factory Physics. That might sound very negative but actually this book is highly recommended. The reason its difficult to read is nothing to do with the writing – which is actually quite easy going – but more down to the size and the maths. It is over 600 pages long and was originally written as a text book so can be a little tedious at times. Add in a bunch of mathematical proofs – essential – which even if you skim can be hard going and you get the picture.

That said its a great book to read with lots of important facts and implications. I have been reading it because I had some loose ends. The operations management module in my MBA all those years gave lots of suggestions and advice but left me wanting to know more. I finally found what I was looking for in Factory Physics. (Specifically queuing, variation and optimisation, but that will wait for another post.)

Now, to get back to the title of this blog post.

Every so often I come across managers who liken their software development teams to factories. They talk about the “software factory” or what happens “in the engine room.”

I think this analogy shows a fundamental misunderstanding of what goes on in software development groups. The idea that you can liken the development of software to a factory line process, with repeatable inputs, defined outputs, known processes, and repeatable activities just isn’t the case.

The difference shouldn’t need pointing but I will: as commonly understood the factory production line repeats again and again, it constructs one (or a limited number of) product(s) many times. There is little innovation or problem solving in the process (as commonly understood).

Whereas, software development constructs one product, for the first time ever; entailing a lot of innovation and problem solving in the first place; and never constructs it again. (After the first time a simply digital copy suffices.)

Now, having digested a lot of Factory Physics there is another reason I dislike this metaphor. Now I have a better understanding of the complications of making a factory efficient it is clear to me that making a factory effective is damn hard work.

Software managers who like to think of their development group as a “software factory” not only have no real understanding of the development process, neither do they have an understanding of how to organise and manage a real factory producing physical things.

In fact, one of the arguments in Factory Physics is that most factory managers don’t have such an understanding either. The authors argue, convincingly, that a lot of factory organisation and management is fashion, or fad driven and not based on a good understanding or science principles.

So I suppose in that way it is like managing software development.

Now before anyone rushes recommend, or attack, Fredrick Taylor and the ideas of Scientific Management the authors of Factory Physics are clear. Yes we should apply scientific thinking to management but, they point out, Fredrick Taylor was not scientific. His studies – from a scientific point of view – do not stand up. They are full of holes.

Just in case I’ve not been clear, lets repeat:

1. It is a mistake to think of software development as a factory line process.
2. Anyone who insists on using the metaphor should find out what is actually involved in running a factory.
3. There are insights from running a factory – to be found in Factory Physics – which apply to software development and many people will benefit from learning them. That doesn’t mean you should run your software development process like a factory though.

In the software development world there are, broadly speaking, two groups of people: those who create the software (coders, testers, etc.) and those who manage the process (project managers, development managers, etc.). When discussing “Agile” I find that both sides think the problem is with the other.

To put it another way, if I’m talking to developers they think its managers who are the block to adopting more Agile techniques. But when I’m talking to managers they say its the developers who resist Agile.

This always reminds me of the old Philip Crosby quote:

“Quality has much in common with sex. Everyone is for it. (Under certain conditions of, course.) Everyone feels they understand it. (Even though they wouldn’t want to explain it.) Everyone thinks execution is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that all problems in these areas are caused by other people.”

(OK, so this blog just got filtered out of lots of feeds and stopped by lots of firewalls but lets continue.)

Lets bring it up to date and make it Agile specific, substitute the work ‘Agile’ for ‘Quality’:

“Agile has much in common with sex. Everyone is for it. (Under certain conditions of, course.) Everyone feels they understand it. (Even though they wouldn’t want to explain it.) Everyone thinks execution is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that all problems in these areas are caused by other people.”

OK, this isn’t news, this study came out a couple of years ago and was covered by many people then. But, I find myself regularly referring to it trying to find the link. So I’m going to blog about it then I’ll always be able to find the link.

The study is by Nagappan, Maximilien, Bhat and Williams and is entitled: Realizing quality improvement through test driven

development: results and experiences of four industrial

teams and is freely downloadable from Microsoft.

This is the key findings are summarised in this table:

Team / Metric	IBM drivers	Microsoft Windows	MS MSN	MS Visual Studio
Defect density of comparable team not using TDD	W	X	Y	Z
Defect density of team using TDD	0.61W	0.38X	0.24Y	0.09Z
Increase in time taken because of TDD	15-20%	25-25%	15%	25-20%

The way to read this is: the researches looked at two Microsoft MSN teams, one team did not use TDD and had a defect density of Y. The second MSN team had a defect density less than a quarter of Y but took 15% longer.

To my mind that proves that which was to be proven, i.e. TDD reduced bugs. But, I’m also aware that other writers have disputed this and I’ve heard of studies which disprove it. (Anyone got a link? Thanks)

Most people who I’ve met, and who have practices, or understand TDD agree it is effective. However there are those who don’t believe it. It reminds me of the episode of Black Adder where Rowan Atkinson’s Black Adder hires a ship Captained by Tom Baker. When there is a problem it plays out like this:

Black Adder: “Someone in the crew will know… you do have a crew don’t you?”

Captain: “Arh, opinion is divided on the subject… all the other captains say you need a crew, and I say You Don’t”

At the end of the day Confirmation Bias will probably decide which set of results you choose to believe.

Once in a while I run across individuals, or even teams, who still think Agile is about just getting on and doing it. Well it is, good for them, but, that doesn’t mean there isn’t a reason for doing it.

There seems to be a myth in some circles that work done using Agile techniques doesn’t require a business case. Lets get this clear: Agile does not excuse you from having a business case for your work.

Of course there are instances were a business case might not exist. Some companies, for better or worse, work without them; those companies aiming at innovation may allow work to proceed to a more advanced stage before asking for some rationale; and products which are in a steady state may just tick over without too much attention to a business case.

But in each one of these cases the lack of a business case has nothing to do with Agile. (Actually, each of the cases does have a business case, its either a tacit business case or a part of a much bigger business case.)

Many development efforts which lack a visible business case can benefit from demanding a business case. If you are not sure why your company is doing some piece of work then maybe the company needs to be clearer about the reasoning.

However, a business case might look a little different on a Agile project. It may well be shorter than a traditional business case, it may lack some of the detail traditionally found in a business case, and it may describe much more a trial-and-error approach.

What Agile projects don’t need are in-depth business plans. Here its a question of detail, a strategic business plan makes plenty of sense. But a business plan which lists many many features to be build and a detailed schedule of when they will be built is little more than an illusion. Such plans give the appearance of certainty but certainly don’t provide it. Indeed, too many plans can actively hinder a project – plans are not benign, they are dangerous. (See An alternative view of design (and planning) for more discussion here.)

That said, there is a very good case for writing a business plan – indeed most forms of plans actually: they are rehearsal time for the real thing. They help you explore and learn about the thing you want to build.

Still, sometime you might skip the business plan. Maybe there isn’t time for a rehearsal, and sometimes learning can be maximised but just getting stuck in .

But, if you do write one, then once that exercise is over, once you start doing it for real, don’t try and hold yourself to a plan. Put the plan in the draw and don’t look at it. The value of planning is not the plan you produce at the end of planning; the value it is the knowledge you acquire in your head.

For that reason you want to maximise the number of people involved in the planning rehearsal so you can maximise the learning.

I’m sitting on a train from York so it seems a good time to share my train-leaving-the-station metaphor with the world. In truth, if you’ve worked with me in the last few years, or heard me speak at a conference I may already have shared it with you. But for the rest of the world, and with full embellishments….

Traditional software projects are like a train leaving the station. There is a big train sitting at Platform 9, we know its due to leave soon, but, well, you know what big long distance trains are like, it may well leave a little bit late. Still, to get a seat we need to be early so we are all rushing to the train.

Since we don’t know when the next train to this destination will go – actually, it is far from certain there will ever be another train – we want to be sure everything we need is on board. So we make sure we put everything we might need on the train. (Think of our requirements document.)

Eventually the train lumbers out of the station, overloaded. Quite possibly it leaves late because we were so busy loading the train, maybe one or two people even argued with the guard to delay departure a little bit so we could get more people and things on the train.

The train – our project – is like one of those trains you see in pictures of the Indian rail network with people crowded on and hanging out of the doors.

We all know the train is overloaded, we all know its going to arrive late, we even think it might miss the destination and arrive someplace else. But nobody wants to admit this.

(Particularly true if you have American management and British engineers because the Yanks view the Brits as being overly negative.)

At some point, beyond the half way mark it can no longer be denied that the train will arrive late. So action is taken. Things are thrown off the train to make it go faster.

By throwing things off the train we hope the train will arrive closer to the scheduled arrival time and the intended destination. However, in so much as there was rationale for putting all the things on the train there is less rationale applied to removing them. Things are thrown off the moving train because we can. (Somethings we can’t throw off because they are too connected to other things.)

In the extreme, those who put things on the train, and really know what is needed for the destination (BAs, Product Managers, etc.) never actually got on the train. Once they put the bits on the train they handed over responsibility to Project Managers to get it delivered. These guys are primarily concerned with dates and since they charged with delivering everything they don’t want to throw anything off.

Eventually, the train lumbers into the final stop – which may, or may not, be the intended destinations – with some random collection of things on board. Everyone gets off and breaths a big sigh of relief. Thank God that is over.

Then the news comes: there will be another train. We begin again.

This time though, we have all been burnt. We are going to make sure we have everything we need on the train, plus all the things we threw off the last train, and some more besides because we now understand the need for bargaining and we need levers.

Thats the traditional view. Now for the Agile view.

Instead of big trains we have a metro system – think of London tube, or better still Glasgow’s Clockwork Orange.

I leave the office at 6pm, go to the tube station and wait for a train. The sign tells me there will be one in 2 minutes, and another 2 minutes after that, and so on.

The train arrives and it is full. I have an option: Do I get on the packed, sweaty train and have an uncomfortable journey? Or do I wait 2 minutes for the next one?

I choose to wait. And while I’m waiting my phone rings. Some friends are in a pub nearby and ask if I would like a drink. I have an option: do I go and have a nice beer (valuable) with friends? Or do I value getting home more?

I go for the beer, something I would never do if there was only one train today. But knowing there will be another train, and another train, and another, means that I can go to the pub safe in the knowledge that I will still get home.

Need I say is? We need software development to be like the metro/tube system and not like the big occasional train.

There is a piece by me on todaysInfoQ – The Limits of Agile.

I’m interested to here your comments.