The first of three posts: this one sets the problem and the next two attempt a solution…

A friend e-mails to say that his team have almost finished their project plan. There is something about the words “project plan” which are like a red flag to this bull… while I’m prepared to admit I may over react sometimes I don’t think I’m completely crazy and I think I can put some logic behind my dislike of “project plans”.

Lets start by defining what I mean. Its not so much a “project plan” – after all I think planning is a great way to explore your problem. As a mental exercise a team planning what they are going to do is good. And I’m all for a bit of critical path analysis to work out what the dependencies are. And I think scenario planning is great. From what I’ve heard war-gaming is another good for of planning.

I should admit to an assumption. When this friend says “project plan” I’m immediately imagining Gantt charts and PERT diagrams. There are many other forms of plan which make a lot of sense. I think product road-mapping is an essential activity. Its specifically Gantt and PERT charts that make me see red – and as layman in the field I don’t see much difference between Gantt and PERT to be honest.

So what is it about Gantt charts? Here’s some:
• The charts are built on estimates and estimates are just that, estimates. They are not commitments, they are not fact, they are not enforceable. Therefore Gantt charts are not an accurate way to plan a project.
• The charts create illusion of control; here is a piece of paper which says the project will be done on 1 May 2010 – or whenever. It won’t. That date is the one date the project will not be ready. It might be ready before, more likely it will be ready after but that one date is fantasy.
• Many – too many – discussions now centre on “why are you deviating from the plan” – “why are the estimates wrong?” – “how can we get back on plan?” – arguments centre over the plan not the thing being produced.
• There is now work to do “correcting” the plan; updating it for what has happened and attempting to re-forecast. This makes work but does not add value.
• The charts are never accurate but they are treated as such.
• The “plan” is taken as a form of commitment.
• The further out the plan the more inaccurate it is: planning for next month is one thing but 5 years out? There is simply not enough information to do it accurately. Plus, things change too much, the further out you look the more things can (and will) change.

Basically, what I’m driving at is: once the plan is in place disucssion moves from what is important (what are we building? when do we need it by?) to the plan. Its goal deferment.

There is a lot more I could say here but I’d be repeating myself. If you are interested read An alternative view of design (and planning) and the section in Changing Software Development.

Fortunately I showed this to my friend and he says they’ve not drawn a Gantt chart, phew! But before he tries I need to give some advice, the next two post will try and do that.

• Project plans (part 1 of 3): Why I don’t like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) – Planning cycles
• Run Scope Creep backwards

One of the criticisms levied at Agile software development is that of Scope Creep. Project managers seem particularly sensitive to this issue, probably because its been fatal to many projects before and thus Project Manager are trained to watch for it and come down hard on it.

However, I’ve come to the conclusion that in a lot of Agile work the secret is to run Scope Creep backwards. Rather than have a project get bigger as more and more requests are added to it what we need to do is have it get smaller over time. Let me explain.

Typically a product/project gets specified and then work proceeds, as it does more and more “change requests” appear, things people hadn’t thought of, ideas for what else it could do – scope creep. (See my piece, Why do requirements change from a few years ago if you want to know more about the why.)

As projects grow in size the chances of them meeting any deadline diminish. This is why traditional Project Managers look at Agile and throw their hands up in the air seeing rampant Scope Creep.

But if you run it backwards, write the spec, write it as big as you like, who cares how big you write it or how many features you put in. The bigger it is the less people will remember. (See my piece on the Documentation myth for why big documents don’t work.)

Allowing everything into the spec means you don’t have to have those hard conversations about what’s in and what’s out. Everyone gets to feel there thing is in.

Now prioritise. If its really big make a first pass with MoSCoW rules. Anything which isn’t MUST have can be ignored for now – you’ve probably got more must’s than you have time for anyway. Generally I’m not a fan of Moscow rules but they are OK as a starting point when you have a lot of requests.

(Note: that’s MoSCoW rules, not Moscow rules – personally my only Moscow rule is avoid the city, my last trip there was awful so future visits are restricted to changing planes.)

Now within the MUSTs prioritise 1, 2, 3…. N. No duplicate priotities. This way when something increases in priority ther things fall automatically, people immediately see the consequences of their decisions.

What criteria you use for prioritisation is up to you. Doing it by value makes a lot of sense, so does doing it by risk. It also makes sense to get some early wins under your belt.

Next work through the items in priority order.

Anything that comes in later (Scope Creep) has to be slotted into these priorities. Add a new item with priority 3 and everything 4..N moves down one.

Notice I’m not interested in change control. I don’t make any effort to restrict the work that is asked for. All I do is demand it is prioritised along with everything else. Valuable ideas can arrive at any point in the work so why draw an arbitary line to differenciate what was “in scope” and what was “out of scope.” The objective is deliver value.

Now as you work through the list you will find that with items 1..N’ you have a a valuable product. Something you can at least show to customers, something people might even pay money for today.

Once you start getting real feedback – especially the sort represented by money – everything changes. Once you know what real customers want you are, literally, in business. You’ll want to add new items and look again at your priorities.

At this point you’ve saved yourself the work N’..N plus all the Should Have’s, Could Haves, etc. This is running Scope Creep backwards.

What all of this highlights is that building a software product isn’t a binary decision. It isn’t “all of this stuff” or nothing. Its an analogue thing not a digital thing. Traditional projects spend a lot of time trying to prevent scope creep through tough change control procedures but really it doesn’t matter when a request comes in. What is important is what is the value it will bring.

Value, not arbitary dates, should be the criteria for deciding what to build.

I was told the following story this week by one of my clients. He works for a large organization with many software development groups. The company believes retrospectives are a good thing and some people in the company, including my client, are trained in facilitation.

A few weeks before Christmas my client got a call from another group lead in the company who asked if he could facilitate a retrospective. “Sure” said my client who went to discuss it with the group lead.

Now it turns out that this group has not been holding regular (iteration) retrospectives during the project so this was to be the end of project retrospective. But the project had slipped a couple of months so wouldn’t ship until February. “Arh” says my client, “wouldn’t it make sense to hold this retrospective then?”

“Yes” says the group lead, “but, one of my objectives for the year is to hold a project retrospective so we need to do it this side of Christmas”. This company operates management by objective and the group lead had this objectives and his manager didn’t want to vary the objectives so an “almost done” retrospective it would be.

There are two schools of thought on when to hold a retrospective and this case was neither of them. The first school, exemplified by Norm Kerth in Retrospectives is the Big End of Project Retrospective (BEPR). The problem with this type of retrospective is that it occurs too late to make a difference to a project. Only if the same people will be working on something similar sometime soon is there a lot of benefit to the participants.

Still the BEPR is a good way of getting started with retrospectives. If you’ve never held one before there is a lot of ground to cover. I find them useful when I’m getting started with a team as a first step towards regular retrospectives and to help loosen the team up. It doesn’t necessarily have to be at the end of a project but to have value it needs to be held in time to make a difference. Do it in “stoppage time” probably makes little difference.

The second school of thought is exemplified in Diana Larsen and Esther Derby’s book Agile Retrospectives is the Regular Iteration Retrospective (RIR). I recently came across Joshua Kerievsky’s “How To Run An Iteration Retrospective” which is a good one page guide to this type of retrospective.

These are smaller, shorter, retrospectives that occur on a regular basis. Typically these are held at the end of an iteration so anything between 1 week and 4 weeks. By occurring regularly such retrospectives can make a difference while work is in progress. There are two problems here, the first is confusion and the second freshness, let me explain.

Simply, people get confused about what is a retrospective at the end of an iteration and what is not. This isn’t helped when you occasionally skip a retrospective.

I’ve been known to skip retrospectives, typically when the team is still settling and I can see changes occurring organically. In these cases I prefer to leave things to improve by themselves.

I’ve also skipped them when I feel that they will sap morale. I recently worked with a large client where I was coaching several teams. It quickly become apparent that the problems the teams were facing were systematic and beyond the teams ability to fix. What was needed was structural change across the organization. This wasn’t going to happen any time soon but until it did asking the teams to reflect on the problems they faced would only revisit the same old ground.

At the end of an iteration there are three types end of reflection to my mind:
• Review: this happens at the end of every iteration. Look at the work done, count the points (if using), talk about what was done and problems encountered. This needs to happen to close the iteration. Not really a retrospective but sometimes its seen as one.
• Informal retrospective: as part of the review or immediately afterwards. A discussion of what has happened, what could be useful and how the team might change. Informal retrospectives are useful when there is not the time for a formal one (they flow naturally after a review), when the team are still settling in or when the team are recovering from a lot of micro-management and learned dependency.
• Formal retrospectives: a formal period of time for reflection with some exercises to facilitate thinking. This format also allows you to keep track of what action was decided on and follow up changes. Ultimately all teams should be aiming for this type of retrospective.

The second problem with RIR’s is freshness and mainly the formal retrospectives. If you are holding these every second week and doing the same exercises they quickly become boring. The trick is to vary the exercises and try new things. Again this is one reason why I’m happy to use informal retrospectives, more variety.

I’ve recently had success using dialogue sheets as a facilitation tool in formal retrospectives. I’ll post my template online soon and blog about it in more detail.

One of the big difference between BEPR and RIR is the timeline. In a BEPR the creation of a timeline is normally a (even the) major event. The team map out what took place and when it happened, and then reflect on it. If you are holding RIR then creating a timeline for, say, 2 weeks then it doesn’t get you very far. Hence the need for more activities to trigger reflection.

As you might have gathered from above, RIR are in many places displacing BEPR as teams move to more Agile development. Generally this is a good thing but I suspect the BEPR is due a comeback and I think the force for this will be the Kanban development process.

Two factors make RIR less suitable to Kanban. Firstly the most advanced Kanban teams don’t bother with iterations. The pull system renders them less necessary, therefore the natural place to hold an RIR disappears.

Second, drawing from Lean, Kanban teams may well adopt a “Stop the line” mentality. That is, if the team see a problem the immediately address it rather than waiting for the retrospective to come along.

Yet I still think there is are occasions when it is worth the team stopping and actively taking time to reflect. In Lean we might think of this as a Kaizen or Kaikaku event. At such time a BEPR retrospective might be exactly right. Still, schedule them every quarter. Make them frequent enough to make a difference and avoid the problems my client had.

Regular readers of this blog will know I have a thing about Conway’s Law. In my mind it makes sense and it links what I did before (software coding and architecture) with what I do now (organization and process improvement). My logic is: If Conway’s Law holds then architecting software starts with the organization.

So it was with interest I ran across a working paper from Harvard Business School entitled “Exploring the Duality of Product and Organizational Architectures: A Test of the Mirroring Hypothesis” (Alan D. MacCormack, John Rusnak, and Carliss Y. Baldwin). The author’s note Conway’s Law as one example of the mirroring hypotheses and cite several others too – which adds to the evidnece that there is something here.

(The link above is to a March 2008 version of the paper, the one I downloaded a couple of months ago was version 3.0 and dated October 2008, I’ve no idea what, if anything, changed in that time or where the October version is for download.)

In this paper the authors try and show statistically that organizational form does effect software architecture. Most of the paper’s 33 pages lays out in detail how they did this, and you could, if you so wish, poke holes in their techniques and question whether the analysis holds up.

Disclaimer, as is my usual practice I’ve read the introduction, the closing discussion and some of the material in between. Life is too short to read 33 pages of academic prose but it could be I’ve missed something, please let me know if I have.

That said, if you are willing to accept their hypothesis and method. There are two minor points that give me concern: 1) all the code they look at is Open Source, 2) we know little about the development practices employed by these teams. Both facts could skew the results, still I’m willing to give them a broad acceptance – then they show the hypothesis stands up.

The authors say:

“Our results reveal significant differences in modulality, consistent with a view that larger, more distributed teams tend to develop products with more modular architectures. Futhermore the differences are substantial – the pairs we examine vary by a factor of eight….”

And later:

“our study provides evidence to support the hypothesis that a product’s architecture tends to mirror the structure of the organization within which it is developed.”

There are many implications of this work and several are discussed by the authors. For me a few things come immediately to mind:

• The work of the architect and the manager are more difficult to separate than is commonly assumed. The organization will always effect the architectural.
• If you want a highly modular system distribute your team.
• This is why broken organizations usually have broken architectures too.
• When fixing a broken organization and/or architecture: fix the organization first.

This paper in common with most other work on Conway’s Law emphasises the way the architecture mirrors the organization. In so much as thisis true and it happens at first, organization determines architecture.

I also believe the reverse can be true. The organization can come to copy the architecture. In a world increasingly dominated by existing “legacy” systems it is common to see development organizations split up to model the software.

For example: Front End (UI) team, Business Logic Team and a Database team corresponding to three layers in a software. This may lead to a modular design in time but it complicates co-ordination. It becomes more difficult to make changes to the code and change the organization.

While most of the software world (including myself when I code) cling to the layered model it goes against object-oriention. In OO the object is should be self contained, too much layering and the object model breaks down. Layering is procedural not OO. The same thing happens in organizations. But all that will need to wait for another blog entry.

More on Conway’s Law:
• What do we think of Conway’s Law now? (EuroPLoP 2005 focus group)
• 2006 Blog posting: Return to Conway’s Law
• The original paper from 1968: How do committee invent?

The new year is just here! And can I suggest a new year’s resolution?

Normally I bite my tongue and don’t mention this. I’ve not blogged about it before because what ever client(s) I’m working with at the time I blog about it will automatically assume I’m talking about them. (Yes, I know people at client sites find and read my blog so I try to self-censor.)

But, I’ve just finished up a major engagement (coaching 4 development teams in the same company) and new stuff kicks off in the new year, so I can say this for once.

Please, don’t take your laptops to meetings. And if you do leave them powered down.

If you go to a meeting it deserves your full attention. If you don’t want to give it, or can’t give it, than make your apologies and say: Sorry I can’t make it, go on without me.

If you do go then give the meeting your attention. OK, occasionally we all need to take a phone call but it is far less often than we think. I remember breaking my own rule a few years ago to take a phone call, but then I was waiting for my farther to die.

Phone calls are different to laptops. A phone call is an interrupt. It says “Someone wants you.” By definition if someone has sent an e-mail it is less important, or at least less urgent. So as much as I dislike meeting interrupted by phone calls I can accept them – personally, I switch my phone to silent and reject calls that come during meetings.

If you are so important you carry a Blackberry (or similar) and urgent (meeting interrupting work) arrives via e-mail it may be a sign that your organization can’t prioritise. But at least a Blackberry is small.

The problem with a laptop in a meeting is it creates a barrier between the user and the other people in the meeting. Every time someone lifts their screen I see a sign that they don’t want to be here, don’t want to hear what is said, and don’t want to be part of the team.

That screen is big. Its a big barrier. Its a sign. It says “I don’t need to interact with you people.”

And what do people do with these laptops?

I tell myself they are answering e-mail, important requests that cannot wait. But… well… I heard about a case recently where the senior manager in the meeting was booker herself a flight on EasyJet. I once went to a meeting which was so packed I had to stand at the back, from where I could see a manager (director level) surfing Amazon.

And people who are typing? Are they really replying to an urgent e-mail or having an Instant Messenger conversation with their significant other?

I once took an IM from a colleague in the New York office. Later in the day I saw him in the London office. He’d landed that morning and spent the first few hours in the London office in meetings. So when I IM’ed with him he was in a meeting on a totally different subject and he could have asked me face-to-face an hour later.

Please, stop taking your laptops to meetings. I’d rather you make your apologies and not turn up at all.

To read the papers one would sometimes wonder if there is a future for software development, indeed IT in general, in Western Europe and North America. Sometimes it seems there is an continual flow of work to India, China and other low cost centres. Personally I think there is a future but only if development groups fix themselves.

First lets be clear: I’m talking about corporate IT here, not independent software developers or other companies who develop software to sell. That is another story, another arguement, lets still to corporates today. Corporate IT departments support business operations by providing them with custom software.

Second there is another force at work: the commoditisation of business processes and the IT that support them. Lets leave that to one side for the moment.

So, I’m specifically talking about IT groups within businesses that need custom software to support some business activity.

Unfortunately such corporate IT departments do not have a good track record. Something like 87% of all such departments are ineffective and do not give the business what they need. (I’ve taken the figure from The Alignment Trap reports which I’ve blogged about before.) In such circumstances IT represents a cost not a benefit. Such companies might be better off not doing IT at all.

If you are doing it, and you are not seeing the benefit, and you are paying the cost then its sensible to look at reducing that cost. Outsourcing and/or offshoring is a valid option.

It might not be nice to hear but if you work in such a group it makes sense to send the work elsewhere. Sending it elsewhere might not fix the original problem but at least it will be cheaper.

So what’s the solution? How do you save your job?

Instead of looking at the cost look at the benefits. Some IT groups have forgotten how to do this simply because their delivery record is so bad. When you only consider the cost side western corporate IT cannot compete, give up now. In order to compete you have to look at the benefit side to.

When things are going well, when an IT group can deliver and can deliver benefits then it is less likely that management will want to change things. When things work, and work well, then any change is a risk. When things don’t work the risk is lower.

Next you need to maximise the benefits. That means looking at how you can add more benefit. And specifically add benefit in ways that offshore low cost centres can’t: quicker turn around, reduced documentation, more interaction with the business.

There is a price to pay in moving work to a low cost centre. Such centres have disadvantages: language differences, cultural differences, distance, time zones, lack of domain knowledge. Sure they can offset these differences but that involves costs. Plus they are getting more expensive (particularly as the pound and dollar fall).

Corporate IT departments can demonstrate their value by responding to business needs more quickly, and by reducing the cost of servicing those needs. When work goes offshore the turn around rate slows because work must be communicated. That means it needs to be more closely defined – this is especially true if the work goes to a third party. Opportunities to change’s ones mind are reduced and the potential for differences increased.

Yes offshore centres can offset these differences but at a cost. Each time the cost goes up the advantage is reduced.

So if you fear your work may go offshore heres what to do:

• Demonstrate the benefit of your work – move the conversations from costs to benefits
• Concentrate on co-operating with your business customers
• Turn work around more quickly
• Reduce the cost of requests and changes of mind – reduce the paperwork
• Work more flexibly
• Understand the business and find ways to innovate

I spent Thursday and Friday last week at XP Day 2008. The conference changed its format from last year. Rather than being a conference aimed at people wanting to do Agile Development it focused on people who do Agile development. This meant the sessions changed rather than being introductory or “how to” material they became Open Space discussions. Instead of being lecturers session leaders were facilitators.

Very few sessions were arranged in advance but one that was was Designing the Agile Company led by Giovanni Asproni and myself. This session used a dialogue sheet technique to promote discussion.

XP Day also had space for Lightening Talks. These were unscheduled talks lasting no more than 5 minutes – so very focused. I briefly outlined the Requirements Trap – or rather the Alignment Trap which I blogged about last year. This short talk got a lot of interest and a lot of people asked for the references so here they are:

• Original Alignment Trap article in the MIT Sloan Management Review
• An abridged version from the Bain Consulting website
• My blog entry on the Alignment Trap and Agile software development

I enjoyed XP Day a lot and learned a bunch of stuff – mainly by having to think about it. Yet I found the conference lacking because I found myself speaking to the same people; I already known most of these people, I read their blogs, I talk to them at XTC and a bump into them at other conferences. One of the nice things about Oredev last month was new faces. I guess you can’t have everything.

Jason Yip works for ThoughtWorks in Australia. He is currently (or maybe was) on a Lean study tour in Japan – I’m jealous, where do I book mine? – and he’s blogging about his findings. Well worth reading, so far several points have stood out for me:

• Andon cords are being pulled all the time: a Toyota factory isn’t the perfect place some of us (myself included) imagine sometimes. Faults happen, things need to be fixed and improvements made – all the time.
• Toyota do pay bonuses: for ideas that lead to process improvement
• Hansei is the Japanese word for serious reflection. In the Agile world Retrospectives are Hansei. As regular readers know I always suggest keeping your own personal journal for reflection, more Hansei.
• Pika Pika is a Japanese term translating to “spic and span”: workers keep their work area clean and tidy. Obvious really, how else can you see the problems? (I’m always shy of people taking cards of the board and putting them on their desks after one engineer I knew lost several this way. We tidied his desk and found work.)
• “The Toyota Production System (TPS) is not a system; it’s a religion” – to all those who have critised Lean and Agile for being too religious, you were right.

If that isn’t enough, Jason also has a nice set of hand drawn sketches to illustrate Agile and Lean!

Turns out books about Product Management are a little likes buses: None then 3 turn up at once!

I’ve been looking for a good book on Product Management for a while so I was interested to see Tuned In by Craig Stull, Phil Myers and David Meerman Scott published. I’ve not had a chance to read it yet (so don’t take this as a recommendation) but I intend to.

Then, at Oredev I met Luke Holmann and had the pleasure to hear him speak. Turns out Luke has a book in the product management space too, its been out for a couple of years but somehow I’ve missed it. Again I’ve not had the chance to read it yet but I intend to, it is Beyond Software Architecture.

Finally, one of the Product Managers I’m working with just now leant me a copy of another book, Expert Product Management by Brian Lawley. This is small book and as such doesn’t cover the role of the Product Manager in full. It simply covers several aspects of product: Road mapping, Beta programmes, Reviews and Product Launch. There is good information here and as its small you can read it quickly. Still, I’m hoping on of the other books will turn out to be the Product Management book I’ve been looking for.