Continuing my follow up to my latest tell of the Alignment Trap in the Agile Journal and on InfoQ.

Lets get one thing straight: I’m not saying requirements don’t matter.

What I’m saying it: when the context is a broken development organization the most important thing is fixing the delivery process.

In which case, you can, and should, take liberties with the requirements process. Then, when delivery shapes up you should come back and fix the requirements issues. Because requirements is where the long term gain is.

Anyone who thinks I don’t value requirements should look at all the postings on this blog about the importance of Product Managers. The requirements is also where the long term future of Agile is.

Look again at those 4 quadrants:
• Maintenance zone: this segment explains why so many of my friends work at dysfunctional IT shops.
• Alignment trap zone: this explains the traditional approach has failed so badly; companies tie themselves in knots trying to do the right thing when they can’t actually do very much.
• Well oiled IT: this is where Agile takes you. But it leaves you there.
• IT Enabled growth: where we all want to be.

Most companies are currently stuck in the maintenance zone. There are three types of company here:
• those who are happy in the mud; i.e. those who lack the will to do something about it, those who believe IT is always like this, those who earn so much money elsewhere it doesn’t matter.
• those who are trying to move to the Alignment Trap; many will still be drinking “do the right thing” cool aid and are making it worse for themselves
• those who are trying to get to the Well Oiled IT, probably via Agile development

The downturn will finish off many of the companies in the first two groups. Those in Well Oiled will probably be OK, and those who can move quickly will hopefully survive. Still, too many companies will survive despite being stuck in the Maintenance Zone.

There is a whole Agile industry (including me!) helping companies make this shift. To be honest, I’ve come to the conclusion that this is the easy bit. (Don’t believe me? Call me.)

But, what do you do once you get to Well Oiled?

The problem is that while Agile as we know it will get you there it won’t take you any further. It runs out of ideas.

Companies could happily stay in Well Oiled, its a (relatively) nice place to be. And you will probably out compete your competitors. But some of the companies that make it – and those who are already there – will look to go further.

This is where Chris Matt’s point on the Agile Journal site are relevant. Agile as we know it, has largely ignored the Business Analyst and Product Manager roles.

The onsite-customer in XP lacks any kind of strategic view. It worked for the C3 team but they were just replacing an existing system, not inventing a new product or doing great innovation. Even so, the original customer nearly had a breakdown.

Scrum is slightly better, it has a full Product Owner role, but it says nothing about how this role is undertaken. It only speaks about the Product Owner in the Scrum process.

And Chris is right about “user stories.” They are a simple tool for communication with developers. They are not sufficient. To make matters worse most teams use them isolation, they do no adopt scenarios, actors or personas.

How do we fix the problem?

Here are three steps to get started:
1. Recognise the importance of the Product Owner role and staff it properly. You need 1 Product Owner for every 3 to 7 Developers. The newer and more innovative your product the more you need. If you have an older less innovative product you might have 7 developers to 1 Product Owner.
2. Stop looking to Project Managers to fill the Product Owner role. They are not Product Owners and their training is very different.
3. Decide whether Product Owners in your company are Business Analysts or Product Managers. If you develop software for your own internal use (or one client) then they are BAs. If you sell your software (shrink wrap or online) they are Product Managers.

If we are to move beyond Agile software development and build Agile companies it is essential we respect requirements. So far our success has all been based on shifting out the maintenance zone, the next challenge is to move on up.

I’m used to getting a reaction from people when I outline the Alignment Trap, and my article in the Agile Journal – and the subsequent report on InfoQ – certainly stirred up some reaction.

Immo Huneke mailed to point out that the Sloan piece doesn’t discuss how you measure alignment. I’m not sure how I would go about measuring alignment myself, after all what is alignment? Can it be measured objectively?

There is a point here, and I myself am a guilty of making a jump from alignment to requirements. I’ve also used the phrases “doing the right thing” and “doing things right” quite loosely.

The other point which we could argue about is “IT”. The Sloan piece talks about IT. There is no information on whether this relates to hardware, software, IT support or software development.

Despite these caveats I think the central message still holds: getting effective is more beneficial than getting.

Still, I want to make it clear I think knowing what you are doing is essential. And in the long term requirements aligned to the business are very important – see the blog entry that comes after this one.

Its all a question of context.

If you want to fix a broken IT organization I claim, and I believe the Alignment Trap supports this claim, it is best to start by making the organization effective then determining what needs doing.

This context implies that there is an organization, and it is doing “stuff”. It knows some of what needs doing. You might be able to remove some “stuff” (or add more) but basically it is not short of work to do.

I believe that in this situation discussions over what to do (alignment, requirements) are not effective because the history of failed deliveries means there is no trust and it means the business side tries to game the system by asking for more than they need because they believe It will always under deliver.

This context also implies that the IT group is, in general, supporting the business. If your company sells Payroll software and your developers are currently writing a competitor to Grand Theft Auto then I think you would be right in question requirements on day one. But in general, if you sell Payroll software it is likely that your development group are working on Payroll software.

In other words: you are better off being generally right than precisely wrong.

Before you can tackle the requirements problem (and alignment) you need to build a delivery machine that can tackle it.

The Machine that Changed the World contains an example I think illustrates the point here.

Producing the body of a car requires a “stamp” to shape the metal for the car body. These stamps are large and take many months to create. In the 1980’s at General Motors a new car would be designed and when the design was finished work on a “stamp” would start. This created a delay of a year or more in producing the new car.

At Toyota the general size parameters of a new car would be agreed early on in the process. The stamp team would start work on the new stamp before the design was finished. Only as the design neared completion would the stamp team complete the stamp. As a result Toyota did not have the same delay.

The Toyota guys started off being generally right and narrowed in on the exact dimensions over time. At GM the desire to be always right held work back.

A provocative piece from me in this month’s Agile Journal – Requirements Come Second.

If you’ve not checked out the Agile Journal recently you might want to have a look, they’ve given the site a make-over and have added blogs, forums and all that community stuff.

Bristol BCS are holding a series of sessions entitled “Spring School – An Introduction to Agile Development”. I will be presenting the fourth and final session (23 March) on the “Future of Agile”.

I’ve not written it yet but it will probably include plenty about Lean and Kanban, the Change Problem and the importance of Business Analysts and Product Managers.

Fellow ACCU member Giovanni Asproni is doing the session on 16 March on Agile testing and TDD.

If your in the Bristol and Bath area I’m sure it will be worth the £30-£50 charge.

After slogging my way into London’s through the worst snow in 18 years on Monday and Tuesday this week to deliver a Agile Project Management course, I was a little relieved when Kingston & Croydon BCS cancelled last nights presentation – because of the weather. Still I was a sorry to miss the evening.

If you were one of those who planned to come along all I can say is Sorry, I believe it will be rescheduled for a later date.

But, when I opened my e-mail I had a very nice surprise…

My book, Changing Software Development: Learning to become Agile is going to be translated into Japanese. I can’t wait to see it. This was quite unexpected and I think goes to show how well the book has been received.

Now if any Japanese readers would like me to do a talk in Japan I’m more than happy to oblige, although I will have to ask for expenses.

Writing that last blog entry on my introduction of Kanban also led me to reflect a bit on my Blue-White-Red Agile method.

I’ve long felt Blue-White-Red was largely derived from XP, thats what the developers were excited about. I called it Blue-White-Red because it was a cut down XP, I was introducing the ideas incrementally. Also, the word “extreme” frightened the managers and “pair programming” frightened most developers.

In the last few years I seen more and more Scrum in BWR. At the time I “invented” it I hadn’t read much about Scrum books so maybe it is surprising how much like Scrum it looks. But, big but, I think this also goes to show how little difference there is between XP and Scrum. I’d have to go back and read the original XP book to be sure but it seems to me the main difference is XP’s emphasis on engineering practices (TDD, refactoring, etc.) while Scrum emphasis the project management bit (just without the project manager).

The other influence was Lean. I’d started reading about lean sometime around 2000. In fact I wrote about “Thinking Beyond Lean” in the April 2002 issue of Overload. I’d also been exposed to a lot of lean on my MBA course.

I distinctly remember telling people the white board was a “Kanban board” and when people asked “What if I have too many cards to put see the board?” | telling them “That is a sign you have too much work on the go” – I was attempting a crude form of work in progress limits.

So what would I recommend to teams in future?

Well as always it depends. But I’m going to favour Kanban. I think Scrum (and BWR) still have their place, they are more suited to environments were large project specifications exist up front. In addition, Scrum has the advantage when the organization needs to see external validation and BWR has the advantage because there is less baggage. There is less written about it.

That last bit might seem a little off but I mean it. The fact that it is less well knows means it brings fewer assumptions, fewer stories about how it failed and perhaps a cleaner sheet. Kanban has many of those advantages today but I don’t expect it to last, success will remove them.

This is the third in a short series of blog posts which started with me explaining why I don’t like Gantt charts as a “project” planning tool and then suggesting an alternative approach.

Short term project planning (1 to 6 months)
A regular release cycle is the first step to delivering. Decide what your cycle is, 1 month, 2 months – probably no more than 3 months and do work in those chunks. Size the work to fit in the cycle. Within the release cycle have several work cycles (iterations) of several weeks length. The more variable or unknown your work the shorter your cycles.

Its OK to sketch out what will be in upcoming iterations and cycles but don’t get too attached to what is allocated where. It will change. And for that reason don’t go into too much detail – certainly no estimates.

All work that goes into the iteration or cycle is rigourously prioritised – 1, 2, 3, … to N. (As mentioned in Run Scope Creep backwards recently.) At the start of each cycle, then each iteration, the team – and business owner – can have detailed conversations about what is to be done and how it is to be done. It doesn’t make sense to have these conversations earlier – because less is know about the need and thing might change. And it doesn’t make sense to have these conversations later because that is too late.

These items aren’t done until they are completely done. All or nothing, no bugs left to fix, no UI to complete, no db schema to revisit, etc. Done as in ready to go to customer.

Ideally each release should have its own small goal but sometimes it ends up being a hotch-potch of things. And ideally the team should commit to doing the work. Many Agile books and methods talk about the importance of commitment, I agree but… its hard to get one day-1, it comes with time.

Roadmapping: Medium term (3 months to 1 year)
For the 3 months to 1 years period create a near term product roadmap. The roadmap is a living document which should be changing. It should be marked with the time buckets of your cycles and no finer. Things can and should move between buckets but everything except the bucket dates is flexible.

A roadmap is not a direct substitute for a project plan. Roadmaps are different. They are need driven not date driven. They are an artefact of product management not project management. They are a learning tool not a commitment. They create discussion not enforcement.

And yes, sanitised version might be shown outside the company to customer but only with the caution that it might change.

Roadmapping: 1 year to 5 years
All roadmaps are rolling but beyond the next year it doesn’t make sense to have so much detail so have another roadmap for the mid-term. You can get more general, more speculative simply because more things can change. So keep your near term and long term roadmaps separate.

This roadmap should show major events inside and outside your company: planned IPO, proposed legislation changes, anticipated competitor products and major event with your major customers.

So why are the roadmaps I describe better than project plan Gantts? Well…
• They are designed to keep the debate open and keep discussion on what is needed
• They do not forecast into the future they are one scenario
• Items in on the roadmap turn into goals supporting the larger visions for the product and company
• Dates are kept vague and planning is separate from commitment

See my Product Roadmap pattern for more about roadmaps, this is part of my business pattern series.

Beyond 5 years: Strategy and scenario planning
These technique tends to work best in the short term. In the long long term where you are going, what your product looks like are a matter of business strategy. Top management should pass down some direction here. Planning for the longer term needs different tools. For the long term – 5 years or more I suggest you start with some scenario planning.

There you go, a rough guide to project planning. Some people would call it Agile project planning, personally I don’t care what you call it. If you want more details it depends on your situation so get in touch.

• Project plans (part 1 of 3): Why I don’t like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) – Planning cycles
• Run Scope Creep backwards

As my last post was a moan I promised to give some advice on how to go about project planning. There is a lot to this and I can’t possibly give all the details in a short blog so I’ve split this into three blog posts. Still there is a lot more I could say.

Turn the question around
Instead of a confrontational question and defensive answer have a rich discussion.

When someone asks “when will it be ready?” turn the question around – (1) When is “it” needed by?

When does your business need a product to use? What are the important dates for the company – end of quarter? end of year? trade show? competitors? contract signing?

I always prefer running a project were there are hard business driven milestones to make. Dates which mean something.

Next (2) What is needed on these dates?

Unfortunately too many conversations in the software business seem to involve someone saying “I need everything and I need it yesterday” – not a very useful position, and if its true you should start looking for a new job because there is no way anyone can make that happen.

Look at each of those date and define what is needed. What are the qualities of the thing that is needed? What is the minimum marketable product or feature for that date?

At this point you know: what is needed, how long you have, and who you have to do it.

Yes, I slipped that last one in because its true: you have the people you have and in the short run your resources are constrained because of Brook’s Law and because hiring people takes time.

So within these parameters the engineering team can now come up with options. How can this need be met within this time with these people? This activity separates the real engineers from the rest. Engineers create solutions within these parameters.

Wikipedia (today) says: “Engineers are concerned with developing economical and safe solutions to practical problems, by applying mathematics and scientific knowledge while considering technical constraints. As such, the work of engineers is the link between perceived needs of society and commercial applications.”

And if you cant? Then go back to the “customer” and discuss it. Find a smaller feature, extend the time (shoot for a different trade show), find an alternative way of delivering.

(Tom Gilb has some good examples of engineering solutions into tight constraints and his impact estimation tables can come in useful here.)

Vision
All work needs to be guided by a bigger vision. You can call it a vision or a goal if you wish I don’t mind. Personally I like the term BHAG – Big Hairy Audacious Goal. You’ve got to know where you are going to give context to everything else.

There are two ways to run a project. The first – exemplified by Gantt charts – is to get all the pieces of string you think need to do the project, lie them end-to-end and see where you get to. If you read my previous post you’ll see why I don’t like that approach.

The second way, I’ll call it the BHAG way, is what I discuss above. Decide where you want to be, when you want to be there, what is will look like and then find a way of getting there. How you get there is called engineering. It is what engineers do.

My favourite example of the BHAG way? The Digital OpenVMS team committed to a delivery schedule and said “We don’t know how to achieve this, but we commit to finding a way.”

• Project plans (part 1 of 3): Why I don’t like Gantt charts
• Project plans (part 2 of 3): Turn the question around
• Project plans (part 3 of 3) – Planning cycles
• Run Scope Creep backwards