I have been preparing some training material for a company which wants to know more about how Agile projects fail. Fortunately I’ve made a habit of recording Agile failure modes in this blog so I have much of the material to hand. (Use the search box above and search on “Agile failure modes.”)

But, I’ve not got exclusive rights to this topic and others have done the same thing, often with catchier names. So, for completeness:

• Wagile from Jason Gorman
• ScrummerFall from Mitch Lacey
• ScrumBut from Eric Gunnerson
• WaterScrum from Scrum Community

As far as I can tell these are the original sources of these terms but I may be wrong. Others have also used the term and some have Wikipedia entries so I could be wrong.

The Agile Journal this month has an article from me this month entitled Dissecting the Product Owner role.

This complements my pieces on the Business Analyst and Product Manager roles which appeared in ACCU Overload earlier this year.

I’ve finally, about 6 months after everyone else, got around to reading the Software Engineering Institutes report on CMMI and Agile. Entitled “CMMI or Agile: Why Not Embrace Both!” the report argues that CMMI and Agile are not only compatible but complement one another. Actually, you can guess at what the report says from the fact that the title ends in a exclamation mark (“!”) rather than the question mark (“?”) you might expect.

The report argues that the two are compatible and complementary. They make a strong case and I for one am glad. CMM initiatives have too often made the words “process improvement” dirty words. They shouldn’t be, it is what we should be doing.

The report argues that CMMI, and its predecessor CMM, have been misunderstood and misused by the industry. Firstly CMM(I) was taken to be a methodology when it was a model. CMM(I) doesn’t tell you what to do, it allows you to evaluate your capabilities.

The second mistake compounds the first by imposing CMM(I) on organizations. Rather that improving an existing process and organization CMM(I) initiatives have replaced what already exists with something else – and in the process destroyed the good that existed. This is the scenario I wrote about a few weeks ago in “A true story about CMM”.

There are a few points were I could take issue with the report.

My first issue is that the report does not define what it means by ”Agile.” I agree this is a problem, and one I’ve been thinking about addressing in this blog. But, CMM and CMMI are very well defined while Agile is not. How can one critique a method without defining what is mean by the term?

Second, the report seems determined to identify an “Agile Institute” to mirror the Software Engineering Institute. It seems to spend a lot of time talking about the authors of the Agile manifesto and founders of the Agile Project Leadership Network. Neither of these groups owns the term “Agile” or speaks for the whole Agile community. There is much under the Agile umbrella which, rightly or wrongly, falls outside these two groups.

Comparing Agile and CMMI is an asymmetrical comparison. One is clearly defined, with a well funded champion to issue standards, revisions and issue certificates. The other is the ill-defined product of an ill-defined community which owes as much to the punk ideal as anything else.

A bigger issue with the report concerns why you might want to bring CMM(I) and Agile together. The report spends several pages early on explaining that the CMM grew out of one type of environment with one set of problems. Namely large projects, often military in nature, often with life at risk, and with a low-trust relationship between supplier and client.

Conversely, Agile came from a different environment: mainly corporate IT groups, small projects (no more than a dozen or two people), with high-trust within the team. Failure to understand these origins and contexts has led to much misunderstanding between advocates of the two.

Given that these two environments still exist, and that Agile has demonstrated success in its native environment and CMM(I) has demonstrated success in its native environment, then why would people in either environment want to adopt elements of the other?

I don’t think this question is really answered. This is a shame because the authors have spent a lot of time setting up a story to explain why we got where we are today but don’t explain how that story ends.

To my mind the story only tells part of the truth. Both CMM(I) and Agile have been sold into many organizations outside their native environments. There are many people who want the respectability of CMMI, and there are others who want the fashion of Agile. Nor am I aware of anyone in the Agile or CMMI camps saying “Don’t do that sort of project with our model.”

I’ll defend Agile here because I believe Agile has, and is, a journey of exploration. The underlying principles are universally applicable but that doesn’t mean all the details are worked out in any given field yet. So if you ask me “Can I do Agile on an X type project?” my answer is “Yes but I may don’t know how immediately.”

However the SEI folks who wrote this report claim CMM(I) has been misapplied. So I ask: which of them was saying “Don’t use CMMI on an X type project” ?

My final niggle is one of clarity. The report has five authors. Several of those authors are quoted and referenced throughout the text to support the arguments made. Yet nowhere in the report, when one of the authors is quoted is there a disclaimer. That is, the report does not say “Smith, one the authors of this report showed the CMMI is effective (Smith 2003).”

The names of the authors aren’t hidden, there is no subterfuge but its not clear and I wish they had been.

On the more positive side there are several quotes in the report worth calling out. About Agile and CMMI:

• “In today’s increasingly dynamic world, CMMI-based organizational process improvement approaches cannot rely exclusively on traditional project management approaches, waterfall-based project lifecycles, and heavyweight analysis methods.”
• “CMMI and Agile can complement each other by creating synergies that benefit the organization using them. Agile methods provide software development how-to’s that are missing from CMMI best practices that work well”

And about CMM(I) and the approach companies have taken in the past:

• “Too often, CMMI has been applied rather than implemented”
• “SCAMPI appraisals are not audits or tests”
• “the complexities of large-scale projects require a more involved infrastructure, this fact is not a license to create unnecessary bureaucracies and unbalanced production at the expense of productivity”
• “Using process experts to create and deploy process improvement activities is not unusual, and often has the advantage of allowing these experts (and the project teams) to focus on their respective tasks. But what makes this approach risky is that project personnel are frequently left out of process design activities and are disinclined or openly skeptical toward the adoption of process improvement activities. This situation is typical of some Six Sigma style approaches to process improvement as well. Quality process teams are created and given the responsibility for quality. As a result, developers abdicate their responsibility for improvement (or worse, for quality).”

Interestingly I also noticed that the SEI’s own Team Software Process (TSP) is now listed as an Agile method. I’ll have to look into this some more.

I am also please that the report highlights the role of organizational learning in software development. Regular readers know this is something I’m very keen on (heck, I wrote a book on the subject!) so I’m pleased to see it explicitly considered.

Even with my reservations I welcome the report. I believe it is a roundabout endorsement of Agile (in its various forms) by the Software Engineering Institute. It is also a very big sign saying “Agile and CMM(I) are compatible.”

At the end of last year, or was it the start of this year, I wrote a blog entry which got a lot of attention: Leave your laptops out of meetings. I was talking specifically about laptops but I could have easily included Blackberries, iPhones and other portable devices.

In todays Financial Times two professors from Toronto Rotman School of Management make the same call: Why e-mail must disappear from the boardroom. Their piece specifically addresses e-mail and boardrooms but its the same argument I was making.

Professors Beaty and Weber wonder what how directors can concentrate on discussions when they are doing something else. And the question what will happen when someone decides to sue a company for decisions taken by directors who are not concentrating.

So there are three good reasons to leave your laptop out of the meeting, switch off your Blackberrry and ignore your SMS messages:

1. You cannot do your job properly is you trying to do two things at once
2. You are opening your company to negligence claims
3. It is rude; you are insulting the people who are in the room asking for your attention

For the record I very really take a laptop to a meeting, only really if I have something on it I want to share. I don’t own a Blackberry, iPhone or other portable e-mail device. And I set my mobile phone to silent when I arrive at an office.

I am guilty of occasionally viewing an SMS during a meeting, and even more occasionally sending an SMS during a meeting. I apologize to everyone in a meeting where I have ever done so.

I fully support Beaty and Weber’s call: think of the message and example it would send if a company board stated that Blackberries and laptops were not to be used in meetings.

I recently came across the expression “The Scrum Wall”, as in the expression “Hitting the Scrum Wall”. Its akin to “the pain barrier”, or “feel the burn” in aerobics workouts of old.

Once I heard it I knew what it was, I’ve talked about this before, now I know it has a name.

The Scrum Wall is the thing Bob Martin described at the ACCU conference. And it is probably why Jeff Sutherland endorses Test Driven Development and other technical practices from XP.

You hit the Scrum wall when you adopt Scrum and everything goes well, then, after a few Sprints things don’t work any more – to use an English expression, they go pear shaped. You can’t keep your commitments, you can’t release software, your customers get annoyed and angry, it looks like Scrum is broken.

This is what happens when you adopt Scrum without technical practices such as Test Driven Development, continuous integration and refectoring. When teams adopt the Scrum process, they go faster, show progress, things look good… and then the quality becomes a problem. Now the team are fighting through quick sand.

The code quality is poor and developers are expected to continue to make progress. Maybe the Scrum Master/Project Manager reverts to past behavior and demands overtime and weekend working. Maybe the team start busting a gut to keep their commitments. Either way the team is heading for burn-out.

That’s the wall.

About a decade ago I worked for a major supplier of financial information in London. The company decided it wanted to improve its software development process and the answer to this problem was to adopt CMM – the Capability Maturity Model. Initially the become level 2 and then move on the level 3.

The little group I worked in interfaced to financial trading institutions in the City of London. My project in particular had to respond quickly to changes at the institutions.

The CMM roll-out was a akin to a neutron bomb in reverse. The people were left in place, but their processes and practices were replaced en-mass. Developers attended CMM training and instructed in how to develop software according to the new process.

The effect was devastating. One manager likened it to pouring quick setting concrete on the railway tracks of progress.

I went out and I bought myself a book on CMM so I could understand it. What I learned was that CMM was a way of measuring maturity. It was a meter, imaging it as a ruler. It was a way of assessing your organization. You could measure yourself as 1 CMM, or 2 CMM, up to 5 CMM. It was not a method of working.

The CMM body of knowledge also contained recommendations, “best practices” and examples of good practices but it was not itself a process. It was a meter for measuring your maturity.

From the book I learned that Watts Hymphrey advised companies to: focus on improving their processes first and let the CMM measures improve by themselves.

What I observed in practice was the destruction of working practices an processes. I never observed any attempt to involve those doing the work in process improvement. The consultants knew best. Opposition was to be overcome. Process improvement was something that was done to others.

This wasn’t the company for me and I left. The company was eventually certified as CMM Level 2 and was expected to move to level 3 in time. Time went by and I don’t think the company ever made level 3. I heard that internally CMM was recognised as a mistake and the work was quietly allowed to die. The company also announced major moves to offshore development.

Fast forward to 2009. I recently met someone who was an outside consultant to the company and advised them on CMM adoption. He told me that he, and the other consultants, had advised the company to build on the practices they had and improve them. They specifically advised them not to write new processes and impose them on people.

He also told me that when the company did its own retrospective on the CMM project it was seen as a mistake to have written and imposed new processes on developers. A far better way would have been to improve what was in place with the people who did the work.

So the moral of the story:
• There is a lot of good in CMM – and the more recent CMMI.
• CMM itself is not a process, it is a meter, it is a means of measuring yourself.
• The key is process improvement, focus on improving and the levels will come.
• Don’t replace, improve, evolve. Involve your own people.

I recently gave some coaching to a big, well known, UK company get an Agile team started. Its interesting, this team have a few novelties all of their own, I’ll blog about that some other time. But they also have all the same questions and doubts that other teams have.

So here is a kind of Agile FAQ, the questions this team asked which all teams seem to ask sooner or later.

Q: What do we do about work items that are too big to fit in one iteration?
A: We will break the work down. We will break business requests into developer tasks. We will implement the minimally marketable feature the business agree to. We will talk to the business customer about what is required, we will implement the smallest thing that will work. It might be new, it might take some time to get the hang of it but it is doable.

Q: Should we have longer iterations? (Because our work items are so large)
A: No, if you have doubts you should have shorter iterations so that a) you see problems sooner, b) you get more practice at breaking work down.

Q: How will I know what to do if I don’t write a technical design?
A: Design is good but do not do more than is needed. Writing a document can be a useful learning exercise but there are other, and faster, ways to do similar things. Technical designs are best done at the white board with a colleague. If you need to document technical knowledge then document it after you have written the code/done the configuration.

There is no place for architects and software designers who are removed from the team. Design and architecture will evolve over time so needs guiding. This means experienced designers need to be involved (preferably coding) with the team daily.

Q: How will the business know what we are doing if we don’t write a functional design?
A: The business will be working with the team in planning meetings and will be seen the work as it is delivered.

Q: How do we manage dependencies between tasks?
A: Identify them and order the tasks accordingly. If need be break them down further.

Q: What if the business wants us to do A and B but not C?
A: Then do A and B but not C. The business pays the bills. If you need to do C for technical reasons – such as system quality – then do it. Don’t do C if its a feature the developers thing is worth having but the business don’t prioritise.

Q: What if the business wants us to do A then B but it would be quicker to do B then A?
A: Explain that to the business but respect their final decision. If they want B sooner then do it first.

Q: What if there is something technical which the business don’t understand and we need to do? The business won’t let us!
A: Explain it in terms the business will understand – this might require some learning and practise.

Q: Shouldn’t we have a detailed plan for future Sprints?
A: There is no point; requirements will change, your knowledge and understanding will increase and you will be better at estimating later. Only plan the next sprint in detail, you can sketch the work beyond that but don’t spend a lot of time on it.

You might sketch out what you expect to do in future sprints in “Release Roadmap|” but remember this will change as you go. Keep the roadmap under review.

Q: Should I include testing a documentation in my estimates? Won’t that make the estimates too large?
A: Yes. If that is work that needs doing it should be included. Not to include it would start to build an additional, hidden, backlog of work. Equally the customer should see the real price of the work, not the price you want them to see with more to come. And if the project manager gets a shock then so be it.

Q: Doesn’t Agile invite scope creep?
A: Yes but in general I find scope reduces on Agile projects. I call it running Scope Creep Backward.

The online Technology Management journal has published an article by me entitled “A new governance model for agile work.”

In the article I argue that Agile based software development needs a new governance model. The old model, based on defined requirements is not applicable – simply because the requirements evolve when working in an Agile fashion. Instead a governance model based on the venture capital investment cycle is more appropriate.

I used to regular review books on this blog, somewhere along the line I stopped doing it – too many other things to blog about I guess. Plus, while I still read a lot I find it hard to find books which say anything news.

And, I increasingly find I’m reading books before they are published. My friends Rachel Davies and Liz Sedley have been beavering away at their book on Agile Coaching for over a year now. I’ve been reviewing the drafts as they go and I’ve seen it change and improve over time.

The book, Agile Coaching is now available as a Beta download from the Pragmatic Programmers press and the print version will be out in August.

For me the book is a book of two halves. As you would expect half of it is about coaching Agile teams. In that you’ll find that it covers similar ground to my own Changing Software Development – both are about introducing change. While I cover more management and background they cover more personal stuff.

The other half of the book is a nice, modern, discussion of how Agile teams work. Its not Scrum, Kanban, XP or any other method. It describes what you find. Its one of the best introductions to current Agile practices you’ll find. Plus, it incorporates a lot of experience which earlier books couldn’t.

Anyway, the waiting is over see for yourself.