It has been a while since I’ve written about Dialogue Sheets here – or indeed anywhere else. So here is a quick update and a request for some guinea pigs – I have ideas for new sheets but I need some teams to try them out. (If you want to be a guinea pig for a new sheet skip to the end of this blog.)

The impetus for writing this blog is a new French translation of the T1 Retrospective sheet. I am most grateful to Nicolas Umiastowski for this translation which is now available with the other sheets.

Nicolas has blogged (“Retrospective dialogue sheet in use”) – with photos – about using the sheet this week with a team. The post is well worth reading, everyone was involved and it sounds like there was a lot of energy in the room.

More generally, the sheets continue to be used by teams, lots of downloads still happen from the Dialogue Sheets website. Unfortunately I don’t get much feedback on what happens after the download. If you have download a sheet and have tried them then I’d love to hear how it went and what you think.

Now some statistics.

There have been over 3000 downloads since 1 January 2012 from 94 countries.

(A word of warning here. I’ve made no attempt to cleanse the data. I know it contains some duplicates and some fake e-mail address (e.g. a@b.com) so I assume some of the downloads are fake. Given there are now over 3000 downloads I think (hope?) this doesn’t make a big difference.)

Unsurprisingly the USA leads the world with 20% of downloads, then the UK (15%) after which numbers quickly tail away: Germany (6.5%), France (5.5%), Sweden (4.75%), Norway (3.9%), Australia (3.8%), Netherlands (3.5%), India (2.8%) and Canada (2.75%).

I am sure one of the reasons France scores so highly is Laurent Carbonnaux’s translation of the Sprint sheet. Nicolas’ translation could see France overtake Germany. (I hope we’ll get some more translations and have added some notes on translation to the website.)

The most interesting story I’ve heard this year about Dialogue Sheets is from Sweden – the country where they were originally invented. I am told that there is a childrens’ nursery school (kindergarten) where the teachers hold a fortnightly retrospective using the sprint retrospective sheet! Wonderful.

The statistic I still find most interesting the retrospective frequency question.

I ask all downloaders “How frequently do you hold a retrospective (at the moment) ?”

Nearly 8% of downloaders say they are retrospective facilitators. Excluding these people, 47% of downloaders claim to hold a retrospective regularly (every two weeks or once a month.) But 44% say they never or rarely hold a retrospective.

Now I think it is fair to say we can expect those who hold retrospectives regularly to be more interested in these sheets therefore I expect these figures to be biased towards such people. In other words, if 44% of people interested in retrospective dialogue sheets say they rarely hold a retrospective I expect the true number of software professionals who hold a retrospective rarely to be a lot higher. Sad.

Moving on from numbers, some other changes.

About 18 months ago I introduced an Iteration Planning sheet, this doesn’t have so many downloads (over 200 in total). I’ve also written a description of how to use this sheet (same page). I know my style of planning is slightly different from everyone else’s – heck no two teams are ever the same! (Since I published Xanpan it have wondered if maybe I should rename this sheet as a “Xanpan planning sheet.”)

When I do training sessions I leave the teams copies of both the Sprint Retrospective sheet and the Planning sheet to help get them started. By all reports they help a lot.

The print on demand service is still there, it is little used and I was recently able to reduce the prices there. But if I’m being honest I don’t think the price of a Dialogue Sheet is a bit issue for two reasons. Firstly unless you are buying a lot of sheets, and unless you live in the USA, you will probably find the cost of postage greater than the sheets themselves. Hence I think most teams get these sheets printed locally.

Second, I think teams that have to ask for money have as much trouble getting $50 as $500. So it is the fact that the sheets cost rather than how much they cost which is the problem.

Now to the innovations….

I continue to use the Agile Thinking sheet at the end of Agile training courses to help teams talk about how they will put the training into action. This has proved very very successful. Steve Smith has recently adopted a similar sheet on his continuous delivery courses.

While I’m writing this I should mention some other changes that happened a while back, I retired the T2 sheet – it was not different enough from the T1, T3 and Sprint versions. And I updated the T1 and Sprint sheets – the Sprint sheet now has time boxes on it to aid with keeping the retrospectives to schedule.

One more thing… but that can wait for the next blog post.

Last night I did a presentation to the Agile 4 Agencies meet up group on the topic of “Agile Contracts” – although I believe “Contracts for Agile” might have been a better title.

This was based on my InfoQ Agile Contracts piece from 4 years ago – also available as a PDF from the Software Strategy website. My thinking and experience has moved a bit since then and some of that was incorporated into the presentation.

When creating the presentation I very much set out to work from the original piece, although as anyone familiar with my writing will know, this territory touches on “Dear Customer, The Truth about IT” (Agile Connection and now the prologue to Xanpan – which is also available from the Software Strategy website – my stuff has a habit of being reused.)

Anyway, the “Agile Contracts” is now online at SlideShare. And I believe a recording of the night might appear before long…

After posting my last post – Estimate or #NoEstimate that is the question? – I felt a little as if I’d stepped on a land-mine. That is to say I had a few comments and a bit a mini-twitter storm. If I’m being honest I have been avoiding some of the estimates/#NoEstimates debate until now precisely because it is obvious feelings on the topic run high.

Perhaps the thing that surprised me most was that a post intended to support making human estimates was interpreted by many Tweeters as part of the #NoEstimates movement! Maybe convergence between #NoEstimates and #NoProjects is already happening in the public mind.

In the meantime it seems to me that a lot of the problem with Estimates lies in what they are, what they are not, how they are used and how they are mis-used.

As is so often the case it all depends on what we mean by the word, in this case “Estimate”.

Generally I find it useful to agree with Humpty Dumpty:

“When I use a word it means just what I choose it to mean—neither more nor less.” (Through the Looking-Glass, Lewis Caroll).

After all, who can forget Bill Clinton saying:

“It depends upon what the meaning of the word ‘is’ is.”

While I am sometimes guilty of the use and misuse of words myself I find it helps to keep an open mind on just what someone means when they use a word or phrase. For example, if a developer says “Unit tests” I try not to jump to assumptions about what “Unit tests” actually are. The fact that such language is used is itself interesting but I also want to know what they actually mean by it.

But back to the word “Estimates.”

On occasions like this I like to check my dictionary, in this case the :

“Estimate

        1.        to form an approximate idea of (size, cost, etc.); calculate roughly

        2.        to form an opinion; judge

        3.        submit an approximate price for a job to a prospective client

        4.        an approximate calculation

        5.        a statement of the likely charge for certain work

        6.        an opinion” (Collins Paperback English Dictionary 2001)

My other usual source is Wikipedia which on this occasion gives:

“Estimation is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is derived from the best information available.”

From these definitions I define a sense that an estimate is:

• Approximate
• A statement of possibility
• Is based on available data which may itself be incomplete or variable

Maybe all estimates should be accompanies by a statement of probability but as Kahneman and Tversky described in the planning fallacy, and has been proved repeatedly since, not only do humans underestimate time but humans are over confident in their estimates. Thus any estimate probability statement would probably itself be an over estimate of probability.

Besides, very few of the “estimates” I’ve ever seen are accompanied by a statement of probability so I don’t think this suggest will get very far.

More importantly these definitions also help tell us determine what an estimate is not:

• As estimate is not exact
• An estimate is not a promise, guarantee or commitment
• An estimate is not a target or deadline

And it is not several other things.

Now in my previous blog post I introduced the idea of “Accurate Estimates” so I was actually sneaking in the idea that an estimate could have a high probability and could be an accurate indicator of what will happen. Perhaps I was guilty of something there.

The trap I fell into is one that many fall into, that of accepting the general usage of the word “Estimate”.

In general usage – in the software community – we misuse the word estimate.

Firstly we automatically equate Estimate with “Effort Estimate”: effort (and therefore cost) estimate proliferate in software development but we overlook other estimates that might be useful, in particular Benefit Estimate.

Second the inherently approximate nature of estimation is too often ignore, estimated are endowed with a sense of promise of what will be rather than recognising their inherent approximate nature. (And as noted in the Planning Fallacy, Vierordt’s Law and Hofstadter’s Law time estimates will always be under estimates.)

This also leads to too much conversation about “Why was the estimate wrong?” – sometime blame may be implied. The answer to the question is really: “The estimate is not wrong because it was an ESTIMATE” That is to say: An estimate is never wrong because an estimate is an approximation and therefore is not binary “Right” or “Wrong”.

Sure you can have a conversation about why the estimate was very different to what actually played out but the nature of that conversation is going to be different depending on what you will do with the findings of the conversation. For example, if the resulting information is used to refine future estimates it will be a very different conversation to one where the result will be punishment for someone. (Yes, people do get punished, I once saw a company where Project Managers were rewarded/punished based on the variance between estimated time spent on work and actual time spent.)

In short too often an approximate estimates based on variable information is used as some kind of exact promise to meet a deadline.

Software developers love to imagine it is evil managers who take their estimates, massage them to be politically correct, promise them to higher ups and then force poor coders to honour the number they first thought, but, big BUT, managers are not the only ones. Even in everyday life the Planning Fallacy, Vierordt’s Law and Hofstadter’s Law hold. Observe yourself next time you have to catch a bus, train, complete a tax return, hand in course work or do something (almost anything) with your kids.

I would love it if I could wave a magic wand and reset everyones’ understanding of the word Estimate but I don’t see it happening.

And I think – although Woody and Vasco might like to correct me – that a large part of the #NoEstimates movement is motivated by this problem. The way I see the logic is:

• Estimates are seldom recognised for what they really are and honoured as such.
• Estimates are misused and used as a stick to beat people and organizations.
• Therefore estimates have become a problem and it is better of finding a way or working without them.

I’m not saying this is the whole #NoEstimates logic but it is part which strikes a chord with me.

Incidentally, because I believe estimates are not a promise I don’t believe in Scrum commitment and because I believe they are approximate that in Xanpan I focus their use on the near term. And because I believe benefits should dictate deadlines not effort I refuse to use estimates as deadlines.

To estimate or not to estimate, to join the #NoEstimates bang-wagon or not, that is the question.

Maybe it is a navel gazing exercise for agile-folk but it does seem to be the reoccurring theme. And I can’t get over this feeling that some of my peers think I’m a bit stupid for continuing to support estimates.

Complicating matters I’m finding my own work and research is starting to be cited in support of #NoEstimate – Dear Customer (PDF version), my publicity for Vierdort’s Law, Notes on Estimation and More notes on estimation. Add my own #NoProjects / #BeyondProjects logic isn’t far removed from the whole estimates discussion.

At Lean Agile Scotland a few weeks ago Seb Rose and I were discussing the subject, in the end Seb said something to the effect:

“How can you continue to believe in estimation when your own arguments show it is pointless?” (I’m sure Seb will correct me if my memory is fault.)

My reply to Seb was something along the lines:

I continue to believe that estimation can be both useful and accurate, however I increasingly believe the conditions under which this holds are beyond most organizations.

To which Seb challenged me to list those conditions. Well here is that list.

I’ve blogged about this before (well, I’ve mentioned it in lots of blogs, see this one Conclusions and Hypothesis about estimates) and I’ve devoted a large section of the Xanpan book to talking about I see estimates working but I think its worth revisiting the subject.

Before continuing I should say: I’m talking about Effort Estimates specifically. There is another discussion which needs to be had around business value/benefit estimation.

Here is, a probably incomplete, list of conditions I think are required in order for effort estimates to be accurate:

1. The people and teams who will undertake the work need to do the estimates
2. Estimates go off if not used: estimates only remain valid for a short period of time (days), the longer the elapsed time between making the estimate and doing the work the less accurate they will prove
3. Estimates will only be accurate if the teams are stable
4. Estimates much be calibrated against past performance, i.e. data is needed
5. Together #3 and #4 imply that only teams which have been working in this fashion for a while can produce accurate estimates
6. Teams must have a record of delivering and must be largely able to undertake the work needed, i.e. there are few dependencies on other teams and few “call outs” to elsewhere in the organization
7. Estimates must be used as is, they cannot be adjusted
8. Estimates cannot be used as targets (Goodharts Law will cut if they are)
9. Estimates made in units of time (hours, days, etc.) are not reliable
10. The tracking and measurement process must measure all work done, not just “project” work
11. Financial bonus should not be tied to estimates or work
12. People outside the team should not coerce the team in any way

There are probably some other conditions which need to be on this list but I haven’t realised. I’m happy to have additional suggestions.

Perhaps this list is already so long enough as to be unachievable for most teams. Perhaps meeting this conditions are unachievable for many, even most, organizations. In which case the #NoEstimate’ers are right.

So… I believe estimation can be useful, I also believe it can be accurate but I believe there are a lot of factors that can cause effort estimates to go wrong. In fact, I know one team, possibly two, who claim their estimates and planning processes is very accurate. Perhaps I cling to my belief in estimates because I know these teams.

When estimates do work I don’t believe they can work far into the future. They are primarily a tool for teams to decide how much work to take on for the next couple of weeks. I don’t expect estimates further out will prove reliable. Estimates for 2 to 12 weeks have some value but beyond the 3 month mark I don’t believe they will prove accurate. So my advice: don’t estimate anything that isn’t likely to happen in the next 3 months, and don’t plan any work based on estimates which extend more than 3 months into the future.

Which means: that even if you accept my argument that estimates work they may not tell you what you want to know, they may not have much value to you under these conditions.

And to further complicate matters I suspect that for mature teams estimation becomes pointless too.

As implied by the list above I would not expect a team new to this (agile) way of working to produce reliable estimates. With experience, and the conditions above, I think they can. One of the ways I think it works is by helping teams break down work into small pieces which flow. As a team get better I would expect the effort estimation to exhibit a very tight distribution. When this happens then simply counting the number of card (tasks, stories, whatever the thing you are estimating is) will have about the same information value for a fraction of the cost.

(For example, suppose a team normally do 45 points of work per iteration, if the teams average size estimate is 5 with a standard deviation of 0.5 then they would be expected to accept 9 pieces of work per iteration. If these statistics are stable then estimation works. But at this point simply taking in 9 pieces of work would also prove a reliable guide.)

So:

1. Effort estimation doesn’t work for immature teams because they don’t exhibit the conditions above
2. Effort estimation does work for mature teams but
3. Effort estimation is pointless for very mature teams

Even given all this I think estimation is a worthwhile activity for teams of type 1 and 2 because it has other benefits.

One benefit is that is promotes discussion – or at least it should. Another is that it forms part of a design activity that helps teams make pieces of work smaller.

But there is another reason I want teams to do it: Credibility.

Estimation is so enshrined in the way many businesses work that teams and those trying to introduce change/agile risk undermining their own credibility if they remove estimation early. And I don’t just mean credibility with “the business” I think many developers also expect estimation and if asked to adopt a process without it will be skeptical.

So its just possible that estimation as we knowing – planning poker, velocity and such – is a placebo. It doesn’t actually help many teams but it helps people feel they are doing something right. In time they may find the placebo actually works or they may find they don’t need it.

Another reason why I like developers to think about “how long will the take” is that I believe it helps them set their own deadline. It helps them focus their own work.

Thus I keep advocating estimates because I think they are useful to the team, the fact that you might be able to tell when something might be “done” is a side effect. Since I find long range estimates questionable I advocate a cheap approach which might be usefulness or might just be a placebo.

However, I do believe, that given the right conditions teams can estimate accurately, and can deliver to those estimates. Increasing I believe very few organizations can provide those conditions to their teams.

We had the Agile on the Beach conference a couple of weeks. In a word: Brilliant – just look at the photos and see how people enjoyed themselves.

OK, I’m biased, I’m one of the organisers, but many people told me it was brilliant or they just didn’t want to hurt my feelings.

There is much I could write about Agile on the Beach but I don’t have enough time. Plus right now I want to focus on just one thing that came out of the conference:

Agile Outside of Software – #AOSW

Just before the conference I published a blog version of my talk to the conference (Agile Outside of Software the blog) and now the slides are online (Agile Outside of Software the SlideShare). Well between the post and the presentation, Agile Outside of Software turned out to be a hot topic.

And it wasn’t just me:

• Belinda Weldlock talked about using Agile outside of software to help various Cornish companies: toy companies, florists and others. Her talk is available on YouTube.
• Belinda also had a video made by Oxford Innovation about Agile in Cornwall: this started with the Cornish Software Mines Agile Programme that I was involved with four years ago. Oxford Innovation have built on that legacy.
• Sabina Renshof from The Netherlands also spoke to me about how she had used Agile with various non-software teams
• Rachel Picken gave a presentation on her use of Agile in a Public Relations company.

One of the points I made was: There is good evidence to think Agile does work outside of software but we lack case studies. I have a few:

• I my own case study from GSMA (writing a specification document with a dispersed team)
• The use of Agile at my client Sullivan Cuff software for everything! Software development, sales, support and well, everything!
• I highlighted case studies from previous Agile on the Beach conferences (Kate Sullivan at Lonely Planet and Martin Rowe at Petroc college.)
• And I discussed the 2007 case study of Shamrock Foods from MIT Sloan Management Review.

(After the conference I discovered this small book which I’ve only just started reading: “Scrum Marketing: Applying Agile Methodologies to Marketing”.)

But we need more.

So Sabina came up with the great idea that we start a collection. And I think we should.

It starts here, right now with the Twitter hashtag #AOSW – well actually, this hashtag was coined at the conference, I’m just a little late announcing it on my blog.

Right now we’ve not worked out how we are going to do this – my guess is a wiki or a LeanPub collection – but right now, if you know of a case study – documented or not – please share:

• leave a comment on this blog
• Tweet with the #AOSW hashtag
• E-mail myself or Sabina

Lets get collecting.

Right now, if you want to know more I’m repeating my Agile Outside of Software talk on 3rd November I’m giving the talk again BCS Bristol “Beyond Software”.

Blow me down, its happening again…

I’m awake. I’m wet, its a cold sweat. Its the small hours of the morning and the dream is horrid….

I’ve been sent to Coventry.

I’m in a clients office waiting for a meeting to start. The development manager is telling me she has selected me to help them become Agile, she checked me out online and recognises that I am pragmatic. Thats why they chose a new tool called Kjsb, its pragmatic too.

Pragmatic.

God does she know how much I hate that word? Pragmatic to me? I recognise that Agile and Waterfall are points on a spectrum and that most organizations, for better or worse fall somewhere in-between. I recognise that every organisation exists within a context and you need to consider that. And even change the context.

But pragmatic? Pragmatic is the Satan’s way of saying “Heaven doesn’t work in the Real World(TM)”.

The CTO enters and is putting down redlines. He knows all Agile, but his people… its his people you see … you can’t trust them, they are like children, you can’t let them have too much say. They need a strong man to lead them.

They had a developer here once who practiced Agile. He did that test driven stuff. He didn’t give out dates. He gave Agile a bad name in the company. The PMO will never accept that.

Fortunately they have just bought Kjsb. This wonderful tool will fix everything. Kjsb has a feature that translates burn-downs into Gantt charts at the click-of-a-mouse. And back again.

The problem is: teams still aren’t shipping on schedule. They need predictability. Predictability is what the one thing really need.

And flexibility. Flexibility is important. Flexibility and predictability, the two things they really need.

And now variation in features. They can’t trade features for time. Fixed scope, Flexibility and Predictability are the three things they need.

But… they have unforeseen technical problems – not bugs you understand, but unforeseen technical problems. They really need to be able to deal with those things. Technical fixes, fixed scope, Flexibility and Predictability are the four things they need.

Nobody expects…

I want to explain queuing theory… a grasp of basic queuing theory is the one thing they need – stick their feet on the ground and cement them to it.

One of the teams runs Agile. It is run by the CTO himself and its good. The other teams… well they don’t really have that much experience. Though the managers are going to get their Scrum certificates real soon now.

How, he asks, can we get everyone else to buy in?

How can we get the PMO to buy?

How can they make the Product Owners buy in?

Mention of the PMO stirs the old guy in the corner, the one who’s hiding behind his laptop, the widescreen laptop with the numeric keypad. And mention of the Product Owners causes the Analyst in the other corner – the one hiding behind the ultra thin laptop – to raise an eyebrow.

Now I see they all have laptops out in front of them… and some of them phones too. In between moving their mouths each of them is staring at their screens.

I’d better say something. “Well,” I start…., “how about we get people from the team who are doing this well to talk about their experience?” Blank looks all round, are they listening

Or doing e-mailing

“Could you them your own case study?”

No – that won’t work because that teams are so very different from everyone else in the company nobody will believe it. They are all individuals.

Besides, the developers won’t be at the buy-in meeting. Its for managers to buy in. Once the managers buy in the developers will be told what to do.

….

I try a different approach: “Instead of talking to the PMO one day, and the Product Managers the next day, and the Development Managers the day after… why don’t we go vertical and take each development team in turn, with the appropriate project, product and development managers?”

No. Managers manage, they are the only ones who need to know. And they are the ones who will be allocating the work with Kjsb.

“Need to know” – “Allocating work” Did I really just hear those words?

Whose version of Agile have they been reading?

O my god, these guys are going on a Scrum Master course next week, there is going to be a bun fight, I don’t know who I worry about most these guys or the poor sod who is teaching the class….

“Can I just check,” I ask, “each team has a project manager assigned, a product manager, a team lead, they will soon have a Scrum Master too?” Heads nod, “and… there are several development managers spanning several teams each?”

Yes

“So if I’m counting right…. each team contains about 4 developers and 1 tester? (Plus UAT cycle lagging several weeks later)”

Yes

“O see…” Am I keeping a straight face? Does my face hide my horror? 3+ managers for every 5 workers? – either this business prints cash or they will soon be bust.

….

“Really,” says the development manager, “we are talking about change, I have 12 years change management experience from call centres to financial services, the CTO hand picked me to lead this change, software development is just the same as any other change”

When did Fred Brooks come into the room, in fact what is he doing in Coventry, he lives in Carolina, why is he wearing a dog collar? And why is it 1974? He’s now standing at the lectern reading from a tatted copy of Mythical Man Month

“In many ways” says Brooks, “managing a large computer programming project is like managing any other large undertaking – in more ways than most programmers believe. But in many other ways it is different – in more ways than most professional managers expect.”

Well this is a dream, what do I expect? Its 2014 again…

“The key is to set the framework,” she continues, “establish boundaries so people know what their responsibilities are then we can empower them”

Fred has gone, standing at the lectern in dog collar is Henry Mintzberg – my management hero – he is reading from another tattered book entitled Managing:

“the later term empowerment did not change [manager control], because the term itself indicated that the power remained with the manager. Truly empowered workers, such as doctors in a hospital, even bees in the hive, do not await gifts from their managerial gods; they know what they are there to do and just do it.”

Empowerment is dis-empowered: using the words say one thing but the message given by using those words is the opposite.

“What we want is consistency across teams” says the CTO who now resembles Basil Fawlty

(What happened to “all my teams are different”?)

“And a stage gate process” says the PMO man, or is it Terry Jones?

“And clear roles and responsibilities” says the Cardinal Fang

“Nobody expects the Spanish Inquisition” says Michael Palin – where did he come from?

….

“It seems to me” starts the Product Owner “that we are making a lot more paperwork for ourselves here”

O the voice of sanity!

“Yes” I begin…. “if you attempt to run both an Agile and a Waterfall process that is what you will have!”

Silence. I continue, “Over time, as you see Agile work, as people understand it, I would expect you will move to a more Agile approach in general and be able to reduce the documentation.”

“No.” The PMO seems quite certain of this, “I don’t think that will happen here, we need the control and certainty that the waterfall and our stage gates provide. We won’t be doing that.”

Poor Product Owner, if he is lucky he’ll be shown the door, if he’s unlucky he’ll be retained.

…

“If you want people to buy in” I suggest, “we must let people have their say.”

The PMO is ready for this: “Yes, we know that, we’ve already arranged for a survey” and she reads the questions:

Q1: “Do you agree our development process needs to change?” Yes or No

Q2: “Our organization wishes to remain in control but we want the benefits of Agile, do you think we should:

a) Embrace Marxism in its entirety,

b) Mandate Waterfall throughout the organization or

c) Create a Managed Agile process?”

Q3: “Have you seen the features provided by Kjsb?” Yes or No.

O my god, its a North Korean election.

I suggest the questions are a little bit leading. “Well we don’t want people being awkward” chips in the CTO.

…

We get up to leave.

“You know,” I say, “when you’ve had a chance to run this process for a while you will want to inspect it and modify it” – but while I’m saying that I’m think “No plan survives contact with the enemy, start small, see what happens.”

“O we’ve already done that. This process is the result of doing that. We won’t be changing it.”

…

Back in my kitchen, a warm milk in my hand. A bad dream.

It was a bad dream. That stuff never happened. How could it? The contradictions are so obvious even a small child could see them. Couldn’t they?

As I climb the stairs back to bed a terrible thought: what if it wasn’t a nightmare? what if it was real? and what if they call me back for help?

Could anyone help such people?

Later this week I’m doing a presentation at Agile On The Beach entitled: “Agile outside of software development”. (I resisted the temptation to call it “Agile Beyond Software”). The presentation will attempt to answer a question which is often asked at, and around, Agile On The Beach: “Is Agile only for Software Development?”

In truth it is not just at Agile On The Beach (AOTB) that this question is asked, I hear it more and more often elsewhere but the origins of AOTB, and target audience for AOTB, mean that the question is very much on topic.

I’ll post the presentation online after I have delivered it – until then I’ll be tweaking it – but right now I’d like to (kind of by way of rehearsal) run down my main argument, so here goes….

Before the term Agile Software Development was coined there was “Agile Manufacturing”. This was this term that inspired the software folks so perhaps the first answer is: “Yes! Of course Agile works outside of software because that is where it came from.”

But, Agile Software Development has far and away surpassed Agile Manufacturing in writing and mindshare. So much so that others in the wider business community have looked at Agile Software Development as a model for other types of working. In a way, “Agile” has come full circle.

Perhaps it is worth pausing for a moment and asking: “What do we mean by Agile?” or “What do we want from an Agile company?”

This could be a big big debate, ultimately it is for each company, each team, to decide what Agile means to them. Rather than have that discussion let me quote MIT Professor Michael A Cusumano:

“I can’t think of anything more important than building an agile company, because the world changes so quickly and unpredictably…

[Agility] comes in different forms, but basically it’s the ability to quickly adapt to or even anticipate and lead change. Agility in the broadest form affects strategic thinking, operations, technology innovation and the ability to innovate in products, processes and business models.” (MIT Sloan Management Review Summer 2011 interview with Hopkins)

Lets add to that a little bit by defining Agility from three views:

• Agile Strategy: Adaptability, listen to customer, leading the market, use change competitively
• Agile Tactically: Experimenting, “Expeditionary Marketing”, live in the now while preparing for the future
• Agile Operations: Deliver fast, deliver quality, deliver value

We could go on but thats enough for now. Now we have an approximate understanding of what we want we can go back to the original question: “Is Agile only for Software Development?”

There are three parts to this answer: Practices, Roots and Case Studies.

Practices

Lots of the practices associated with Agile actually come from elsewhere. Examples of stand-up meeting proliferate – bars, healthcare, the military, Japanese local Government and so on. Many companies operate regular status or planning meetings, Agile just elevates this practice. WIP limits are well established in manufacturing.

Agile picks up some practices directly – visual boards (“information radiators”) are nothing new.

Some practices it picks up and changes – Retrospectives have long existed as “Lessons Learned” or “After Action Reviews.”

Some practices Agile (might have) invented – Planning poker but this is itself version of wide band delphi.

And some Agile plays back to the business – TDD and BDD are writ large in Lean Startup (which is itself an extreme version of Expeditionary Marketing from 1991, Hamel & Prahalad).

Roots

As already mentioned: Agile Software Development was inspired by Agile Manufacturing.

I’ve described my Agile Pyramid model before (Agile and Lean – the same but different, How do you make Lean Practical?) and my argument that “Agile is Lean thinking applied to software development” and I wrote whole book saying the software development, and Agile software development specifically, is an example of Organizational Learning.

Well, Agile Software Development is not the only application built on these foundations. Obviously the Toyota Production System is. And so too is its close cousin the Ford Production System. Look further afield and you will find Last Planner in construction. I could continue but I think my point is proved.

While not every Agile practice can be taken out of software development and used someplace else the roots of Agile mean that the principles, values and ideas which Agile is built on can be. In your domain Agile as now known might work quite well, but in someone else’s domain there may be more need to think deeper.

One caveat: I believe the more deeply you look at Agile and more Agile is applied outside of the software development the more it looks like Lean. Ultimately the distinction between Lean and Agile breaks down in my book.

Case Studies

Good news: There are case studies of teams using Agile outside of software – and if you know of any more please tell me! Add them in the comments on this blog post.

Bad news: There are not that many case studies. Unless you are a software team you probably can’t find one.

For the Agile on the Beach conference I have deliberately sought out Agile outside of software development in the last few years. This has resulted in two good examples.

Two years ago Kate Sullivan described how the Lonely Planet legal department adopted Agile working. In fact Kate said all of Lonely Planet adopted Agile.

Last year Martin Rowe talked about how he used Scrum to manage the Foundation Computer Science course at Plymouth University.

Elsewhere, six years ago the MIT Sloan Management Review carried a piece by Keith R. McFarland entitled “Should you build strategy like you build software?” in which he described how Shamrock Foods of Arizona used Agile to plan and execute their business strategy.

Myself, I have seen the marketing team at Sullivan Cuff Software Ltd use an Agile like approach.

And last year I helped the GSMA (the people responsible for SIM cards) use Agile on a project writing a document for cellphone manufacturers, mobile network operators and umpteen suppliers and partners to allow mobile phones to be used for loyalty coupons.

So, back to the original question: “Is Agile only for Software Development?”

Answer: No

Question: Will Agile work outside software development?

Answer: Yes

But, the detail may vary.

Finally, and please excuse the pull. As a result of this discussion my company has adapted its successful Foundations of Agile Software Development 2-day course for companies outside software. Please take a look at Agile Kick-Off (for non-software teams) and let me know what you think.

In the early days of this blog I used to regular blog book reviews. As I found ideas of my own to blog about – and found fewer books I wanted to read – so book reviews ceased. But right now I want to blog about a book because I want to recommend this book.

The book is: Scaling Up Excellence by Robert Sutton and Huggy Rao.

I’ve been a fan of Sutton’s work for a while now – specifically The Knowing Doing Gap. In this book he continues some of that theme (“Why don’t we do what we know is better?”) but he looks specifically at scaling.

Now scaling is a hot topic in software development (I wrote a little about this myself, Scaling Agile Heuristics – SAFe or not!) so its a book which should find a ready audience in the software community. Although I sometimes feel this is a book about change and on the whole I’m no longer a fan of books about change (there lies a long story for another blog). As a book about change what makes this book worth reading is that it is particularly focused on scaling up changes.

I hope this book becomes more widely known in the software development community, and particularly those concerned with “Agile” and “Scaling Agile.” For now here are some of the key points I want to remember from the book and I hope will wet your appetite:

• Sutton and Roa suggest “the problem of scaling” can also be defined as “the problem of more”. For example: how do we get more teams working in this excellent fashion? Or, how do we get this excellent team to be even more excellent?
• “In order to scale excellence you need some excellence to scale”: as I said in my post last year: If you can’t do it in the small then you can’t do it in the large.
• They introduce the idea of Buddhist and Catholic change models: in the Buddhist model the aim is to help every one/team come to their own understanding of improvement while in the Catholic model the aim is to help every one/team replicate an existing example of excellent. They don’t suggest either one is inherently better but they do suggest that sometimes one model if the context better.
• Continuing on the Catholic theme, they also spend a lot of time discussing standardisation. This is something the has troubled me in the past – I don’t like standardisation, I naturally lean towards the Buddhist model. But I also feel I need to think more about this topic, this book has given me more to think about, watch out for a future blog.
• There is good discussion of the role of hierarchies
• Scaling is a marathon, not a spring

They are just a few bullets from the first chapter or two but I could go on through the whole book. My (e)book copy is covered in highlights and comments.

Let me add a story of my own.

Early on in the book they give three reasons why scaling initiates often fail. I think these three are worth considering in the Agile domain:

• Illusion: leaders believe scaling will happen more easily than the facts suggest
• Impatience: lenders scale too soon
• Incompetence: leaders start scaling without really understanding the thing themselves

Arguably these three things are intertwined and overlap. The third one reminds me of some bank executives I heard speaking a few months ago. The 4 or 5 senior bankers were talking about why they loved Agile and why they wanted more people in the bank to adopt Agile. They were good. They came across as honest. Then they took questions.

The first question was: “Agile says collocated teams work best, how do we reconcile this with a bank policy of remote working and offshore development?”

Rather than say: “I understand your concern, this is something we are working on and we need your help, we might need to change policy or find new ways or working” the executive replied: “With modern tools this isn’t such a big problem any more…” Immediately it seemed that not only did the banker not understand Agile and why Agile said this but he set out to show that as a banker he understood more about modern technology than the technologist asking the question (who was wrong.)

Shortly afterwards someone said: “How do we reconcile regulation with Agile?” and a senior managers then started to wrap themselves in knots about allowing teams to choose Waterfall or Agile during which time one of them said: “Look, as long as your documentation is in order you can do what you like.” Again they demonstrated a lack of understanding.

With those two answers the managers not only shot themselves in both feet but destroyed all the credibility they had spent 30 minutes gaining.

Rather than continue, let me just say, if scaling (anything) interests you then read it – Scaling Up Excellence.

The summer lull has finally allowed me to get a lot done. In particular I’m very pleased to have finished a piece of writing I began some months ago: Bugs – Management Strategies.

This is an important piece for two reasons. Firstly I’ve been wanting to write this essay for a few years but never found the time. It has taken about four months in total and it runs to about 30 pages, now I know why it took so long.

Secondly, this is one of the corner stones for Xanpan volume 2. Volume 2 was going to be subtitled “Management Heuristics” but I’m now thinking it will deal more with organising so the sub-title might become “Management Heuristics and Organisation” or “Heuristics for organising software development” or some combination.

When I have a couple more chapters of volume 2 in place I’ll make it available on LeanPub. Until then you can register your interest in volume 2 right there.

In the meantime please read “Bugs – Management Strategies” and let me have your feedback. At this point I’m really posting it to get some feedback.

And if you haven’t already bought and read Xanpan volume 1: Team Centric Agile Software Development please do today!

(And if you prefer the a hard copy then get Xanpan at Lulu rather than LeanPub.)

I’m awake. I’m lying in my bed. I’m sweating but I’m cold.

Its the small hours of the morning and the dream is as vivid as it is horrid….

I’m standing in a clients offices, I’ve been here before, I know whats happening.

They are building an website. Quite a complex one, this will be the primary purchasing venue for many customers. This will project the company image – and with the right bits it can up-sell to customers – it can even help reduce costs by servicing the customers after the sale. All good stuff.

But it is atrociously “behind” schedule, someone said it would be finished in a year, that was three years ago before any code was written. Now its two years to completion but in my dream people say 2+3=2.

How can that be?

I can’t say it but the only way out I can see is cancellation. If I was suddenly in charge of the client I’d cancel the thing. I’d salvage what I could and I’d launch a new, smaller, initiative to replace the website.

But its too big to fail, even the board knows how much money they are spending. Who’s going to walk in there and say: “Scrap it.”

Saying “Scrap it” would be to admit one failure and invite a messenger shooting.

And if I was the head of the supplier I’d say the same thing. I’d say to my customer: “I know I’m earning oodles of cash out of this, I know its a high profile feather in our cap but really its out of control you really shouldn’t continue.”

But of course they won’t.

Forget the money they’d lose, they weren’t hired to answer back – like my tailor friend.

And of course I’m neither of those. I’m just the guy having the nightmare and in the nightmare I’m the consultant who is trying to fix, it. In the nightmare I’m not fixing it I’m providing cover, while I’m there its Agile, while its Agile its good, Agile is a good drug and I’m the pusher.

“You can’t cancel it because all the competitors have one and so we must have one” tells me a ghostly apparition.

“We must be best in class” says another apparition.

“We must be head-and-shoulders above the opposition” says third – aren’t the opposition seven times the size? And don’t the competition buy large parts of their solution off the shelf?

But every time I look the work seems to grow. Every discussion ends in more stories. Not just stories, epics, super-stories, sub-epics, mezzanine-stories.

But its OK, this is Agile.

The business keeps throwing new requests at it which are just accepted – because they are Agile!

Some of these are quite big. But that’s OK because the team are Agile. And Agile means the team do what the business want right?

I watch the Analysts work over the stories in the backlog, as they do each grows and replicates like an alien parasite. The Analysts find more edge cases, extra detail which need to be included, more scenarios which need to be catered for. Each becomes a story itself.

But that’s OK because the team are Agile.

And those damn competitors don’t stop adding and improving their site which mean the client must add that too.

But that’s OK because the team are Agile.

And the points…. points are the new hours, in the dream I have a book “The Mythical Man Point”. The backlog is measured in thousands of points. The burn-down charts go down – but only if you look at the sprint burn-down, hunt around Jira and you can find a project wide burn-down, O my god, no….. its full of stories!

This is not a burn-down chart carrying us to a safe landing, its a fast climbing interceptor…

The backlog is a demon… its… its… undead.

The faces of those who’ve seen the chart are prematurely aged. Open Jira and show someone the chart and…. their hair turns grey, the wrinkles appear, in moments they are….

One man is immune. As the points grow his power grows, he is… he is… The Product Owner.

He introduces himself: “Snape is the name, Severus Snape” – I knew I’d seen him somewhere before.

In the planning meeting, he sees the poker cards pulled out, he focuses on the developer with the highest score, there is a ray of cutting sarcasm… he withers. The developers submit, the numbers are lowered. The Product Owner chuckles to himself – no over estimating on his watch!

One of the developers suggest “Maybe we should wait until we finish the current work”

Snape sneers: “I thought you were Agile boy?”

“If you can’t handle it I have some friends in Transylvanian who are really Agile…. do you want to lose the contract boy? … Off-shore is so so cheap…”

There is a reality distortion field around the Product Owner. Show him a burn-down chart and it looks good, his presentations to the steering committee always show perfect burn-down.

I’m in my pyjamas standing outside the building at night: a sinister looking figure is breaking and entering, he sneaks into the building, he opens Jira and … inserts stories! His mask falls, it is….The Product Owner! Of course, without stories in the backlog he would cease to exist, his power comes from the size of the backlog, more stories more power.

Ever since his boss came down with a rare form of chronic flu a link in the reporting chain has been missing. Made worse when the next man up was dismissed for inappropriate behaviour in the canteen. Since when the Product Owner reports to the COO, a COO who doesn’t really have time for him and only has a shaky understanding of any IT related topic.

I do the maths. The backlog isn’t so much a backlog as a mortgage, and the team are under water! The payments they make against the mortgage aren’t even covering the growth in stories. The backlog growth is an interest rate they can’t pay.

It takes months for stories to progress through the backlog and reach developers. When work finally gets to developers they too uncover more edge cases, more details, more scenarios, more of just about everything. Why didn’t the Analysts find these? Did they find them and then lose them?

Then there is a stream of bugs coming in – oozing through the walls. The technical practices aren’t solid, they are… custard! Bugs get caught but more get through!

Bugs can’t be fixed because: “bugs are OpEx and we are funded from CapEx.”

Someone has slain the Bug Fixing Fairy, her body is found slumped in the corner, a nice your girl straight out of college. They are hiring another fresh young graduate to send to the slaughter, fortunately Bug Fixing Fairies are Plug Compatible with one another.

Release dates can’t be honoured.

Woody Allen and Anne Hall walk in – since when did Woody Allen do horror films?

‘two elderly women are at a Catskill mountain resort, and one of ’em says, “Boy, the food at this place is really terrible.” The other one says, “Yeah, I know; and such small portions.”’

I have X-Ray vision: I can see WIP where it lies, there are piles of it on the floor. Its stacked up like beer barrels in a brewery. But the beer isn’t drinkable. Its a fiendish plan. If anyone drinks that beer, if the WIP is shipped, they will discover…. its full of holes! Quality control is… offshore.

Why is there so much WIP lying around? Why is the WIP rising?

Because they are Agile comes the reply… the business can change their mind at anytime, and they do.

I’m drowning in WIP.

WHIP to the left of me, WHIP to the right of me.

The developers are half way through a piece of work and the team are told to put it to one side and do something else. Nothing gets delivered, everything is half baked. WHIP – work hopefully in process that is.

When, and IF, the team return to a piece of WHIP things have changed, the team members might have changed, so picking it up isn’t easy.

WHIP goes off, the stench of slowly rotting software.

But that’s OK because the team are Agile.

Arhhh, the developers are clones, they are plug compatible, you can switch them into and out as you like… but they have no memory….

It gets worse, the client has cunningly outsourced their network ops to another supplier, and their support desk to another one, and the data-centre to the another… no one contractor has more than one contract. Its a perverse form of WIP limit, no supplier is allowed more than one contract.

O my god, I’m flying through the data centre, the data centre supplier has lost control, the are creepers everywhere, each server is patched in a different way, there is a stack of change configuration requests in a dark office, I approach the clerk, its its…. Terry Gilliam, the data centre is in Brazil….

Even when the business doesn’t change its mind the development team get stuck. They have dependencies on other teams and on other some other sub-contractor. So work gets put to one side again, more WIP.

All roads lead to Dounreay in Scotland, a really good place if you want to build something really dangerous, but why does this project require a fast breeder nuclear reactor?

But that’s OK because the team are Agile.

The supplier is desperate to keep their people busy, if The Product Owner sees a programmer who’s fingers are not moving on the keyboard he turns them to stone. The team manager is desperate to save his people, he rummages in the backlog and finds… a piece of work they can do.

(With a backlog that large you can always find something even if the business value is negative – and there are plenty of them.)

You can’t blame the development team, they need to look busy, they need to justify themselves so they work on what they can.

But that’s OK because the team are Agile.

Get me out of here!!!!!

I’m in my kitchen.

My hands are wrapped around a hot-chocolate, I need a fresh pair of dry pyjamas but that can wait while I calm down. I’ve wrapped a blanket around me and have the shivers under control.

Are they Agile? Undoubtedly it was sold as Agile. It certainty ain’t pretty but it is called Agile.

They have iterations. They have planing meeting. They have burn-downs. They have a Scrum Master and they have Jira. They have User Stories. They have some, slow, automated acceptance tests, some developers are even writing automated unit tests. How could it have gone so wrong?

Sure the development team could be better. You could boost the supply curve.

But that would be like administering morphine. The pain would be relieved for a while but the fever would return and it would be worse. The real problem is elsewhere. The real problem is rampant demand. The real problem is poor client management. The real problem is a client who isn’t looking at the feedback.

The real problems are multiple, thats what is so scary about the dream. They are all interconnected.

In the wee-small hours I see no way of winning, its a quagmire: To save this project we need to destroy this project. But we all know what happened in Vietnam.

What is to be done? – I can’t go back to sleep until I have an answer.

Would the team be better off doing Waterfall?

The business would still change its mind, project management would put a change request process in place and the propagation delay would be worse. There would probably be more bugs – testing would be postponed. Releases would be held back. This would look better for a few months until they came to actually test and release.

If they did waterfall, if they did a big requirements exercise, a big specification, a big design, a big estimation and a big plan they might not choose to do it. But frankly Agile is telling them clearly this will never be done. In fact its telling them with a lot more certainty because they are several years in and have several years of data to look at.

Agile is the cover. Because they are Agile they are getting more rope to hang themselves with.

But all this is a dream, a horrid dream, none of this ever happened.