Quick as a flash Eben Halford on Twitter pointed out the mistake with my last blog (Question for self-organizing teams). I was mixing up self-organizing teams with self-directing teams. Well maybe I was… much of my post still stands either way, and as Eben himself pointed out, we might be trying to split a hair here.

Frankly, I suspect many people think the whole “self-organizing team” thing is one-size. The idea that there are actually “self-organizing” and “self-directed” and possibly “self-managing” hasn’t occurred to most people and, if they have noticed, they don’t know the difference. In fact, I’m not sure I know the difference!

If you take time to look at some of the literature on self-organizing/directing teams you find that very little of the pre-agile stuff talks about self-organizing teams, rather the common term in management literature is self-managing teams, most of the serious research relates to this rather than the (to my mind) weaker idea of self-organization.

To untangle this mess lets define a hierarchy here:

• A self-organizing team is a team where team members get to decide among themselves who does what, the team gets to work on problems and have some power to remove their own blockages. Clearly there are teams who are more self-organizing than others and teams which have more authority than others.
• In a self-managing team there is no active day-to-day management of the team. The team are effectively left to manage their own work. To my mind this is a stronger form of self-organizing.
• A self-directed team is a team which sets its own goals, decides its own objectives and determines its own priorities.

Does anyone know of any serious literature (i.e. research led) which defines these terms better? I’d love to have some better, clear, more well defined, separation.

Clearly some, but not all, of the questions I posed in my last blog relate to self-directed teams rather than mere self-organizing teams.

But while I have, belatedly, made this distinction it seems to me that not everyone does. It seems to me that making this distinction would help by removing a lot of the confusion that sounds self-organizing teams. And making this distinction would actually help with the questions I posed in the last blog post because teams and their managers would be able to draw lines and discuss where authority and responsibilities lie.

You see, one of the problems with labels, especially labels like self-organanizing teams, is that they mean different things to different people. Indeed different authors define them differently.

It is because of this confusion that I prefer to avoid using these labels and prefer to leave the idea of self-organizing teams alone.

Instead I prefer to talk about authority, I want team members and teams, however they are organized and managed, to be given authority. Sometimes authority is vested in the team members equally, so for example each member has the authority to make a decision. In other case authority is vested in the collective team, in that case as long as the team can agree a decision they can make it,

In other cases authority is vested in specific individual team members. This usually occurs where specialist skills or knowledge are needed. The Product Owner role is a great example here: product owners need to have the authority to make decisions on prioritisation, they need authority to make trade-offs and make compromises. The more authority you give the product owner the better they can do their job.

I want teams and individuals to have authority for two reasons. I believe the soft, fuzzy, reason that is usually sighted: people get more satisfaction from work when they have autonomy (i.e. they can make their own decisions) and thus are more motivated and more engaged as workers.

The second reason is very hard, not fuzzy: when developing software there are thousands of decisions that need to be made. Many of the decisions require specialist knowledge and the people with the greatest knowledge, both of the technology and the situation in hand right now, are the people doing the work. The testers and programmers at the code face have more information about what needs to be decided than anyone else.

Pushing decisions down to the lowest level means decision making can be made more efficiently, i.e. in a more timely fashion. But in order to do that the workers need authority.

Now notice I’m saying authority, I’m deliberately avoiding the word “empowerment”. Empowerment is a horrid word and it describes a horrid concept. Don’t talk to me about empowerment, its nasty.

On that cliff hanger, if you want to know why I don’t like empowerment, well tune in next time, the post is already drafted.

Try this thought experiment.

You are a software development manager. You learn about agile and you think it is good. You adopt agile and you make all your teams into self-organizing teams. (Leave aside the question of whether you then quit in a fit of “no managers needed” – we can talk about that later.)

Most of your teams work well. Productivity goes up, people are happier. They organise their own work. Occasionally teams come to you and say they would like extra people on the team. Provided they can back this up with evidence that they are productive and add value, and that adding more members will increase the value they deliver then they are allowed to expand.

But, one team doesn’t perform as expected. They don’t achieve the productivity of the other teams, they don’t demonstrate the value other teams do.

Question: Do you intervene?

If you intervene you are not allowing them to self-organise, you are playing big bad manager.

If you intervene in one team what will the other teams think?

Perhaps its not as clear as that. Perhaps the team are performing but it becomes clear they have achieved a local optimum and are not improving like other teams.

Question: How do you nudge the team to move to a higher performance level?

And what if when you challenge them about things like pair programming, honouring deadlines, maybe predictable velocity, etc. etc. they tell you they have tried these things, or at least talked about them, and they have decided that these are not good for them.

Question: What do you do if they self-orgianise to not do the things you hoped they would? Things you think might improve performance?

(Potentially if you allow a team to self-organize they might even start requesting complete requirements documents up front, they may resist change requests.)

Now suppose that rather than play big bad manager you play facilitator, Uber-Scrum-Master, you go to the team and say: “I think this team could perform better, I’d like to help you improve.”

And someone on the team says back: “Could you please give us an example of when you think we could have performed better?”

Now you have a specific instance in your mind where the other teams (the performing teams) think this one messed up. Do you tell them? To tell them is negative, to tell them may make them defensive, to tell them really make you seem like a big bad manager.

You could hire an agile coach, scrum master or such and ask them to do pose these questions and make these changes. But isn’t that just delegating? You are still the power behind the action.

You could go to the other teams and ask them to challenge the under performing team and raise concerns with them directly but that might be seen as manipulative, and now you are interfering in multiple teams. Are you allowing them to self-organise?

Now suppose things take a turn for the worse. The company looses money and the shareholders ask for redundancies.

Do you communicate this to the teams and ask for volunteers? – you may hit employment law problems here.

Do you ask everyone (yourself included) to take a pay-cut?

What is someone crunched the figures and found it was the underperforming team that was dragging everyone down, and cutting the whole team could return you to profit. Should you ask the teams to do this themselves? Or do you do it?

If the team crunched the figures would the underperforming team decide to offer themselves up for redundancy? Would it be fair it you – or the teams themselves – decided to cut one person from each team instead?

In short: if you have an team where the performance isn’t satisfactory what do you do?

I suppose the ideal is that someone on the team, probably a product owner/analyst type person, is regularly analysing the benefit delivered by the team and the cost of the team. If the benefits fall below the cost and the team can’t come up with a plan to change it then the team members request dissolution. But has anyone ever seen that?

I pose these questions because I don’t know the answers. It is questions like this that add to my scepticism over self-organizing teams, or rather, cause me to see limits in the model.

I find it hard to image any team offering to dissolve themselves. O, there might be the odd story of teams doing it but I find it hard to image this is the norm. I find it easier to image teams getting more defensive and seeking to explain away their failings by reference to others (I’m avoiding the word blame.)

Arie de Geus in The Living Company says that companies, and by extension teams, are living beings, they exist to continue their own existence. In capitalist coiuntries markets will eventually force poor performing companies out of business. In the closed ecosystem inside a company how do we replicate that mechanism?

I see limits in self-organzing teams: One of manager’s roles is to determine these limits and act at the limits.

One more thing, run this thought experiment again, but this time imagine that you (the enlightened manager) chose to dismiss yourself once you have set up the teams. The questions still stand, the problems still exist, but removing the manager removes the last person who might take action.

In the summer my training and consulting business gets quiet. That allows me to take some holiday and to pursue some stuff I don’t get much time for otherwise. This summer I’ve picked up a programming idea I started last summer, worked on over Christmas and still haven’t finished. This means I’m programming!

It is getting close to 10 years since anyone paid me to programme but I dabble a bit, I keep my hand in (a little). Right now I’d like to share some observations I’ve made. Programmers out there – please agree or disagree, just leave comment below. And if you haven’t programmed for a while then think about these points.

A bit of background, I’m working in Python (2.7) for the Google App Engine. It is about 2,300 lines of HTML and 4,000 of code and unit tests. I’ve set myself a challenge of doing this without a debugger. Just unit tests and some debug logging/printf’s if needed. I won’t have an IDE either, I’m using TextMate. Giovanni Asproni persuaded me to try PyCharm but the thought of learning another IDE stopped me installing it.

So,programming…

1. Its addictive – hard to stop, when you see your code working its hard to step back, not to play with the kids, not to eat and even not to go the toilet!
2. Sometimes it feels like your brain will explode, you are just trying to keep so many things in mind at one time and make sense of something (or somethings) new, and understand how it all hangs together and…. you just feel full.
3. It scares me – knowing that I will be so immersed scares me, I don’t pick it up and do little bits when I can be cause I know I’ll get sucked in.
4. It requires large chunks of time – its not like answering e-mail, its not something you can do 5 minutes here, and 20 there, you need a few hours at a time, partly because of #1 and #2 above.
5. Them tests – TDD development makes it so much more enjoyable, yes its hard to keep yourself honest but when you see a little bit more working it feeds the addiction; it also means you spend less time with those annoying little bugs and trivialities. Once or twice I’ve strayed from writing tests for too long and ended up in a mess.
6. Trivial tests Yes! – when I evangelise TDD I often get asked by people “But surely you don’t need to write tests for all the small, obvious stuff, getters and setters say”. (Leaving aside the “you should’t have getters and setters” argument for a moment). Its hard to argue with them. But its perhaps with the trivial stuff that I find TDD makes the biggest difference, perhaps in part because I’m in Python and not a compiled language. Its those occasions when you think “this is trivial I’ll just do it” that often catch you out. I am not thinking, I am on auto pilot, or, heaven forbid, I’ve done a bit of cut and paste.
7. No you can’t TDD everything (particularly in the UI or a third party system) but that means I go slower there, I’m taking more risk. I wish could test that stuff. Why should “not being able to test everything” be an objection to “testing everything you can?”
8. Bigger problems, multiple things: some of the bigger problems you face are where several things come together, and sometimes it far from obvious that they have! (This is the same as my entry in 97 Things Every Programmer Should Know, Two Wrongs Can Make a Right.)
9. Taking a break: sometimes when you are in the middle something, particularly if your brain is about to explode (#2), and/or you are convinced you are on the edge of completing something, or accomplishing a breakthrough (particularly if something is not working like you expect it to) then… you just can’t take a break. Not so much addition but the feeling that if you let go now you will never get it all back into your head again. But, big but, a break is just what you need. Only when you come back fresh can you actually make it work. Only a break will let you step back and think without the overload.
10. Nothing else: When I’m programming, when I’m on a roll, I don’t want to write more Xanpan, or continue the User Stories series, or chase down some leads for work. Yes I should do all these things but… programming is good. There is only so much time in the day. Wouldn’t it be nice if I had someone else to take care of those things? If I was a team I could…. all those programmers who want to do away with managers should think about this. If you do away with managers much of that management work still needs doing and now the programmers are going to have to do it!

Its easy to see what attracted me to programming all those years ago. The deep immersion and challenge, man v. machine, it soaked up so many of my teenage days – particularly summer holidays. Its perhaps fitting that it is mainly during the quieter summer months – when my training and consulting work isn’t so busy – that I get time to do a bit of programming.

Something else thats funny, when I’m programming I really like listening to old 70s and 80s music, stuff I listened too when I was a teenager immersed in programming. Guess I’m just a middle-aged man seeking his youth!

I’ve updated the Team Start Dialogue Sheet. Many thanks to Marta Kolenda who used the sheet with two of her new teams and sent me some feedback. Incorporating her feedback gave me a chance to make some more changes to the sheet too.

I’m always glad to have feedback from people and teams who try any of my Dialogue Sheets – getting feedback from Agile teams can be like getting blood out of a stone!

Plus I’m always keen to sit in and observe teams using the dialogue sheets – particularly the still new Customer Discussion sheet – so please invite me!

And while you are looking at the sheets… you might notice that Software Strategy – my company – has a shiny new website. Bblogs and books don’t pay mortgages so Software Strategy is my commercial side. If you know anyone who wants some advice or training on software development or Agile then point them here!

Hopefully most of my readers have noticed by now that I regularly stray from the Agile Gospel, or rather, I have a tendency to go against the common form of Agile. (And if you haven’t noticed you probably haven’t read My warped, crazy, wrong version of Agile post and its philosophical successor, Xanpan.)

Today I want to take issue with Iterative and Incremental, in particular the definitions of Iterative Development and Incremental Development used by Jeff Patton. It’s not that I think Jeff is wrong, its just that I use these terms differently and if you look closely Jeff and I use them in almost opposite ways. Seb Rose picked up on this a few years ago but recently I’ve had two or three e-mails form concerned readers but it was David Lowe’s blog that finally moved me to write.

I have long talked about “3 Styles of Agile”, one of these styles being Incremental and one being Iterative, to recap:

Iterative development occurs when the wider organization don’t care about Agile – or if they do they see it as something to make coders in the boiler room work faster. Consequently the team do good Agile stuff (stand-ups, iterations, TDD, refactoring, etc.) but they are working from a big old requirements document. In so much as they have a Product Owner the role is probably filled by a Business Analyst who has little authority, he operates a bacon slicer which turns the requirements document into small stories for development.

The organization wants all or nothing. They don’t care about early partial deliver. They may even see deviation from requirements as a problem because they still operate traditional governance (time, cost, features and/or compliance with a plan.)

Incremental development is fairly similar, the team do the good stuff but the organization doesn’t really care for Agile and probably still has traditional governance. However they team do make regular deliveries, at the very least they demo their latest creation and more likely they provide early versions for clients to use. As a result the benefit starts accruing far sooner but that also means they get extra requests for functionality and some stuff is removed.

Because they are delivering something, and because they are increasing the value delivered I call this incremental. It is increasing. In contrast an iterative team are basically running in tight loops. It better than the alternative but the business see no added value. However in doing this the team has created forces which will result in a future conflict – they accept change but governance does not.

So my model is clear?

(For a longer discussion see my Agile Spectrum piece from a few years back.)

Now Jeff Patton uses the two words differently. Jeff’s analogy is the Mona Lisa: A development team could paint the bottom left corner of the Mona Lisa perfectly, you would have a few square inches of perfect Mona Lisa and the rest of the canvas would be blank. Once that is done the team move to the right a bit and paint the next few square inches, and so on. He calls this Incremental because the Mona Lisa is appearing in increments.

Jeff suggests that a better approach is to take a blank canvas, sketch a woman and a background with lines only. Show that to the customer and get feedback. Next add some colour but don’t worry about the details. If the customer likes what you are creating keep going. Each time you increase the fidelity of the picture. This is what Jeff calls Iterative.

In Jeff’s language feedback occurs on each Iteration. In my language feedback occurs with each Increment.

You see the confusion?

I’ve tried applying Jeff’s language to my model, I’ve mentally tried to say “OK Jeff, you are right, I’ll change” but the words don’t work for me.

The thing is, and this isn’t always obvious: Jeff and I are discussing different problems.

His context, the problem Jeff is addressing is new product development where the customer doesn’t know what they want.

My context, the problem I am addressing here is: Teams trying to be Agile within an existing organization.

I use the words Iterative, Incremental and Evolutionary to describe the different approaches to implementing Agile within organizations. I try desperately not to say one is better than the other (although I fail miserably usually). Armed with this language I can describe what I see in companies, in teams, and I can advise accordingly.

I have no problem what so ever with Jeff’s use of the words in his context. In fact I agree, if you are doing new product development it may well make sense to increase you fidelity each time round the loop.

Which leads to the interesting question: Could you combine the two models?

Lets consider the four cases:

1. Jeff Incremental, Allan Iterative: This would be a team trying to perfect a small part of system from the requirements document before moving on to the next with little or no feedback. In other words, pretty much what most companies who have 100% faith in the requirements document try and do and fail badly at.
2. Jeff Iterative, Allan Iterative: This couldn’t exist, in Jeff’s model Iteration allows for feedback while in my model Iterations are devoid of feedback.
3. Jeff Incremental, Allan Incremental: This is case #1 but with added feedback, this would be problematic; A recipe for technical debt and an inconsistent UI. The team would complete one section to perfection, feedback would cause the next section to be a bit different, it would be perfect when it was complete but it would be inconsistent with the first part. As you progress across the Mona Lisa the style would change. It would be like having Leonardo De Vinci paint the picture over the course of his career. The final parts would be very different to the first.
4. Jeff Iterative, Allan Incremental: This could work, it could be good. The team would be producing low fidelity versions which are deployed to live as they go along and feedback is incorporated. The only problem here is that the requirements document (upon which governance is based) would rapidly become out of date, there would be tension. If you reformed governance then you’d probably be at the model I call Evolutionary.

So there you go.

In a nutshell: we use the same language to address different problems and in different context.

Sorry, the world is complicated.

I recent message from a friend tells me he is putting on his parachute. He can’t take it anymore. He’s tried hard, very hard, to change a challenged company. Actually, I know a little bit more about this company than I should. I once told him that working there was like being aircrew in Catch-22: Only the sane should work there because it was cruel to employ ill people, but it was such a crazy environment wanting to leave was a sign of sanity.

Unfortunately this is not the only company I have heard of, even seen inside, in the last few years that have such a business model. In fact, I’ve now seen so many that I’ve named it: The Prolonged Death Spiral Business Model.

Let me explain… this requires two or three building blocks but I’ll get there…

In Business Patterns for Software Developers I wrote:

“Most [software] companies never make it past the conception stage: ideas stay in people’s heads, business plans fail to be funded or people start but lack the energy to follow through. Even those plans that get past inception regularly flounder – products fail to make it to market, and those which do fail to sell. …

Companies can fail at any time during their existence. However the younger a company is, the more likely it is to fail. Infant mortality is high: most companies under one year old fail. However, once a company is past its first year, survival chances improve remarkably. …

Death comes to a software company when its sales do not cover its costs. Once a software product is developed and sold, companies can reduce costs and continue selling the product. Companies that have an installed user base, particularly when customers are paying regularly for support services, can stagger on for months or even years. This process can be especially prolonged when customers have data locked up in a proprietary data formats or on servers.”

When I wrote this I was thinking of the growth of successful software companies, the kind of companies I wanted to work for. I first saw a prolonged death play out when I was working at Quadratron in the 1990s. What I never realised was that a prolonged death spiral could actually be a viable business model itself.

Quadratron was dying, it eked out its last few years collecting maintenance royalties from legacy customers – one customer in particular. In fact it was dying when I joined, they lured me in with a plan to spend a lot of the remaining cash on a new product. But things were worse than that.

Like so many companies Quadratron found that once you have survived the first few years, once you conquered the risk of developing a product and have an installed user base you can continue milking that base for a long time. Provided you don’t do anything silly like trying to develop a new product that is! Quadratron had been very successful, it had a lot of customers to milk.

When you reduce a software team to care and maintenance only your costs are much lower. Importantly your risks are much lower too, and you have cashflow. Yes, an accounting term I know but it means: money is coming in. And money continues to come in because while some customers may dump your product the longer you are in place the more difficult it is to dump. This is what accountants call “free cashflow”, its how much money the company has to play with, its a better measure than profit because profit can be gamed.

Now, like many others in the software industry I think of venture capitalists as nice people who live in Sand Hill Road, Palo Alto and invest in software companies with the capacity to grow very very fast and make everyone rich. Call this the Californian model of venture capital.

OK, these guys are not so lovable that you want to take them home to meet Mum and Dad but they have financed a lot of good software companies over the years – Lotus, Yahoo, Google. They invest in 100 companies, they expect 90 to lose money – they try to minimise it – 9 to break even and one to make so much money they make up for all the losses.

Then there is another type of financier. Call them the private investor or private equity. I think of this as the European model of venture capital. They buy existing companies and milk them. In buying they look for two things: lots of free cashflow (i.e. day to day profitability) and minimum risk (i.e. they really want to limit their down side.)

As far as possibly they buy the company with debt. With other types of company assets can be used to secure debt but software companies have few assets. So debt is serviced from the cashflow, and the cashflow comes from an installed user base who cannot, and do not want to, change.

Since debt is serviced from cashflow and cashflow comes from existing customers the emphasis is on keeping customers – don’t loose them! So most development work is driven by what individual customers want. Backlogs are stuffed full of customer specific requests.

Growing the company, especially developing new products, would be really risky. The owners and managers are into revenue protection. They want any new product development to be paid for by customers, software alchemy.

Software Alchemy: When a software company believes it can develop a “product” for one customer (preferably paid for by the customer) and then sell the same “product” to multiple other companies.

You can certainly develop technology for one client, you may solve their problems, but developing a product for multiple customers is a different undertaking. Very occasionally, at the start of a technology cycle, companies pull of this trick (usually by accident or by having a very stupid customer.)

Of course keeping customers means you need to persuade customers your product has a future so you need a bit of a roadmap, a bit of a vision but its weak and flexible. If an existing customer doesn’t like it change it! Forwards the end, like when I jumped from Quadratron, even maintaining the pretence of a future for the product is jettisoned to save money. The final few customers are trapped.

Because the company is now loaded down with debt (and needs to pay interest) they find making any sort of profit impossible. Actually, if there was some more free cash then the private equity owners would leverage it by taking on more debt and paying it out as dividend, or they would re-leverage the company – possibly by selling it to another private buyer.

Another trait you sometimes see with these companies is acquisitions: when they can these companies buy related companies who also have products which are past their best but have an installed base. Again they can do this with debt. They hope to get some technology crossover but in reality the people who run these companies don’t have the skills to do that so the mess gets bigger.

By the way: since the companies are loaded with debt they pay no tax because tax can be offset against debt.

These companies chase EBITDA – Earnings Before Interest Tax Depreciation and Amortisation. Because that is the only valuation which makes sense when the company has been leveraged to the hilt.

In short the company is being run as by “financial engineers” – and thats the polite term.

These people know a lot about debt structures and taxation but nothing about software businesses. Today they may be running a software business, tomorrow it could be toothpaste.

Its worth noting these companies want to be Agile because even these guys have heard that Agile will result in lower costs, faster delivering (faster cashflow), and happier customers (even fewer losses and even more reoccurring cashflow.) But they aren’t prepared to make the necessary changes, perhaps a little Agile training or coaching but anything that requires serious investment (e.g. TDD) is off the table.

These companies are a success by some criteria: the people running them and the people who buy them stand to make lots of money. Financially they look good – except the debt. And customer continue to use the products they want to use. They exist, they employ people. By some criteria they are a success, we should not forget this.

They can be miserable places to work in because real engineering is not a consideration. And pity the poor customers who are being led up the garden path about future products.

It would be criminal for any of these companies to take advantage of an ill person by offering them employment. As a sane consultant I’ve even tried to help a few of these companies. However, not wanting to work with these companies – as an employee, contractor or consultant – is a sign of sanity.

If you work for a company which is run by financial engineers in Britain, Europe, America, London, Cambridge, Cheshire or anywhere, you know what you should do for the sake of your sanity.

Question: What is the opposite of Slack?

Or, perhaps a better question: What is the problem to which Slack is the answer?

Lots of software people like to advocating the benefits of slack for development teams. I’m not talking about the popular collaboration tool Slack, rather I’m talking about Slack in the sense discussed by Tom DeMarco in his book. DeMarco’s is a good book and has some good arguments but on the whole I’ve never been a fan of introducing random slack into the development process. (Have a look at my comments on Seb Rose’s blog post from a few years back to see what I mean.)

Now I’ve found a book, and two academics, who have done the hard work of creating an anti-dote:

Scarcity: Why having so little matters so much by Sendhil Mullainathan and Eidar Shafir (this is the UK edition, in the US the publishers have a different name slightly, Scarcity: The New Science of Having Less and How It Defines Our Lives)

Scarcity is the opposite of slack, and just maybe, scarcity is the problem which requires slack.

This book is highly recommended. Although it has nothing to do with the software community this book deserves to be essential reading for everyone who claims to know something about software management and in particular Agile.

The authors are ambitious if nothing else: they set out to create a new discipline, the study of scarcity. Although most of their experiments, examples and studies concern scarcity of money and/or time they endeavour to project their arguments into a more general space, scarcity of anything.

As you might expect, I read this as a software engineer. I read the book applying their thinking to the world of software and specifically Agile Software development. And after all, so much of managing software development comes down to the scarcity of time and the scarcity of money even if it manifests itself as a scarcity of people and excessive demands.

Now there is good news and bad news for software people.

First a little theory: the authors suggest that when people encounter scarcity they “tunnel”. In other words their mind starts devoting a lot of capacity to the resource which is in short supply. This has advantages: our focus increases, our productivity increases, some of judgement (yes estimating) skills improve and overall our effectiveness increases. Yes, scarcity can create productivity improvements. Scarcity is not all bad, it can be beneficial.

But there is a downside. For people in the scarcity tunnel their mind devotes a lot of time to thinking about this problem. As a result there are negative effects. Anything that doesn’t effect the short term tunnel is pushed to one side. So, for example, the short term benefits of taking a payday loan displace the long term problems which will be made worse.

This creates what the authors calls a “bandwidth tax”: so much of the mental energy is devoted to the tunnel and scarcity that less capacity is available for other things. For example, parents who have to juggle childcare arrangements for children are less effective at work.

The authors present serious evidence that the bandwidth tax reduces intellectual capacity, i.e. IQ, intelligence quotient. Perhaps controversial this leads them to suggest the poor are poor because they are poor, a vicious circle develops which is difficult to escape and traps people.

OK, enough of the book, read it yourself. What about software development? What does this book mean to software?

I see that tunnel a lot with development teams, they are doing stuff they know they shouldn’t but they feel they can’t afford to do the right thing. As a result they never get out of the mess they are in.

It is immediately obvious that one of the ways Agile Software Development – specifically iteration based Scrum, XP and my own Xanpan – work is by creating a short term scarcity of time (two weeks!) which causes tunnelling. Effectiveness goes up as items outside the tunnel are pushed out.

Work in progress limits (whether flow based or velocity/points based) are another form of induced scarcity. They force managers and product people specifically to tunnel on what is important, what needs going an they push less important items out.

But this comes at a cost.

One clear implication is that activities with benefits which will fall outside the tunnel will be dropped, skipped or minimise. And the obvious thing here is Technical Quality and improving the Technical Infrastructure. For example, adding automated tests, applying test driven development, improving the build system, refactoring, the list could go on.

(Indeed, it was worry about not investing in these which causes Seb Rose to advocate Slack.)

Mullainathan and Shafir give examples of poor farmers who save money by not taking out insurance against crop failure, drought or other possibilities. They also show how the same farmers would benefit and save money by taking out insurance.

They give example of vaccination programmes where those who will benefit – and know they will benefit – do not partake.

When they need the benefits of such insurance it is too late, they are tunnelling and cannot afford to invest. When they have the time or money to invest in insurance it seems unnecessary. People are optimistic and under estimate the risk to small problems occurring.

This is the same as teams who don’t do practices such as Test Driven Development. They are tunnelling.

There is more evidence here about the role of deadlines and routines: deadlines and routines help us manage our time and ensure things happen. The research presented in the book complement my own notes on time estimation (Notes on Estimation and Retrospective Estimation and More notes on Estimation Research). (It is because of such finding that I cling to fixed length iteration in Xanpan rather than going abandoning them like Kanban.)

And those vicious circles which keep poor people poor parallel software teams too. Poor people find problems which can be solved with a little money become very disruptive because even a little money is hard to find, they tunnel more, more problems are ignored. The current fire is extinguished but other problems are worse now.

I see the same thing with software teams. Projects run late and over budget so managers tunnel on the project, while this is happening other problems are building up, quality is neglected, technical debt is incurred, support tickets are left unanswered, changes which could help are put off until the fire is extinguished.

Sometimes I think the answer is simple: choose not to enter the scarcity trap, choose not to firefight, choose to do the right think. It reminds me of the talk JB Rainsberger gave on his 2015 European tour “Just Stop It.” It reminds me of Renton in Train Spotting: Choose Life. It reminds me of Joe Bergin’s (in)famous “Do the Right Thing.”

Perhaps easier said than done, personally I have one piece of advice for these companies: Stop Thinking Projects. Project thinking obscures the problems and solutions.

The authors have some advice, some solutions, but they don’t have a complete solution. I find myself in the same boat: I can see how their arguments play out in software and I have some solutions but I don’t have them all.

Importantly they do identify Slack as part of the answer to Scarcity. Anyone who has studied queuing theory knows this.

Not only does this book deserve to be widely read in the software community – and specifically the Agile community – but I hope to see these two authors appearing at software conferences before long.

Finally, I consider myself lucky to have found this book. After several years where I struggled to find books which said something new I’ve been lucky enough to read three in the last year. (I blogged about Scaling Excellence last year (another must read), I didn’t blog about Joy by Richard Sheridan but his book deserves reading just for the share life affirming enjoyment, a reminder that software development can be and should be joyous! OK, I enjoyed it for another reason, Richard and I both had our lives changed by the same events on 2 January 2001, but that is a long story!)

From time to time I’ve been heard to say: “Agile is Punk.” But I’ve never explained myself.

I’ve also been heard to say things life “Agile is about democratising the workplace” but I’ve never explain myself there either.

Let me try…

What I mean when I say this is: Agile (software development) has a lot in common with Punk rock.

To me the important thing about punk rock was that it was about people trying it – music, their own thing. Anybody trying to play music, anybody forming a band, anyone who had a novel idea trying to get a record contract. Yes, even if they couldn’t play an instrument they could have a go, and who knows, maybe they would learn as they went.

(I should say that while I’m old enough to have been around when punk was I’m not old enough to have been there. Both post-punk and post-disco influences were at work in the New Wave music which was common when I came music listening.)

Punk had a democratising effect on music. Music has aways been of the people, anyone can listen, anyone can try to sing – although I’m not very good at it even I can try! But the music industry was something different, performing, recording – there were barriers there! Punk tore down barriers.

Punk opened up the recording industry. Punk opened up music.

Agile opened up the software process industry.

Before Agile official software processes were pretty locked down. You had to be an academic or expert/consultant to dabble in that space. Programmers who worked in under official software processes were on the receiving end.

Agile said: “Your opinion is important too.”

In truth music has always was been open to everyone, just not the recording industry. And in software development processes were open to anyone, most programmers did not work under an official process, mostly it was common practices which, if they worked, were probably more effective than official ones.

Unfortunately these unofficial work practices came with guilt: because we did not do it the way the books said we should. When faults occurred we blame ourselves for “not doing it properly”.

Agile says: “Everyone involved with software development should have a voice in deciding how to work, it can be improved and you don’t need to be an expert, academic, consultant or certified member of some body to express that view.”

That also makes it democratic.

I don’t mean democratic in the sense that we all get to vote, I mean democratic in the sense that it is power is vested in the hands of the collective people. Everyone has a voice, everyone can participate, and those who hold executive power do so by the will of the people.

Agile is about giving everyone a voice. Like Punk that means accept that those who don’t have much skill are also entitled to a voice.

Funnily enough, I’ve long held that any punk band that made a second album weren’t punk anymore because they were part of the industry, they were now experts! The same is true with Agile, hang around for long enough (like me) and you are no longer an outside but an insider, an expert.

Increasingly we see Agile heading outside of software development. When this happens it becomes necessary to ask: What is Agile?

My answer is: Democracy.

Agile is about valuing everyone, agile is about giving everyone a voice, agile is about putting the power to change the workplace (process, systems, norms) into the collective hands of the people who work there.

Yes at times it feels revolutionary, but there are fellow travellers, it is all a Theory-Y movement.

There is a technique sometimes used in retrospectives, and sometimes called “Taking the temperature” but in truth it can be used in many settings and I’m sure it goes by various names. Basically, you pose some question to a group, for example:

“How well do we feel the last iteration went?”

“How much do you know about Agile?”

“Are you confident we can meet this objective?”

Everyone takes a piece of paper – usually a sheet of post-it paper – and writes a number from 1 to 10. Depending on your question the number 1 means: bad, nothing, unlikely, while 10 means something like: brilliant, everything, certain.

The papers are folded (its anonymous), collected and opened.

The anonymous bit is important: I once sat with a team where everyone answered 6 to the question “How did this sprint go?”. In my opinion the sprint was horrid, indeed the whole initiative was a mess but I too answered 6. Maybe I answered 6 because the people before me had anchored my answer, maybe I answered 6 to conform with the group, maybe I didn’t want to rock the boat. But probably I answered 6 because in my opinion the manager running the team, the initiative, and this meeting was the cause of most of the problems and I felt that anyone who tried to speak up would be quizzed as to “Why do you say that?” and the answer “Because you really don’t get it you silly old …” was not very useful.

The thing is, once you have those numbers they are a talking point. I like to post them up on a board, in order, where we can see them and discuss them. If the question is “Are we confident in our ability to meet this goal?” and I am faced with a lot of scores around 6 or 7 I might say: “What could we do to increase our confidence?” and we could have a general discussion about this.

I usually use this technique at the start of a training session to judge the knowledge of Agile in the room. That helps me adjust my level of delivery to the audience.

The thing about this technique that I wanted to bring up today is this: the mean isn’t particularly interesting, i.e. summing all the number and dividing by the number of votes to achieve a mean average isn’t very useful. In part this is because the average does’t tell us much, regression to mean will happen and it doesn’t really give you anything to talk about.

Sure, its a bit of a vanity metric: “Look we scored 6.8 which is up from 6.65 last iteration!”

Median (ordered middle) might be more interesting. And mode (most common) is certainly more interesting. They are things to talk about. Faced with 8, 6, 6, 6, 5, 3: “Why do most people think this iteration went well but not everyone?”

There may be value in recording an average to track the ups and downs over time. I’m not sure there is but if I was I’d rant to track the median rather than the mean, to my mind that will contain more information than a mean. (However even here I’d get worried about Goodhart’s Law and goal displacement, someone might think the aim of an iteration was to have a good iteration.)

But what is really interesting, and the reason why all averages aren’t that interesting is because it is the extremes which are the most interesting.

“Why do we think that someone thought the iteration was not very good?”

“What could have caused someone to rate the iteration as high as 8?”

“How could we have a good (8) and a poor (3) in the same iteration?”

The extremes, particularly when there are opposing answers, make a good talking point. They are something to talk about. Remember of course to keep it anonymous: don’t ask “Will who ever scored 3 explain why they dissent?”, ask instead “What do we think made someone rate the iteration as a 3?”

So if you are going to use this technique please:

• Keep it anonymous
• Share the results with the whole group
• Avoid the mean
• Talk about the extremes
• Talk about how you could score higher in future