Interesting piece in last weeks Economist which suggests the Hawthone effect doesn’t stand up to analysis. (For good measure the Financial Times covered the same story over the weekend.)

For those who don’t know, the Hawthorne effect is named after the Chicago telephone factory were it was originally observed. The researchers suggested that the act of experimenting on people changed their behaviour.

Back in 1924 researchers changed the lighting at the Hawthone factory and observed productivity increased. Then they reduced lighting and to their surprised it increased again. The conclusion of the study is widely cited to show that when workers know they are experimented on their performance increases. Another spin on the research suggests it is when workers feel valued their performance increases.

Now it seems the original data has been re-analysed using modern econometrics techniques. What it actually shows is the workers were more productive at the start of the week (lighting changes happened on Sunday.)

(Have a look at the Economist piece or the research itself to understand the subtleties I’ve glossd over.)

Anyone who saw another Economist report from last year knows already that most published research is wrong. That study showed that most of the research published in the top journals is disproved within three years.

Which raises the question of whether less well known and influence journals are actually more reliable but lets leave that to one side for now.

Perhaps the only surprise about the Hawthone effect is that it took over 80 years to be exposed.

And what lesson are we to draw from what we now know?

Well, maybe, if workers are more productive at the start of the week then keep that first two or three days clear for core work. Push meetings such to the end of the week.

I can never decide if I actually like foie gras as a food. The taste is always secondary to the thought of the force feed ducks. This production process has led to attempts to ban foie gras – as in Chicago a few years ago.

Personally I’d like to end a practice I’ve seen too many times in the software industry and which I’ve come to call foie gras recruitment.

Foie gras recruitment is the forced expansion of a development team beyond the ability of the team to absorb new people. It happens when an organization decides it needs more software produced and decides to do so by hiring lots of new developers.

For example, an internet company I worked with a few years ago. The company faced a rapidly expanding market and needed more from the development team. I was asked help them improve their development processes and practices (“get agile”). When I turned up on day-1 I found four new developers were starting that day and another the following week.

This was on top of the two new developers who had been hired two months previously. And as if that wasn’t enough the company was opening a US development office and had three new recruits starting there the following week.

Given that the original development team was only four strong that is more that is more than a 100% increase. Unfortunately it was too late for me to stop any of this, I just had to pick up the pieces.

This was not the first time I had seen this problem and I’m sure it won’t be the last. The reason why companies do this is because they feel the need to get more done. However the immediate effect of stuffing the development group with new people is to slow everything down. The more you force feed the group the greater the slow down. Hence foie gras recruitment.

Most developers know Brooks Law: “Adding developers to a late project will make it later.” This can be generalised as “Adding developers slows work down.”

This is because new developer need time to learn their way around the system, the tools, the process and practices. When they don’t know they have to ask. People need to tell them things. Those people are the current developers. So they get less done. The more people you hire the slower things go.

(A related problem is hiring just developers rather than a balance of developers, testers and BAs/Product Owners but we’ll leave that for another day.)

In my experience a new developer needs at least three months to become productive. In the book Organization Patterns of Software Development Jim Coplien and Neil Harrison suggest the figure is closer to one year.

I’m sure someone will say: “But if the business want that” or “If we really must deliver more we need more people” but the reality is: more means slower. It is one of those occasions were someone need to explain to “the business” that more isn’t what they think it is.

This gives rise to the latest Kelly’s Law of development: In the short term your development resources are fixed – they may only be reduced.

Of course in the long term you may need to increase your development team and you should hire new developers. The right way to do this a rolling programme of recruitment. Aim to hire, say, one new developer every six months.

The frequency with which you hire will depend on things like: your current team size (big teams can hire more frequently) and the structure, number and interdependency of the existing team(s).

In any company there will be some level of natural loss and you may well want to set you hiring level to allow for this.

In the short term, if you really need to improve your through put then you need to look inside the team at the process and practices and see what can be changed there. Even here it takes time to improve throughput.

So, good bye to foie gras recruitment, hello to a little and often.

Please forgive my slight departure from my usual posts on software development, Agile, patterns and so on. But I’ve had cause in the last month to think hard about what the internet means to us all.

I’m probably one of the last generation of Europeans to remember life without the internet. My first contact with e-mail was about 25 years ago when a friend and me looked after the school computers and got access to the first inter school e-mail system – The Times Network for Schools.

(Access didn’t last long once the school saw the phone bill. Just long enough for us to discover that none of the other local schools had not changed their default password.)

As my professional career has developed so has the internet. But I can remember time before the internet so I know the world could survive without it again. A different world yes but it would survive.

Last month the internet took on a dark side and my relationship with it changed. Death came to cyberspace.

I’ve been following the blog of that old school friend for a few years. Nothing that interesting but it felt like I was in contact with him. Then in late April his post read like this: “I am an alcoholic, I have been for nearly 10 years. At the weekend I attempted suicide. The police broke into my flat and saved me because someone saw my update on Facebook.” Presumably his Facebook updated read like “Dave is … attempting suicide with pills and beer.” Since then he’s been blogging every few days about his attempt to give up drink.

Think about that. A suicide attempt on Facebook. A confession of alcoholism in a blog. A public dairy of a recover attempt.

Sure you could say it was a very big plea for help. Publicity for himself yes, and maybe it was self indulgent but it also a sign of how we live out lives.

Then yesterday, a Facebook message from someone I didn’t know. I though it was spam and almost ignored it but I didn’t. It was that friend’s sister. She wanted to tell me he’d died.

I’ve long thought e-mail was the worst possible way to give people bad news but I was thinking of news like “Your project is late” this is an order of magnitude more.

Facebook was probably the only way she had of finding me quickly and I’m glad she did. But that changes Facebook for me. Until last month Facebook was about friends, pictures, smiles and so on. Its now changed.

I guess its like the telegrams we used to get: “Regret to inform you Bill Smith passed away STOP.”

And what is the protocol here? Do I go and delete him from my friends list? Or do I leave him there Zombie like? How long do I leave his blog on my reader?

Who tells Facebook to remove his account? Do they need to see a death certificate?
How long does Blogger leave his blog there before it is deleted by some robot?

Web-Archive will always have his posts and the websites he created.

Gradually he’ll be removed from friends lists, links to his blog will be dropped and the great garbage collector in the sky will remove his cyber-soul.

For now Dave still lives in cyberspace, he just hasn’t posted for a few days. Soon it will be a week, then a month, then a year. He’s already left more of a life history than most people who have ever lived, and its far more accessible. In 500 year historians will have a better understanding of everyday life because of the likes of him.

Jeff Sutherland was back in London last week and was good enough to give a talk to ACCU London. As one of the organisers of the ACCU London events I feel I’m justified in giving myself (and my co-organiser James Slaughter) a well deserved pat on the back. This was something of a coup for us.

Over 80 people registered for the event – in fact there was a waiting list of people we couldn’t fit in – and I counted at least 70 people in attendance.

As this event was somewhat bigger than our usual events we had to find a bigger room. Fortunately JP Morgan came to the rescue with a room at the top of their London tower. So not only did we have Jeff but we had a great view of London. Thanks JP Morgan!

Jeff is a good presenter and had some interesting things to say. Although I know most of the Scrum-stuff I still picked up some good information. A few points I thought writing down:

• When Scrum was introduced at Palm it was found that 1 hour of postponed testing resulted in 23 hours of testing later.
• The Chaos reports from the Standish group find that 76% of requirements change after a project has started.
• Data from Yahoo shows that teams adopting Scrum on their own show a 35% productivity improvement. Teams which are coached through Scrum adoption shows a 300-400% improvement. Good news for those of us like me who make a living as an Agile Coach, usually with Scrum. (For those who want it the original source of this statistic is “Rolling Out Agile at a Large Enterprise” by Gabrielle Benefield at the Hawaii International Conference on Software System, 2008.)
• Not only does Jeff endorse the addition of XP technical practices (TDD, refactoring, pair-programming, etc.) to Scrum he believes these are what allow Scrum to scale up. This is important as it fits in with Bob Martin’s comments at the ACCU conference last month.

At the end of the session there was a very good Q&A session – so good we ran out of time and had to cut it short.

In response to one question Jeff mentioned a paper he wrote a couple of years ago called The Future of Scrum. What is interesting here is, Jeff foresees changes in the way Scrum is practices. This contradicts with Ken Schwaber’s statement that “There is only on Scrum.”

I don’t want to get into fighting about whether there is one Scrum, or a future Scrum or what ever. What I think is interesting here is that there are different views on the subject.

While the Agile community as a whole sometimes seems unable to agree on “what is Agile” the Scrum community seems steel like in their understanding of what Scrum is. The truth, for both groups, is somewhere between the two extremes.

I stumbled across a programme called English Heritage on BBC 2 last night (Friday 15 May). Enjoyable for three reasons: first it was an insight into the work of English Heritage, second it was about renovation of Kings Cross station but while both of these were interesting the what I found fascinating was; third – the work of architects.

That’s physical, building, architects. Not the software engineering type.

Like many others in the software engineering community I’ve been guilty in the past of throwing the architectural metaphor around without really understanding what building architects do. Sure I know a little but much of what I think they do is guesswork.

So this programme grabbed my interest because it showed real architects at work. Here are some points that stood out to me:
• Work had begun on the project and the architecture plans were far from complete. Design was continuing and evolving as work continued.
• There was a budget and work had to be within the budget. It looked like the budget (and I suspect time) was set and design and construction occurred within that budget. I saw no sign of a up front master plan which had been broken down item by item, then estimated for time and cost and the sum total agreed.
• In some cases demolition had occurred and it wasn’t clear what would be build in place.
• Building areas where being found and possible work suggested.
• Most of the work was on the old Kings Cross station, some was redesign, some was addition, some was removal and some was simple refactoring.
• We never saw the architects at their drawing boards; we did see them walking around the site, investigating ongoing work; arguing, discussing and negotiating with clients and authorities.
• We saw architects having to change their designs because of business constraints (not enough money to renovate one area) and regulating authorities (English Heritage restricting what the architects could do and how they did it).
• We saw one set of architects disagreeing with another set (Network Rail’s architects wanted to do things want way and English Heritage’s architects thought it should be done another way.)

All in all I found it a fascinating insight into what architects do. As I long suspected, the practice of architecting buildings is not what a lot of software people assume it to be.

The BIG question came up the other week during my presentation to HP: “How many organizations are doing Agile?”

As I said at the time: Thats the $64,000 question – or perhaps in todays money, $64,000,000,000.

Of course, it depends a lot on how you even define “Agile”. Sure if your following everything Kent Beck says in the XP books your probably Agile. But in most places adoption is patchy, a bit of TDD here, some iterations there – stand up meetings seem to be very common now but one practice does not an Agile team make.

Within organizations Agile exists in some places and not in others. There is a large Swiss bank in London which has one of the best Agile teams in the country – well, at least it contains many of the best Agile people in the country.

As I said, Its a LARGE bank. I met someone from elsewhere in the bank at a BBQ last week. The very mention of Agile upset him. His team claimed to be doing Scrum but it didn’t sound anything like the Scrum I know – requirements changed everyday.

“What is Agile” is big question in its own right, maybe we’ll come back to it another day. Back to the original question.

When Jeff Sutherland spoke in London last month he mentioned that there are 83 certified Scrum Master trainers. Bob Martin at the ACCU conference mentioned that there are now 50,000 certified Scrum Masters.

Of course we don’t know how many of those Scrum Masters are practicing, how many work with multiple teams and how many unofficial Scrum Masters there.
Neither do we know how many other teams there are so knowing how many Scrum Masters there are doesn’t help.

In late 2006 Gartner issued a report (“Agile Development: Fact or Fiction”) which estimated less than 5% of development groups were doing Agile. If think its reasonable to assume more teams are doing it now.

I heard from a developer who was looking for a new job in Moscow recently that about 1 in 10 companies claimed to be Agile. That would make it 10% of companies, or 10% of teams.

To get an idea of the UK market I did a series of searches on two job boards: JobServe.com and Monster.co.uk. I searchied for all jobs asking for Java, C# and C++ then I repeated the searches for the same languages with Agile.

Monster.co.uk
Jobs
% of Agile
Java
501

Java and Agile
68
13.5%
C#
411

C# and Agile
67
16.3%
C++
1686

C++ and Agile
19
1.1%

I was quite surprised by the results for C++ here, but if you want to work Agile you are going to have to look hard for an Agile C++ shop. Another surprise, especially for C++, is that different jobs boards might be better for different languages (I’ll leave that thought for someone else to follow up.)

JobServe.com
Jobs
% of Agile
Java
1077

Java and Agile
135
12.5%
C#
960

C# and Agile
127
13.2%
C++
705

C++ and Agile
51
7.2%

Interesting while there are fewer C# jobs than Java jobs the percentage of jobs asking for Agile is similar. So bang goes the theory that there is more Agile Java than Agile C#.

These are very crude counts, I made no attempt to adjust for duplicates (which certainly exist), location anything else. I don’t even know if the searches included JavaScript in the Java searches. (I suspect not because the number of Java jobs is not so far ahead of C# jobs.)

It is worth pointing out that, in general, only those jobs which are difficult to fill get listed so these stats are not representative of the job market as a whole.

Crudely, there seems to be a cluster around 13%. If we ignore the highest and lowest results the answer is between 7.2% and 13.5%. Which isn’t far from that Moscow report.

Certainly there will be teams who are asking for Agile but don’t do Agile. For some teams they may be looking for Agile developers in the hope that they bring Agile into the organization.

For me one of the tests of Agile is: Is it working?

If your development is not effective then I don’t think you are Agile. You might be following a process called Scrum, XP, DSDM or something else but if your team is not effective then you are not filling the business need and you aren’t Agile. How can you be Agile if you are failing?

To be Agile you must be effective.

(That is a little sneaky of me because I am equating Agile with success. It also means the questions “How many people are doing Scrum?” is different to “How many people are doing Agile?” but lets leave that to one side.)

Back to my favourite study from Bain Consulting (The Alignment Trap) which estimated that only about 15% of organizations had effective IT. Now not all these IT groups are doing software development and some will be effective without Agile.

But, if we put those two statements together we get 15% as the upper bound of adoption. That is: at most 15% of oranizations are effective, therefore at most 15% of organizations are Agile. (If they are not effective they cannot be Agile.)

None of this data is conclusive, or definative, but I think it does give us a range:

Between 5% and 15% of all software development is done using Agile software development.

It its worth pointing out that both a high number and a low number is good.

The more companies adopt Agile the safer it is to adopt. The more companies adopt it the more mainstream it is and the more reason to adopt it.

But, while only a few companies are working Agile there is a real competitive advantage in companies adopting it. If your company can produce software in a fraction of the time, or with a fraction of the people of your rivals then your onto a good thing.

It is a risk reward ratio. If you wait until its adopted by 80% of the industry there won’t be an advantage. You’ll be playing catch up.

Given that, the 5-15% range is probably good news. Its common enough to be safe but rear enough to be advantagous.

I often describe myself as an Agile Coach. In fact coaching teams in Agile development is only part of what I do but its the only bit that fits in two words. For some people the title Agile Coach is self-descriptive but for others it leaves the questioner none the wise.

With, Clark Ching’s recent blog about bread as a prompt, here is what I do as an Agile Coach.

The Coach helps teams adopt and improve Agile methods and practice. A Coach helps people rethink and change the way they go about development.

The Coach role is part Consultant and part embedded Trainer. My task is to help individuals understand how to apply Agile ideas in practice. Part of this involves painting a picture of how the Agile world works by telling stories. It involves explaining how to tackle specific problems, in a specific environment.

I tend to work from the process and management side, so I spend most of my time with team leaders, managers, business analysts and product managers. I attend their regular meetings and hold one-on-one sessions to help them try new approaches to the problems they face. That could be project planning, talking to the business, talking to the development team, or about ongoing problems.

Some Agile Coaches get involved with the technical side. They will pair program with developers and help them write tests for TDD. I don’t normally get involved with the actual coding any more but I do spend time with developers. Sometimes these are one-on-one conversations to address specific issues raised by managers, and sometimes they are conversations initiated by developers themselves. They have a question, they come to me and together we find an answer.

I also work with the entire team – managers, developers, testers, etc. – to help improve the team performance. This typically starts with a discussion about Agile, or a process mapping exercise. When a team is first starting out with Agile I like to run what I call “future-spectives”, or “reverse retrospectives”, in which the team choose the practices they would like to adopt.

Initially I run planning meetings, stand-up Scrum meetings in the mornings, reviews and retrospectives. Over time I like to hand these meetings over to team members to run themselves.

As the team improves I introduce new practices and new exercises. Gradually the team starts to look like a true Agile team. I think you need about six weeks to lay the foundations – hence my Scrum Foundation Programme.

I also help remove block and impediments when I can. It can also be as simple as buying books, or (as on a recent engagement) persuading the company to spend money on more technical training for the teams.

There are hundreds, even thousands, of small decisions made everyday during software development. These decisions are based on people’s mental models of how development works – or how it doesn’t work. Part of the Coach’s role is to help people unlearn many of these models and relearn models based on Agile values.

I’m often heard to say that it is in these thousands of small decisions that success or failure hides – “God is in the detail” or “The Devil is in the detail” if you prefer. By definition big decisions aren’t made that often and you usually know they are being made but small ones are made without thinking. Its no use switching to Agile if you keep making the small decisions based on some other model.

Its these small decisions that you can’t see in advance that can derail work. That’s part of the reason I prefer working as a Coach with a team rather than as a Trainer. Yes I deliver training but when I’m gone do people use it? When I’m there as a Coach I walk people through the changes, and I can provide any training as we go.

Each team is different so the approach has to be different every time. It also means that while I describe myself as an Agile Coach I sometimes work in a more interventionist manor. On occasions I will take over the management of the team – as a Development Manager or Project Manager – and help the team change that way.

Working this way is actually harder. Working as an external Coach you are looking into the team and you can see what is happening more clearly. When you are inside the team its often more difficult to see what needs doing. That said, once you have decided on action it is usually easier to make it happen.

On other occasions I’m more of a hit-and-run Consultant. I arrive, I talk to people, I make recommendations – perhaps write a report – and move on.

Finally, working as a Coach isn’t, and shouldn’t be, a full time job. I’m always conscious that the team shouldn’t become overly dependent on me. I prefer to spend only couple of days a week with a team – although when a company has several teams I may end up switching between them during a full week.

So that – briefly – is what this Agile Coach does for teams. And if you think some of this would help you please give me a call – 0845 310 8366 (UK), allan@allankelly.net. (Yes, that last paragraph is self serving but this is my blog!)

My suggestion that C++ should split in two prompted an e-mail from Robin Williams. It seems there have been similar thoughts in the Fortran community about extensions to that language. The result is the F programming language.

Its not clear whether F is an active language or whether it is a moribund experiment. As Robin pointed out, one could consider Fortran 77 as the simpler version of Fortran.

I suppose a similar thing could happen with C++, people don’t have to use the new standard, they could just stick with C++ 1998. Hopefully some compiler vendors will offer a switch to switch off the C++ 200x features.

Or of course, we could just stick with C.

I was contact by a company a few weeks ago looking for some Agile and Test Driven Development training. At first my heart sank when they said they still had a lot of code in Pascal, or rather Object Pascal (Borland 7). But in the last couple of weeks I’ve been wondering if that’s a good thing.

I used to program in Borland (Object) Pascal 7 and it was a nice language. Sure we didn’t have meta-templates or reflection but it could do most of what I wanted without half the complications of modern C++, Java or C#.

As you might expect from the ACCU conference there was plenty of C++. But the language was far from the only one in town – there were at least 8 others. This year most of the C++ sessions looked at the forthcoming C++ 200x standard. It now looks unlikely that this will be completed this year so we might want to rename is C++ 201x. As to exactly when, well, your guess is better than mind.

Some years ago I said that this standard would be “the longest suicide note in history.” I stand by this comment – indeed I found other people repeating it, either to agree or disagree.

I say this because I believe the language is now so big to be un-teachable, and possibly unusable. I don’t see how making the language so much more complicated will encourage more people to use it, indeed I see the reverse. And I believe this standard will only add to the confusion around C++.

I’ve also been saying for the last few years that we have seen the last generation of C++ programmers. A recent meeting made me rethink this position – a recent graduate who has to learn C++ for his first job. But then, at the conference and without promoting from me, Andrew Holmes said he didn’t think anyone would learn C++ after 2006 so maybe I’m not completely wrong.

Anyway, learning the new C++ will only be an academic exercise for the next decade. It will be a while before any compiler implements it and much longer before a substantial code base exists to work in.

I believe things need to be removed from language to simplify it. So I have also been suggesting for a while – to anyone who will listen – that C++ should split in two. There should be one backward compatible version with minor fixes and enhancements. A second version should break backward compatibility, remove features, fix elements of syntax and make it easier to learn.

After speaking to several people at the conference I believe this has now happening. Not officially but by defacto.

Walter Bright’s D Programming Language is gaining traction. This is the simpler C++ which I – and others – have been looking for. Unfortunately I didn’t get to talk to Walter at the conference, I hope he will return next year with more D and I will get the chance to speak to him. (Walter, if your reading this, can you please present D in 180 minutes?)

Its not a forgone conclusion yet but I hope D will succeed. Actually, I think Oracle’s purchase of Sun could help here. Oracle are quite clear why they are buying Sun: to get Java. This means things will change in the Java world – if only because Oracle want to make money out of Java.

Its too early to tell yet how this will plat out but I expect to see many people rethinking their languages choices as Oracle’s plan plays out.

Attendance was high. People came from as far away as California and Tokyo. Since the move from the Randolph to the Barcelo hotel the conference has been able to cope with 350 attendees. The last couple of years it has been sold out, this year it fell short by about 5. I believe 15 or more bookings happened in the last week.

Given that other conferences are reporting falls of 30% or 40%, and some are cancelling altogether that is remarkable. The success I believe is down to two reasons.

First the obvious: the conference had an extremely strong programme.

Second: community.

The ACCU conference is not just the best software development conference in the UK – possibly in Europe and indeed the world – but it is also the high point of the ACCU year. The ACCU has close to 1,000 members thoughout the world, it publishes 12 journals a year, runs several active mailing lists and holds numerous local events in the UK and USA.

For many members the conference is a chance to meet people face-to-face who they only meet electronically otherwise.