Time fly’s, I still feel young but when I realised this morning that this time 10 years ago I was working on a project for Railtrack I felt a bit older. That project taught me a lot about how not to run a project, it also lead to a lot of thinking over the years.

In the mid-1990’s the British Government set about privatising the railway system. At the time British Rail owned and ran everything to do with trains in Britain. It was too big to privatise in one go and the Government wanted to bring about competition on the railways so it was broken up into many smaller pieces.

A set of Train Operating Companies (TOCs) where established to run the trains, and one company, Railtrack, was given the track to run. The BBC have plenty of stories on what happened to Railtrack, for example this one, but none of these talk about APLAN.

To bring about this vision Railtrack needed a new timetabling system, the old one kind of worked, it was a MSDOS based computer system (PROTIM) and lots of paper. This didn’t matter much because the timetable didn’t change much, just twice yearly increments. But for the new world of competition something much bolder was needed… and so project Access Plan, APLAN for short, was born.

By the time I joined the project in September 1995 it had failed once already and had been passed to a second software development company – Sema. At its height there where about 120 people on the project. The odd thing was that few of these where programmers: at most 12 C/C++ developers, 2-3 Visual Basic developers and 4-6 database DBA’s / SQL developers. The rest of the team was business analysis, testers (system and user acceptance), architects (who didn’t code and came from a COBOL background) and managers: lots of managers.

The project was priced on time and materials so there was money to be made. I was a contractor working on an hourly rate (6 days a week), there was my agent (adding 20-25% for themselves) and then Sema, with their own staff living in hotels and working on a T&M basis plus the a mark-up on the contractors. Railtrack was paying the bill but they where still owned by the Government so it was really the tax payer paying.

Just to make sure we all got the job done to a suitable standard we worked to ISO-9000. We had think process manuals and even thicker requirements, architecture, specification, functional specification and test plan documents. Unit testing happen religiously but it wasn’t automated. (This was the project where I realised that ISO 9000 was evil.)

The project was a “success” – it shipped code, Railtrack used the system to produce a new timetable.

The project was a “failure” – very early on the most ambitious ideas about train companies bidding to run trains had been dropped. Towards the end of “phase 1” even more features where dropped and much of the final timetable was produced on paper.

Trouble was: who was going to tell the Prime Minister the system was a failure? Sure everyone at the code-face knew things weren’t going to work like people envisaged, even the first line management knew that. But the message was watered down as it got passed up the chain. If the CEO of Railtrack or John Major knew the true extent of the problems the privatisation couldn’t happen.

And if it didn’t happen when it was scheduled to happen well… everyone knew that Tony Blair would be Prime Minster in a matter of months so it might not happen at all. So the project was a classic Death March.

“Phase 2” was cancelled before it even got started. Yet with the bulk of the system in place Railtrack had to continue to use it. I even rejoined the project two years later to develop parts of it – but that is another story.

Of all the lessons I learnt on this project two stand out.

Lesson 1: Inside every large project is a small one struggling to get out

This project created its own size, more people required more people. It was T&M so there was an incentive to add people. We could have lost 80% of the people and achieved an awful lot more than 20% of the work.

Yes, 20%, 30% or even 40% would not have been what the Government wanted but that leads to lesson 2.

Lesson 2: Some projects shouldn’t be done

I often asked myself: what would I do to fix the project? By the time I was on the project it was already too late.

I often asked myself: how would I have run the project? I knew a lot of techniques to improve things, and I know a lot more now, but I don’t know any that could make this project a true success. The only conclusion I can give is: the project should never have been done in the first place.

Some projects are too big, too complex, too ambitious, too time constrained that they should not be attempted. As a supplier you should not accept such projects.

Of course there will always be someone, somewhere, who given enough money will attempt it but these are actually the worst people to attempt such a project. If the customer will not listen to advice and does not realise what they are getting into it is their own problem.

No, if someone comes to you with such a project you should try to persuade them of a better way: extend the time allowed, break the project into chunks, reduce scope. Infact, this is more than just a “should” it is your duty!

But if at the end of the day you can’t persuade them to accept an alternative don’t do the project: you don’t need the money that much and your staff will thank you for not making them do the project.

In ten years a lot of things have changed: Neither Sema or Railtrack exist any more, I’m not a contract programmer any more, our business models where not sustainable. Unfortunately death march projects still occur.

I’ve used some of the Christmas time to rush through to the end of my latest read – Good to Great by Jim Collins (Random House, 2001). I’ll admit upfront I wasn’t going to read this book but the senior management at my company have been reading it over the last year so I thought I should join in too!

Its easy to see why the book has been such a big seller. Its easy to read – short and a chatty style – and it makes you feel good, it suggests you do nice things in your company not nasty things.

Personally I don’t find it adds very much to the discussion that hasn’t been said already. Yes it introduces a few new metaphors (the Hedgehog concept and the Flywheel) but nothing you can’t find elsewhere really. In part this might be because the book is a “prequel” to Jim Collins and Jerry Porras’ Built to Last.

Yet I found the book reminded me more of another book, The The Living Company by Arie De Geus (1999). This should not be a surprise because De Geus discusses the similarities between Built to Last and Living Company so we should expect a few.

Unfortunately Good to Great lacks a good bibliography or further reading sections. This seems to be a common failing in many business blockbusters, they strip out the endless referencing that academics love and is necessary for hard research and journals to make it easier to read but at the same time you loose the context of where the book fits in or where to go next.

For example, Collins tells us that his Great Companies “confront the brutal facts” but doesn’t discuss how they do this or why they ignore them. Arie De Gues does, he explains why companies ignore known facts and what they can do about it – his solution is to “create future memories” by “planning as learning” and “scenario planning.”

(One could also refer to Pfeffer and Suttons Knowing-doing Gap on this subject but again Collins doesn’t.)

One more tiny example I have to put in so I can recommend a another good book: Collins mentions the importance of project post-mortems but that’s it. Norm Kerth’s Project Retrospectives goes into great detail on how to do this.

Why bang on about referencing? After all, references do make the book harder to read. Well, references add credibility for one. Second, I’m unhappy when an author seems to be claiming all the ideas for themselves – not that Collins does, but some other authors do. But perhaps the most important reason is so you can learn more about a particular issue.

Anyway, enough about referencing, what does this book say? The big themes are:

• No single event that creates a great company, not hiring a particular CEO, discovering a technology or launching a product. Its emerges over time.
• Get the right people, get shared values and the rest will come
• Confront the brutal facts: most companies fail to pick up on the important, life threatening or enhancing, facts and events.

Put all this together I found the book a pretty convincing argument for the argument that business strategy is more emergent than planned. Collins more or less says that most companies didn’t recognise their strategy until they were in the middle of it. Again, overtones here of Henry Mintzbergs’ The Rise and Fall of Strategic Planning but Collins doesn’t tell us this.

So if your looking for an easy read then this book is worth the money. If your looking for more depth then I recommend you read The Living Company . Among other thing both discuss:

• Getting the right people, and getting rid of the wrong ones
• Develop your people, they are there for the long term
• Quality discussions between people
• Recognising and acting on facts and events

In fact, one of Collins interviewees, at Kroger supermarkets, actually says “the company had a will to live.”

On Wednesday night I was out drinking with ACCU members in London. For those of you who don’t know the ACCU I’d better explain. It is an association of professional developers, for my money the members are among the best software engineers in the world. Great people, if you ever get the chance to hire an ACCU member then do so, they share a passion for software development, improvement and learning.

But, as I’ve said here before I am no longer a software engineer – I am a Product Manager. And I got my legged pulled a bit on Wednesday night for that!

Perhaps I should give up my ACCU membership and my association with so many engineers. Certainly I don’t find articles in the ACCU magazines as interesting as I did – too many curly braces! – my interests have changed.

Now there is no rule that Product Managers can’t associate themselves with such groups but there is no such thing as a free lunch, if I am to embrace my new role and identity I need to give up some stuff. Every time I associate myself with my old role I’m not embracing the new.

But actually, I am still a software developer, I’m still helping develop software, I’m just doing it differently.

I’ve always preferred the title Software Developer to Software Engineer. Two reasons really, firstly, I’ve always had my doubts about how much engineering really goes on when writing software. Secondly, and more relevant here, I’ve always been aware of the other stuff that goes on.

Developing software isn’t just about cutting-code. Its about customer requirements, their problems, packaging, delivery, marketing, solving problems and introducing change.

Now I’m a Product Manager I’m concerned with: My customers, their problems and what software can do to solve their problems (which means change.)

So I’m still developing software but I am at a different place in the food chain.

I’m not really that unusual, a lot of product managers have a engineering background – and a lot have an MBA so really I’m not unusual – I’m dangerously close to being typical!

In some ways my engineering background might be a disadvantage. If I don’t jettison some of the old identity – the bit that wants to engineer and change code – I risk focusing on the wrong things. It is so much easier to stay in the comfort zone of what I have done before, to hide behind the technical rather than focus on the customer and do new things.

Failing to focus on the customer is probably one of the oldest (the oldest?) failure modes there is – both in software and in business generally.

I’m back in the USA for a few days attending some seminars on product management. They very good and highly recommended – check out the website links, http://www.pragmaticmarketing.com/ and http://www.productmarketing.com/productmarketing/.

As I have said before his blog I have been trying to get to grips with product management and what is it product managers do for while so the seminars most useful. About half material in the seminar is like an MBA refresher course – specifically my marketing modules. The other half is really what do product managers do? And how do they do it?

So to save you the suspense I’ll summarise the answer… product managers identify what customers needs and get it delivered.

But there are one or two twists…

First is, it is not what one customers says it is what a set of customers say – that is what the market says not what an individual customer want.

Second twist: it isn’t the features the customers ask for that we should build but a solution to their problem. So we have to look beyond the mere feature request to a deeper level to the problem that they are encountering and then we need to build the solution.

So if a customer asks for the colour to be changed you have to ask: why? If the answer is simply that this customer doesn’t like the colour then probably it is insignificant.

But if you find that many customers are asking for the change and when you look into it you find it is because you’re product appeals the colour blind users then there is a legitimate reason to change the colour – there is a problem to solve.

But to properly solve the problem you have to make sure you change the colour in such a way that colour blind users can use the software. There is no point changing from one colour to another if it doesn’t improve situation.

Picking up another theme of this blog, nothing I have heard here contradicts what I’ve written about Lean Thinking. In fact I think to ideas are completely complimentary.

• Lean thinking says: reduce waste – specifically don’t do work that you don’t need too, and when you do solve the problem solve it completely, no half measures.
• Product management says: only do the work that directly benefits the customer, work that we have qualified and will help the customer. And when you do it solve the problem completely there’s no point in only solving half the customers problem.

So seems to me that both sides are saying the same thing don’t do work that nobody wants. That may seem pretty obvious but it seems pretty difficult to actually do.

John Maynard Keynes, the economist once said:

“The ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is commonly understood. Indeed the world is ruled by little else. Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist.”

Ideas can seem small. They can seem weightless. How many ideas fit on the head of a pin?

But Keynes was right. Consider Karl Marx, his ideas where hatched during the middle of the nineteenth century but led 60 years later to the Russian revolution, and from there to the cold war that only ended in the 1990’s. Millions still live with his ideas in China, Loas, Korea, Cuba and elsewhere.

That is a powerful idea.

Maybe if Marx hadn’t had these ideas someone else would, but mankind had gone several thousand years before he came up with the ideas. Maybe it would have been another thousand years before someone had the ideas. Still, apply the Jackson Pollock test, it was Marx who had these ideas when he had them.

Something similar happens to ideas in companies.

Often it takes time for a new idea to work its way into the system, people need time to think about the idea, see if it makes sense to them and try the idea. Sometimes the people who embrace the new ideas aren’t in a position to do anything with them only in time as these people move into positions of influence can they do something with a new idea. And sometimes it takes time for people to see the applicability of an idea, only over time as they now view the world with the new idea in their head does it start to make sense of them.

So as with Marx there may be a the gap between the idea and affect.

Ideas can often seem very abstract and they can be difficult for people to grasp. This is where stories role to play. By embedding an idea in a story that tells how it changed people, and what was done, the idea is less abstract and people have an example to better understand the idea. This can help in internalising the idea and seeing work can be applied in an organization.

Still it can take time for an idea to have an affect. We can speed up the work of an idea if we support it but we can also smother it. I suppose a I have been guilty once or twice of having a great idea, or at least an idea I thought was great, and by being over enthusiastic about the idea I have made people sick of the idea.

Another thing that can slow down theory adoption of an idea is the need to change some of our existing ideas. If we are to embrace a new idea we often have to give up something else – if I want to embrace the idea of a red car I need to get rid of my idea of a blue car.

And so back to Keynes who also said

“The difficulty lies not so much in developing new ideas as in escaping from old ones.”

Look at the difficulty some countries have had in getting rid of Marx’s ideas.

I spent a couple of years working in California, in Silicon Valley itself to be exact. Unfortunately instead of getting .com boom I got .com bust, still it was a very interesting experience.

One of the differences I found between London and California was the existence of Product Managers. I’m not saying product managers didn’t exist in London but they were few and far between, whereas in California they were plentiful. These are the people actually charged with ensuring the product develops in the right ways and they seem to be intrinsic to high-tech companies.

(As I’ve noted before product managers exist outside of the software arena but these are often marketing rules concerned with branding, advertising and image.)

Of course back in London we had Business Analysts who performed some of the same role but the two are very different beasts – product managers are much more outward looking and business analysts are normally inward looking. Of course some of this is the difference between a product company and the bespoke development organisation.

As I’ve noted here before this is the year when I became Product Manager. Last week somebody asked me an interesting question: “what is it product managers do?”

So I waived my hands a bit and I said something about understanding the customer, understanding customers needs, customer problems and what they are looking for a product.

And the product manager needs to talk to the technical people who developed the product so they can understand what is possible, what new technologies are coming, and how the product might be able to meet customers needs.

In a way that product manager is a translator, translating what the developers say to the customers and translating what the customers say to the developers. But there’s more to it than this.

There’s an element of creativity, seeing beyond the customers immediate concern to what could be, and imagining how the different technologies can be put together to create something new.

There is also the question of product strategy, where is the product going? What will look like in a years time? What about the competition? Is there even a competitor?

Then there’s the question of marketing: so-called inbound marketing (finding out what the market wants) and outbound marketing (presenting the product to market). The marketing and strategy questions are very closely related.

And you can throw in here something about product vision too.

Of course all this these be aligned with company strategy, so you might well get involved with setting the comely strategy to. Normally the product strategy will support the corporate strategy, if product strategy and corporate strategy are different there will be problems.

Then there’s the question of project management. In some organisations that product manager might be quite close to the project management, in fact they may do themselves. The product I look after has a small development team so I get involved in a lot of project management decisions. On bigger teams than maybe for a project manager and a product manager.

However I’ve also seen situations where there is a product manager, a project manager a software development manager and maybe a technical lead too. When this happens you have too many cooks. It is often said that the best software development teams a small but there is no point in having three or four software developers and another four or five managers.

But I digress….

In many ways the product manager is a product champion. In this part of the reason why think a product manager’s role is essential. The product without a champion is unlikely to move forward and advance. Of course there should be room from more than one champion, the more people who are enthusiastic about the product the better.

So being a product manager is a mix of all these things and probably a few more.

Unfortunately that is rather longer answer than some I would like – I’d like to have a more succinct answer to the question. So I need to keep thinking about this and see if I can come up with a nice short answer.

The naming of patterns can be a tricky business. There are all sorts of rules of thumb, one can use, for example: favour and nouns over verbs, tell the reader what to build and describe what you get rather than what you do.

My first proper pattern, Encapsulate Context, didn’t follow these rules. In fact, there was a lot of debate over the pattern name, if I remember rightly. I originally called it a Program State, then during the writing it became Encapsulate Exclusion Context, and when it was workshop at EuroPLoP the group felt the name Encapsulate Context was best. So it became Encapsulate Context.

Well a while back, I rrealised the name Encapsulate Context broke a good many of the rules of thumb, but I felt that it was too widely known to change.

Earlier this year, I submitted the pattern to the editors of the forthcoming patterns book Pattern Languages Of Program Design (Volume 5). The pattern was anonymously peer reviewed by two other writers, and in the best tradition of anonymous review one of these thought the name was fine, and the other one wanted a radical change.

The one who wants a change wanted the name changed on the grounds they it confuse Smalltalk programmers. Since the pattern is aimed at C++ and Java programmers this wasn’t a big concern of mine. And again, I felt the naming Encapsulate Context, had a certain history.

( The book, by the way, probably won’t appear until early in the New Year but you can pre-order it already Pattern Languages of Program Design 5.)

So when the book appears the pattern will be Encapsulate Context but since then, the name been on my mind more and more. Finally I decided to change it.

There is now a new version on my web site using the new name. Well, I say, a new version. Apart from changing the name, i.e. adding a single d, there isn’t really any change.

Moral of the story? Embrace change, don’t let the past, confine you.

Encapsulate Context is dead, long live Encapsulated Context!

I’ve raced through my latest book, sign of a good book, an interesting read, and most of all, a sign of large print!

This month’s book has been Lean Solutions by Womack and Jones (2005), two of the original authors of The Machine That Changed the World (Womack, Jones & Roos, 1991).

In some ways, Lean Solutions is an update of the earlier book for the start of the 21st century. It updates the ideas with some good examples from outside Toyota. I particular like the Tesco example, and the study of the sports shoes.

Did you know that training-shoes (sneakers) need to be ordered 5 months in advance? Or that Nike alone produces more sample shoes than the fourth largest manufacturer? (And that was before Adidas bought Reebok, so I guess it’s more shoes in the third largest manufacturer now.) It all because the industry is not lean.

Order have to be made so far in advance so they can be sent to Asia for manufacture. They have to manufacture enough to ship a large container to the US or Europe. And that transport takes time. That needs to be divided into smaller loads and distributed. There are delays and buffer stocks at every stage.

So, not only is there a lot of surplus in the system but the company isn’t very responsive and customers often find items out of stock. In fact, it puts the whole economics of lowest-cost manufacturing into doubt – and all that implies for offshore production.

The book focuses on customer rather than manufacturing. The authors identify the main customer problem today: lack of time. So it is fitting that several of their studies are drawn from service industries such as medical care and car maintenance.

But the authors go beyond lean thinking and case studies. They envisage a world where lean is the norm, they discuss how the world could be and how all our lives could be improved. The author’s set an agenda that you massive change the way the world works.

If you know a bit about lean this will teach you a lot more. If you know about lean from older books this will update you. And if you want to be lean this will give you some good examples and stories to tell.

Continuing on the theme of innovation, there is another common technique used by companies to produce innovation. Often it used to develop somebody’s innovative idea and is sometimes used to generate innovative ideas as well. This is the skunkworks model – or to give it a less jazzy title: separate your imaginative team.

In this model, the team that is to produce the innovative product is separated from the main organisation. The people involved a ring fenced, they may work at a separate location, they are removed from the day-to-day life of the company, and in particular, the politics and blocks on innovation that exist normally. Sometimes these teams are kept secret.

This technique has been documented in countless stories, indeed, Lockheed Martin have trademarked the term skunkworks. If you want to know more about this method you could read Coplien and Harrison’s Skunkworks pattern and in their book, Organizational Patterns of Agile Software Development, 2005.

(I have also discussed the technique in pattern form, Separate Imaginative Teams.)

Hamel and Prahalad also noticed this technique in their Harvard Business Review article, Corporate Imagination and Expeditionary Marketing – May-June 1991.

There is something macho about this technique: the image of a bunch of brave souls going off to design and create a new product, cut off from the Corporation, free from the politics and infighting. And sure it does work, companies do create new products this way, however, this technique also has a downside.

This technique may create a new product, it may bring about an innovation, it may get you out of a hole right now, but it does little to make the overall organisation more innovative. In fact, it may detract from the overall company’s innovative ability.

To start with the innovative people are separated from the rest of company so none of their expertise or experience is directly accessible by the rest of the company. Neither do they form role-models for other people in the company. Many times, the new product development is invisible to rest of the company – they just get on with their regular work.

And when the product is produced in must somehow be folded back into the company. The rest of the company may not understand the new product. Indeed it might be quite different from the company’s main products. Therefore there is a learning curve, while the new product becomes part of the stable.

The people who have created the new product also need to be folded back into the company. But while they have been outside the mainstream, they may have got used to a different way of working, a freer environment, a lack of politics or structure. To these people re-entering the corporate fall might be difficult. Indeed it might be easier to leave the company altogether.

Meanwhile, the people who remained with the company working on the old products are not working on the shiny new thing, they may become resentful of those working on the new product – especially if the skunkworks team are seen to be given privileges or better resources.

And there’s not forget one the points made by Arie de Geus which I noted a few days ago: it is not just the taking of decisions that takes time but the acting on them too. The people outside of the company have made all sorts of decisions, and when they returned to the company they expect those others to act on them – there will inevitably be a delay. Indeed, there may be a repeat of the learning curve.

Perhaps, one of the most famous examples of this approach was Xerox’s Palo Alto Research Centre – or PARC for short. Xerox set up a research centre on the other side of the country, stuffed it brilliant people and gave him plenty of money. The project succeeded, it invented most of the features we find in the modern PC.

But the project also failed, the team was so far removed from the main Xerox Corporation and the company could not usefully exploit their innovations. Still the researchers found a way, and many of them left Xeroxto found successful start-up companies in Silicon Valley, for example Adobe and 3Com.

So, on balance, I am not a fan of the skunkworks approach to innovation. If you want your company to be innovative you have two embed, within the company, and within the values.

There is a lot of hot air spoken about innovation. Indeed, there is probably more talk about innovation than there is actual innovation itself.

I started to get all excited about innovation on Friday, when one of my managers said:

“Allan, do you know anything we can do to improve innovation?”

Of course there is one obvious answer: 20% personal projects, its the 3M example – and now Google. Just about everyone seems to have heard this example so I’ll be brief in my comments: at 3M engineers are encouraged to spend about 20% (figure varies depending on who you read and which company it is) on “personal projects.” Some of these projects eventually make it to full products, like Post-it pads and Google News.

But are there other things you can do?

I was excited so I went home and started looking through some of my textbooks and journal archives, my question: just what do you do to produce innovation?

On the one hand innovation is just an extension of problem-solving, which is itself an example of learning. So doing innovation is firmly within the knowledge and learning agenda I keep banging on about. But on the other hand, innovation is so specific it is a subject in its own right.

Regular readers of this Blog will know I don’t have much time for “big brains”: I don’t believe that the CEO, CTO and a few managers can sit in the boardroom for six hours and come out with a new product. Most innovation is bottom-up.

Nor do I believe you can schedule innovation: Ever seen a project plan with a date pencilled in for innovation? No, you can’t timetable it.

So, how do you do it?

Looking at my books I find advice like: improve you ability to learn, trust your employees, organize your business structures to promote innovation, value innovation, align your HR policies (reward innovation and risk taking), don’t punish failure, and so on. These are all big, macro solutions, they may be necessary but they are not sufficient, you need something else.

You can set your business environment up to encourage innovation with these ideas, you can show people it is valued, but what do you do?

This is where the 20% personal project comes in. It is something you could do today. It is easy to see how you could get a new idea out of it – whether that is a product or process innovation.

A few months ago I was able to speak to someone who works at Google and they described how this works.

Engineers need to spend 20% of their time on a personal project. But many of them don’t know what to do, so most of them are open to suggestions. Meanwhile, the product managers have the opposite problem. They are identifying things the company could do, but without a prototype or proof of concept they can’t get any official resources.

So the product managers look around and find engineers who need projects. They then have to interest the engineers in working on their idea. If the project goes well they can then go official and ask for full project status.

(By the way, read my lips: No project managers!)

The trouble with this example, the 20% example, is the one everybody cites. It seems. When asked: “how do you do innovation?” People reply with a 20% example. What is actually happening is that this example is getting in the way of other ideas and examples of how you do innovation!

So, dear readers, the challenge for you:

what does your company do to encourage innovation?

I need your ideas and experiences.