Programmer, software engineer, software developer, coder, developer, code-monkey, build engineer, tester, software tester, test automator, test engineer, quality assurance engineer, analyst, business analyst, system analyst, product owner, product manager, requirements manager, subject matter expert, domain expert, project manager, change manager, programme manager, development manager, iteration manager, release manager, deployment manager, scrum master, team leader, technical lead, development lead, architect (system architect, solution architect, software architect), consultant, coach, designer, user interface designer, user experience designer, product designer.

Did I miss any? – There are a lot of roles in software development teams and even more titles. Does your team have a full set? Do you need them all? How do you choose which ones you have, and which you don’t?

And everyone of them offers a solution to your problem. So….

In walks the Programmer (Software engineer, software developer, etc. etc.) and says: “Yes, I understand your problem, what you need is some software writing. Only us programmers can do that so don’t worry about that testing lark, or analysis. Just introduce me to some customers and I’ll work it out.”

The Business Analyst walks in next: “Its all very well writing something, and its all very well making sure it works. But, is it the right thing? How do you know its what you need? Us Business Analysts are trained to understand what you need and give those requirements to the programmers and testers.”

Next in is the Tester. “You see, the problem is that Programmers don’t really know what you want, and when they do they make mistakes. So I’ll test everything and ensure it works, and that it is what you want.”

Very convincing. Next you talk to a Project Manager: “yes, these programmer and testers are very good but have you noticed how long they take? Unless there is someone here to organise them, and dare I say push them, then you’ll never get anything.”

You see, everyone (every role) offers to solve your problem. Maybe by creating a solution, maybe by ensuring its the right solution or maybe by making sure that it all fits together.

And these roles overlap. Programmers can talk to you about what you need, so can Testers. Project Managers like to have a chat too and Business Analysts claim its their right.

As you get more people things get more complicated. They seem to make work for themselves. So you need an overall Development Manager to manage you teams. You hire Architects to ensure the Programmers really do it right.

More people begat more people: Release Managers, Programme Managers and Team Leaders appear to organise people. Designers, Build Engineers, and Consultants appear to specialise in parts of the system.

They all claim to have the answer to your problems but things look worse.

Then someone tells you about Agile. You need a Product Owner, a Scrum Master and a Team. More roles, and the people you already have don’t want to give up their titles so you add some more.

And the holders of all these roles claim they have the solution to your problem.

They can’t all be right.

Thing is: there is only one role you absolutely must have on a software development project. That is the guy who does the coding: Programmer, Software Engineer, Software Developers, call them what you will. If you are not getting code written you aren’t doing software development.

Which of the other roles you have depends on the complexity and nature of the work, the size of the team, the approach you take and your wider organisation.

This doesn’t stop the specialise from telling you they have the solution to your problem. Which makes it all confusing.

I intend to keep with this theme of roles for a few blog entries, so watch this space.

A while ago I tried to answer the question “How many software teams are using Agile?”. My best efforts put the answer at somewhere between 5% and 15%.

In the search for some other data I’ve recently been look through Capers Jones “Applied Software Measurement” (2008). It is packed full of interesting statistics and I really must buy a copy and read it in more detail. Anyway, Jones gives his hypothesis of project in this table:

Type of development	Number	%
End-user	2,500	2%
Agile	20,000	13%
Web	12,500	8%
Systems	8,750	6%
MIS	70,000	45%
Commercial	3,250	2%
Outsource – US	17,500	11%
Outsource – offshore	15,000	10%
Military	7,750	5%
Total	157,250	100%

(In case that table gets mangled, the first column is the type of development, the second the total Jones looked at and the third the percentage of the total.)

Jones puts the number of Agile projects at 13%, firmly inside the range I give. Of course there are two caveats here. Firstly, the ways Jones has sliced and diced the numbers leaves me wondering how he has counted military work running Agile, and outsourced work running Agile.

Second, how do we know if a piece of work is Agile or not? All the evidence I have tells me that an awful lot of teams are saying they are “Agile” and frankly aren’t. As Kevlin Henney suggested on re-twittered (from somewhere or other) recently a better question to ask is “How are you Agile?”

I’ll get Jones book and read it some more. One other thing he mentioned stood out: getting metric on software projects is hard. He says its like searching through debris, lots of rubbish then occasionally a nugget. Which sums up how I feel about software metrics now. Its quite like I imagine archaeology.

I recently stumbled across Les Hatton’s site. I’ve owned one of Les’ books for probably over decade (Safer C, a classic) but this was the first time I’ve looked at his website. I highly recommend a perusal of the site and reading some of his papers. Apart from anything else he’s an even more interesting guy than I thought he was (and I already though he was interesting!).

One set of his papers looks at the power-law nature of software code. I’ll admit I couldn’t follow all of the academic speak and advanced maths terms – which might disqualify me from reading some papers but it
more a sign that I didn’t take the time to work it out in depth. (Most of his papers are written in a very accessible style.) The key point I took away was that he extends James Noble’s work to non-OO systems. Hopefully I’ll find time to go back and read more of these papers in depth and make sense of the bits I skimmed this time.

Another paper which stood out was “How to build successful complex systems: lessons from open source” with his suggestions for how to build the NHS IT system. Until I read this I was unaware that the Welsh
Health Service had taken a different route to the English one, had spent far less money and had actually produced a successful system.

Les, like many others, sings the praises of the Open Source movement. He’s right, there are some very successful Open Source products out there, but, as I’ve pointed out before, most Open Source
projects fail. And when I say MOST I mean tens of thousands fail for every success.

Fortunately Les’ suggestions allows for this; he suggests a Darwinian process with seeds 100+ development efforts and over a series of rounds reduces them to one winner.

Although Les suggests this in an Open Source context what he is actually suggestion is also a Venture Capital model. He proposes to give over 100 teams a little bit of money (seed money in effect) to create an initial system. Year on year winners are selected to continue development, these are given a bit more money. That sounds exactly like Angel funding, VC Round A, VC Round B, etc. etc. funding. OK, Les model is “open” because the winning teams get to pick over the remains of the loosing but even that exists in commercial activity: failing companies get bought by the winners who keep the best bits.

Either way the result is the same: rather than one “too big to fail” NHS IT project we have a number of small projects which compete and can fail. If we remove the ability to fail from the process then we have to
pay for failure.

Whether it be evolution, capitalism, IT projects or even banks, if failures are not removed from the system then change is inhibited because the winners cannot really win.

This is the important thing about failure. Forget that myth about learning from failures, the important thing is that it eliminates something that doesn’t work.

As regular readers may recall (one of my favourite expressions!)… I regularly ponder my subscription to the MIT Sloan Management Review. After a few dull issues – which leave me wondering if it is worth the money – it has once again come up with an issue with a couple of articles which justify the price for the whole year.

One of these is “Why Forecasts fail. What to do instead.”

The insights I find illuminating here are:

• Sophisticated, complex, models are good at fitting past data (“forecasting wit hindsight”) but they only not very accurate in predicting what will happen in the future (they tend to extrapolate)
• Simple models are not so good at explaining past data but are better at forecasting the future
• Human judgement is worse than statistical models at predicting the future
• Experts don’t predict any better than the average person
• Averaging the predictions of several independent individuals is more accurate

The article cites research to back up these assertions.

This fits in well with what I tell teams when I’m coaching and training in Agile methods. The simple estimation and velocity measures I advocate are better than the complex models which are too often preferred.

And it is no use asking a system expert (architect, senior developer, what-ever) how long it will take, their estimate is no better than anyone else. But, getting several different estimates (e.g. using planning poker or similar) is.

One technique suggested by the article is something I’ve tried in “future-spectives.” You say to the team, or individual, “Imagine we are at the end of the project, we have finished on time, in budget, everyone is very happy, what did we do right?” and the opposite: “Another failed project, what did we do wrong?” Imagining yourself in that situation can produce useful insights.

One of the ideas I talked about in my Jax London presentation is something I call the Scrum hegemony and it deserves a few notes.

In the early days of Agile there was a tendency to equate Agile with XP, that changed a few years ago and Agile become (almost) synonymous with Scrum. I’m not saying Agile was XP or Agile is Scrum, just that to the uninitiated it can seem that way. (I blogged about this nearly 2 years ago now, see “Scrum is the new XP”.)

In many ways the Scrum people did a fantastic job of making Agile acceptable to the corporation. They had data and Harvard Business Review articles to cite, they didn’t ask the corporation to get into technical details (like TDD) and they had a friendly (English) name which avoided the word EXTREME! And most of all they had Certifications. O, don’t forget a pretty good marketing machine.

All this had the effect of making Agile acceptable to suited corporate types who didn’t know the first thing about software development but knew projects were always late. Ironically Scrum isn’t much more than XP, indeed, it is less than XP.

Consider XP: you can basically divide it in two. The bits about engineering (continuos integration, test driven development, refactoring, etc.) and the bits about managing the work (iterations, stand-ups, stories, etc.). Scrum, as documented concerns itself with the management side.

Granted Scrum expands on roles, Scrum adds some concepts like self-organising teams, adds some terms like backlogs and renames others (iterations to sprints) and adds burn-down charts but the management side of XP is basically Scrum, and Scrum is XP.

Purists might like to argue about which stole from which but the point remains: they are the same.

Scrum is devoid of the engineering practices, but as I’ve noted before in this blog: Scrum without the engineering practices is heading for trouble.

XP’s success, and the even bigger success of Scrum had the unfortunate side effect killing off most of the other Agile methods: FDD, ASD, Crystal, etc. Pockets still exists (especially with DSDM) but that is all they are, pockets. That was good for understanding but bad for experimentation and learning.

That’s now changing. The Scrum hegemony is now ending. Kanban, and perhaps other methods, are now offering alternatives. David Anderson’s Kanban insurrection is again offering an alternative. Kanban is again allowing the experimentation and variation in process that the Scrum hegemony has been stifling.

Don’t get me wrong, I don’t think for one moment Scrum is going to roll over and disappear, or that Kanban will dominate. Scrum will continue to be the Agile method of choice for corporations, it will be the 800 pound gorilla to use a phrase. But it will no longer be the only show in town.

Kanban is on the rise and drawing more attention to Lean, Software Craftmanship is on the rise and Tom Gilb’s work is being re-examined. There has long been a divide in Scrum between those who believe in “one and only one Scrum” and those who see “Scrum A, B and C” (I was going to post a link here to Jeff Sutherland’s blog but it appears he’s removed the post). Now there is a schism in Scrum: there are two bodies awarding Scrum certification, Scrum Alliance who’ve been around for a while and a Scrum.org backed by Microsoft and Ken Schwaber.

One of the good things about Scrum was that it was clear about what it was and was not – unlike Agile. This increasingly looks in doubt. As Scrum has grown more popular variations have set in, differences in certification and types of Scrum only add to those differences. The danger for Scrum is that it goes the way of the word Agile and becomes all things to all men.

That risk is echoed in the wider Agile family now. I welcome the rise of Kanban, not just because I think its a good system but because I think it is offering opportunities to think again about how we do things. But the end of the Scrum hegemony could leave the Agile as a whole fractured and incoherent, and decidedly not the type of thing corporations should be involved with.

Worst of all, it could see a new methodology war. There would be no winners here, only looses. Scrum and Kanban, and all the other methods, shouldn’t be rivals just alternatives. Unfortunately between the method zealots and in the commercial market I fear that message will be lost.

Time to bring this mini series to an end. I’ve talked about

• Segmenting Architects
• Architects who aren’t
• Architecture or Design?

As a quick rule of thumb consider:

• The Product Owner (Produce Manager/BA): is concerned with the What
• The Project Manager: is concerned with the When
• The (your type of) Architect: is concerned with the How

And some closing advice:

• When you have a software system to design or fix, look to an architecture team not an individual. The team can share the vision in creating and exploitation.
• Never, ever, give someone the title Architect: give them the role yes, but never the title. Once you give them the title it is difficult to remove. And since you don’t know how their ego will respond you risk creating an problem. Architect is a role, not a title. Give people architect responsibilities but make sure they are still defined as a Software Engineer or whatever term you use were you work.
• The moment an “Architect” asks for an expensive tool to draw UML in then its time to let them go. UML can have a role but it isn’t worth spending money on. Expensive tools allow people to hide, architects shouldn’t hide. A cheap tool might be acceptable but don’t let them hide behind it.

Continuing my examination of the architect role I think we need to point out there is not one architect role my several. There is no such thing as an “Architect”, only an “Architect of something.”

Rather than talk about an “Architect” with one word we really should use two words. The first word describes what they are an Architect of, and the second, well thats the word “Architect”.

Some examples should help:

An Enterprise Architect: One who is concerned with the systems of the enterprise as a whole and how those systems fit together. By definition this architecture in the big and it means making architects at the grand level and not being concerned about details.

So: the Architect might rule that all new systems should be in Java, and not C# but they should not concern themselves with Java coding standards or which design patterns the Java developers are using.

A Solution Architect: these architects get involved in the early discussion about what the solution will look like. They may be responsible for sketching it, they may devise a prototype, they may be involved in customer conversations around requirements because you don’t know what you want until you see the solution; and this means they have some business acumen.

You might say these are the guys who have the initial vision for the final product. Importantly though: they need to stay with the development from start to finish. They can’t walk away once they have sketched out the initial vision. They can hand over to someone else with time but they need to keep skin in the game to have legitimacy and so they learn how effective their solutions were.

Ideally, when you start working on a product/project you want one of these guys involved but you want them to change hats as the work increases and become: a software architect.

A Software Architect: This is were my interest really lies, these are the guys who are custodian of the design vision for a piece of software, or application if you prefer. They are responsible for ensuring the whole team shares the same idea of how the software is designed. As such they are more responsible than most for how the software looks (inside) now and in the future but they are not solely responsible. Software Architects should have more experience than other team members but their responsibility is to lead through teaching.

Software Architects must implement, they need code under their finger nails if they are to retain their knowledge and legitimacy.

Their work includes:

• Being a Senior Developer, hands on, they should have code under their finger nails
• Educating junior developers, sharing their knowledge, mentoring, teaching and training
• Guiding development towards a consistent and sustainable architecture
• Holding responsible for the shared technical vision and ensure it is shared
• Dealing with Conway’s Law: interfacing with non-technical manager types to ensure the technical and organisational architecture are compatible

You might like to note I’m not saying they create the design, they do not. Their responsibility is to help the team create a shared understanding which is the design, and then become custodian of that vision.

I’ve avoided mention of roles like Network Architect, these might well exist but not always. Maybe you can think of some more architects in your organization.

I’m also avoiding the term System Architect because you have to define what you mean by “system” – where does it start and where does it end? Potentially System Architect is “Architect of Everything”.

These roles will also vary by organisation size. In a large organisation you might find these roles exist as distinct roles, in small organisations they are likely to overlap. So an Enterprise Architect also needs to do some Network Architect.

This is understandable but problems come when one individual is so busy wearing many hats that they neglect some aspect of one of the roles. They try and do two or more roles and end up doing one (or all) badly. Consider an Enterprise Architect who combines his role with being a Software Architect at the same time. If the Enterprise aspect dominates they may neglect involvement with coding the application so they cannot speak with experience about the application. Or, the other way round: they are so busy being a Software Architect that they don’t keep up to date with emerging trends and neglect the Enterprise side.

I mentioned the A word in my last post (Are there any System Analysts out there?). As regular readers might have noticed, I have over the years of this blog taken the odd pot-shot at Architects. But I’ve avoided direct comment on architect and architecture. Somehow it feels the time has come to address this debate.

I should say before we go too far that when I talk about Architecture I’m thinking software architecture. And when I talk about Architects I’m thinking about the architecture of software. I’m not thinking of network architecture, or business architecture, of course these topics are connected with software architecture but we need to narrow the topic down a little.

Nor am I talking about system architecture and system architects. System architecture is a particularly confusing concept because we first have to define the size and scope of our system: a piece of software is a system, so too is the computer it runs on and so too is the combination of both. So lets agree to leave system architecture to one side too.

Before we can tackle to subject of Architects I think we need to get a better understanding of architecture. So I’ll defer architects to another day and think about the nature of architecture today.

If we look up architecture in the dictionary then we find its all about design. Architecture is design. The word “architecture” (like the word “design”) is both a noun and a verb. Architecture is something you do (we may create a proposed architecture) and it is something you have, you software has an architecture whether it was design or not.

As an experiment, whenever you hear someone talking of “software architecture” mentally substitute the words “software design”. I don’t think you’ll find any loss of meaning.

Both design and architecture are, certainly in software systems and frequently elsewhere, part a deliberate attempt to create a particular outcome, they are also part emergent. How much is deliberate and how much emergent varies. As a result, the architecture (design) we finish with is something different to that which was planned.

(As an aside, we’ll talk about another time, that diagram is based on one from one of my favourite books, The Rise and Fall of Strategic Planning. There are close parallels to the way we relate to, and go about creating software architecture and business strategy.)

So architecture is design. Is it anything else? Anything more?

Architecture is more than design because the word architecture implies something bigger. An architecture is more than a design. You architect buildings but you design tents. Some of this “architecture as grand design” is just that, aggrandisement. It an attempt to make design into something grander. Personally I think design is a worthy and grand thing itself, you don’t need to aggrandise it any more.

In a software context there is the question of inside and outside. The term “software design” is often used to describe the external properties of the software, how it looks, feels, perhaps the features it offers. While
architecture often refers to the inside: how the thing hangs together and work. So there are “user interaction designers” but I’ve never heard of a “user interaction architect”.

I have long argued that software design exists in every line of code. Writing code is designing software. Certainly this fits with the “architecture is the inside” line of thinking but it doesn’t fit with so well with “architecture as bigger”. Somehow, while I happily argue that the choice between a FOR-loop and a WHILE-loop is an act of design, it seems wrong to argue that this is an act of architecture. But, according to own logic, that is what I am arguing.

The architect Ludwig Mies van der Rohe is reported to have said: “God is in the detail”. He wasn’t the first to use this quote but it does summarise his approach to architecture. He was an architect who was concerned about details. There was no lower bound for his architecture. So on that basis, Yes, whether you use a FOR-loop or a WHILE-loop, a bunch of IF-statements or a CASE-statement, you are making architectural decisions.

Architecture is design, it scopes up to the very big but it also includes the small details, because you just don’t know when the small details are going to become important.