Has anyone met a System Analysts lately?

I ask because I’ve been looking for over a year and can’t find one. I should say immediately I’m not looking to hire one, rather I want to find out what they do. I’ve looked and I’ve asked and the role seems to have disappeared from organizations. Or rather, perhaps I should say the title has disappeared; the role still exists in some places, for better or worse.

My belief is that the System Analyst role has been divided up between the Business Analyst and Architect roles. Like a System Analyst, a Business Analyst will look at what is required from a computer system but they (should) look predominantly from a business perspective, not a technology perspective. Like a System Analyst, an Architect will look at the current systems and how it they work and fit together and how they should work and what new work is needed.

(Note: Of course the Architect’s role is itself a complex issue and needs subdividing, that will need to wait for another blog entry.)

I’ve run this idea past a number of people in the last year and I’ve yet to have anyone seriously disagree. In order to validate it further I went to my old (1992) copy of Roger Pressman’s Software Engineering [[link]] and found the following definition of System Analysis:

“… system analysis defines the roles of each element in a computer-based system, ultimately allocating the role that software will play”

Pressman doesn’t define the System Analyst’s role still, it seems logical that a System Analyst is someone who does system analysis. Later in the book he gives the list of the objectives of system analysis:

1. Identify customer need
2. Evaluate the system concept for feasibility
3. Perform economic and technical analysis
4. Allocate functions to hardware, software, people, database, and other system elements
5. Establish cost and schedule constraints
6. Create a system definition that forms the foundation for all subsequent engineering work

If you look at that list 1, 3 (economic) and 5 fit squarely in the BA function while 2, 3 (technical) and 6 fit in the Architect role. The BA is concerned with the people element of item 4 (and the implied process change) while the Architect is concerned with hardware, software, db, etc. Of course the two functions need to work together here because there are trade-offs.

So that’s it, System Analysts no longer exist, an open and shut case. Except…

I’ve also observed what BAs do and I think some BAs are filling a System Analyst function while holding a Business Analyst title. This is wrong. A Business Analyst should be concerned with the business need and what will deliver value to the business.

In other words: BA should be concerned with the “what”, not the “how”.

When a BA steps into the “how” they are engaging in design and enter the world of Engineers and Architects. There are, at least, three good reasons why they must not do this:

1. By specifying the “what” they restrict the options available for the development team which means they prematurely close off discussion of how needs can be best and most effectively satisfied.
2. Engineers and Architects tasked with designing solutions have the training, experience and skills necessary. One wonders if the BA who suggests “how” has equal competences, and if they do, why are they working as a BA rather than an engineer?
3. Specifying the how undermines the engineering team and encourages them to take a box-ticking approach. This in turn affects quality.

In undertaking design (synthesis) they are removing their attention from analysing the what – making more work for themselves. Synthesis and analysis are very different activities and are deliberately separated.

(Of course, on a small team the same person may find themselves engaged in analysis one week and the following week tasked with creating the solution. In such cases it is still worth considering the two as separate activities and getting some mental separation.)

I’m commenting here on what I’ve observed, its not that I have a dim view of System Analysts – if there are any out there I’d love to hear your comments. In fact, there are two reason why I’d like to see the return of the System Analyst role.

Firstly, as Tom Gilb would be quick to point out: there is an important role for System Engineering in the IT development. IT work which is too tightly focused on a piece of software, or a point solution, risks ignoring the role IT plays in the wider system. There is a need for more System Engineering in our IT development efforts and part of that is System Analysis (although maybe a different type of system analysis.)

Which raises the question: who in a typical IT development team is best positioned to take on the wider system, interdisciplinary and engineering questions which arise in large development?

That sounds like an Architect to me.

Secondly, in so much as a System Analyst is one type of Architect I think it’s a better title because it is less prone to ego-bloat. Unfortunately, some people let the title “Architect” go to their head. Once they get to be “An Architect” they see no reason to code any more or get their hands dirty with detail. They prefer to sit alone, draw UML diagrams and issue edicts. I’ve heard them called “space cadet” or “astronaut” architects. They become divorced from that is really happening.

So purely on the grounds that I suspect the “System Analyst” title less prone to pomposity than “Architect” I think it should be used.

There is a re-occuring question I am asked from time to time, and I see on discussion forums. It again appeared last week:

“We are new to Agile and are looking to adopt a tool for managing the process… which tool would you recommend?”

There are variations on this question (replace Agile with Scrum/XP/Kanban/whatever, or replace “managing the process” with requirements/development/testing) or maybe its put as “Does TFS work with Scrum?” or “Is Target Process the right tool for XP?” but its the same basic question.

Anyway, I try not to take the bait on these but last week I couldn’t resist and answered it. My answer solicited a positive response from several people so I thought I’d share my answer with more of you and expand on it a little:

For anyone new to Agile (all flavours, Scrum, XP, etc.) I strongly advise avoiding all software tools for managing the process until you get the hang of working in the new manner manually. You will have a faster and more fulfilling learning experience if you do it by hand – pens, cards and white board – for a few weeks before you look at a tool.

There are several dangers with tools:

• You may mistake the tool for the process: e.g. using Rally doesn’t make you Agile
• The tool might not work quite the way you need to work: in your company the tool might not do something you need to do
• The tool might be too strict: when your learning you need a bit of slack
• People may communicate through the tool rather than face-to-face
• You may spend time customizing the tool when you should be focusing on the actual work
• And, when you find things don’t work quite the way you want them to it is much easier to change a card or a board than change the software.

Something magical happens when people work with cards. Abstract “work” becomes real. We relate to it in different ways when it has a physical presence and you can move it about. You don’t get that with a tool.

Once you have a tool the tool easily becomes an impediment to further change:

• To change you need to change the tool
• You need time to reconfigure the tool (if you can)
• You need time to move your data to another tool (if you can)
• You need to justify the change of tool to those who paid for it: “We paid $10,000 for these licenses 6 months ago, what do you mean you aren’t using it any more?”
• You need to justify spending money on another tool: “And you want me to spend another $5,000 on another tool? Will you use it this time?”
• And you have invested yourself in the tool, you don’t want to admit you got it “wrong”.

Nobody ever got fired for spending too much on index cards, or for throwing a pack away.

Yes I know there are genuine reasons people want to use a tool to help with their work. But choosing the tool before you change is putting the cart before the horse. In my opinion people and teams which spend time evaluating and deciding on their tool are resisting change.

Damn the torpedos just do it!

Do it by hand to start with, give it about 3 months like then then look at some free tools, don’t waste time on customizing anything. Give it another 3 months then you should have the experience to find the best tool for your team.

Don’t be scared to change tools – and don’t invest so much in one tool that you can’t change in a few months when you realise its not right.

I’m not saying “Don’t use tools”. To be honest my preference is not to use them, stick to paper, pens and boards. However, I recognise that on some occasions they can help (remote working springs to mind.) But, before you start using one be clear what you need to get from it.

The thing is: Agile is about learning, and you have a much better learning experience when you are manually managing the tasks. If you need a tool to manage them you probably have an over complicated process. When you insert a tool between you and the process you loose sight of what is happening. The tool is a box labelled “magic happens here” and complexity can set in.

There is a story. Its about Jack Kilby – not heard of him? You should have done, he was co-inventor of the Silicon chip. A few years after inventing the chip he helped Texas Instruments develop the calculator. But he regretted the passing of the slide rule.

He said: the slide rule gave engineers the numbers but they needed their intuition to know were the decimal point went. Was the answer 123,456,78 or 123.45678? The calculator took that away and he thought that made for poorer engineering.

Using an tool to manage your Agile process is like that. It removes part of your intuition, part of your understanding.

I stopped reviewing books in this blog a while ago but, having said a few words about Domain Driven Design Using Naked Objects last time I want to say a few words about another book I’ve just finished reading: Wired for Innovation: How Information Technology Is Reshaping the Economy.

For anyone who is concerned with the future of IT, technology in general, and improving the state of the art this book is well worth reading. At a little over 100 pages and £10 it is worth the investment.

The book is written by two academics, and it is a bit academic in tone and approach but there are some gems in here. Not only have the authors studied how IT is used they have made serious attempt to measure it. In fact, if you have read anything by Brynjolfsson before there is a good chance it was his work on the “productivity paradox.”

The authors know about measuring IT and have some fascinating statistics. For example, did you know that in the last couple of decades “IT intensive” companies have grown much much faster than those which are not IT intensive?

I don’t want to review the book but I do want to highlight some of the points the authors make for the benefit of those who won’t read the book. In particular, there are some facts here which need to be considered by those who advocate a focus on “Business Value” when developing software (of which I include myself).

Firstly the book resolves the “productivity gap”. This was the observation in the late 1980’s that despite all the investment in productivity enhancing IT there was no resulting increase in productivity in the production and GDP statistics.

The explanation turns out to be: time. There is a time lag between investment in information technology and when the benefits are seen. This has closed somewhat in recent years, instead of decades it is down to years but it still exists.

So: problem number 1 for the business value crowd, How do you measure business value when the benefit will not be seen for two or three years? Or longer still? How do you create a feedback loop when your developers are working today and you can’t measure the value for two years?

The second issue we need to pay attention is “the complementary nature of IT.” It turns out, when you look at the numbers that IT by itself doesn’t delivery a whole load of benefit. To see real benefit you have to combine it with other practices. There are seven:

• Move from analogue to digital processes
• Open information access
• Empower the employees
• Use performance based incentives
• Invest in corporate culture
• Recruit the right people
• Invest in human capital

If you want to increase “business value” then do all these things as well as your IT work. Trouble is, many of these things fall outside the bounds of your typical IT project. (And in part doing these is why there is a time lag.) If you don’t do these things then you will fall a long way short of maximising business value.

Put these together and you discover: a successful IT project requires organisations to change their processes. Which leads to another issue the book brings out: should business processes be counted as assets for a company?

After all, we count the hardware and software used for the process, why not the actual process? We invest in it, we spend money on it, why not count it?

This may sound irrelevant but it is important because: failure to count processes as assets means companies don’t invest in them. It processes were counted it would become easier to justify the seven things listed above. If processes were assets then money spent on them would come off one account and appear on another so it is easier to justify.

Where all this is leading is a stark reminder, next issue, that in many businesses, and among many managers, IT is considered intangible. The authors don’t challenge this assumption, to be honest, there is good reason why IT is seen as intangible. But it does mean that someone who talks about “Business Value” looks a bit out of place when “everyone knows” its intangible.

(Why IT is seen as intangible is probably another blog entry.)

Next, a lot of IT has delivered “value” it isn’t countered. Take for example a Google search. How much is a search worth to you? How much time and effort would you need if you had no Google?

But, while Google’s revenue is countered by the Government those searches aren’t. In fact, nobody, anywhere, attempts to put a value on Google searches.

Or take Wikipedia. How much is it worth? If I want to know how the steam engine works I can go out and buy a book. That will be counted. But if I read it on wikipedia it isn’t.

These are just examples of how IT systems change things and add value which isn’t countered. It is the kind of challenges we are up against when we try and measure “business value” in IT work.

And its why, for anyone who claims to know about measuring business value in IT, this book is worth reading.

I am very excited about the next ACCU London meeting when Dan Haywood will be speaking about Naked Objects. The meeting is open to all and is free, just check the ACCU website for details.

I’m excited about this talk because Naked Objects is one of the few technology based approaches to software development which I think holds real promise for better development. This is because the Naked Objects philosophy is based on a very different approach to most software development. Rather than view the end user as some kind of inconvenience Naked Objects treats them as a problem solver who wishes to solve something.

I read the original Naked Objects book by Richard Pawson and Robert Matthews a few years ago and the idea has been with me ever since (I reviewed the book in an earlier blog entry). Its the difference between treating users as adults and not children to my mind. Rather than the designers defining processes in the software they give users the tools to resolve the issues their customers raise. This approach respects the person using the software. It moves the conversation from business processes to customer service; and it fits far better with piecemeal growth over big-bang change.

Dan has just published a new book on Naked Object, Domain Driven Design Using Naked Objects and I’ve been lucky enough to have a sneak preview. This is the first technical book (i.e. one with lots of code) I’ve read for a while and I enjoyed it. So here is potted review… Dan sets out to explain the Naked Objects framework. Yes, Naked Objects is not just a philosophy it is the framework too. No need to invent the wheel, here it is; and it seems that framework has matured quite nicely.

He is passionate about his subject, I’m almost itching to download this stuff and start coding myself – arhh, if I could only prioritise coding for fun above the other things going on in my life. The writing is very clear and to the point with a little bit of humour. His instructions for getting the environment set up for the example code and case study is clear.

There are plenty of code samples and screen shots to illustrate what is going on which is nice but… On the downside the book is nearly 400 pages long which means it yo are unlikely to carry it around much. I read a PDF version so I didn’t suffer too much.

I sometimes seem to be the only person I know who has not read Eric Evan’s Domain Driven Design book. Initially I thought I might be at a disadvantage reading Dans book but he explained just enough to get me through. Indeed, I’m now intrigued and plan to get a copy of the DDD book.

Sometime ago I asked Steve Freeman “Whats all the fuss over domain driven design? What is it?” to which Steve said: “Its about doing objects correctly, the way we should have been doing them all along.”

Everything I’ve learned about domain driven design since then fits with Steve’s description. It seems to me that domain driven design and naked objects were made for one another so I’m glad to see Dan’s book.

A recent McKinsey quarterly has a piece on organizational Agility, “Competing through organizational agility” by Donald Sull. It s a reminder that “agile” means something outside of the software world.

That said, the piece is a little disappointing. The piece doesn’t say anything particularly new. The only insight I gained from it was separation of agility into: Strategic agility, Portfolio agility and Operational agility. It would have been nice to know if you can have one without the others.

The main article also contains a sidebox which suggests some companies are centralizing decision making to improve agility. Personally I find that idea a little crazy. As far as I am concerned centralized decision making runs counter to agility for two reasons.

First it moves power away from those closest to the action. Those who deal with the customers, market and problem are the best place to make decisions. Moving decision making to the centre is disempowering and adds several layers of message passing.

Second centralized decision making is too often slow, bureaucratic or politicized. It takes time for the centre to realise a decision needs making, time to gather the information, time to make the decision and time to send out the decision.

Interestingly some of the examples given in the piece (e.g. Zara) are the same example cited by the Lean guys. More evidence that Agile is Lean by a different name.

An interesting prediction from Kevin Rutherford for 2010: “I predict that 2010 will be the year that eXtreme Programming returns to centre stage.”

I tend to agree with Kevin, while his logic is sound I would offer an alternative rationale for the return of XP. It goes like this…

XP can be viewed from two perspectives: project or engineering practices.

The first of these is basically Scrum. The cognoscenti might debate which came first and which borrowed from which but basically the approach is the same: short time-boxed iterations. Yes Scrum adds in Scrum Masters, Product Owners, self-organizing teams and such but while XP’s description is more basic they have broadly the same approach.

As to engineering practices, there is a growing Software Craftsmanship movement which is aiming to put these back on the agenda. While Scrum has been centre stage in the Agile world many of the engineering practices have been absent. Uncle Bob Martin has been talking about this for a couple of years and I’ve observed it myself (see my post about the Scrum Wall.)

Scrum gained a lot of popularity during 2008 and 2009 but too many of those teams adopted the project management bit without the engineering practices. While that approach can bring benefits it also stores up problems for the future. As more teams take this approach more teams will see these problems and more teams will need to add back the engineering practices and take craftsmanship seriously.

Add engineering practices to a Scrum team and it isn’t very different from XP.

The question Kevin leaves unanswered is: XP 1 or XP 2? For me its Extreme Programming, first edition, “the old testament.”

Now, to extend Kevin’s prediction. I hope we will see the return of “requirements” in 2010, or rather, more focus on “business value” delivery.

The “what are we building?” question has been underplayed in Agile too date. This means the Product Owner, Customer, Business Analysts and/or Product Manager roles (call it what you will) needs to have more attention. Fixing development is the easy bit, the difficult bit is doing the right thing. But, you can only really address that question once you can deliver.

Living and working in London means a lot of the software people I meet and talk to work in financial services. It amazes me the number of programmers etc. that banks employ. And you hear stories, some really awful stories about the state of IT in banks. And I’ve seen a few in my time.

So, for all those readers out there who have to work with banks and other financial services companies I recommend you get hold of a copy of this weeks Economist and read “Silo but deadly.” This article lends some weight to those stories I hear in pubs, at conferences and during training courses.

Some highlights:

• Financial services companies are estimated to spend $503bn on IT this year, thats more than Governments, more than manufacturing and more than any other sector
• The chief risk officer at Deutsche Bank says: “Banks are essentially technology firms”
• SOA architecture is a problem, that’s Spreadsheet Oriented Architecture
• Data quality is a recognised problem

The articles suggests that the woeful state of IT in banks may have contributed to the recent problems. This is something I’ve suggested other links before now, e.g. the Moody’s bug.

Personally, I think one of the reasons why banks and other financial companies have such poor IT goes back to that quote above: Banks are technology firms but they don’t view themselves that way. As a result the way they don’t necessarily take the right approach to people or projects, and they don’t take the right short-term v. long-term view and so on.

Lest we get too negative about our industry you might also want to read this recent survey from McKinsey which suggests IT isn’t doing such a bad job, and outside the IT department corporations are pretty happy with IT on the whole.

Twice this year I have visited companies which want to try a more Agile way of working but who have requirements coming of their ears. I’m talking hundreds of pages of requirements. Sometimes as use cases and scenarios. Sometimes as big meat documents written by expensive consultants and bearing names like “Requirements Specification” or “Business Design.”

Its hard to argue with such requirements documents, in part because its hard to read them and understand what they are about. The thing they have in common is a level of detail. They go into great detail on what needs doing and how you might go about doing it.

Its not the first time I’ve seen such documents. I’ve been on similar projects before. One variation is a project I saw last year were the documents were not requirements documents but strategy documents. Several big name consultancies had been into this (media) company and do strategic work around a new product. Lots of presentations, lots of SWOT analysis and so on.

What all these documents have in common, apart from being expensive, is that they are worse than useless. They give the impression that things are know but they aren’t know. They knowledge walked out the door, what you have is a snapshot of someone’s understanding. But its a snapshot that nobody can really take in because its so big. And it is a snapshot that is rapidly aging.

As I’ve said before: the bigger a document the less likely it is to be read, and if it is read, then the bigger it is the less that will be remembered of it.

To which we might add: the bigger the document is the more there is to go out of date.

Yet because these documents exist people feel compelled to use them. They may even be mandated to use them. They don’t feel they can say to their boss: this was a useful learning exercise, no we will do it for real. So we then embark on a process that try to use documents which only get in the way.

Having someone investigate requirements or strategy can be useful. But think of it as a learning and feasibility exercise. And above all: don’t let that person leave the project.

At best these documents are historical reference source; as worst they are an obstruction.

Set a clear goal for the work and work out what you need to do to meet it. As Tom Gilb says, this should fit on one page of A4.

As you work towards this goal you will uncover requirements. You might want to note them down but remember there is no point in running far ahead of the development team.

Once you have something up and running – even in demo – what people think they want will change. So start small, get something working and get a team that delivers.

Just say No to big requirements documents. Say Yes to short, concise, goal to aim at.

Continuing about patterns… At the beginning of the month I went out to Bulgaria to teach pattern writing to some academics and students at the University of Sofia. It was the first time I’d been to Bulgaria so I was interested to see the country which physically, historically and culturally sited between Europe and Russia.

We made the class very practical, no PowerPoint, in fact apart from a 5 minute video it was computer free. Yes we talked, yes we used flip charts but most of all it was practice, this was about writing after all. Only about an hour in and we had our participants putting pen to paper to write some patterns.

For this we used a video about milk packaging from TetraPak, there is an English version on You Tube, as it happens we used the short German version but it was the pictures not the words which were important. After watching this we did some brainstorming to identify what was happening in the video, i.e. what solutions were being used to solve which problems.

Initially we asked the participants to think about three things: the state before the solution, what the solution was and how things were different after the solution. Once the groups had grasped this we expended these into the traditional parts of a pattern: context, problem forces; solution and implementation; consequences and all the other bits (known uses, etc.)

During the writing myself and my co-presenters (EuroPLoP regulars Klaus Marquardt and Didi Schueltz) circulated between the groups to answer questions, provide advice and deliver a bit of shepherding.

And then we repeated the exercise with different subject material. We did some brainstorming to identify ideas which the participants would like to write patterns about, problems in their own domain. We repeated the writing process and finished the day with a writers workshop.

So in the space of one day we covered all the pattern basics. Although not in as much depth as maybe we could have. We carried on into a second day but only used half of the day. Next time I’ll be sure to use two whole days.

On the second day we covered the ideas behind shepherding, talked some more about pattern form, answered some questions and ran a second writers workshop, this time on an existing pattern by an experienced writer.

More than ever I left feeling that patterns are a generally applicable knowledge management technique. Not everything belongs as a pattern but the form is useful in many domains. And pattern thinking (or perhaps pattern analysis is a better term) is useful even when the thing you are writing (or just analyzing) is not a pattern.

And what did I learn? Well…

• I really enjoyed teaching pattern writing and hope to do it again
• Using a video to provide source material worked well
• Not using slides worked well, it requires confidence but the result was better
• You still need to give the attendees something to take away so having support material is necessary. Still, I’m always disappointed with slides which are little more than bullet points after the class. We did hand out some of my writing about patterns and patterns form, some patterns about patterns and some links to websites with more information and patterns.
• Having co-presenters was good. This was an intensive class so it needed more than one person
• You really need two days to do this properly
• Having attendees experience what you do was far more valuable than just lecturing them

Next time I would like to include a pattern reading exercise too. Take a well written pattern and read it through as a group. I’m going to be delivering a software patterns course this week and I plan to try this exercise in miniature.