Outsourcing banana skins: Warning signs that your supplier isn’t as good as they claim

So you think you want to contract out some development work? – yes, you know this area is full of banana skins to slip on, and you know others have problems but you still want to do it.

And you want to contract it out to an “Agile” development shop?

There are no laws against opening a development shop – a digital agency, an outsourcer, a consultancy, call it what you like. That is the beauty of capitalism, it allows pluralism. The hard part is choosing one that is competent, some outsourcers are pretty awful.

Everyone who can spell “Agile” can claim to be Agile, and most hire a copywriter to give them an online spin so they all end up making the same claims.

Those who are half good can coach their staff in what to say. And in truth, most of them genuinely want to do the best possible job for you – and be agile too. But how can you really tell?

Well… when I’m being cynical I think you can’t. The only way to really tell is to give them some work and see what happens. Of course once you’ve engaged someone you need to be both legally and mentally prepared to walk away.

So to help you here are some warning signs that you have stepped on a banana skin and your supplier isn’t as good as you – and they – want to be. You might even say these are warning signs that they aren’t really Agile.

1) Customer involvement

They don’t want customer involvement. They don’t want your people on site. They claim that your people get in the way. They want to be left alone to do things.

Obviously I’m thinking primarily of actual Customers, Users, Product Owners, Business Analysts and so on. These people should be working closely with the suppliers people. They should have direct contact, they should be discussing stories.

If your supplier isn’t embracing your people and treating them as their own team members something is probably wrong.

The supplier should be challenging your people – after all the suppliers are the experts, if they are simply accepting everything you ask for then something is wrong.

The same is true of other people you might want involved: a consultant or Agile coach should be welcome too. And if you decide to ask a third party to inspect their development then they should be open to this too.

Naturally they should also be open about the code too. After all the code will be yours one day.

2) Regular demonstrations

The supplier should be providing regular demonstrations – “show and tell” – as a very minimum. Every couple of weeks I expect the supplier to show the latest work. You – and your people – should be able to see working software offering more than the previous demo.

If the supplier is NOT providing regular demonstrations then you should be worried. Likewise, if the demonstrations don’t show progress get worried. Most of all, if the supplier doesn’t want to talk about why demonstrations aren’t happening, or how they can show progress then something is wrong.

3) Release, release, release

Show and tell demonstrations are good but the real test is to release to live. Releasing all the way to your live system. You might hide it on an obscure URL that nobody knows, or call it a beta release or something, I don’t care, the closer they can get to real live the better.

You supplier should be releasing to a live environment – or an exact copy – very very regularly.

The longer the supplier goes without an actual release the more nervous you should be. Sure, once in a while things go wrong and nobody is perfect. They may go two weeks with nothing to show for it. I don’t like it – and neither should you – but an occasional gap is OK.

Going four weeks without a release… I suppose it might happen in the early stages of the work. But it is in the early stages that you most need reassurance that they can deliver something – anything!

Six weeks with no release… well here we are into “3 Strikes and you are out.” Sure they will be able to give technical reasons why things messed up three times in a row. But take it from me, something is wrong.

The longer they go without an actual release to more concerned you should be and the more you should offer to work with them to address the issue.

Eight weeks? – eject, eject, eject.

4) Show me the tests

Maybe this should be warning sign #1 but for this you need someone technical, someone who knows what a test looks like. In the show-and-tells your supplier should be able to show you automated tests executing. Not very exciting perhaps and certainly not meaningful to the business but if they can’t show you then how do you know they even exist?

And if your supplier doesn’t have an automated test suite then it is certainly time to get out.

Ultimately the system they are building is yours. Your people will need to maintain it, or you will need to pay someone else to maintain it. Without automated tests that is going to be hard. Skipping tests now might make it look like you are saving money but you are not, even in the short term the lack of tests will bite you hard, it will push up costs and destroy schedules.

5) “Feature complete”

The phrase alone should be a warning sign. Equally the words “75% feature complete” (or any other percentage) is a big red flag.

If the supplier doesn’t have a test suite, if they can’t show working (preferably releasable) code then its probable they are feature stuffing. They will say they are making progress because “60% of features are done”. They may even start to claim they are feature complete but remember: a feature without a test isn’t done.

A feature without a test is pure risk. At any time a defect can put a hand up and say “Fix me!”

An automated test isn’t a guarantee of bug free code but without automated tests then I guarantee you have defects waiting to appear.

If you are in a relationship exhibiting any of these five sign then it is certainly time to talk. It may be time to end. But how do you avoid getting into that position?

Let me be as clear as I can: both you and your supplier should prioritise working, usable, functionality over more functionality. As the old saying goes “A bird in the hand is worth two in the bush”, working, deliverable (even better released) features are the priority, there should be less work in progress, fewer incomplete features, fewer “almost done” and as few as possible defects.

While cynical me thinks you might not be able to avoid it that doesn’t mean you shouldn’t try, so here are four warning signs that you are about to step on a banana skin:

1) “Agile is not that different”

Don’t let them tell you Agile isn’t different. In many ways it isn’t but if a supplier is trying to persuade you that you don’t need to change the way you work then it is a sign that either a) they don’t appreciate the magnitude of the change or b) they will tell you anything to get the contract signed.

Since you want a supplier who will challenge you it might not be a good idea to hire one who doesn’t like challenging you or doesn’t prepare you for difficulties.

2) “We are certified”

An extension of warning sign #1 is that the supplier is proud of how certified they are: ISO-9001, PMI, PRINCE-2, CMMI – some in the Agile world would regard these certifications as warning signs in their own right.

Scrum Master Certified, Agile Project Manager Certified, Scrum Product Owner Certified: these are slightly better but anyone who can’t tell you the flaws in all these certifications has myopia.

Question any organization that offers up badges rather than working products.

(Disclosure: for better or worse I hold a couple of Kanban certifications, while I think Lean Kanban University have done a better job than many in making their certifications meaningful I don’t think they are a panacea either. Anyway, Kanban certifications aren’t as recognised as those I just mentioned.)

3) Fixed or long term contract

IT suppliers have a long history of locking clients into “fixed contracts” – fixed scope, cost and time. These contracts are utterly flawed. Anyone claiming they can fix everything is a charlatan. Give them a copy of “Dear Customer: The Truth about IT” (the Xanpan prologue) and say goodbye.

Similarly locking yourself into a long term contract with a supplier before you have done some work with them is a bad sign. Do a small piece of work, for a small fee, with your potential supplier and see if they are as good as they say.

In my experience the best – most “agile” – digital suppliers can pick and choose their customers. If your supplier needs you more than you need them then it is a bad sign.

Read more? Subscribe to my newsletter – free updates on blog post, insights, events and offers.