Consulting Engineers

For the last couple of years I've lived at the bottom of Syndenham Hill (on the Kent side). At the top is the place to where the Crystal Palace was moved after the Great Exhibition. The Palace and the park built around it had many water features and for this and other reasons water towers were erected to give sufficient head of pressure, something otherwise hard to achieve at the top of the biggest hill for miles around. Big park, lots of features, huge towers. 

The engineer who built these towers proved to be mentally unsound and the towers structurally unsound, so Brunel was brought in to rebuild them (which might happen again). The story goes that he was at first most reluctant to pass judgement on the work of another engineer, however bonkers. Gentlemen didn't do that to one another. 

This was in 1855, when engineering as a profession was young and about twenty years before the Tay Bridge disaster firmly fixed in people's heads the idea that one engineer won't do. Much as even accountancy firms themselves have to get an accountant in to audit their books, on a big enough job engineering firms get one another in to check they've got their sums right. This the work of Consulting Engineers. 

In the IT world we bandy the word "consultant" around with a certain amount of levity. In the US a consultant doesn't really seem much different from what in the UK we call a contractor: in everything but name a temporary employee. In the UK there is a slight difference, which a contractor once expressed to me (a consultant) in this way: contractors don't have to write reports.

More generally contractors build the system they way they are told to, consultants have an opinion about how the system should be built. This distinction is not lost on the tax authorities.

Anyway, what with all this talk of "craftsmanship" there's been about the place recently thoughts naturally turns towards the more mature disciplines. A significant aspect of those, in many cases, is that it's awfully hard to get certified to practice. 

That's in part so that individuals can bear some liability if the job goes tits-up because the certifying bodies certainly will. This is as imperfect as any human institution, but at least they try. Another aspect of this is genuine consultancy. If you, you personally, are going to bear liability for it all going pear-shaped then maybe you sould spread some of the blame load by getting someone else in. 

Notice that in the medical field, if you aren't sure of a diagnosis then you can ask for a second opinion. If your doctor isn't sure, they'll go and get one by themselves.

This seems not to happen much in the IT field. Perhaps another symptom of our immaturity? When was the last time you worked on a project where the prime contractor brought in a competitor to check that they'd "got their sums right"?

Have we had our Tay Bridge yet? Maybe. In some domains, avionics, for instance, you do hear of multiple independent implementations. That's not quite what I mean, though. 

Why don't IT companies running projects (of a certain size or complexity) routinely get the competition in to express an opinion? Why don't clients demand this as part of their risk mitigation strategy? What is it going to take for folks to bring a genuinely professional standard of conduct to IT?

Update: it's two days later and no-one has thrown down the gauntlet. I was expecting some bright individual to come back at me with a "you work for a consultancy, so you tell us why this doesn't happen" But no-one seems to be biting....


Paul Keeble said...

OK I'll bite.

Personally I think the reason for this is that the process of writing software is actually the process of understanding the domain rather than the actual cutting of the code.

Not all projects but most of them have as their major issue the understanding/definition of the scope of the project. How do you bring in a third party who at that point knows even less about the problem than you to audit what you have done?

What you can do is review the code, the overall architecture and even the process, but its much harder to review the core of the system without substantial investment for the life of the project.

There are a tonne of other reasons but this feels like the gotcha and just another example of why cutting code is nothing like building a physical structure. There are no sums to get right.

Anonymous said...

I routinely offer to bring in "non-aligned" consultants to vett my proposals to clients. I am delighted when my clients accept this offer. Most, however, are unwilling to pay for this quality check. What most clients want, in my experience, is a final, authoritative, definitive answer to their "problem," regardless of whether their problem (as they themselves express it) actually exists.

This is the space that IBM, Oracle and Microsoft have long exploited. I know quite a few "ethical" consultants who have no compunction against having others knowledgable in their particular domains review their proposals and progress. I know a LOT of clients unwilling to pay for that that review.

Anonymous said...

Because the reviewing consultant would most likely be focused more on positioning themselves for new work than rendering an objective opinion. Sad, but reality.

Anonymous said...

I would suggest variants of 3 things: 1. the "methodology" used for systems development, many of the methodologies do not have room for second opinions. Iterative processes seems to allow more opportunity for this type of action. 2. The lack of internal skilled resources most organizations have to apply to the 'project' which leads to 3. C'mon computer experts know-it-all and get it right...right?! Oh ya, Timelines... its gotta be done quick.

JS said...

The medical and architecture professions bring in consultants because there is a legal liability they assume when providing a service. For IT, there is no liability attached to poor workmanship in this field. Since the customers of IT products seem fine with this arrangement - cheap products with extremely poor quality, there won't be any change to this status quo.

Nat Pryce said...

One problem I can forsee is that all the big consultancies have a terrible track record of delivering (UK public-sector) IT projects. So would bringing in one of the big IT consultancies to get a second opinion on the work of another big IT consultancy produce any useful results?

Rob Brown said...

In my experience, some big companies actually DO get second opinions, usually by creating project structures with competing entities working within them. Unfortunately this creates competitive forces within big high risk projects, and ultimately leading towards Death March circumstances as people struggle more and more to deliver. As a consequence these become wealthy environments in which to assign blame to any or all of the consultants who remain magically unaligned, and thus the permanent management structures of the client protect their "jobs".

I've only seen 1 senior manager in 15 years actually "take a fall" when the project he was owner of went well over time and budget (double) due to his bad structuring by hiring 3 different entities to do 1 bleeding edge project. Not sure what others' real experiences in this area might be.