Applying the techniques developed for manufacturing to IT projects is fundamentally flawed and a new approach is needed to manage them, says Sankarson Banerjee

On my birthday this year, I decided to be good and eat healthy. No rice, potatoes, meats, or desserts, and no deep fries for a month. This was my declared intention. A couple of euphoric days later, I realised that my supposedly simple diet was not that simple after all. While aloo paratha and fried rice were excluded, gobi paratha or macaroni with cheese were welcomed. Tiramisu was out, but pizza was in; so was Sushi, out, but butter-laden maska khari was in. None of these pointed towards my stated intent of eating healthy, and I responded to my doubts in proper MBA style. I drew up a bulleted list of what I needed to do, including clarifying definitions, adding exceptions, and refining rules. I invested the better part of two hours in codifying the intent of my simple diet into a process, indeed, the holy grail of all processes, which is, making it ‘idiot-proof’.

All of us have relatively simple statements of intent regarding what we want from technology, but translating the same on the field (especially in a nationwide environment) is far more challenging

What does success in a technology project mean? It is usually defined as being on-time, on-budget, and meeting end-user requirements. The kinds of technology projects are the plain vanilla software development where someone describes a problem, you write the code for it and voila, the problem is solved. Then there is the implementation kind where you buy something readymade and then get a bunch of people to tweak and twist things to fit your particular situation. Finally, there are the pesky technology services, but what to do when one of the myriad devices and systems that every employee uses daily, decides to misbehave, or leaves the building entirely. These are three broad classes of technology projects. This rest of this article broadly touches upon each of these three.

Let’s face it, we have a problem…

The success number - 16 per cent - is as amazing today as when the Standish Group released its influential report in 1995. A failure rate of a few percentage points should be serious cause for alarm but investing in a technology project seems to be riskier than Vegas. Many other studies on software projects, ERP implementations, and suchlike have presented other numbers but the studies all agree on one basic conclusion – failure is more likely than success in any software project, and the larger the project, the more likely the failure. Personal experience bears this out; I cannot remember a single unqualified success in my career.

Our technology project management methods are modelled on processes of the mass manufacturing era that came before it. Factories and assembly lines dictate our thinking, and the rules for software are the same - start with detailed specifications, measure progress, and deviations at every step and you will end up with a high-quality product repeatedly, and consistently. The focus is on the two words – repeatedly and consistently. We have to achieve the exact same output all the time, every time. This thinking was applied to everything from pins to aeroplanes, and triggered an improvement in manufacturing like never before seen. It was applied to software ‘engineering’ in software ‘factories’, the intent was to make a good process that starts with the specs, and ends with a defect-free software product or technology service, and the output will be consistent and repeatable. This approach is the thinking behind the major quality standards such as, ISO and SEI-CMM, and is widely called the Plan-Centric method.

Our technology project management methods are modelled on processes of the mass manufacturing era that came before it. Factories and assembly lines dictate our thinking, and the rules for software are the same

The fundamental flaws to this approach are not unknown. There are significant differences between moving bits and moving atoms. When moving atoms (manufacturing) you are building processes and machines that move the same atoms millions of times over, with the same precision. When building software to move bits, the tool that actually moves the bits (the computer) is already capable of moving the same bit with exact precision millions of times over; there is no doubt on that front. A technology project is not the moving of the bits themselves; it is the creation of instruction sets that tell the machine (the computer) which bits to move. The two key fundamentals of engineering processes – repeatability and consistency – do not apply directly to the technology process itself, but to its eventual output. This is because you do not repeat what you built (the same instructions) again and again - that will make a very dull computer indeed. Software or tech services are meant to be general purpose – a large range of inputs can be processed to give different outputs.

Build me a woman, make her 10 feet tall
One of the foundations of high quality manufacturing is exact specifications; from the curvature of the hood to the shape of the screws on the cup holder. Every part in a car is specified exactly, assembled together exactly and tested to very narrow tolerances; it usually takes years to get all the specifications right for every model. This initial investment in detailed specifications can then be allocated over the hundreds of thousands (or even millions) of copies of the car that is built. Cars that are not mass marketed – such as the Ferrari or the Lamborghini – go through a similarly detailed exercise but cost 10 times as much because the initial investment is now spread over fewer cars. Formula One cars are even more extreme, in which millions of dollars of design go into just two cars.

Software engineering cannot do this. It is indeed possible to arrive at perfectly detailed specifications – as precise as the sound of that door on your Beamer – but it will be just as expensive and time consuming. Unfortunately, instead of hundreds of thousands of BMWs, just one single software project will use the specifications; the next software project will do something else and have to gather its own specifications. This is patently infeasible; especially in a world where technology changes rapidly and specifications that take years to build will be obsolete.

This reliance on exact specifications is the big cause of failure in most projects, and the solution is not really to produce better specifications. We already know that it can be done but is usually neither cost nor time-efficient. Any software process that relies on detailed specifications is going to be too expensive, take too long to deliver, or be obsolete because the world has moved on. What is needed is a way to accommodate fuzzy specifications that does not throw a large hiccup every time it faces a change.

On the face of it, repeatability is not related to specifications. Repeatability is an attribute of the process; a good process should have enough checks and balances so that whatever the inputs, it will provide consistent output. To ensure repeatability, processes are made as predictable as possible – do the same things for the same period of time and achieve the same result. It is also directly related to estimation; repeatable processes are easy to estimate while processes that cannot be repeated are not. Accurate estimation means that one should be able to estimate with a fair degree of accuracy what inputs are required and how long it will take to achieve the outcome. By the same token, a process that is difficult to estimate accurately cannot be repeated with any confidence.

Manufacturing processes are commonly able to estimate every input to a high degree of accuracy. A modern automobile plant will tell you down to the last second and last dollar how much it takes to produce each car. They are estimates because in an individual car small things may go wrong costing more time and money. However, such variances in a mature process are very small and very well understood.

Process measurement is the tool of choice for traditional software processes, to ensure that there is enough data to make sure estimates are accurate. Deming’s most famous statement (a little paraphrased) is that you always measure to make sure you know what is happening. This led to a large number of metrics being used to control processes and ensure that the process is repeatable. This means that the next time round, the same measurements can be achieved to within a small margin of error – and they can be used as the basis for estimates. In manufacturing, this works wonderfully well because the same car is produced thousands of times. When the manufacturing line is new, the estimates are not nearly as accurate and overruns are common. Over time, however, they become precise.

Software is built in layers… Each layer of software has its own set of bugs and quirks…. To top it off, the programmers are human and will make different kinds of mistakes

The whole point of metrics and repeatability is, given the specifications, the ability to estimate how much time and resources are required to produce the output. You can take a feature you just added which took let us say ‘X’ hours of work, and say that this second feature is 10 per cent more complex so it should take (say) 10 per cent, or 20 per cent, or 32.775 per cent more than ‘X’. The process can then flex to accommodate it. Unpredictability, the traditional method says, is due to errors in estimation which can be reduced, or eliminated through iteration - a mature model has reliable estimates gathered from repeated experience of fixing the error. Underlying this belief is that there is some constant relationship between input change and output change and experience will tell us what the relationship is.

Exactly, my dear Watson
The output of every software project is a little different. You build a word processor once; you are not going to build the same word processor again. This time, you will build a slightly different one with more features, probably a little more integration with a spreadsheet, maybe even a team function. It looks like a small change in output – one feature added onto hundreds of existing features – but it may turn out that the small feature is enormously complicated and expensive to add.

Thus the fallacy that experience brings reliability to technology. In nearly every project I do, there are algorithms, technologies and products involved that I have never seen before. This is not an issue of wider experience or more brains; I would state somewhat daringly that every technology team in the world faces at least one such unseen challenge for every non-trivial project. The problem with repeatability is not an error of estimation; it is a systemic effect brought on by rapid technological change. This speed of change forces something new on every project – a new version of the language, new versions of tools, services, databases, operating systems, or hardware. A small variance in any of these can bring on many hours of additional work and there is no way to predict this beforehand. Software development, in this sense, represents a classical chaotic system.

It is not difficult to see why the system may be chaotic. Software is built in layers – application software built on servers, built on protocols, built on the operating system, and many other layers in between. Each layer is in turn software, not more than a few years’ old; and it is not perfect with its own set of bugs and quirks. To top it off, the programmers on the team are human and have different strengths, meaning that they will make different kinds of mistakes. Each layer is not, in itself, unstable but none of the layers are rock solid. This leads overall to a system that is stable most of the time, but has unexpected catastrophic vulnerabilities because some odd actions cause unexpectedly large ripple effects. Think of it as balancing bricks; if you could align every centre of gravity perfectly, you could get really high without any instability at all. In real life, however, they will never align perfectly and a very small wobble may be enough to bring down the tower. The larger the software project, the higher the tower, the more the wobble.

Potatoes and pigs
The key components of an IT project are the people. When making steel, there are lots of inputs to be managed, and labour is only a component of this; IT projects are all people. Factories treat units of labour like any other input, controllable and interchangeable with any other of a similar kind. Intel did once introduce the designation of ‘Commodity Specialist’ to interact with outsource technology vendors, but I find it hard to imagine that the average technology person fits either criterion – controllable or interchangeable. Unfortunately, people are inconsistent and unpredictable - knowledge professionals even more than assembly line workers.

Oscar Wilde described decadence as the subordination of the whole to the parts; I feel technology is becoming more of that – a collection of process steps more important than the intended outcome

Then there is the person-day; a theoretical unit of work that our hypothetical ‘commodity’ person completes in one day. This metric is used extensively to estimate how much time, how many people, and what cost in any technology project? But can such a unit exist at all? Neither the person nor the unit of work is definable; only the day manages to stand up. It is not uncommon to find orders of magnitude difference in productivity between two members of the same team. Across a million people in technology with different intelligence levels, social backgrounds, education, and work environments solving different problems, is there any possibility of a common unit of work? All our metrics have implicit or explicit dependency on this one measure. A lot of time gets spent in reporting around this – attendance, timesheets, performance reports, yet the link to intended outcomes remains elusive. Many companies that neither have timesheets nor attendance seem to have much better outcomes – known mavericks such as Facebook and Google, but also innumerable individual teams inside corporate IT do so. Other unintended consequences are creative people do not join the mainstream, employees game the system, and meeting the process becomes more important than the end result.

Some people have re-thought the process, and they have addressed some of these questions by inventing different approaches that target some of the deficiencies, which are collectively called agile approaches. Ironically, they grew out of pioneering work done at the ultimate of assembly lines - a car company – in the very year that the Standish Group was making its bold recommendations.

However, Agile has not been the promised saviour we thought it would be; even the original Chrysler C3 project had mixed success and was eventually discontinued. I was a big fan of Agile a decade ago, and it did indeed produce better results under certain conditions. But, I have since realised all kinds of limitations. Agile methods work well for a restricted subset of projects, but they have large challenges as a universal methodology. The intervening decades have thus continued the tyranny of the micromanaged plan - in part because people are still convinced that it is just a matter of ironing out the rough edges.

Last words
For all the failure of planned implementations, the technology world is littered with successes. For every Microsoft that releases lemons years late, Google or Facebook or a host of startups rapidly produce high-quality, largely bug-free, software, with tiny teams and few control systems. Most Open Source projects seem to thrive on chaos with neither control nor metrics looming over their heads. Oscar Wilde described decadence as the subordination of the whole to the parts; I feel technology is becoming more of that – a collection of process steps more important than the intended outcome.

Somehow, we need to get to this balance in technology projects. We need to assume intelligent actors (idiot-proof processes are for idiots) and allow for judgements on final intent rather than process steps. We need to tie this all up with the corporate need for budgeting, projections, and performance evaluations; creativity, but on demand, and under control. Any ideas?