Software delivery management is a technical discipline

Core to the delivery manager role is to oversee and manage the resources of a project up to completion. One of the most important resources is time and how much is needed to complete the project. This task leads traditional project managers to obsess over time estimates, Gantt charts including milestones, and collecting and reporting on the status of the project against those milestones.

When estimates are inevitably wrong and timelines slip, people will always behave in the same way: find a way to deflect blame from themselves while also trying to lay the blame at someone else. This culture of creating timelines and assigning blame is evidence of dysfunction in an organisation.

This dysfunction is not just that hostility is rewarded with the reputation boost (or at least the evasion of a sanction) that comes from deflecting blame, but also that the organisation is one unable to restructure itself to most effectively deliver new software. This is a solved problem.

We know as an industry that agile is the most effective way to deliver software. Lean processes in manufacturing are well documented. The most successful software development organisations publish how they operate, for anyone to copy. What could be better evidence of dysfunction than an organisation that continues to operate ineffectively, ignoring the industry’s known, published solutions to organise successful and effective teams.

If the above is not enough to convince you of the irrationality of assigning blame for missed estimates here are two more reasons:

An agile organisation will not make long term plans around estimates. It is impossible to accurately estimate the timeline for software delivery. This is not a skill that develops with knowledge or experience. It is simply not possible to estimate the time to deliver non-trivially large, complex software.

The primary challenges to providing estimates are uncertainty and complexity. The lean movement provides a great deal of wisdom from the world of manufacturing, with many ideas that translate to software delivery. Estimates are one area where these ideas break down.

Software is always developed for the first time. There are design patterns, libraries, frameworks, but ultimately every application or feature is developed once. Manufacturing is the process of repeatedly creating the same component or product multiple times. Software will always feature some complexity or challenge that could never have been discovered with any a priori knowledge or reasoning about the problem.

Add that software is just more complex than manufacturing. Of course a direct comparison is impossible but we have a hand-wavy attempt at one anyway. Consider and compare each component in a piece of software: a text string, an integer value, a data structure; with the number of individual pieces in a manufactured good: a rivet, a sheet-metal panel, an assembled component. Software works out to be orders of magnitude more complex.

This combination of complexity and the uncertainty of developing new code makes for an impossible challenge if trying to estimate the time to complete a project. This marks a sharp departure from conventional manufacturing where time estimates, Gantt charts, milestones and long-term plans make sense.

How then can an organisation plan, allocate budget, accept the risk of a large project, without estimates? Given it is not possible to estimate the time to deliver software, vociferously demanding estimates brings them no closer to being.

Instead, a solution presented by Dave Farley in this video is to follow the agile principle of releasing often, with small pieces of functionality. Features are prioritised in favour of technical risk and the value they bring to stakeholders, represented by the product owner.

This process decreases the risk to management and investors because the riskiest parts of the project are completed first. It minimises time and resources invested before we discover whether the most difficult technical challenges will be insurmountable. Stakeholders will also see steady progress as features are rolled out over time. They receive a steady stream of value returned for their ongoing investment.

The decision then becomes one of continuing to fund an ongoing project, rather than an all-or-nothing proposition. If the project is stopped before completion the investor is not left empty-handed. Value was delivered up until the project was stopped. Without regular small releases the only decision is an all-or-nothing proposition to either stop or keep throwing good money after bad with only the possibility of a final release that sooner or later returns value. Instead, stakeholders can stop a project once they determine the reward will not meet the continued cost of investing.

W.E. Deming has a famous 94–6 rule that 94% of problems in a business are the result of processes (a product of management) and only 6% the result of people.

If we accept this then assigning blame when things go wrong can at best be considered irrational. In practice it becomes a game of political manoeuvring in which those most adept at political machination within an organisation can have blame assigned to those less politically adept.

The gemba is a Japanese word to describe the place of interest. In lean manufacturing it refers to the production line. A similar idea, genchi genbutsu, describes the practice of solving problems by going to the place where work happens and seeing for oneself. It is the idea that managers get their hands dirty and be directly involved in the problem solving process. What does this mean for software development?

Solving software problems like bugs or performance issues can be even more difficult than writing the software for the first time. New code requires an understanding of technology and frameworks. To debug that code requires these skills in addition to the investigative skills to read logs, profile code, and the tenacity to question and test the assumptions used when first developing software.

There really is no meaningful help that someone can provide unless they have the skills and experience of solving software problems themselves. Arriving at the gemba is only the first step. It is necessary to also be able to contribute and to investigate. Toyota’s group managers (2nd level managers) are expected to perform tasks just like the line workers.

The above does not say anything about developing software in the first step. There are important decisions about which frameworks and design patterns to use, software engineering, and other best practices. These are all technical areas in which technical skills, experience, and a willingness to continually grow to learn about new technologies are needed if a leader is to be of any help to the team.

Non-technical managers can at best contribute on ceremonial tasks and not much else.

Absent the technical skills to meaningfully contribute, delivery managers may play the role of facilitator. This is the person that will organise meetings between disparate groups to solve problems, use their authority to get things moving, file front-door requests, and other administrative tasks.

This role should not exist. It is a band-aid over the real problem of a rigid, bureaucratic organisation. An organisation should make this role unnecessary by removing the roadblocks the facilitator works to get through.

The words, “front-door request,” should not be spoken in an organisation. These are bureaucratic roadblocks to slow down the rate at which a team can be tasked with work. It is anathema to an ambitious organisation that would seek to make meaningful achievements, get work done, and deliver for their customer.

A manager’s authority can sometimes open doors closed to lower staff. This is another sign of dysfunction in an organisation. Activities should be organised and executed on their merits. The threat of blame, reputation damage, or worse yielded by a senior manager and tied to their authority should not be the currency by which work is prioritised. Again, this is the mark of an organisation driven by internal politics instead of delivering for their consumer.

The most important people on a project are those creating value for the customer. Leadership and management roles are necessary but are not directly creating value. Their use should be judicious and the roles need to be filled by those skilled enough to provide meaningful leadership.

A role needs to always be aligned with how an organisation delivers value to the customer. A misaligned role should be removed. The use of non-technical staff in the leadership and management of software delivery does not serve the customer.

What will be the role of the non-technical manager in a reformed organisation?

Toyota’s famous practice, kaizen, seeks to make operations as efficient as possible, including where possible to remove people from a process. Those removed are never dismissed from the organisation. New roles are found elsewhere. Staff are valuable and the best effort needs to be made to retain them.

There are roles for non-technical people in software development: product owners, business analysts, technical writers and otherwise. The difficulty is when these roles are mixed with the technical practice of delivering software, especially in leading this delivery.