Every analytics business case ends with a number. Millions in avoidable scrap. A percentage off direct labor. An idle-time reduction quoted to two decimal places. The number goes into the deck, the deck gets approved, and the work starts.

Almost nobody goes back and publishes what fraction of that number actually landed.

I have two I can talk about. On a predictive-failure program in continuous-process manufacturing, validated improvement reached about eighty percent of the targeted run rate by the second month. On a scheduling and process-efficiency program, about seventy percent by the third. Both were counted by finance, not by me.

Neither is a hundred percent. I would still rather be judged on those numbers than on the targets.

The modelling is the short part

The predictive-failure work started with roughly fourteen thousand sensor tags coming off a plant historian. The job was to reduce that to something that predicts a process failure early enough to act on. Gradient-boosted trees, time-series regression, anomaly detection, tested against a named opportunity.

That part took weeks. The realization took months. Almost everything that determined the outcome happened after the model was already good.

What actually decides the realization rate

Lead time, not accuracy. A failure predicted twenty minutes out is a curiosity. Predicted four hours out, an operator moves a setpoint and the failure does not happen. How much warning a model gives is the metric that decides whether it is worth anything on a plant floor, and it is the metric that almost never appears in the business case. Accuracy does, because accuracy is easy to put in a deck.

Somebody has to change what they do. A prediction that lands on a dashboard nobody owns changes nothing at all. This is change management, and it is the part that gets handed off with an assumption attached. The assumption is that operations will pick it up. Operations already has a job.

The baseline has to survive finance. If you cannot say what would have happened anyway, you cannot validate a saving, and finance will not book it. Roughly half of the realization work is measurement design rather than modelling, and it has to be designed before the intervention, not reconstructed after it. This is the most common reason a model that works produces a zero on the books.

Some of the opportunity was never there. Business cases get built from the top of the range, because that is the number that gets funded. So the realization rate has a numerator problem and a denominator problem, and only one of them is the model's fault.

The discipline that closes the gap

None of those four is a data-science problem. They are process problems, and there is an old, unglamorous method for process problems.

DMAIC puts measure before improve, and control after it. The measure phase is where the baseline gets designed, before anyone has an incentive to flatter it. The control phase is where the improvement is held: the setpoint change becomes standard work, the alert gets an owner, the variance gets watched.

Control is the phase everyone skips, because by then the model works and the interesting part is over. It is also the only phase that decides whether the number survives the next quarter.

The two programs got to seventy and eighty percent because the control phase was scoped as work, with a name against it, before the modelling started.

Why publish this

Because the alternative is a case study with a target in it, and a target is not a result.

I would rather show the realization rate. It is a worse number. It is a more useful one.

Back to writing