The failure is upstream
In the post-mortems we've run, the model was rarely the problem. The project failed months earlier — when nobody defined what “working” meant in business terms, when the data turned out to live in six systems with three owners, or when the pilot was scoped to impress rather than to learn. By the time accuracy numbers disappoint, the real damage is already done.
The five failure modes
One: no KPI, just a vibe (“we should be using AI”). Two: data readiness assumed, never audited. Three: a pilot designed as a demo, so it proves nothing about production. Four: no owner on the business side — the project belongs to IT and dies in IT. Five: vendor lock-in that turns every iteration into a change order.
What the 5% do
Successful teams invert the order: they pick a number they want to move, audit whether the data can support it, and only then choose a technique — often embarrassingly simple. They ship something narrow to real users within a quarter, and they keep the spec portable so they can change builders without starting over. None of this is glamorous. All of it is repeatable.