What keeps a document pipeline alive in production

An AI system that works in a demo and an AI system that runs every day look nothing alike. And it is not a question of model: the model is the part you replace most easily.

In a demanding industrial context, I designed and put into production a pipeline that processes hundreds of scanned documents a month. Paper timesheets, digitised, that used to end up in manual data entry with all the transcription errors that implies.

A document’s journey, and what it does not tell you

The journey has three stages.

The document lands in a network folder and goes into processing on its own: nobody clicks, nobody uploads anything, the trigger is the file arriving. It is then read by an extraction engine and cross-checked by a vision-capable model, with normalisation, a confidence score and a match against the employee directory. Finally it is written into the business database transactionally: all or nothing, with a full rollback on the slightest error.

That is the part that goes into a diagram, and the part that impresses in a meeting. It is not what decides whether the system will still be there in two years.

Choice 1: nothing reaches the database without human validation

The document and the extracted data are shown side by side, and someone validates. Never a fully automatic import, even when the confidence score is excellent.

This is not excessive caution, it is a design choice with two effects. The first is auditability: every record in the database has an identifiable owner, which is a hard requirement in any regulated or audited context. The second is adoption: the business side agrees to trust a system it still controls. A system nobody dares to contradict is a system nobody uses.

Validation costs time, obviously. Choice 2 is what makes that cost decrease.

Choice 2: every correction is remembered and fed back

When an operator corrects an extraction, the correction is not lost in a form. It is reused in subsequent processing.

The system therefore improves with use, with no retraining, no new project and no extra invoice. That mechanism is what makes the validation workload fall month after month instead of staying flat, and it is what turns a fixed operating cost into a decreasing one.

In practice, it is also what decides the real profitability of a document automation project. A pipeline whose review workload never drops is a pipeline that moves work around instead of removing it.

Choice 3: extraction engines are interchangeable

The engine that reads the documents sits behind an interface. If a better engine ships next year, or if the vendor triples its prices, you swap it without touching the rest of the system.

That architectural decision is taken on day one, and today it is worth more than the choice of model itself. In a market where extraction engines change every six months, a system welded to one vendor is a system whose exit cost rises every month.

Seen from the executive floor, the translation is simple: when the market moves, you do not pay for the project twice.

None of these three points is an AI topic

Traceability, feedback loop, decoupling: these are software engineering topics, the same ones you apply to the rest of a serious information system.

That is exactly why so many proofs of concept never reach production. They solved the most visible question, feasibility, and none of the three that decide what happens next. That is not a failure of the teams who built them: it was neither their mission nor the right moment.

The question to ask any proposal

When a diagnostic, a supplier or an internal team tells you a use case is feasible, the real question is this one: feasible once, or feasible every day for two years?

The difference between those two answers never shows up in a demo. It shows up in the three choices above, and they have to be made before the first line of code.