Most generative features demo beautifully and erode trust within a fortnight. The demo shows the good case. The fortnight shows the other three.
We have shipped enough of these now to know where they fail, and none of the failures are about model quality.
One: confident output from thin evidence
The model does not know what it does not know, and by default it will not say so. A surveying assistant that writes "no damp was observed" when the surveyor simply never looked at that wall has not made a small error. It has made a legal one.
The fix is not a better model. It is refusing to generate anything that cannot be traced to an input, and rendering the gap explicitly where evidence is missing. On Fieldnote, unsupported sections are a visible, countable state in the interface — and they turned out to be the feature surveyors valued most.
A system that says "I don't know" twice a day earns more trust than one that is right 95% of the time and silent about the other 5%.
Two: no way back
Generative features tend to ship as one-way doors. You press the button, the text is replaced, and the thing you wrote is gone. Users learn this once and then stop pressing the button.
Every generative action we ship is reversible, and the original is visible while the suggestion is on screen. This is unglamorous and it is the difference between a feature people try and a feature people use.
Three: invisible cost
Token cost is real, variable, and usually hidden from the person triggering it. A team discovers the bill at month end and disables the feature. We surface the unit — generations, pages, minutes, whatever the customer can count — in the interface, before the action, every time.
What we actually build
The model call is perhaps a tenth of the work. The rest is:
- Provenance. Every output links to the inputs that produced it.
- Reversibility. Nothing is replaced without a visible way back.
- Disclosure. The user knows what it cost and that a machine wrote it.
None of that is a prompt. That is the product.
