"But LLMs are non-deterministic" is one of the most common objections to AI in production. It's usually aimed at the wrong thing.
Yes, models produce different token sequences on repeated runs. But ask yourself: is that actually the problem?
Ask ten support agents the same question. You'll get ten different wordings. Nobody calls that non-deterministic - they check whether the answers are correct.
Ask an engineer to rebuild something. They won't produce identical code. Different variable names, different structure, maybe a different language. That's fine if the result is functionally equivalent.
We've never required character-level determinism from humans. We require correctness and consistency of outcome.
The same should apply to AI systems.
What people actually want when they say "determinism" is:
- Repeatable, correct outcomes
- Auditable decisions
- Compliant, explainable results
- Reliable behaviour at the system level
None of that requires identical token sequences. All of it requires good system design.
In fintech, the bar is not "did the model say the same words?" It's "did the system produce the same compliant, correct, auditable result?"
The fix isn't making models repeat themselves. It's:
- Better context engineering (narrowing the valid output space)
- Reasoning and planning (structured convergence)
- Tests and validators (executable specifications)
- Business rules and controls (guardrails that catch what matters)
Good AI products don't eliminate model uncertainty. They absorb it through context, orchestration, and checks.
Exact-output determinism is the wrong metric. Deterministic satisfaction of intent is the right one.