Common mistakes
- No acceptance criteria or output schema.
- Prompts that hide requirements instead of stating them.
- No retrieval grounding for factual answers.
- Missing refusal rules or safety rails.
- Unversioned prompts; changes ship without tests.
- No evals or only manual spot checks.
- Ignoring latency/cost by overusing heavy models.
Production patterns to use
Schema + criteria: JSON outputs with explicit fields and quality bars.
Retrieval: Provide sources; require citations; refuse if insufficient.
Tooling: Use function calls for structured actions, not free-text.
Safety: Refusal rules, PII stripping, and approvals for risky actions.
Evals: Golden + synthetic tests before every release.
How we ship safely
- Define success: schema, constraints, and edge cases.
- Add retrieval, citations, and refusal logic.
- Version prompts + track lineage in Git.
- Run evals and regression tests; gate releases.
- Monitor logs, alerts, and cost; iterate quickly.
Prompts are product code. Treat them with specs, tests, and monitoring—or expect surprises in production.
FAQ
Do small prompts need tests?
Yes—lightweight evals catch regressions and cost spikes early.
Which models to target?
Use small/fast models for simple tasks; reserve heavy models for reasoning.
How to manage versions?
Keep prompts in Git with changelog and linked eval results.
What about multi-turn?
Constrain memory, reset state intentionally, and test flows end-to-end.