An AI agent that works in a demo can still fail in production because production adds ambiguous requests, stale data, permissions, timeouts, duplicate actions, changing interfaces, and accountable users. The central design challenge is not making a model call tools. It is building a bounded system that knows what it may do, exposes uncertainty, survives partial failure, and lets a human stop or reverse consequential actions.
This guide presents a practical production framework for ChatGPT-style agents. It applies to research, support, content operations, internal assistants, and developer workflows. Product interfaces and model capabilities change, so treat the architecture as a control pattern rather than a promise about one provider.
Start with the outcome, not autonomy
Write the user outcome in measurable terms: resolve a supported question, prepare a sourced research brief, classify a ticket, or draft a change for review. Define what counts as success, failure, escalation, and abstention. If a deterministic rule or simple script solves the task, use it; adding an agent creates variability and a larger security surface.
Choose the lowest autonomy level that delivers value. A read-only assistant may retrieve and summarize. A copilot can propose a tool action for approval. A bounded agent may execute reversible low-risk steps within an allowlist. Publishing, payments, deletions, credential changes, legal claims, and production deployments should remain behind explicit human approval unless a rigorous risk process says otherwise.
Turn the workflow into a state machine
Do not hide the entire process inside one prompt. Represent states such as intake, validation, planning, retrieval, draft, review, approval, execution, verification, and completion. Define allowed transitions and terminal failures. Store a durable run identifier so retries resume safely instead of starting another action.
A state machine makes behavior inspectable. If retrieval fails, the system can move to “needs source” rather than asking the model to improvise. If approval expires, execution is blocked. If an external API succeeds but the response times out, verification checks whether the side effect exists before retrying.
Design tools as narrow contracts
Each tool should perform one clear operation with a typed input, typed output, permission check, timeout, and safe error. Prefer `create_draft(title, body)` over a generic command shell. Validate URLs, identifiers, enum values, size limits, and destination scope outside the model. Never rely on the model to remember a security rule that code can enforce.
Return structured errors such as `not_authorized`, `rate_limited`, `validation_failed`, or `already_completed`. Avoid dumping secret-bearing stack traces into the model context. Include enough information for a safe next step, and attach an idempotency key to actions that must not be duplicated.
Idempotency prevents expensive retries
Networks fail at the worst moment: after a remote service accepts an action but before your agent receives confirmation. Generate an idempotency key from the run and action, store the intended request, and query the destination before retrying. A second “send,” “publish,” “charge,” or “create” should return the first result rather than repeat the side effect.
Design compensation for operations that cannot be atomic. A draft can be archived, a reservation can be canceled, or a configuration can be restored from a snapshot. Record which compensations are automated and which require an operator. Test the recovery path before relying on it.
Keep memory scoped and evidence-based
Separate short-lived run context, user-approved preferences, retrieved evidence, and durable business records. Give each item a source, owner, timestamp, sensitivity, and expiration. Do not let an old model summary silently become authoritative memory. Refresh facts from their systems of record.
Retrieve the smallest relevant context rather than sending an entire database or conversation history. Remove secrets and unnecessary personal data. When projects or customers must be isolated, enforce that separation in storage and authorization, not only through instructions. Our ChatGPT Projects workflow explains how to organize persistent context while retaining human review.
Defend retrieval from prompt injection
Webpages, documents, emails, and tickets are untrusted data. They can contain text that tells an agent to ignore rules, reveal information, or call a tool. Label retrieved content as evidence, isolate it from policy instructions, and prevent it from granting permission. The agent should cite it, not obey it.
Use destination allowlists, content-size limits, file-type checks, malware scanning where appropriate, and policy filters before retrieval reaches the model. Test with documents that contain hostile instructions. Success means the agent extracts relevant facts, reports the suspicious instruction, and keeps its original scope.
Build an approval experience people can use
An approval dialog should show the exact action, destination, changed fields, evidence, uncertainty, and rollback. “Approve agent plan” is too vague. Let reviewers edit or reject the action and require fresh approval if important inputs change after review.
Match approval strength to risk. Low-risk reversible drafts may use batch review. External messages need recipient and final-body previews. Production changes need diffs, tests, owner identity, and rollback. Expire old approvals so a queued action cannot execute after the environment changes.
Evaluate with realistic tasks
Create an evaluation set from real, permission-safe examples: common requests, difficult edge cases, ambiguous inputs, missing data, conflicting sources, malicious content, unavailable tools, and partial failures. Score task success, factual support, policy compliance, correct abstention, tool selection, argument accuracy, and recovery behavior.
Keep a hidden test set to reduce overfitting. Run evaluations after model, prompt, tool, retrieval, or policy changes. Compare against a simple baseline and human process. An agent is not better merely because it completes more tasks; unsafe confident completion should score worse than appropriate escalation.
A staged production test plan
- Run offline against saved inputs with every external action mocked.
- Use shadow mode on live requests while humans continue the real work.
- Enable read-only tools for a small internal group and review every trace.
- Permit reversible draft actions with explicit approval and idempotency.
- Expand one capability at a time after predefined quality and safety targets are met.
- Keep a kill switch, rollback, owner, and support path at every stage.
- Re-evaluate after changes instead of assuming earlier results still apply.
Shadow mode is especially useful because it reveals real language and data variation without letting the agent affect users. Compare recommendations with actual outcomes, but protect privacy and disclose monitoring according to organizational policy.
Observability should reconstruct a decision
Assign a trace ID to the run and log state transitions, model and prompt version, retrieved source identifiers, tool requests, validation outcomes, approvals, timing, token use, errors, and final status. Redact secrets and minimize personal data. Operators should be able to answer what happened without reading a user’s entire private conversation.
Track more than uptime. Useful metrics include completion rate, escalation rate, unsupported-claim rate, correction rate, tool error rate, duplicate-action prevention, approval latency, cost per successful outcome, and incidents. Segment by task type because an average can hide a dangerous workflow.
Control latency and cost deliberately
Set budgets for model calls, retries, retrieved documents, context size, tool calls, and wall-clock time. Stop loops that do not improve the state. Cache stable, non-sensitive results with version and expiration. Route simple classification to a smaller method when evaluation shows that quality remains acceptable.
Show users progress and allow cancellation. A fast wrong action is not useful, but a silent ten-minute agent also creates poor trust. Measure end-to-end time, including human review and correction, rather than advertising only model response latency.
Handle failures as normal states
Classify failures into invalid input, missing permission, unavailable dependency, model uncertainty, policy refusal, timeout, and unknown side effect. Give each class a safe transition. A timeout after an action should trigger a read-back check. Repeated validation failure should escalate rather than encourage the model to invent new fields.
Use circuit breakers when an external service is unstable, rate limits per user and tool, and queues with dead-letter handling. Preserve the original request and diagnostics for an operator. Avoid automatic fallback to a more privileged tool or a different destination.
Security and privacy baseline
Use dedicated service identities, least privilege, short-lived credentials, encrypted transport and storage, secret managers, tenant isolation, and audited access. Keep secrets out of prompts, logs, source repositories, and generated error messages. Threat-model account takeover, prompt injection, data exfiltration, excessive agency, insecure tool output, and supply-chain compromise.
Review retention and deletion requirements for prompts, files, embeddings, traces, and feedback. Explain to users when AI is involved and where a human can correct the result. For personal workflows, our ChatGPT privacy guide offers a complementary checklist.
Release checklist
- A named owner, documented scope, and measurable success criteria exist.
- Tools are narrow, validated, least-privilege, timed out, and observable.
- External instructions cannot expand permissions or reveal secrets.
- Consequential actions require a clear preview and accountable approval.
- Retries are idempotent and partial side effects are verified before repetition.
- Evaluations cover edge cases, attacks, abstention, and tool failures.
- Alerts, kill switch, rollback, incident response, and user support are tested.
- Cost, latency, privacy, accessibility, and retention are acceptable.
A launch decision should cite evidence from evaluations and staged operation. Record accepted residual risks and a review date. If the team cannot explain how to stop and recover the agent, it is not ready.
Frequently Asked Questions
Which agent framework is best for production?
Choose based on required tools, state durability, observability, deployment environment, security controls, and team expertise. A simpler framework with clear state and tests is often safer than a feature-rich system the team cannot debug.
How much autonomy should an agent receive?
Start read-only, then add reversible capabilities one at a time. Grant autonomy only when evaluation, external controls, monitoring, approval, and recovery match the consequence of failure.
Do agents need long-term memory?
Many do not. Use durable memory only for a defined user benefit, with consent, source, scope, expiration, correction, and deletion. Retrieve current facts from authoritative systems.
How can we prevent duplicate actions?
Use durable state, unique idempotency keys, destination read-back, and action-specific status. Never retry a timed-out consequential call blindly.
What should trigger an immediate rollback?
Unauthorized access, data leakage, repeated unsupported claims, uncontrolled loops, duplicate external actions, policy bypass, or inability to reconstruct important decisions should stop expansion and activate the incident plan.