An agent demo does not show how the system will behave under production conditions. It must interpret imperfect requests, choose among tools, handle stale or hostile data, survive network failures, respect permissions, and leave an understandable record of what happened. The hard part is not making a model call a function. It is making every possible action safe enough, observable enough, and recoverable enough for real users.

This guide presents an engineering workflow for production AI agents. It focuses on the design choices that remain important even as models and product interfaces change: narrow outcomes, explicit tool contracts, approval at the point of consequence, repeatable evaluation, operational visibility, and controlled rollout. It draws on OpenAI’s current agents overview, function calling guide, guardrails and human review guidance, agent evaluation guidance, safety best practices, and production best practices. Treat those official pages as the source for current API details.

Begin with a job that deserves an agent

An agent is useful when a language model must manage a multistep workflow, decide which tools to use, and adapt to context. Not every AI feature needs that freedom. Classification, field extraction, a fixed transformation, or a known sequence of API calls may be simpler and safer as ordinary software. OpenAI’s practical guide to building agents recommends agents for work with nuanced decisions, difficult rules, or substantial unstructured data. That is a better starting test than asking where an agent might look impressive.

Write the job as an outcome a user can recognize. “Help with support” is too broad. “Answer questions from approved support articles, collect missing account details, and prepare a refund request for human approval” is testable. Name what the agent may read, what it may change, and what it must never do. Define success, abstention, escalation, timeout, and cancellation before selecting a model.

A useful scope document fits on one page. It lists the user, trigger, allowed data, permitted tools, completion condition, approval points, and owner. It also names foreseeable harm. Could the agent disclose another customer’s data, send an incorrect message, duplicate a payment, overwrite a record, or follow an instruction hidden in a web page? These questions turn vague safety concerns into requirements engineers can implement.

If you are still choosing between chat, agents, scheduled tasks, and conventional automation, the PChatGPT comparison of AI productivity tools and workflow automation offers a user facing view of those categories. The important decision is how much initiative the system needs, not how many features it can advertise.

Design the workflow before writing the prompt

Draw the happy path and the failure paths. A support agent might identify intent, retrieve policy, check account state, propose an action, request approval, execute once, verify the result, and summarize. Beside each step, record the input, output, authority, timeout, retry rule, and evidence to retain. This map becomes the basis for tools, logs, tests, and the user interface.

Keep deterministic work deterministic. Authentication, authorization, money calculations, date validation, policy thresholds, schema validation, and idempotency belong in code. Let the model handle tasks that benefit from language understanding or flexible reasoning, such as identifying a request, choosing relevant evidence, or explaining an outcome. The model can recommend an action, but the service should enforce whether that action is allowed.

Start with one agent when possible. A single bounded loop is easier to inspect than a network of specialists. Split the system only when separate instructions, permissions, or context clearly improve reliability. Multiple agents add handoffs, more state, more opportunities for inconsistent decisions, and a harder debugging story. Architecture should follow measured need rather than fashion.

Make every tool a narrow contract

Tool access is where an agent becomes consequential. OpenAI’s function calling guide describes a loop in which the application provides tool definitions, receives a tool call, executes it, returns the result, and lets the model continue. Your application remains responsible for the execution. A tool call is a proposed structured action, not proof that the action is correct or authorized.

Give each tool one clear purpose. Prefer get_order_status and request_refund over a broad manage_order tool. Use typed parameters, required fields, enumerated values where appropriate, and strict schemas. Validate arguments again on the server. Never build database statements, shell commands, file paths, or destination addresses by blindly concatenating model output.

Separate read tools from write tools. Reading a product catalog does not carry the same risk as changing a price. Use least privilege credentials for each tool and derive user authorization from trusted application state, not from a claim inside the conversation. A model should not be able to increase its own permissions by asking for a different role.

Return structured results that distinguish success, rejection, retryable failure, permanent failure, and unknown outcome. Include stable record identifiers rather than relying on prose. When a timeout happens after a request was sent, the agent may not know whether the side effect occurred. The recovery path should query the system of record before trying again.

For side effects, use idempotency keys or an equivalent deduplication design. A retry should not create a second refund, ticket, email, or reservation. Record the proposed arguments, policy decision, approval identity when applicable, execution response, and verification result. Redact secrets and unnecessary personal data from that record.

Put approvals beside consequences

Guardrails and approvals solve different problems. OpenAI’s current Agents SDK guidance describes input guardrails for requests, output guardrails for final responses, tool guardrails around function calls, and human approval for side effects. A general instruction such as “be careful” is not a substitute for a control at the action boundary.

Require approval when an action is costly, difficult to reverse, external, privileged, or sensitive. Examples include sending a message, changing an account, deleting a file, publishing content, purchasing an item, cancelling an order, and running a command. Show the reviewer the exact action, target, important arguments, supporting evidence, and expected consequence. “Approve agent plan” is too vague.

Approval should pause the run before execution. After a reviewer approves, recheck authorization and any time sensitive preconditions. An old approval should not permit an action after the account, price, policy, or destination has changed. Record who approved what, and give the reviewer a clear way to reject or edit the proposal.

Prompt injection needs architectural defenses. Treat text from web pages, documents, email, and tool results as untrusted data. Do not let retrieved text redefine the system’s permissions. Limit the tools available for the current step, keep sensitive values outside model context when possible, validate tool arguments, and require approval for meaningful side effects. Readers evaluating agents that operate in websites can also use PChatGPT’s AI browser agent safety guide to think through the added risk of acting through interfaces.

Build a failure model, not just a happy path

List failures by layer. The model can misunderstand intent, select the wrong tool, omit a required step, or produce unsupported text. A tool can time out, reject arguments, return stale data, or report a partial result. The surrounding service can lose state, exceed a limit, or receive two copies of the same event. A person can approve the wrong target. Your design needs a response for each class.

Set bounded retries with backoff for failures that are genuinely temporary. Do not retry validation errors or denied actions. Cap model turns and tool calls so a confused loop cannot run forever. Give the system a terminal state such as needs human help rather than forcing every request to complete autonomously.

Make cancellation real. The interface can stop future steps, but it cannot pretend an action already completed never happened. Track committed side effects and define compensating actions where practical. For example, a created draft might be deleted, but a sent email can only be followed by a correction. The final response should state what completed, what failed, and what remains uncertain.

Persist state at meaningful checkpoints. A long workflow should resume from verified state instead of replaying every action. Store the minimum context needed to continue safely, with retention and access controls appropriate to the data. Version instructions, tool schemas, policies, and model configuration so an incident can be reconstructed.

Evaluate the whole trajectory

Final answer quality is not enough. An agent can produce a plausible response after using an unauthorized source, calling an unnecessary tool, leaking sensitive data into a log, or attempting the same action twice. Evaluate the path: tool choice, argument quality, evidence use, policy compliance, approval behavior, stopping behavior, and final communication.

OpenAI’s agent evaluation guide recommends starting with traces while debugging, then moving to datasets and repeatable eval runs. A trace provides the sequence of model calls, tools, guardrails, and handoffs. Turn observed failures into durable test cases. Your dataset should include normal requests, ambiguous requests, missing data, malformed tool output, prompt injection, permission failures, outages, duplicate events, and requests the agent must refuse or escalate.

Use deterministic checks whenever possible. Did the agent call only allowed tools? Did every write have approval? Were required fields present? Was the final status verified? Was private data absent from the response? Human review or model based graders can assess softer qualities such as relevance and clarity, but calibrate them against examples and inspect disagreements.

Choose metrics tied to the job. Useful measures include successful task completion, correct abstention, unsafe action attempts, duplicate side effects, escalation quality, tool error recovery, latency, and resource use. A single average can conceal a severe failure class. Break results down by intent, risk tier, language, customer segment, tool, and workflow version where those dimensions matter.

Observe production without collecting everything

Operators need to answer four questions quickly: what did the user ask, what path did the agent take, what changed in an external system, and why did the run stop? Use a correlation identifier across model calls, tools, approval events, and application logs. Record timing, status, schema version, and error category. Keep an audit trail for consequential actions.

Observability should respect privacy. Do not log API keys, authentication tokens, raw secrets, or full sensitive documents by default. Classify data before launch, redact fields at ingestion, restrict access, and define retention periods. Separate diagnostic detail from analytics. More logging is not automatically better if nobody can safely search or interpret it.

Alert on symptoms users feel and controls you cannot afford to lose: elevated task failures, rising tool timeouts, approval bypass attempts, repeated actions, unusual permission denials, and sudden changes in escalation. Dashboards are useful, but sampled trace review often reveals behavior that a count cannot explain.

Treat security and operations as product features

OpenAI’s safety guidance recommends adversarial testing, human review where possible, constrained inputs and outputs, clear limitation communication, and issue reporting. Apply those ideas to the complete service. Authenticate users, authorize every resource access, rate limit expensive or risky operations, scan uploaded content where appropriate, and offer a visible route for reporting bad behavior.

Keep credentials in a secret manager or protected environment, never in prompts or source code. Use separate development, staging, and production projects. OpenAI’s production guidance recommends protecting API keys and separating environments as systems scale. Give production access only to people and services that need it. Rotate compromised credentials and test the rotation process before an incident.

Plan capacity and degraded behavior. Rate limits, provider incidents, and downstream outages are normal operating conditions. Decide whether the agent should queue, retry, provide a read only response, or hand the work to a person. Communicate status instead of leaving the user watching an endless spinner. Cost controls should cap runaway loops while preserving enough diagnostic evidence to understand them.

Roll out in stages and earn autonomy

Begin in shadow mode if the workflow allows it. Let the agent propose decisions while the existing process remains authoritative. Compare proposals with real outcomes and capture disagreements. Next, allow it to prepare drafts for review. Grant limited execution only after the relevant failure rates and approval behavior meet your acceptance criteria.

Use a small audience, narrow tool set, and reversible actions first. Establish a kill switch that operators can use without deploying code. Keep a deterministic fallback or human queue. Increase autonomy by action class, not with one global switch. An agent might safely look up an order while still requiring approval to refund it.

Every release should identify what changed: model, prompt, tool schema, retrieval source, policy, or orchestration code. Run regression evals before deployment and monitor the same risk slices afterward. Roll back when a change degrades important behavior. Production quality is a continuing practice, not a launch milestone.

A practical production readiness checklist

• Task: The outcome, non-goals, completion state, and escalation path are explicit.
• Authority: Every tool uses least privilege and server-side authorization.
• Contracts: Arguments and results use validated schemas with clear error states.
• Side effects: Writes are idempotent, recorded, verified, and approved when consequential.
• Untrusted data: Retrieved content cannot silently change permissions or policy.
• Failure: Retries are bounded, uncertain outcomes are reconciled, and cancellation is defined.
• Evaluation: Tests cover trajectories, edge cases, attacks, abstention, and regression.
• Operations: Traces, alerts, ownership, rollback, privacy controls, and incident response exist.
• User experience: The product communicates progress, limitations, approvals, and final state clearly.

If any answer is vague, keep the system in a lower autonomy mode. Human review is not a sign that the agent failed. It is a deliberate part of the product when consequences require judgment.

FAQ about production AI agents

What makes an AI agent different from a chatbot?

A chatbot may simply generate a response. An agent manages steps toward an outcome and can select tools that read data or take actions. That extra authority creates a need for explicit permissions, tool validation, traceable state, failure recovery, and approval before consequential side effects.

Should a production agent use multiple specialist agents?

Not by default. Start with one bounded agent because it is easier to evaluate and operate. Add specialists when separate context, instructions, ownership, or permissions produce a measured improvement. Each handoff adds state and another place where errors can occur.

Where should human approval be required?

Place approval immediately before actions that are external, costly, privileged, sensitive, or hard to reverse. The reviewer should see the exact target, arguments, evidence, and consequence. Recheck authorization and changing preconditions after approval and before execution.

How do you know an agent is ready for production?

Readiness is evidence, not a successful demo. The system should pass representative and adversarial evals, preserve permissions, handle tool failures, prevent duplicate side effects, pause correctly for approvals, produce useful traces, protect sensitive data, and support rollback. Launch narrowly and confirm that production behavior matches the tested behavior.

Decide what can ship

Before release, ask an operator to trace a failed run, reject a proposed action, and stop the workflow. If the product cannot support those tasks, keep it in draft-only mode until it can. Use evaluation and operating evidence to decide which actions can run without review.