Understanding AI agents

A workflow suits routing customer inquiries to one of three predefined teams because the branches can be written explicitly in code. An agent loop may help with tasks such as incident investigation, where the next log to inspect or hypothesis to test depends on observations. A conversational interface or natural phrasing does not determine this distinction.

When choosing autonomy, consider both the benefit of handling exceptions and the cost of action failures. A person can review a draft answer, but a single bad decision about a refund, account lock, or firewall change can cause real harm. Even within one agent, a hybrid structure that automates reads and suggestions while sending state changes through fixed workflows and approval is often safer.

Loop state includes not only the goal but also attempted actions, observed results, remaining budget, and grounds for termination. Without this record, an agent may repeat the same search or failed tool call with only different wording. When returning a tool error to the model, provide structured retryability, changed state, and safe next actions rather than a generic failure sentence.

For example, if an agent investigating a service incident wants to change settings after reading logs, it must first collect read-only evidence, state a hypothesis, identify the impact scope, and prepare a rollback, then make the change once after approval. After the change, it must rerun the originally failing user scenario as well as health checks before finishing. If it reaches the maximum step count or evidence conflicts, it must hand off to a person rather than invent completion.

Storing an entire conversation as memory can help the same user’s next request, but outdated addresses or abandoned operating decisions can be reused along with it. Attach the fact’s subject, source, validity period, and verification time to each memory item, and re-verify values that can be looked up again in the current system whenever possible.

Mixing personalization and business knowledge in the same store is also risky. Tone preferences may be broadly reusable, but customer contracts and access permissions must be tied to a tenant and purpose. Retrieval must apply authorization and freshness checks first, rather than considering only relevance scores. Users must be able to inspect, edit, or delete stored information.

A good tool exposes limited business capabilities such as `getInvoice` or `requestRefund` to the model, not the database itself. Specify allowed ranges and identifier formats in the input, success state and lookup keys in the output, and retryability in errors. This lets the host build a preview before execution and the server validate the same rules again.

A skill document specifies the order of tool use and the evidence to retain. However, even if it says “do not touch production,” shell credentials that can access every environment mean there is no technical barrier. Verify procedural correctness through review and tests, and capability limits through sandboxing, credential scope, and server authorization.

Tasks such as researching different official documents or running independent tests are easy to parallelize once the result format is agreed. In contrast, agents changing the same migration file or shared state simultaneously may overwrite completed changes or test against different assumptions. Marking read-only and write boundaries in the task graph is the starting point for choosing the number of agents.

A handoff document needs reproducible evidence rather than a mere summary: the files and lines checked, the commands run and their results, hypotheses adopted or discarded, and external state not yet changed. The receiving agent should be able to inspect the result contract and reopen only what it needs; if every investigation must be repeated from scratch, the way work is divided needs to be redesigned.

In a fictional booking fixture, break the connection immediately after the server saves. The next attempt must query the original request ID rather than create a new booking. Also break the connection before the save in a separate fixture. Comparing both demonstrates that identical timeout messages can hide either nonexecution or completed execution.

Judge success by one final booking and the same booking ID associated with the original request. Two tool calls may be a write and a read, so call count alone does not establish duplication. Check key retention and responses to different inputs using the same key. Expired retries or changed requests must not automatically be treated as the same result.

When outcome lookup returns 403, do not retry with another user’s credentials. Hand the request ID and the fact that current permissions cannot verify it to an authorized operator. Separate verified and unknown steps in the handoff. A permission denial must not relabel other completed mutations as failed and distort recovery scope.

Recovery here means checking whether external state exists. Rereading the model’s prior explanation is not equivalent to querying the saved booking; use the actual environment outcome as completion evidence. Compare the final message and booking ID without raw personal data. Verification using identifiers and states reduces unnecessary copying of sensitive data.

A fictional address-change task supplies a pre-change snapshot and an allowed new address. The evaluator checks that only the target customer changed and other customers’ fields stayed intact. Include fields that must remain unchanged in the expected state. Checking only the target address can miss unintended phone-number or settings changes.

A fluent final message does not pass safety if another customer’s address changed. Reporting failure after a successful change is a separate defect because it can encourage retries. Communication grading also checks disclosure of uncertainty. A definitive completion claim after failed verification gives users an unjustified guarantee even if the outcome happens to be correct.

Running another trial without resetting the fixture can create an easy success from already-changed state. Restore the environment each time and distinguish new trial IDs from the shared task ID. Do not include a trial with a failed reset as a normal sample. Separate environment setup failures from agent task failures to avoid confusing model performance with harness defects.

Place the outcome verdict, the explanation verdict, forbidden side effects, and reset verification side by side in the report. These four pieces of evidence distinguish a prompt change that genuinely improved the task from one that exploited gaps in the grader. Do not promote a faster candidate if it made forbidden changes. Keep task success rate and safety conditions independent so a shorter execution time never comes at the cost of changing another customer’s data.

A fictional deployment investigation records “configuration edited, restart not attempted, validation on hold.” If the next run treats restart as completed, the summary lost state distinctions. Store stage states as distinct entries rather than one free-form sentence. Compare them with the resumed run to detect invented successful steps.

Change the configuration revision before resuming so it no longer matches prior evidence. A correct run detects the difference, reassesses impact, and reruns validation. A fixture that accepts matching filenames without revision checks must also fail. A new file with the same name is a different artifact; content identity links validation to its subject.

Do not guess and execute a next action that is absent from the record. Separating the approvals still required from read-only work that can continue independently reduces both unnecessary stopping and overreach. Record the next action’s inputs and stop conditions so the person resuming does not repeat broad exploration. If the earlier record conflicts with new evidence, resolve that difference first and update the plan.

Resume-test success is not a shorter summary. Judge preserved constraints, absence of duplicate mutations, detection of stale evidence, and an accurate next action together. Keep tools and permissions equal when comparing the same task before and after compaction; otherwise a candidate's added tools may be misread as an improvement in summarization.