Prompt and agent workflows

“Summarize this well, like an expert” reads naturally, but each reviewer understands success differently. Start by writing down whom the result is for, which decision it supports, which numbers·conditions must be preserved, and which guesses must be excluded. For customer inquiry classification, specify the allowed labels and which takes precedence when they overlap; for document summarization, specify paragraph-level evidence and preservation of dates·amounts. Only with this information can you compare model candidates and human reference answers against the same criteria.

Input contracts need fields and sources too. User questions, approved database records, retrieved external strings and past model answers have different trust levels. Use delimiters and roles to distinguish instructions from data. Do not promote 'ignore previous rules' inside a document into a command, and restrict tool, secret and network permissions in the application.

Specify both the output format and the failure behavior. Do not just write JSON; define field names, types, enums, and required fields, and whether missing evidence should return `unverified`, request a retry, or hand off to a person. Even when a model matches the schema shape, the factual accuracy of the values is a separate matter. Record which failures the parser and schema validator, calculation code, comparison with the source, and authorization checks each prevent.

A long prompt is not automatically a good contract. Conflicting rules, unverifiable declarations, and repetitive wording increase tokens and latency and obscure essentials. A short prompt may suffice if task boundaries are clear and failure tests pass. Refine it based on which sentences change results and which validators catch errors in representative, boundary, and failure cases, rather than targeting character counts.

Why does this happen?

Because only by defining success and failure first can prompt changes be compared on the same task criteria rather than on taste.

When is it a problem?

Elaborating a vague request can lead to approval based on one natural-sounding answer while missing omitted numbers, fabricated evidence and empty-input failures.

Common beginner misconceptions

Assigning a role or writing 'be accurate' does not by itself provide fact verification or authorization controls.

How to verify it yourself

In the current prompt, mark the purpose, input sources, prohibitions, output schema, and failure states, and check whether a test or code verifies each one.

Put long-lived rules, such as the overall purpose of the service, its allowed scope, and failure principles, in the system message. Put the current user's task, target, and input in the user message. Assistant history is a record of earlier outputs, not always-true knowledge or new operating rules. Tool results and retrieved documents are external data, so mark their boundaries so that sentences inside them do not gain higher authority even if they look like instructions.

Explaining role priority in words does not eliminate attacks. Models are probabilistic string generators and may mishandle boundaries in complex contexts. Keep secrets inaccessible to the model even when untrusted documents request them. Gate external sending·deletion·payments through server allowlists·authorization checks and human approval. Prompts are one defense layer; the application makes the final execution-authority decision.

Test role conflicts deliberately. Evaluate cases where the user asks to change the system's output limits, a retrieved document claims a different purpose, a previous assistant turn stated a wrong label definition, and the system prompt is empty. With only normal cases, you cannot tell whether the model happens to comply or the boundary actually holds. Record the failed output together with the layer where it was stopped.

Distinguish task rules from user choices. 'Answer in Korean' may be a service default, but if users can request English translation, specify override conditions. Giving absolute prohibitions and default preferences equal force increases conflicts. Define each rule's owner, allowed overrides and violation handling, and rerun role-conflict regression sets after changes.

Why does this happen?

This is because distinguishing input authority keeps instructions in external data and past model errors from being promoted to operating rules.

When is it a problem?

Merging retrieved documents and system rules into a single string lets prompt injection act like a command and makes it hard to trace which source changed the result.

Common beginner misconceptions

The system role is not a security boundary the model can never violate, nor a safe vault for secrets or tool permissions.

How to verify it yourself

Insert strings that tell the model to ignore rules and send data externally into an untrusted document, then test whether the application's tool, network, and secret access is blocked, not just the model's answer.

As Hugging Face's official chat-template documentation explains, chat ultimately becomes a token sequence processed by the model. Objects such as `{role: user, content: ...}` are application representations; the tokenizer template adds model-specific control tokens such as `<|user|>` or `[INST]`. Applying a different template to the same conversation can produce a sequence unseen during training and substantially degrade performance. Matching the model-family name does not guarantee the correct input format.

Compare the official tokenizer’s apply_chat_template path with actual runtime rendering. Check generation-start markers, end-of-turn tokens, system-role merging, and multi-turn order. Duplicate tokens may appear if the tokenizer adds BOS and EOS to a rendered string already containing special tokens. Preserve rendered output or its hash, token IDs, and lengths as evidence, rather than only UI messages.

Templates also affect the context budget. More role markers and examples leave fewer tokens for task data, and long inputs may be truncated. Measure truncation direction with the exact tokenizer and check that system rules and the latest user question are retained. Do not confuse the template’s maximum length, the model architecture’s context maximum, and the runtime’s currently allocated context.

Run golden render tests when changing the model or runtime. For fixed messages, check expected control-token order, generation markers, and token count, then compare actual normal·conflicting·empty-input results. A changed runtime default template or a server wrapping an already-rendered client string again can cause regressions despite identical prompt text. Manage weight·tokenizer·template·runtime as independent revisions.

Why does this happen?

This is because the model reads token sequences produced by a template, not role objects; incorrect serialization can change the meaning of the entire prompt.

When is it a problem?

If only UI messages are saved, regressions caused by duplicated special tokens, missing generation markers, and runtime default changes cannot be reproduced.

Common beginner misconceptions

A model name containing Instruct or chat does not mean that every runtime automatically selects the correct template and keeps it fixed permanently.

How to verify it yourself

Render representative messages using the official tokenizer and deployment runtime. Compare strings, token IDs, lengths and end markers, and pin them with hashes.

Few-shot prompting uses a few input-output pairs to demonstrate the desired format and judgment. First, people define labels and task rules, then select representative examples from actual incoming requests. Rather than adding several simple refund questions, use cases that reveal decision boundaries: both refund and exchange clues, a missing required order number, or an `unknown` output. Examples supplement the rules but do not replace the taxonomy itself.

Reviewers should be able to assess the correct answer and why other labels do not apply. However, including long reasoning verbatim is not always better. Unnecessary internal explanations consume tokens and may be copied by the model. Provide only a short rationale field needed by users and verifiable evidence. De-identify sensitive real customer data or replace it with authorized test fixtures.

There is no fixed right number of examples. Consider model context, example token length, task diversity, and evaluation improvement together. If adding an example improves boundary quality but worsens input truncation and p95, retrieval can be used to select only verified examples relevant to the current question. Compare an instruction-only baseline without examples, a small set, and a large set on the same gold set to check marginal utility.

Check for example leakage and order bias. If few-shot sentences are nearly identical to test items, scores rise above actual generalization. Split by customer, document, and time, and check whether judgments hold when label order and proper nouns change. Use a confusion matrix to check that predictions do not skew toward the majority label just because its examples appear last, and store the passing example set as a hash together with the prompt version.

Why does this happen?

Examples specify task boundaries, but poor choices can distort evaluation by encouraging imitation of surface wording, order and majority labels.

When is it a problem?

Filling the prompt with an arbitrary fixed number of easy normal examples still fails on mixed or insufficient inputs where the real cost is high, and only wastes context.

Common beginner misconceptions

More examples do not always mean higher accuracy, and answering correctly on examples included in few-shot prompts does not establish generalization to new task cases.

How to verify it yourself

Compare no examples, a small set and the candidate set on the same separate test, inspecting order changes, boundary labels, tokens and p95 together.

An autoregressive model repeatedly scores next-token candidates using all tokens so far and selects one. In official Hugging Face generation strategies, greedy decoding chooses the most likely token each step, sampling draws from the probability distribution, and beam search maintains multiple sequences. These describe generation paths, not evaluators of correspondence with external facts.

Greedy decoding can reduce variation in the same environment, but does not guarantee the best task answer across the whole sequence. High-probability early choices can lead to repetition or incorrect conclusions later. Sampling can explore different expressions and ideas, but can also select low-probability tokens that are inaccurate or violate the format. Choose the strategy based on task error costs and evaluation results.

For tasks with a small output space and easy validation, such as classification and extraction, test a low-variation baseline and schema first. For generating creative candidates, measure the usefulness, duplication, and prohibited content of repeated outputs and the cost of human selection. Conversation may call for a middle range between stability and variety of expression. Rather than copying universal numbers such as “always 0 for precise tasks, always 1 for creative work,” inspect the actual distribution for each model and runtime.

When comparing, pin the prompt, template, model, and evaluation set. Do not run greedy decoding and sampling once each and pick the nicer sentence; run each several times on normal, boundary, and failure inputs. Save correctness and evidence, schema, omissions, repetition and diversity, and p95 and tokens for each raw output. Change one variable at a time so decoding changes are not mixed with hidden prompt or model changes.

Why does this happen?

Because only by not mistaking the selection strategy for a factuality switch can you choose the evidence retrieval, validator, or model improvement that fits the error.

When is it a problem?

Lowering only the temperature when a wrong answer repeats can make the model produce the same mistake more consistently and hide missing evidence or prompt errors.

Common beginner misconceptions

No single ranking, such as "greedy is always most accurate and sampling is always creative," applies to every model and task.

How to verify it yourself

Collect repeated raw outputs for each strategy under the same manifest, then compare quality, schema compliance, diversity and latency by subset.

Temperature generally changes the relative sharpness of logits, affecting how much probability concentrates on high-scoring candidates. Top-p restricts sampling to a candidate set within a cumulative probability threshold. Allowed values, handling of 0, and order of application with other filters may vary by runtime and version. Check the request and resolved configuration to ensure that client parameters were not ignored or replaced by defaults on the server.

Changing both values substantially at once makes it hard to tell which setting changed the results. Start from a supported baseline and adjust either temperature or top-p gradually, one at a time. Measure distinct answers, self-repetition, and prohibited-content rates alongside quality and schema compliance. Do not read numbers directly as a “creativity score of 70”; document them as the range of results observed for that model, prompt, and task.

Output length is a separate contract. `max_new_tokens` caps the number of newly generated tokens, while the context maximum concerns the total space for input and output. Too low a limit can truncate JSON closing braces or evidence; too high a limit can increase repetition, latency, cost, and resource occupancy. Set the limit by measuring the expected output-token distribution and the longest normal result, and define failure handling that does not consume partial results when the limit is exceeded.

Also test EOS tokens, stop strings, and schema completion conditions. Check whether a stop string inside a normal sentence or JSON value causes early termination, and whether the streaming client correctly handles a stop split across multiple tokens. Also confirm that the limit applies when the model emits no end token and that server resources are reclaimed after the client cancels. If the template's end-of-turn and the API stop list overlap, verify with actual output.

Why does this happen?

Candidate distributions and stopping conditions cause different failures. Separate them to identify format errors, insufficient diversity and unbounded generation.

When is it a problem?

Adjusting several values at once leaves only the output that happened to look good and fails to reproduce JSON truncation, stop-sequence conflicts, and ignored server parameters.

Common beginner misconceptions

Temperature and top-p values do not indicate the same creativity level across models and runtimes, nor are lower values a universal measure of factual accuracy.

How to verify it yourself

Save resolved generation configs and raw responses. In repeated tests changing one variable at a time, check schema termination·stop behavior·limits·p95.

As Ollama’s structured-output documentation describes, requests can provide JSON schemas and responses can be validated with tools such as Pydantic·Zod. JSON Schema describes JSON data structure and validation semantics. Explicit required fields, types, enums, array items, and rules for additional fields make application contracts more reliable than free-text parsing. Check the supported schema subset and error behavior for the exact runtime version.

Passing the schema can confirm that `amount` is a number, but it does not prove that the amount matches the source, reflects the latest policy, or belongs to a record this user may see. Look up whether `source_paragraph` actually exists in the document, compute totals and date ranges in code, and recheck authorization on the server. Do not treat model-generated confidence numbers as real probabilities without a calibration evaluation.

Design fields that expose failures. When evidence is missing, allow `status: unverified`, `missing_fields` and empty evidence rather than forcing invented required values. If null and empty strings mean different things, keep schema and task-code interpretations consistent. Use separate error codes for parsing failures, schema violations, evidence mismatches and policy violations to decide retries, human review or stopping.

Compare token costs and conflicts when schemas are repeated as prompt text versus combined with native API schema options. Even with structured output, evaluate normal·boundary·long-string·Unicode·escape cases and streaming completion. Limit retry counts and time; do not repeat the same incorrect result indefinitely. Contract-test schema-version changes with consumer code and prepare migration or rollback for previous versions.

Why does this happen?

Separate structural errors from factual and authorization errors to avoid overtrusting parser success and choose recovery paths appropriate to each failure.

When is it a problem?

Counting only valid JSON can record model-generated incorrect numbers, fabricated evidence, and unauthorized records as successes.

Common beginner misconceptions

Using a JSON schema or a low temperature does not guarantee that the output content is deterministic or factual.

How to verify it yourself

After the schema validator, chain checks for source existence, calculations, business rules, and authorization, and test that each error fixture stops in a distinct state.

In sampling comparisons, recording the seed can help rerun the same pseudo-random path. However, as PyTorch's official reproducibility documentation explains, complete reproducibility may not be guaranteed across different releases, platforms, or CPUs and GPUs. Also check whether the runtime actually supports and applies the seed, and whether request order under batching and concurrency affects the random state. Having a seed parameter name alone does not make a service deterministic.

A reproduction manifest includes the model and weight hashes, tokenizer and chat template, prompt, example, and schema hashes, generation config, input order, and runtime, library, driver, and hardware. Quantization, attention backend, and parallelism can also cause numerical differences. Preserve the container digest and environment, and link the resolved server config, rendered prompt, and raw output. Do not use a moving latest tag or main branch as a comparison baseline.

A deterministic algorithm option can improve repeatability, but it may raise errors on unsupported operations or change performance. If the research reproduction environment differs from the production throughput environment, specify both conditions and check the quality difference. When CPU and GPU results differ slightly, rather than insisting on exact string matches, decide in advance whether task correctness, schema, and critical failures must match and what numeric tolerance is allowed.

Do not select one seed’s good result as representative. Examine quality, schema compliance, diversity and p95 distributions across seeds or repeated runs, recording the worst subsets and variation. Reproduce bugs with exact seeds and stacks where possible, but do not erase production failures merely because they do not reproduce. After model/runtime updates, run both the previous seed set and new random samples to reduce overfitting to fixed fixtures.

Why does this happen?

Because besides random state, libraries, kernels, and parallel execution also affect generation results, so a seed alone cannot explain the cause of a change.

When is it a problem?

Approving reproducibility because a single run with the same seed matched misses failures and performance differences that appear with platform updates·concurrent load.

Common beginner misconceptions

Fixing the seed does not guarantee that text stays identical forever on every device and runtime version, or that the model's answers are accurate.

How to verify it yourself

Repeat the seed set on the exact manifest, and compare the task metric distributions across other machines and candidates, as well as the latency cost of deterministic options.

First pin the baseline to an immutable version and define gates before seeing results. Separate task correctness and evidence, schema and required fields, refusals and omissions, diversity target ranges, p95, tokens and errors. Set minimum criteria for normal, boundary and failure subsets so high overall averages cannot hide essential failures in number extraction, missing-evidence handling or safe refusals. Approve targets and judges before viewing candidate outputs.

Run each candidate multiple times using the same model, template, gold set, runtime, and hardware. When comparing prompt wording, do not also change the model, quant, or temperature. Link each output, schema error, latency, and token usage to the request manifest, and define a rubric and a process for handling disagreement in human judgments. Self-check or model judge results are only supporting signals; they do not replace calculations, evidence, or an independent novice reviewer.

After passing every offline gate, observe limited canary traffic while preserving the production distribution and privacy. Compare schema failures, holds, refusals, user corrections, p95, and downstream errors against the baseline. If a predefined stop condition is exceeded, return traffic to the previous prompt, template, and config automatically or through an approved procedure. Rollback readiness is proven only when the same failing input actually recovers on the previous version.

Evidence records the owner and reviewer, purpose and scope, exact source and version, raw result location, limitations, and decision date. Review it again when the model, runtime, template, schema, task taxonomy, or input distribution changes. Do not overwrite previous versions; manage deprecation and retirement dates. The final conclusion should not be “this prompt is better” but which gates passed under which exact conditions and which failures are on hold.

Why does this happen?

Because prompts and sampling change user behavior and downstream systems without code changes, they must be handled as deployable and recoverable units of change.

When is it a problem?

Overwriting a prompt after one good example loses regression coverage, template and sampling causes, and the recovery version, leading to lower standards in production.

Common beginner misconceptions

A prompt being a string does not make it safe to change immediately without review·version·canary, nor is it approved permanently and independently of model updates.

How to verify it yourself

Repeat the same manifest, run the subset gates and canary stop, then restore the same failing input with the previous version and record the evidence ID.

The official OpenAI function calling flow has distinct steps: tools are presented to the model, the application that receives the model's call executes the function code, and the result is passed to the next model request. The model producing a `send_email` JSON does not mean an email was sent or that the current user gained permission to send it. The dispatcher looks up only registered exact tool versions and hands a call to the executor only when name·arguments·caller context and policy all match.

Tool selection can also fail. The model may answer without any call, choose the wrong tool, or issue interdependent calls in parallel within the same turn. The application explicitly handles the zero, one, and many cases and determines whether parallel execution conflicts with the order of side effects. Instead of assuming “the model will pick one on its own,” contract-test normal, ambiguous, and conflicting requests.

Some model arguments are values the client must not send. If the model is asked to regenerate the current user ID, tenant, permission scope, or an order ID the server already knows, it can guess other resources. Inject these authoritative values from the authenticated session·server state, and let the model make only the limited choices the task actually requires. Tool names·descriptions should also state clearly when to use and not use the tool and what its output means.

Show distinct execution states to users: proposed, awaiting approval, running, succeeded and partially failed. Do not collapse them into one completion message. Verify success through authoritative API responses and resource state, not model-generated prose. Link run/call IDs, caller, tool/schema version, validated argument hash, policy, approval and side-effect IDs in audit traces.

Why does this happen?

Separating proposals from execution prevents model errors from directly causing external state changes or privilege escalation.

When is it a problem?

Passing tool call JSON directly to a function can let a single model output execute an unknown tool, act on another tenant's ID, or make a duplicate payment.

Common beginner misconceptions

A model that supports function calling is not an execution engine that understands and guarantees application permissions, business rules, or the actual success of a function.

How to verify it yourself

Send fixtures with unregistered names, another tenant's resources, and zero, duplicate, and parallel calls, and confirm 0 actual side effects and a clear error state.

OpenAI documentation explains that strict mode makes function calls conform to a schema and requires conditions such as `additionalProperties: false` on objects and required fields. This is a more reliable starting point than free-text parsing. Use enums, numeric ranges, and nested objects to narrow allowed structures, and clarify optional-field meaning. Check the API’s supported JSON Schema subset and strict-mode fallback behavior in actual responses.

Valid arguments do not imply valid actions. Even if `{invoice_id: "A-19", amount: 50000}` passes the schema, it does not establish that the invoice belongs to the current tenant, has not already been refunded, is within the user’s limit, or uses the correct currency. The server checks the current database version and caller permissions and returns distinct errors for stale resources and business conflicts. Do not authorize using role or owner fields supplied by the model.

Narrow tool interfaces reduce invalid states. Use separate `get_invoice` and `request_refund` tools instead of `execute(action, payload)` or raw SQL. Bind an invoice ID already selected in the UI to server context, and delegate only necessary parameters, such as a reason category, to the model. Two safe steps that always run consecutively may be combined into one transaction tool, but keep operations with different approval boundaries or rollback behavior separate.

Outputs also require schema and semantic validation. Check tool-result field types, sizes, and required values, and sanitize HTML·scripts·external instructions and secrets·PII. Even with `success: true`, re-query actual resource IDs and state when needed. Give the model only minimal fields and stable error codes needed for its next decision; do not expose raw internal stack traces·credentials·entire database rows.

Why does this happen?

Structural validation and task authorization prevent different failures, so connect the two to avoid mistaking valid JSON for safe execution.

When is it a problem?

If passing the schema alone counts as success, another user's valid ID, an amount outside policy, and an already processed resource all execute as normal calls.

Common beginner misconceptions

Strict mode improves schema compliance of model arguments, but does not certify factual content, permissions, current state, or side-effect safety.

How to verify it yourself

Run correctly formatted fixtures with a different tenant, an excessive amount or a stale version. Confirm that domain and authorization gates each reject them after schema validation.

A larger tool catalog increases both the candidates the model may select incorrectly and the capabilities an attacker may target. Expose only namespaces and tools needed for the current turn. Separate read-only queries, additive writes, destructive actions, and arbitrary network or code execution into risk tiers. Do not give production agents general-purpose shell, browser, or SQL access without a sandbox and allowlist. Even when using tool search, verify trust in the discovered server and tool version.

The MCP 2026-07-28 tools specification states that servers perform input validation, access control, rate limiting, and output sanitization, and that clients consider confirmation of sensitive operations, tool-result validation, timeouts·auditing. Tool annotations are untrusted hints unless they come from a trusted server. Do not skip actual endpoint review·approval merely because a remote server claims `readOnly`.

Authentication identifies callers, and authorization determines which actions they may take on current resources. Even with transport frameworks such as MCP authorization, applications must implement invoice- and document-specific policies. Bind access tokens to target resources, audiences, and scopes, and do not pass another server’s token unchanged downstream. Do not place master credentials in browsers, model context, prompts, or tool results.

Rate and resource budgets are also permissions. Read-only searches dumping entire customer databases or calling arbitrary URLs can harm confidentiality·availability. Limit query scope, rows·bytes, egress domains, result sizes returned to context, and per-user·run quotas. Check versions and expiry during credential rotation·revocation, server compromise, and tool-list updates so existing runs do not automatically inherit new permissions.

Why does this happen?

Because even if prompt injection succeeds, keeping the capabilities the agent can see and execute small limits the actual scope of damage.

When is it a problem?

A master token and broad SQL·URL tools can let a single line in an untrusted document trigger tenant-wide reads, external transmission, or resource exhaustion.

Common beginner misconceptions

A successful MCP connection, an OAuth login, or a single readOnly annotation does not automatically authorize every tool argument·resource or downstream permission.

How to verify it yourself

Call with an unauthorized user and tenant, an out-of-scope tool, bulk reads, and disallowed egress, and confirm that the server returns 0 data with 0 side effects and enforces rate limits and auditing.

Tool failures are not all the same natural-language error. The model may correct malformed arguments to match the schema, while 429 and transient 5xx errors may permit bounded retries after backoff. Errors 401 and 403 concern credentials and permissions, and business conflicts such as 409 may indicate changed resource state. Define maximum retries, human handoff, and publicly disclosed details for each error class in the contract.

Use idempotency keys for mutation requests. Bind each key to the run·logical action and have the server return the previous response·side-effect ID. Retrying a payment with a new key after a timeout may duplicate a first request that actually succeeded. Resolve unknown outcomes by querying a status endpoint or resource state. Do not casually promise exactly-once execution; document duplicate detection and compensation procedures.

A tool result is an observation the model uses for its next plan, so clearly include success or failure and authoritative state. Distinguish protocol, tool-execution, and domain errors so the model handles fixable missing fields differently from permission failures. Returning errors containing internal topology, stack details, or secrets may help attackers probe the system. Give users an error ID and a recoverable next action.

Do not hide partial batch failures. If only one of three items ran, the result is not completion; record lists of executed, failed, and not_attempted items along with side-effect IDs. For each tool, indicate whether compensation is a true reversal or a separate opposing transaction, or whether the action cannot be undone, as with external email. The acceptance test checks the full ledger so the agent cannot narrow the goal or drop failed items and report success.

Why does this happen?

Because network timeouts and model repetition can occur in normal operation, duplicate harm results unless mutations are made into a safely retryable protocol.

When is it a problem?

Retrying every error three times can cause authorization-bypass attempts, duplicate payments and runaway request rates, while incorrectly marking partial success as complete.

Common beginner misconceptions

Writing “Do not run duplicates” in the agent prompt does not replace server idempotency keys, status lookups, and transaction design.

How to verify it yourself

Disconnect before the response, resume the same logical action, and verify that exactly one side-effect ID exists and batch succeeded·failed·unexecuted states are accurately visible.

An agent looks like a loop that plans toward a goal, runs tools, and uses observations to decide the next action. In production, the loop’s state must not live only in chat text. Keep the immutable goal and acceptance criteria, caller and tenant, plan revision, current step, tool calls and results, approvals, and side-effect IDs in a durable store. The model context is a necessary view; the system of record is the application state.

Each step has explicit state transitions such as pending·approved·executing·succeeded·failed·held. If the process crashes, find the mutations that already succeeded from the checkpoint, reuse them via their idempotency keys, and continue only the steps that have not yet run. Do not put “probably succeeded” into the next prompt; query the authoritative API or resource version. If the tool·policy version has changed at resume time, new approval·evaluation may be required.

Parallel agents and human edits create stale state. Detect conflicting changes with resource ETags·versions, compare-and-set, or locks. Bind approvals to resource versions so the first agent does not execute against a balance·document that another agent has since changed. Record observation timestamps and provenance, and validate cache freshness·scope.

Separate memory and audit purposes. Do not treat long-term memory such as user preferences as task authorization or established fact. Keep input·schema·policy decisions and state IDs needed for reproduction in traces, without retaining raw secrets·personal information·all reasoning indefinitely. Define retention·access·redaction policies and incident-related legal requirements. Use tool·policy logs as primary evidence rather than model output.

Why does this happen?

Because agents cross multiple stages and failures, they must manage actual state durably to recover from duplicate execution, partial completion, and stale observations.

When is it a problem?

Using chat history alone as the ledger means that after a crash no one knows what ran, so the same action is repeated and the model's incorrect completion messages are stored as facts.

Common beginner misconceptions

Long context or memory features do not automatically provide database transactions, resource versions, or an auditable side-effect ledger.

How to verify it yourself

Immediately after the second step runs, stop and resume the process, and test for 0 duplicate mutations and detection of the exact current state and changed-resource conflicts.

Agent budgets are not just token limits. Set separate maximum model turns, tool calls, wall-clock time, model and API costs, read bytes, output tokens, and mutation counts. Define them in advance based on user-requested task size and risk tier, and separately control authority to increase limits. Do not omit remaining work and mark completion merely because the budget is nearly exhausted.

Loops are hard to detect from the tool name alone. Track patterns of identical argument hashes, identical errors or observations, no change in resource state, and changes only in plan wording. Retrying a 403 with different IDs, or slightly rewording a search query to read the same document, is also repetition without progress. Combine per-tool retry counters with a progress metric for the whole run, and block new calls at the threshold.

The stopped state provides verified progress so far. Show the goal, the succeeded, failed, and not_attempted steps, actual changes and side-effect IDs, the last error class and evidence, and the exact next action for a person to approve or modify. Do not request raw chain-of-thought; pass on only auditable facts and decisions. After handoff, the new caller re-verifies permissions and context.

A kill switch must block new tool calls and credential issuance through server policy, not just a UI button. When canceling an in-progress call, check whether the downstream action already executed, and drain or revoke batch queues and tasks. Record timeouts, user cancellations, incident stops and normal successes as distinct terminal states. Regularly practice the kill switch, resumption and rollback to the previous orchestrator.

Why does this happen?

Because stochastic planning and external failures can cause loops, and without explicit limits and handoffs, costs, resources, and side effects keep growing.

When is it a problem?

If max steps are written only in the prompt, the model may ignore the limit or the application may keep calling, and remaining items may be hidden just before the limit to create a false completion.

Common beginner misconceptions

An agent declaring completion or apologizing for the same error does not verify actual progress, budget, or a safe terminal state.

How to verify it yourself

Return the same 403, timeout, and unchanged search results repeatedly, and verify that new calls are blocked at the threshold and that an accurate partial ledger and handoff are produced.

Approval UI should explain actual effects rather than function names. Instead of “allow send_email?”, show exact recipients, subject, body, attachments, external domains, and irreversibility. For payments, show currency, amount, payee, fees, and before-and-after changes; for deletion, show total targets, retention, and recoverability. Summarized batches need a path to review the full list, filters, and totals.

Prevent time-of-check/time-of-use problems where arguments change after approval. Store the authenticated caller, tool/schema version, canonical argument hash, resource version, expiry and one-time nonce in the approval record. Execute only when all match the current request; require fresh approval if recipients, amounts, files or resources change. Do not trust a client boolean `approved` or a model's statement that the user agreed.

Set automation scope through risk-based policy. Public reads may be automatic; private reads need authorization and audit; reversible internal drafts need previews and idempotency; external sending, monetary actions, deletion, publication and permission changes may require explicit approval. Apply two-person rules and delays to high-risk or large actions. Small tools, safe defaults and dry runs reduce unnecessary confirmations.

Test the approval mechanism itself against attacks. Check expiry and reuse, a different caller, a one-character argument change, stale resources, clickjacking and hidden targets, and execution after cancellation. When a user rejects an action, store the decision scope so the model does not reword and propose the same action again. For irreversible actions such as external email, do not promise recall; separate test recipients and drafts from the final send.

Why does this happen?

Because approval works as a real check of intent and risk only when what the person saw is the same as what the executor runs.

When is it a problem?

If the approval token is not bound to the arguments, the model can change the recipient or amount after approval, or reuse an old approval for a different action.

Common beginner misconceptions

Showing a confirmation dialog once or having the client send approved=true does not guarantee server-side authorization, current state, or meaningful consent.

How to verify it yourself

After approval, change the recipient, amount, resource version, and caller one at a time, and test whether the executor requires re-approval with 0 side effects.

In direct prompt injection, the user tries to change the operating instructions; in indirect injection, strings inside retrieved documents, webpages, emails, or tool results steer model behavior. Blocking only the phrase “ignore previous instructions” misses other languages, encodings, images, and more natural requests. Mark system, user, and data boundaries, but do not assume the model will honor them perfectly; restrict capabilities at the execution layer.

Do not expose unnecessary secrets or broad tools to the model. The executor uses a secret vault only for exact destinations·scopes and never returns secret values to model context. Apply egress allowlists, DNS·redirect validation, URL-scheme·IP-range restrictions, and response-size limits. Separate customer reads·arbitrary POST·credentials so one agent does not possess all three, even when a retrieved document instructs it to send a customer list to an external URL.

Even a trusted API can return a tool result containing an attack payload. Sanitize and quote HTML and Markdown links, hidden instructions, and file content, and attach source and trust labels. Passing the result schema does not promote instructions inside text fields to system commands. Every subsequent tool call must again satisfy the original user goal, server policy, and authorization. Do not directly execute tool names or arguments recommended by results.

Following OpenAI safety best practices, red-team adversarial inputs intended to break the system as well as representative user behavior. Measure whether unauthorized data reads, secret exposure, non-allowlisted egress, approval bypass, and mutations actually remain at 0, rather than whether the model says it refuses. Also evaluate encoding·multi-turn·tool-result·partial-permission·long-context attacks and false positives that obstruct legitimate tasks.

Why does this happen?

Because model-level defenses can fail, limiting available capabilities and data paths is necessary to prevent actual harm even when an attacker injects instructions.

When is it a problem?

With only a phrase filter and an “ignore external instructions” prompt, variant attacks can combine broad tools, secrets, and egress to send data outside.

Common beginner misconceptions

A model saying “I refuse” to an attack prompt does not mean background tool·network·data access was safe.

How to verify it yourself

Plant encoded exfiltration instructions in untrusted documents and tool results, and confirm in traces that secret reads, disallowed egress, and unapproved mutations are all zero.

Pin the model and prompt, exposed tools and schemas, dispatcher and policy, credential scope, runtime and gold trace set in the evaluation manifest. Include normal success, ambiguous goals, missing inputs, 401/403, rate limits and timeouts, malformed outputs, duplicate calls, stale resources, direct and indirect injection, and partial failures. Disclose candidate-specific tool or permission differences and preserve a common baseline.

Split metrics into task success·correct final state, schema·authorization, unauthorized read/write·exfiltration·approval bypass, duplicate mutations, max steps·cost·p95, and human handoff completeness. Mandatory safety gates such as unauthorized actions and duplicate mutations must be zero and cannot be offset by average success. Set thresholds and reviewers before seeing results, and run at least several times to observe variation.

A trace links caller and goal through calls, policies, approvals, tool results, side effects, and terminal state. Redact personal information and secrets, while retaining source, argument, and result hashes and resource versions to reproduce incidents. Start in shadow mode or a sandbox without real mutations, then verify the kill switch, queue draining, and credential revocation in a limited canary. Feed production feedback, near misses, and errors back into the evaluation set.

Rollback does not mean only switching back to the previous model. Restore the prompt, tool registry and schema, dispatcher, policy and credential configuration, and state migration to the previous compatible stack. Test whether unauthorized actions, duplicates, p95, and handoff recover on the same failure trace. Following NIST principles for pre-deployment testing, human oversight, change management, and incident management, record the owner, limitations, approval scope, and re-review triggers as evidence.

Why does this happen?

An agent connects the model, tools, permissions, and external state, so general answer quality alone cannot determine its actual risk and recovery capability.

When is it a problem?

Raising only the average task success can hide even a single unauthorized transmission, duplicate payment, or kill-switch failure, and the previous compatible stack is lost as well.

Common beginner misconceptions

Passing a model benchmark or a few prompt-injection examples does not certify application-wide tool permissions·failure handling·canary or incident readiness.

How to verify it yourself

Run fixed normal, attack, and failure traces at least three times, and use the side-effect ledger to verify every required gate, the kill switch, and rollback to the previous stack.

Start with users’ repeated tasks, not a model list. Observe whether agents enter invoice suppliers, dates and totals, developers draft boilerplate and tests, or meeting organizers extract action items from transcripts. Record input formats, frequency and variation, current time spent and common errors. Document both the time automation may save and the risks it may introduce.

Express success as reviewable output. Instead of “a good summary,” require that all mandatory clauses, numbers, and exceptions are present and the source paragraph can be found; instead of “accurate OCR,” require that the total, tax, and date fields each pass a threshold. Before seeing results, the owner approves the critical fields that must not hide in averages and the boundary between automatic processing and human hold.

Do not extend one model’s capability to the entire workflow. A vision model describing a receipt is different from accurate field-level OCR, and even with an ASR transcript, action items and speaker attribution are separate stages. Even when a code model writes tests, compiler, security, and repository review are still needed. Design the intersection of specialist models, deterministic code, and human judgment.

Start small pilots on low-risk, repetitive tasks with clear validators. Save time through drafts, recommendations and review queues before automating final external sending, payments or merges. Measure the existing human workflow's quality, time and corrections as a baseline. Compare total processing-time savings alongside review burden and newly introduced errors.

Why does this happen?

Because only with user outcomes and error costs can different model outputs be compared in terms of the same task value and risk.

When is it a problem?

Starting from one natural-looking demo output can lead to failures on actual distributions of empty inputs·tables·noise·dependencies and increase human review time.

Common beginner misconceptions

Being a small local model does not by itself make the task scope safe or automatically resolve verification, permissions, privacy, and human accountability.

How to verify it yourself

For 20 current manual cases, record inputs, results, time spent, correction types and critical errors, and compare the same items for the candidate.

During source ingestion, check file type and size, source and owner, consent, license and malware. Separate image rotation/cropping, audio resampling/segmentation, text normalization and code-context selection into deterministic preprocessing. Store exact tools, parameters and output hashes to explain whether source, preprocessing or model changes caused different results.

Preserve source coordinates in model output: OCR fields link to page·bounding box, ASR sentences to audio start/end·speaker, summary claims to paragraph IDs, and code patches to base commit·file lines. The pipeline attaches this provenance through authoritative mappings, not IDs invented by the LLM. The next stage receives schema·source references and validation state as well as text.

Each stage clearly defines success·hold·partial·error. Do not turn corrupt files·unsupported languages, truncation, low confidence, or rule conflicts into empty strings or guesses. Record timeout·retry and partial batch state, and give the human queue the original·candidate·error reason and an editing UI. Save corrections as a separate version with the reviewer·timestamp instead of overwriting the original.

Record preprocessing, model and runtime, prompt and schema, validator, glossary, pronunciation dictionary, and UI versions in the deployment manifest. Even when only one stage changes, rerun full regression on downstream metrics. Preserve the previous compatible pipeline and intermediate schema migrations, and confirm that the same failing sample recovers after rollback. Trace the full lineage from final assets and records by evidence ID.

Why does this happen?

Because when multiple AI and code stages are chained, the final error alone does not reveal the cause, so source, version, and stage state are essential.

When is it a problem?

Copying only the intermediate text leads to misdiagnosing OCR digit errors as summarization-model problems and makes it impossible to find the original location, preprocessing, and editor, or to roll back.

Common beginner misconceptions

A final output that passes the schema or reads naturally does not mean the upstream stages and provenance are correct.

How to verify it yourself

From one final field, sentence or audio item, trace back through the source page/span, preprocessing, model and validator, corrections and approval, and rollback artifacts.

Provide a narrowly scoped target issue, base commit, modules to change, and acceptance criteria. The model does not know repository-wide invariants, dependency versions, or hidden configuration, so it can invent nonexistent APIs or unsafe defaults. Accept output as patch suggestions, and reject unexpected files, generated binaries, and large lockfile changes. Keep each diff small enough to review in one pass.

Use a disposable sandbox with minimal filesystem and network access. Do not put repository secrets, production credentials, personal information, or signing keys in prompts or the environment. Control dependency sources, hashes, and egress so installation scripts and builds cannot send data externally or modify host paths. Read the exact target and effect of generated commands instead of blindly pasting them into a terminal.

Run compiler and typecheck, formatter and lint, unit and integration tests, and existing regressions on a clean checkout. When a model writes both the implementation and its tests, the same misunderstanding can be copied into both, so use the spec, existing behavior, and human-written boundary and negative fixtures. Following the secure development practices recommended by NIST SSDF, integrate prior vulnerability regressions, input fuzzing, and static/dynamic analysis into the pipeline in proportion to risk.

Reviewers check requirements, architecture, errors·authorization·data handling, and dependency licenses. Passing tests does not guarantee maintainability, task intent, and security. After merge, observe production-like inputs and metrics under feature flags·canary, preserving issues·base commits, patch·artifact hashes, toolchain·tests, and approvers. Roll back to previous artifacts·configs on regression.

Why does this happen?

Because generated code is an executable that handles real permissions and data, every check in the existing secure SDLC must be applied even more strictly than for natural-language answers.

When is it a problem?

Taking model-written code·tests from the same prompt and deploying them immediately misses shared misunderstandings, secret exposure·dependency risks, and production-only errors.

Common beginner misconceptions

Code that compiles or a model that calls it “safe” does not mean authorization, business invariants, and vulnerabilities have been verified.

How to verify it yourself

Run builds and existing, negative, and security tests in a clean sandbox, then verify that the code owner approved the minimal diff, dependencies, and rollback together with the evidence.

Hugging Face's official task description defines summarization as a sequence-to-sequence task producing a shorter version of a long text, distinguishing extractive and abstractive approaches. Extractive summarization selects original sentences, adding less new wording but potentially breaking context. Abstractive summarization generates readable new text but may invent connections or causality absent from the source. First choose the type and permitted editing scope for the intended use.

Long inputs may exceed model context limits. Use token maps to see which sections truncation removes and chunk according to document structure. Summarizing chunk summaries can accumulate small omissions and errors and merge exceptions from different sections. Preserve tables, footnotes, appendices, and the latest revision, and attach a source-paragraph ID to every summary claim.

Separate evaluation into core coverage, faithfulness, forbidden additions, critical omissions and readability. Compare dates, amounts, units, proper nouns, negations, conditions, exceptions and disclaimers using parsers and source lookup. An overlap metric such as ROUGE or a single model judge cannot substitute for these task-risk checks. Review source and output side by side, recording pass, edit or reject for each claim.

Classify user corrections as matters of preference or factual errors. Fixes for excessive length or tone and fixes for missing conditions or invented facts have different root causes. Evaluate subsets by document type, length, and language, along with empty, duplicate, and contradictory sources. Pin the exact source, model, prompt, chunk, and generation config, and recover documents that fail on the candidate with the previous summarization pipeline.

Why does this happen?

Because a summary deliberately reduces information, the task must define what may be omitted and ensure that nothing absent from the source is invented.

When is it a problem?

Evaluating only fluency and length can award high scores despite one omitted amount, exception or negation, potentially reversing an actual decision.

Common beginner misconceptions

An abstractive summary that reads better than the original or has a higher overlap score does not mean every claim is supported by the source.

How to verify it yourself

Open the source paragraph for each summary claim, compare numbers, conditions, and exceptions, and confirm that the truncation map includes every required section.

Translation converts text sequences into another language. Define exact source and target languages, scripts, locales, domain, tone, and audience. Even for English-to-Korean translation, contracts, UI, technical documents, and marketing differ in terminology, honorifics, and sentence structure. Check the model card’s supported languages and an actual domain gold set, and put unsupported or mixed-language inputs on hold.

Version approved glossaries, product and API names, and do-not-translate lists. Automatically compare counts and identities of numbers, currencies, units, dates, placeholders, HTML tags, Markdown links, and code spans. Check that BiDi text, Unicode normalization, and line breaks do not break output formats. Validate table, caption, and cross-reference numbering at document-structure level beyond sentence translation.

Human reviewers judge accuracy, fluency, terminology, style and locale, and critical omissions and additions segment by segment. Specialized fields need reviewers who know the domain and language. Back-translation or agreement between different models can offer clues to errors, but it can also reproduce the same wrong meaning, so it does not replace source-target comparison. Legal, safety, and medical output remains a draft until approved.

Check Document consistency: terms, actors, and pronouns may change between chapters, while table headings or footnotes may disappear. Use translation memory, glossaries, and context windows, checking that older approved phrases do not conflict with new source revisions. Preserve source/target hashes, segment IDs, model/configuration, automatic flags, and reviewer edits, and run regressions by language and domain.

Why does this happen?

Because translation is not word substitution but preserving document and domain meaning in another language, fluency alone cannot justify approval.

When is it a problem?

Checking only sentence-level fluency can miss inconsistent product terms, placeholders, table numbers and contractual negations or exceptions across the document.

Common beginner misconceptions

A back-translation that resembles the source or a high bilingual model score does not make professional review and comparison with the source unnecessary.

How to verify it yourself

On a gold document, check both automated tests for glossary, number, and placeholder parity and a professional reviewer's segment error taxonomy and document consistency.

OCR output differs from a multimodal model’s document description. Describing something as a “restaurant receipt” does not guarantee vendor·date·tax·total accuracy. Tesseract’s official guide explains how rescaling, binarization, noise removal, skew·borders, and page segmentation affect quality. Classify image DPI·focus·rotation·background and table·single-line·sparse layouts.

Version preprocessing recipes and preserve both raw images and the images actually seen by OCR. Auto-deskew may crop small text and thresholding may remove thin strokes, so do not apply one setting to every document. Record exact page-segmentation mode, language data, and OCR version, evaluating receipt·form·table profiles separately.

Separate character·word errors, field exact match, and table structure metrics. One digit in a total, a bank account·invoice ID, or a checkbox state can be critical even with low overall CER, so use independent gates. Preserve bounding boxes·pages, and validate subtotal+tax=total, date ranges·ID checksums, and allowed currencies in code. Send conflicts·low-confidence cases to a human queue with the source crop.

Vision-LLM post-processing can make OCR corrections read naturally, but may also introduce values absent from the source. For each correction, record the original OCR, normalized value, and rule or reviewer. Include actual scanners and mobile cameras, blur, shadows, rotation, small fonts, and personal-information redaction in the test set. Restrict access to and retention of raw documents, and roll back failed candidate preprocessing or models to the previous pipeline.

Why does this happen?

This is because OCR errors arise from image strokes·layout and preprocessing, and a single digit can change task outcomes, requiring field-centered evaluation.

When is it a problem?

Looking only at overall CER and natural-language explanations can hide errors in totals, IDs and table columns in the average, while LLM post-processing plausibly finalizes incorrect values.

Common beginner misconceptions

Using a larger vision model or stronger binarization does not automatically solve every document layout, small text, and field calculation.

How to verify it yourself

For each Document type and capture condition, inspect raw images/crops for critical-field exact match, bounding boxes, and calculation conflicts, and track why humans make corrections.

In Hugging Face's official description, the ASR task maps a speech signal to text output. Do not generalize success on one audio file to an entire meeting room. Record sample rate, channels and codec, language and accent, microphone distance, noise and overlap, and segment length. Keep the recipe for stereo-to-mono conversion, resampling, and silence splitting, along with the exact ASR model and decoder, in the manifest.

WER summarizes substitutions, deletions, and insertions relative to the number of reference words, but varies with normalization. Pin punctuation, number-spelling, and casing rules, and preserve both raw and normalized transcripts. Separately measure product names, people’s names, dates, amounts, negations, and commands using critical-term recall and exact match.

Diarization and timestamps are separate error sources. Incorrect speaker attribution changes action owners, and overlap can erase a short objection. Link segment start/end and speaker IDs to transcript and summary claims so reviewers can play the source audio. If an LLM changes numbers after ASR using context, do not finalize them without source-audio verification and reviewer approval.

Meeting workflows need action-item precision and recall, reviewer correction time and final approval as well as transcript readability. Include subsets with noise, remote speakers, code-switching, long silence and empty or corrupt audio. Apply consent, recording notices, voice-data access and retention controls, and rerun previous failing clips and the full regression suite after model or vocabulary updates.

Why does this happen?

This is because even with the same WER, errors in names·numbers·negation and speakers cause different business harm, so source span and critical term evaluation are needed.

When is it a problem?

Clean-studio WER alone misses overlapping speech, noise, proper names, and incorrect action owners in real meetings; summaries may hide those errors.

Common beginner misconceptions

A readable transcript, or an LLM correcting sentences after ASR, does not mean the numbers, speakers, and intent of the original audio have been recovered accurately.

How to verify it yourself

Measure WER, critical terms, speakers, timestamps, action items and human correction time separately on gold audio covering actual microphones and noise conditions.

Hugging Face's official TTS task description explains generation of natural-sounding speech from text and the availability of models for different languages and speakers. Pin the exact model, speaker/reference, language, sample rate, text normalization and synthesis configuration. A returned audio array does not guarantee pronunciation in the target language, consistency across long paragraphs or clarity on actual playback devices.

Include numbers, dates, units, English acronyms, Korean and foreign names, URLs, code, punctuation, and quotations in test scripts. Evaluate pronunciation errors, intelligibility, unexpected pauses and speed, clipping, silence, and loudness separately. Check waveform peaks and audio duration with code, and have people listen through headphones and phone speakers and with background noise. An ASR round-trip is an auxiliary signal, not a substitute for independent listening.

Voice-cloning references require explicit speaker consent, licensing and allowed purposes, retention, deletion, withdrawal, and compensation terms. Do not mistake publicly available audio of colleagues or celebrities for permission. Strengthen explicit approval, disclosure, access controls, and auditing for calls and public announcements with fraud or impersonation risks. Treat voice files and embeddings as sensitive assets comparable to biometric data.

Output audio appropriately discloses whether it is synthetic and its publisher context. Link script·model, voice authorization, edit actions, and the final asset hash as provenance. Content Credentials such as C2PA can provide source·history trust signals, but they do not prove that the pronunciation or content itself is accurate. Collect issues from a canary audience and keep the ability to roll back to previously approved audio.

Why does this happen?

Because TTS produces audible media associated with human identity, it must address actual pronunciation and audio quality, consent, and impersonation risks in addition to text accuracy.

When is it a problem?

Listening to only one naturalness sample means number·name errors, clipping in long audio, and reference-speaker rights·disclosure issues are discovered after deployment.

Common beginner misconceptions

Public audio or local synthesis does not automatically grant consent·a license to use a voice clone or the right to publish it.

How to verify it yourself

Check critical pronunciation and audio metrics, independent listening, voice consent, allowed purpose, deletion, disclosure, and asset provenance.

An evaluation manifest contains the raw source set, expected stage outputs, exact preprocessing, model, and validator, and the reviewer rubric. It includes not only normal cases but also corrupt, long, table, and small-text inputs; dialect, noise, and overlap; numbers, names, and negation; insecure code; and cases with missing voice rights. Run the same split at least several times for each candidate, and prevent data leakage and benchmark-only tuning.

Split metrics into stage quality, critical fields·terms, end-to-end task success, p50·p95, resources, and human correction rate·time. Mandatory gates such as code security regressions, invented facts in summaries, OCR totals, ASR action owners, and TTS consent may require zero failures. Record disagreements between automatic scores and human rubrics, and do not lower thresholds after seeing the results.

As W3C WCAG guidance describes, prerecorded synchronized media should provide captions that include not only dialogue but also important sound information. Do not publish ASR drafts as is; review speaker labels, time sync, and meaning. Transcripts and captions provide hearing access and also serve as search and review evidence, but they are subject to privacy and retention rules. Provide the necessary alt text and source views for images and documents so reviewers can check the originals.

After offline gates, observe actual user corrections·drop-off and incidents in shadow·limited canary. Roll back to previous preprocessing/model/validator/UI and approved code·document·audio assets, then test recovery from the same failures. Record owner, limitations, rights·consent expiry, and re-review triggers such as input·model·policy changes. The overall conclusion identifies the exact workflow version and task-specific pass·hold states.

Why does this happen?

With multiple stages and people involved, operational success depends on correction burden, accessibility, rights and complete compatible rollback as well as final quality.

When is it a problem?

An average end-to-end score can hide critical-stage errors, increased human corrections, caption and voice-consent issues, and incompatibility with the previous schema.

Common beginner misconceptions

A high public benchmark score for each model or a successful demo of each stage does not mean the actual connected pipeline, human UX, and publishing rights have been approved.

How to verify it yourself

Check the stage, critical, correction, and p95 gates and captions and rights for the same source set at least 3 times, and recover from failure with the previous full pipeline.