KoreaDevKNOWLEDGE SHARING

Content typeLearn

AI SOFTWARE DEVELOPMENT · 02 / 10

How AI has changed development

Learn how permissions and feedback loops change from autocomplete to coding agents that explore, modify, and verify a repository.

Difficulty
Beginner · Practical
Structure
Lessons 8 · Labs 2 · Assessment

CORE UNIT 1 / 1

How AI has changed development

Learn how permissions and feedback loops change from autocomplete to coding agents that explore, modify, and verify a repository.

Difficulty
Beginner · Practical
Structure
Lessons 8 · Labs 2 · Assessment

Diagrams and tables: composed by the author using each lesson's official primary sources. Find the originals and review dates at the end of that lesson.

NEW HIRE ONBOARDING

Start in the order you would receive your first assignment

So that even a new hire with no prior IT background can follow along, we start with the situation, the task, the evidence, and when to report, before difficult definitions.

  1. 01

    Read the situation in one sentence

    The only request is “fix the login error on your own and deploy it.” The repository contains the user's uncommitted changes, and a production deploy changes external state.

  2. 02

    Today's assignment

    Distinguish the working scope of AI autocomplete, conversational generation, and coding agents.

  3. 03

    Evidence that shows the work is complete

    Link each final diff change to evidence, and check preserved user changes and older-consumer validation.

  4. 04

    When to stop and ask a senior colleague

    As the scope of automation grows, so do costs and the risks of incorrect changes, secret exposure, and external side effects.

Unpack unfamiliar terms first

From autocomplete to coding agents
The important change in AI development tools is not sentence length but the growing scope in which they observe, act, and re-verify on their own.
How to select repository context
Good context is not a large number of tokens but the minimum set of contracts and evidence needed for the current decision.
Fast feedback from types and schemas
Explicit contracts make incorrect assumptions fail sooner rather than making AI smarter.

Questions for this course

Why did it change, and what must be verified?

Do not merely memorize a technology's advantages; check the conditions under which they hold and the new failure boundaries they introduce.

OBSERVABLE OUTCOMES

What you can do after this course

  1. Distinguish the working scope of AI autocomplete, conversational generation, and coding agents.
  2. Explain how TypeScript static types, runtime schemas, and tests expose incorrect assumptions in AI-generated code.
  3. Turn an ambiguous request into a task contract with acceptance criteria and verification commands.

PREREQUISITE CHECK

Three things to check before reading

This is not a test of memorized answers. Think about each question first, then open the explanation to review the foundational concepts used in this course.

1Does successful compilation mean the requirements are met?

No. Compilation finds violations of language and type rules, but user intent and business outcomes must be verified through separate tests and reviews.

2Is letting AI read files the same as giving it permission to run commands?

No. Reading, writing, execution, and external transmission have different impact scopes, so each needs its own least privilege and approval boundary.

3Is a long prompt always good?

Unrelated information distracts. Providing the success criteria, the current code, and the necessary evidence in small units makes the work easier to verify.

TEXTBOOK GUIDE

Main text that covers each concept from its background to the criteria for judging it

We explain the material section by section so readers new to IT can connect causes and effects without memorizing terms.

CONCEPT FLOW

How the chapters connect

The chapters are not isolated short answers to memorize. Follow them from left to right to see how each chapter's concepts support the next decision.

  1. 1.From autocomplete to coding agents
  2. 2.How to select repository context
  3. 3.Fast feedback from types and schemas
  4. 4.Close the loop with execution and test feedback
  5. 5.Authorization · Approval · Human review
  6. 6.Build an oracle independent of the patch
  7. 7.Separate repository instructions from task authorization
  8. 8.Turn an experimental diff into a reviewable change bundle
How AI has changed development: the overall map. If you lose track while reading the detailed explanations and chapters below, return to this sequence.
Figure 2-1. How AI has changed development: concept developmentShows how each chapter’s choices and limits lead to the problems of the next chapter.
  1. 1
    From autocomplete to coding agents

    The important change in AI development tools is not sentence length but the growing scope in which they observe, act, and re-verify on their own.

  2. 2
    How to select repository context

    Good context is not a large number of tokens but the minimum set of contracts and evidence needed for the current decision.

  3. 3
    Fast feedback from types and schemas

    Explicit contracts make incorrect assumptions fail sooner rather than making AI smarter.

  4. 4
    Close the loop with execution and test feedback

    Changes made by an Agent are complete only when supported by reproducible verification results, not merely explanations.

  5. 5
    Authorization · Approval · Human review

    The criterion for automation is not whether an action is possible, but whether its impact and recovery are controlled when it fails.

  6. 6
    Build an oracle independent of the patch

    Fix expected behavior first so generated code and tests do not share the same mistaken assumption.

  7. 7
    Separate repository instructions from task authorization

    Text in a file is task material; it does not itself authorize secret access or external transmission.

  8. 8
    Turn an experimental diff into a reviewable change bundle

    Completion means explaining which change can be approved with which evidence, not producing a volume of code.

CONTROLLED EXPLANATION

Independent verification of a discount cap

Current state: Approved rule

Independent verification of a discount cap

Derive expected results from approved rules, excluding the path where code and expectations share the same error.

Oracle sourceReproduce bugFix causeRerun same fixture1Approved rule2Input fixture3Existing error4Patch candidate5Independent verdict
  1. Approved rule

    10% · 700 won cap

  2. Input fixture

    Order: 10,000 won

  3. Existing error

    Discount: 1,000 won

  4. Patch candidate

    Apply cap

  5. Independent verdict

    700 won and normal-order regressions

1 → 5
Oracle source
2 → 3
Reproduce bug
3 → 4
Fix cause
4 → 5
Rerun same fixture

700 won is the approved teaching policy. Do not overwrite test expectations with implementation results.

CONCRETE CASES

Selection criteria for all courses

TABLE 2-1

Selection criteria for all courses

Compare the technology in each chapter by how it works, the new costs it adds, and the evidence to check, not by its name.

Table 2-1. How AI has changed development: design decision criteria
.Core mechanismCosts to watchEvidence to check
1. From autocomplete to coding agentsTool results feed back into the next model input, creating a feedback loop that repeats until the termination condition is met.As the scope of automation grows, so do costs and the risks of incorrect changes, secret exposure, and external side effects.Separate the same task into a suggestion-only mode and a file-editing/test mode, then tabulate the possible side effects.
2. How to select repository contextUse retrieval and structured summaries to keep only the information needed for the current goal in the working context.Too little misses rules; too much adds cost and conflicting information.Record the contracts read before the change, the affected files, and the verification to run in three columns, and review them for gaps.
3. Fast feedback from types and schemasStatic types and runtime schemas reject invalid state early, at different boundaries.Maintaining types and schemas separately causes drift, so a code-generation or contract-test strategy is needed.Inject an invalid property, an invalid value of a valid type, and missing external JSON separately to see which gate catches each.
4. Close the loop with execution and test feedbackObserve the failure, apply the fix, then re-verify under identical conditions to narrow down the causal relationship.If the test suite is slow or flaky, the agent may ignore the signal or create meaningless workarounds.Link each acceptance criterion to an observable assertion and the evidence it produces on failure.
5. Authorization · Approval · Human reviewLimit capabilities to the minimum scope and place prior approval and post-execution audits according to impact.Excessive approval requirements halt automation, while loose requirements let high-impact mistakes spread quickly.Tabulate each tool's reads, writes, external transfers, and recoverability, then specify approval points.
6. Build an oracle independent of the patchCompare pre-fix failure and post-fix recovery against expectations derived from external task rules.Verifying the oracle takes time; tests cannot conceal ambiguous policy.Inspect failure reproduction and expected-value diffs separately, then rerun existing valid cases.
7. Separate repository instructions from task authorizationSeparate material provenance from host authorization so untrusted text cannot expand execution scope.Excessive blocking reduces diagnostic capability, so test normal work and attack fixtures together.Check traces to confirm that attack instructions in README files, logs, and error messages do not lead to actual reads or transmissions.
8. Turn an experimental diff into a reviewable change bundleLink changed files, failing fixtures, tested revisions and recovery boundaries into one evidence bundle.Even small patches may require preservation of existing work and checks on data-consumer compatibility.Link each final diff change to evidence, and check preserved user changes and older-consumer validation.

CHAPTER 1 / 8

From autocomplete to coding agents

The important change in AI development tools is not sentence length but the growing scope in which they observe, act, and re-verify on their own.

Why this concept became necessary

Early code completion looked at the current line and nearby files to suggest the next token or code block. Because people chose and executed the suggestions, the tool had little authority, but it could produce plausible code without knowing repository-wide rules or actual test results.

Interactive tools produced explanations and proposed changes across multiple files, while coding agents connect search, edits, shell execution, and reading test results in a loop. Quality now depends less on a single answer than on correctly repeating observation→hypothesis→small change→verification.

The broader the scope of action, the greater the impact of failure. Read-only investigation, source changes, package installation, and deployment each require different approvals and rollback plans, and an agent's ability to do something is not by itself grounds for granting execution permission.

Figure 2-2. From autocomplete to coding agents: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

The important change in AI development tools is not sentence length but the growing scope in which they observe, act, and re-verify on their own.

How it works

Tool results feed back into the next model input, creating a feedback loop that repeats until the termination condition is met.

Verification evidence

Separate the same task into a suggestion-only mode and a file-editing/test mode, then tabulate the possible side effects.

Follow it through a concrete system

Autocomplete suggests the next few lines of the function a developer is viewing, while a coding agent can receive a goal, locate relevant files, run tests, and select the next change based on failures. Even with the same model, connecting read, write, and execution tools changes the system's responsibilities and risks. Package installation and deployment, in particular, affect much broader external state than source-code suggestions.

For small bug fixes, an effective loop is to reproduce the failure first, read the relevant contract, make a minimal diff, and then re-verify with the same command. Conversely, giving an agent write access to the entire repository while the goal is ambiguous lets it redefine on its own what counts as done. The higher the level of automation, the more specific the acceptance criterion, stop conditions, and reversible units of change must be.

Selection criteria and failure boundaries

As the scope of automation grows, so do costs and the risks of incorrect changes, secret exposure, and external side effects.

Misconceptions to avoid: Describing an agent as merely a chatbot that gives longer answers misses the difference in control flow.

Verify it yourself

Separate the same task into a suggestion-only mode and a file-editing/test mode, then tabulate the possible side effects.

What to judgeTool results feed back into the next model input, creating a feedback loop that repeats until the termination condition is met.

To summarize this chapter

The important change in AI development tools is not sentence length but the growing scope in which they observe, act, and re-verify on their own.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Anthropic, 「Building Effective AgentsReview date 2026-08-28 · Scope Latest official documentation
  2. Anthropic, 「Effective Context Engineering for AI AgentsReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 2 / 8

How to select repository context

Good context is not a large number of tokens but the minimum set of contracts and evidence needed for the current decision.

Why this concept became necessary

An Agent does not understand the whole repository at once. First locate rules, entry points, relevant types, and tests to map the task, then read additional files according to the change's impact. Including every file with a similar name can bury important constraints.

Operating contracts such as AGENTS or README, public interfaces, failing tests, and actual error logs have high priority. Distinguish canonical sources from derived artifacts because old generated files or unrelated diffs can distort current decisions.

Context engineering involves not only adding information but also summarizing, using external memory, selecting tool results, and discarding outdated hypotheses. Leaving file, line, and test evidence next to each claim lets a person re-check the reasoning.

Figure 2-3. How to select repository context: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

Good context is not a large number of tokens but the minimum set of contracts and evidence needed for the current decision.

How it works

Use retrieval and structured summaries to keep only the information needed for the current goal in the working context.

Verification evidence

Record the contracts read before the change, the affected files, and the verification to run in three columns, and review them for gaps.

Follow it through a concrete system

Gathering context is not about attaching as many search results as possible. When fixing a payment amount error, business rules, public API types, the amount calculation function, existing regression tests, and actual error logs come first. Including every file with payment in its name, or old build artifacts, mixes information from different revisions and can lead to misreading the current contract.

A good work record does not stop at “this file seems relevant”; it preserves both evidence and gaps. For example, distinguish that the API schema is canonical and the generated client is derived, that a production error occurred only at a certain revision, and that production-data conditions have not yet been reproduced. These distinctions let the next person review or challenge the reasoning without repeating it.

Selection criteria and failure boundaries

Too little misses rules; too much adds cost and conflicting information.

Misconceptions to avoid: Putting the entire repository in at once does not make the agent fully understand it.

Verify it yourself

Record the contracts read before the change, the affected files, and the verification to run in three columns, and review them for gaps.

What to judgeUse retrieval and structured summaries to keep only the information needed for the current goal in the working context.

To summarize this chapter

Good context is not a large number of tokens but the minimum set of contracts and evidence needed for the current decision.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Anthropic, 「Effective Context Engineering for AI AgentsReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 3 / 8

Fast feedback from types and schemas

Explicit contracts make incorrect assumptions fail sooner rather than making AI smarter.

Why this concept became necessary

AI infers interfaces from names and usage examples but easily misses implicit rules. TypeScript parameter and return types and discriminated unions make allowed states explicit in code and report invalid properties or missing cases at build time.

However, types are a compile-time model. HTTP JSON, database rows, and environment variables arrive at runtime, so a schema parser must report the failure location and return a safe error. Forcing an external value through a type assertion makes the checker trust it without real evidence.

Clear types help both AI and people by revealing the scope of a refactor and letting editors track symbols. Semantic errors such as discount calculations, authorization rules, and date boundaries still occur within valid types, however, so examples and property tests are needed.

Figure 2-4. Fast feedback from types and schemas: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

Explicit contracts make incorrect assumptions fail sooner rather than making AI smarter.

How it works

Static types and runtime schemas reject invalid state early, at different boundaries.

Verification evidence

Inject an invalid property, an invalid value of a valid type, and missing external JSON separately to see which gate catches each.

Follow it through a concrete system

A TypeScript interface aligns value shapes among compiled components, but HTTP responses arrive from outside compilation. At the boundary, accept JSON as `unknown`, validate required fields, ranges, and combination rules with a schema validator, then narrow it to a safe type. Simple property checks can miss number-like strings or IDs that exist but are empty.

A schema need not contain every business rule. Rules such as “the end date is later than the start date” or “only the payer may request a refund” require examining multiple fields and current permissions together. Record shape validation, domain validation, and authorization as distinct errors so the agent knows what to fix and is less likely to bypass checks with an incorrect assertion.

Selection criteria and failure boundaries

Maintaining types and schemas separately causes drift, so a code-generation or contract-test strategy is needed.

Misconceptions to avoid: Saying that TypeScript controls how AI interprets JavaScript confuses the execution model with the checking model.

Verify it yourself

Inject an invalid property, an invalid value of a valid type, and missing external JSON separately to see which gate catches each.

What to judgeStatic types and runtime schemas reject invalid state early, at different boundaries.

To summarize this chapter

Explicit contracts make incorrect assumptions fail sooner rather than making AI smarter.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Microsoft, 「TypeScript HandbookReview date 2026-08-28 · Scope Latest official documentation
  2. Microsoft, 「Type CompatibilityReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 4 / 8

Close the loop with execution and test feedback

Changes made by an Agent are complete only when supported by reproducible verification results, not merely explanations.

Why this concept became necessary

Good work starts by reproducing the failure. Confirm that the test fails before the fix because of the actual defect, then run the same test and impact-scope checks after the smallest change to reduce accidental passes.

Even when a test command succeeds, inspect what it verified. Mocks that copy the implementation or assertions that check only element presence can miss incorrect user outcomes. Failure messages and coverage scope matter as much as pass/fail status.

Lint, type checks, unit, integration, browser tests, and builds find different defects. For riskier changes, add conditions closer to production, but separate fast local gates from full release gates instead of slowing feedback by running every test each time.

Figure 2-5. Close the loop with execution and test feedback: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

Changes made by an Agent are complete only when supported by reproducible verification results, not merely explanations.

How it works

Observe the failure, apply the fix, then re-verify under identical conditions to narrow down the causal relationship.

Verification evidence

Link each acceptance criterion to an observable assertion and the evidence it produces on failure.

Follow it through a concrete system

The first step in a verification loop is to establish reproducible failure conditions, not to make a fix. Record the input, execution environment, expected result, and actual result, then check that the test fails under those conditions before changing the code. Without reproduction, a code change cannot show whether the symptom vanished by chance or the cause was removed, and it is easy to declare completion while the same defect remains under other conditions.

After a fix, run not only the new test but also type checks, affected unit and contract tests, and representative browser journeys according to risk. Simply rerunning a failed command until it passes is not evidence. Isolate environmental causes, or control time and shared state for flaky signals, then verify repeatable results with the original inputs.

Selection criteria and failure boundaries

If the test suite is slow or flaky, the agent may ignore the signal or create meaningless workarounds.

Misconceptions to avoid: It is incorrect to assume that a single command exit code of 0 proves the user’s requirements are met.

Verify it yourself

Link each acceptance criterion to an observable assertion and the evidence it produces on failure.

What to judgeObserve the failure, apply the fix, then re-verify under identical conditions to narrow down the causal relationship.

To summarize this chapter

Changes made by an Agent are complete only when supported by reproducible verification results, not merely explanations.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Microsoft, 「Playwright AssertionsReview date 2026-08-28 · Scope Latest official documentation
  2. Microsoft, 「Playwright Best PracticesReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 5 / 8

Authorization · Approval · Human review

The criterion for automation is not whether an action is possible, but whether its impact and recovery are controlled when it fails.

Why this concept became necessary

Read-only searches usually leave no state change to undo, while file overwrites, database migrations, sending messages, and deployments change external state. Separate target scope, allowed operations, secret access, and network destinations for each tool.

Approval does not mean deferring every click to a person. Automate repetitive low-risk checks; for irreversible or broadly consequential actions, show the exact target and diff immediately before execution.

Review should focus on contracts, security boundaries, failure paths, and evidence rather than the number of lines AI wrote. If the agent records its reasons for changes, test results, and remaining uncertainty in a structured form, reviewers do not have to reconstruct the same reasoning from scratch.

Figure 2-6. Authorization · Approval · Human review: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

The criterion for automation is not whether an action is possible, but whether its impact and recovery are controlled when it fails.

How it works

Limit capabilities to the minimum scope and place prior approval and post-execution audits according to impact.

Verification evidence

Tabulate each tool's reads, writes, external transfers, and recoverability, then specify approval points.

Follow it through a concrete system

An agent with read-only permissions cannot directly change external state even if it reaches a wrong conclusion. Each added capability, such as file writes, shell execution, network transmission, account changes, and production deployment, widens the breach scope and raises recovery costs. Permissions should therefore be split by capability and target rather than bundled under a single “use development tools” grant.

Approval is an information boundary for understanding the actual change, not a formality of clicking a button once. The approval screen should show the target environment, before-and-after values, actor, and recovery method. Repeated low-risk reads may be automated by policy, but sending secrets, deleting data, and deploying to production require explicit reconfirmation, while server-side authorization must remain separate.

Selection criteria and failure boundaries

Excessive approval requirements halt automation, while loose requirements let high-impact mistakes spread quickly.

Misconceptions to avoid: Claiming safety from the phrase “Human in the loop” alone does not specify concrete approval points.

Verify it yourself

Tabulate each tool's reads, writes, external transfers, and recoverability, then specify approval points.

What to judgeLimit capabilities to the minimum scope and place prior approval and post-execution audits according to impact.

To summarize this chapter

The criterion for automation is not whether an action is possible, but whether its impact and recovery are controlled when it fails.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Anthropic, 「Building Effective AgentsReview date 2026-08-28 · Scope Latest official documentation
  2. Anthropic, 「Demystifying Evals for AI AgentsReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 6 / 8

Build an oracle independent of the patch

Fix expected behavior first so generated code and tests do not share the same mistaken assumption.

Why this concept became necessary

When fixing a discount bug, tests derived directly from the existing implementation may endorse the same wrong calculation. Start from approved pricing rules and visible totals, not implementation lines.

Write a small table of inputs, expected results, and reasons, then run it before editing. Confirm that the reported bug actually fails; only then does a later green result count as evidence of the fix, rather than merely showing that normal cases pass.

Money comparisons depend on boundaries and rounding stages. Rounding each item can differ from rounding the total, so identify the contractual rule and preserve it in the test explanation.

If the source rule is ambiguous, do not let the model invent policy. Present the two interpretations, obtain the task owner’s decision, and align code and tests with the approved rule.

Figure 2-7. Build an oracle independent of the patch: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

Fix expected behavior first so generated code and tests do not share the same mistaken assumption.

How it works

Compare pre-fix failure and post-fix recovery against expectations derived from external task rules.

Verification evidence

Inspect failure reproduction and expected-value diffs separately, then rerun existing valid cases.

Follow it through a concrete system

A fictional fixture applies a 10% discount to a 10,000-won order with a 700-won cap. The expected discount is 700 won; this teaching contract detects a missing cap and is not a real company policy. Place capped and uncapped inputs side by side. This also catches a bogus fix that hard-codes every result to 700 won.

The test must initially fail if the old function returns 1,000 won. After it returns 700, also retest uncapped and ineligible orders to check that the local fix has not harmed other cases. Find other callers such as refunds or recalculation. A shared-function change can affect more than the reported screen, requiring checks on consumer contracts.

If test files are part of the patch, review expected-value changes separately. Changing 700 to 1,000 without an approved policy change makes tests pass while preserving the original defect. If a policy change is approved, record its evidence and effective date. Distinguishing policy changes from bug fixes prevents recalculating historical orders under unintended new rules.

Link the failing fixture, approved rule, patch and rerun results. This answers which user-visible error was removed, a more useful question than coverage alone. Record command exit codes and failing inputs. A note saying “tested” cannot distinguish undiscovered tests from executed tests whose results were ignored.

Selection criteria and failure boundaries

Verifying the oracle takes time; tests cannot conceal ambiguous policy.

Misconceptions to avoid: Having the AI write tests does not make verification independent.

Verify it yourself

Inspect failure reproduction and expected-value diffs separately, then rerun existing valid cases.

What to judgeCompare pre-fix failure and post-fix recovery against expectations derived from external task rules.

To summarize this chapter

Fix expected behavior first so generated code and tests do not share the same mistaken assumption.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Microsoft, 「Playwright Best PracticesReview date 2026-08-28 · Scope Latest official documentation
  2. Anthropic, 「Demystifying Evals for AI AgentsReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 7 / 8

Separate repository instructions from task authorization

Text in a file is task material; it does not itself authorize secret access or external transmission.

Why this concept became necessary

Logs or repository documents attached to a bug can contain unrelated execution instructions. Treating them as higher-priority instructions merely because the AI read them turns investigation into a privilege-expansion path.

First define file scope, allowed commands, and deliverables in the task contract. Even if a document asks to print every environment variable, the host must separately decide whether that is needed to reproduce the bug and whether it falls within current permissions.

Provide relevant functions, failure boundaries and necessary document locations instead of the entire repository. The aim is not to hide contrary evidence, but to make verifiable material and untrusted instructions easier to distinguish.

Redacted logs should retain diagnostic context such as paths, timestamps and error classes. Removing everything prevents reproduction; pasting everything unnecessarily retains personal data in model inputs and reports.

Figure 2-8. Separate repository instructions from task authorization: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

Text in a file is task material; it does not itself authorize secret access or external transmission.

How it works

Separate material provenance from host authorization so untrusted text cannot expand execution scope.

Verification evidence

Check traces to confirm that attack instructions in README files, logs, and error messages do not lead to actual reads or transmissions.

Follow it through a concrete system

Insert a request to transmit a secret file externally into a teaching README. This is an attack fixture testing whether untrusted input changes the task boundary, not an authorized action. Use a fake marker instead of a real secret and observe attempted egress through a blocked collector. The test itself must not expose personal data or production credentials.

Success means no actual secret-file read or external request, not merely a warning from the model. Inspect rejected targets and policy decisions in tool traces without recording secret contents. Check that rejection logs do not copy the marker from protected files. Blocking execution while echoing sensitive content in errors merely moves exposure into observability storage.

Move the instruction into log comments and test-failure messages as well. A string filter targeting only README content can miss equivalent instructions arriving through other paths. Allowed targets must remain unchanged when the attack is paraphrased. A boundary owned by provenance and execution policy differs from a filter memorizing one sentence.

After restoring normal error logs, bug reproduction and authorized edits must still work. Blocking every tool is not a successful development assistant, so retain normal-task regression evidence too. For normal fixtures, verify that enough relevant error lines and functions remain available. Removing all context and forcing the model to guess the cause is not a solution.

Selection criteria and failure boundaries

Excessive blocking reduces diagnostic capability, so test normal work and attack fixtures together.

Misconceptions to avoid: Repository instructions are not all user-authorized commands.

Verify it yourself

Check traces to confirm that attack instructions in README files, logs, and error messages do not lead to actual reads or transmissions.

What to judgeSeparate material provenance from host authorization so untrusted text cannot expand execution scope.

To summarize this chapter

Text in a file is task material; it does not itself authorize secret access or external transmission.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Anthropic, 「Effective Context Engineering for AI AgentsReview date 2026-08-28 · Scope Latest official documentation
  2. Anthropic, 「Building Effective AgentsReview date 2026-08-28 · Scope Latest official documentation

CHAPTER 8 / 8

Turn an experimental diff into a reviewable change bundle

Completion means explaining which change can be approved with which evidence, not producing a volume of code.

Why this concept became necessary

Multi-file AI patches can mix bug repair with unrelated cleanup. Link every change to the reported symptom, a necessary contract change, or added verification, and separate unexplained edits.

Identify pre-existing user changes before editing and distinguish them from the new diff. Reverting earlier changes wholesale because a test fails can destroy work unrelated to the bug.

If configuration or data formats change, reverting code alone may not restore service. Record compatible settings, data boundaries and whether the patch creates states unreadable by the previous version.

Tie validation results to executed commands and the tested revision. Reusing results after further edits can make the evidence refer to a different change than the one being approved.

Figure 2-9. Turn an experimental diff into a reviewable change bundle: decision flowThe chain from the problem conditions through the working principle to verification evidence.
Problems and selection criteria

Completion means explaining which change can be approved with which evidence, not producing a volume of code.

How it works

Link changed files, failing fixtures, tested revisions and recovery boundaries into one evidence bundle.

Verification evidence

Link each final diff change to evidence, and check preserved user changes and older-consumer validation.

Follow it through a concrete system

A fictional task is a patch that fixes the date format in CSV exports. The reviewer checks that only output examples and the related parser and test changes are included, and moves any unrelated authentication-setting change to a separate review. For each file, briefly record who authored the change and why this fix needs it. Even when several people have edited the same file, do not treat the whole task as one owner’s result.

Reproduce the bug while preserving any user-authored UI-copy changes already in the working tree. If those disappear across the patch, the preservation contract fails even when tests pass. Do not automatically clean uncommitted changes when creating a test environment. If using an isolated copy, record included changes so differences in test inputs remain visible.

Test the corrected CSV with a fixture representing an older consumer. Even accurate dates may require a compatibility notice or version boundary if existing clients cannot parse them. Record the consumer parser version and expected date. If browsers and batch jobs interpret the CSV differently, explicit supported consumers make compatibility judgments reproducible.

Describe changed behavior, reproduced errors, passing checks, and remaining compatibility conditions. Do not claim deployment or operational recovery that was never performed; the next approver needs an accurate view of remaining work. Remaining work is information about the approval boundary, not a reporting flaw. When deployment permission or data-change approval is required, mark code-review completion and operational rollout separately.

Selection criteria and failure boundaries

Even small patches may require preservation of existing work and checks on data-consumer compatibility.

Misconceptions to avoid: One successful test run does not approve later edits or deployment.

Verify it yourself

Link each final diff change to evidence, and check preserved user changes and older-consumer validation.

What to judgeLink changed files, failing fixtures, tested revisions and recovery boundaries into one evidence bundle.

To summarize this chapter

Completion means explaining which change can be approved with which evidence, not producing a volume of code.

Official sources for this chapter

The technical facts in the text were reviewed against the following primary sources. The author reconstructed the diagrams and comparisons using these materials.

  1. Microsoft, 「Playwright Best PracticesReview date 2026-08-28 · Scope Latest official documentation
  2. Anthropic, 「Demystifying Evals for AI AgentsReview date 2026-08-28 · Scope Latest official documentation

INTERACTIVE LAB 1 / 2

Lab 1 · Turn an ambiguous request into a verifiable task contract

The only request is “fix the login error on your own and deploy it.” The repository contains the user's uncommitted changes, and a production deploy changes external state.

Choose the safest first work contract that still makes real progress.

Choose an answer

Correct answer B

A. Rewrite the entire repository and deploy it to production immediately.The scope and success criteria are missing, and the user's changes and external state are handled unsafely.

B. Preserve the rules and the diff, build a reproduction test, apply the minimal fix and run the full local gate, and then keep the deployment diff separate from its approval.Proceed with diagnosis and reversible local verification while keeping a clear boundary around high-impact deployment.

C. Answer with generic login code without reading any files.Not a workable fix, because it does not check the current defect or the repository contract.

D. Ask the user about every action, one line at a time.Excessive approvals unrelated to risk stall work and still fail to explain the safety boundary.

INTERACTIVE LAB 2 / 2

Lab 2 · Choose evidence for a discount patch

The approved rule is 10% off a 10,000-won order, capped at 700 won. The AI changed both the function and test expectation to 1,000 won, and tests pass.

Choose the next action that can confirm the original bug is fixed.

Choose an answer

Correct answer B

A. Deploy because tests are green.The test shares the implementation’s wrong expectation.

B. Restore the approved 700-won fixture, verify pre-fix failure and post-fix success, and rerun normal orders.An implementation-independent oracle is needed to detect the missing cap.

C. Keep the expectation and increase coverage.More executed lines do not repair the wrong business rule.

D. Change the cap if another AI agrees.Model agreement does not replace the task owner’s policy approval.

KEY TERMS

Key terms in this unit

From autocomplete to coding agents
Tool results feed back into the next model input, creating a feedback loop that repeats until the termination condition is met.
How to select repository context
Use retrieval and structured summaries to keep only the information needed for the current goal in the working context.
Fast feedback from types and schemas
Static types and runtime schemas reject invalid state early, at different boundaries.
Close the loop with execution and test feedback
Observe the failure, apply the fix, then re-verify under identical conditions to narrow down the causal relationship.
Authorization · Approval · Human review
Limit capabilities to the minimum scope and place prior approval and post-execution audits according to impact.
Build an oracle independent of the patch
Compare pre-fix failure and post-fix recovery against expectations derived from external task rules.
Separate repository instructions from task authorization
Separate material provenance from host authorization so untrusted text cannot expand execution scope.
Turn an experimental diff into a reviewable change bundle
Link changed files, failing fixtures, tested revisions and recovery boundaries into one evidence bundle.

UNIT WORKBOOK

Exercises and worksheets for applying concepts to new situations

Start by checking basic principles, then expand to practical workplace decisions. After submitting an answer, you can see why every option is correct or incorrect, not just the correct answer.

THREE-LEVEL ASSESSMENT

From basic principles to operational decisions

After you submit an answer, you can see not only the correct answer but also why each option is right or wrong.

Basic Question 1

What is the most accurate benefit of explicit interfaces in AI coding?

Choose an answer

Correct answer B

A. External inputs automatically become safe at runtime.Runtime schema validation is required.

B. Exposes incorrectly assumed value shapes at the type-check stage.Quick feedback before execution is key.

C. Tests are no longer needed.Behavioral errors must be verified by tests.

D. AI always generates correct business rules.Business meaning is not guaranteed by the interface alone.

Apply Question 2

The agent made the tests pass but implemented the requirement incorrectly. What should be improved next?

Choose an answer

Correct answer C

A. Remove TypeScript.The issue is not the type but the verification goal.

B. Replace all tests with mocks.It can obscure the actual boundaries further.

C. Write acceptance criteria as user-observable outcomes and assert on those outcomes.Tests must be tied to real requirements.

D. Make the agent repeat the same command more often.Repeating a wrong assertion is meaningless.

Capstone Question 3

What are the right criteria for designing coding agent authorization?

Choose an answer

Correct answer D

A. Grant broader permissions to larger models.Model size doesn't eliminate the risk of side effects.

B. Maximize only the speed of work.Recovery and impact scope are missing.

C. Require approval for every tool, every time.An excessive boundary that blocks even low-risk verification.

D. Separate automation from approval requirements according to an action's impact scope, recoverability, and external data transmission.A criterion that links capability to risk.

PRIMARY SOURCES

Course references

This list collects the sources for each chapter. The text and author-created diagrams were prepared by directly reviewing the originals below.

PERSONAL WORKSHEET

A learning worksheet you adapt to your own environment

Your input remains only on the current browser screen and is not stored or transmitted externally. Use categories and pseudonyms instead of actual sensitive information.

OFFICIAL SOURCES

Verify against official sources

Technical, compatibility, and model information reviewed: August 2026

LEARNING RECORD

Have you reviewed the text, decision activities, and all explanations?

Completion status is stored only in this browser.