Data and fine-tuning

A Supervised Fine-Tuning (SFT) dataset is not a repository for large amounts of text. Each row is a training signal demonstrating expected output behavior for a specific input. Converting an entire company-policy PDF into rows does not create a system that reliably answers using current evidence. RAG may be appropriate for frequently changing facts and source citations, while SFT is a candidate for adjusting repeated behaviors such as categories, JSON schemas, tone, and refusal rules.

Write training goals as observable acceptance criteria: categories within an approved enum, required urgency and reason fields, a hold when order IDs are missing, and no final refund approval. First run the base model on 100 production-like normal, boundary and failure cases, recording accuracy, format, critical recall and human corrections as the baseline. Without a baseline, you cannot distinguish training improvements from lost original capabilities.

An error taxonomy distinguishes wrong labels, omissions, unsupported additions, PII reproduction, unsafe compliance, format and language errors, and excessive refusal. High-harm classes such as money, permissions, and safety get independent gates that average scores cannot offset. When evidence is missing or labels conflict, provide examples of terminal behavior such as insufficient or escalate instead of an answer that plausibly picks one.

Build a Data-plan matrix covering normal, boundary, insufficient-information, refusal, and actual channel/language/length conditions. Set row targets from each cell's frequency, harm, and current failures, not a fixed number such as 100,000 rows. Start with a small pilot reviewers can read completely to check loading, templates, tokens, loss, training, and evaluation; add cases with new causes according to remaining failures.

Why does this happen?

Because the behavior to train and its failure costs must be defined first before sources, schema, labels, required coverage, and the evaluation set can be designed against a single standard.

When is it a problem?

If you scrape data first, rows of irrelevant text and unclear rights accumulate, and even if train loss falls, actual task errors cannot be explained.

Common beginner misconceptions

Adding many recent documents to SFT does not automatically replace versioned RAG and source citations, nor does it guarantee new general capabilities.

How to verify it yourself

Record the base model’s fixed normal, boundary, and failure set and critical gates, and link each planned row to the matrix cell or failure it strengthens.

Being published on the web or shared by a colleague does not by itself grant rights to train on or redistribute material. Check the exact license and terms for commercial use, derivative works, redistribution and attribution, and prohibited uses. For internal material originally collected for other purposes, such as support conversations, email, and meetings, review individual and organizational expectations, contracts and policies, and consent. Do not put material that needs a judgment call into training on the assumption that it is probably allowed; quarantine it in rights_pending status.

The source registry holds the source ID, owner, collected date and revision, acquisition method, rights basis and allowed purpose, and reviewer and expiry. Rather than exposing full source paths, each dataset row links a limited source reference and a transform version. A hash helps identify the same file, but it does not prove legal permission or content accuracy. When sources and rows have a many-to-many relationship, make it explicit in a lineage table.

Distinguish human-written, crowdsourced, machine-generated, translated, and augmented rows. For synthetic outputs, record the generating model, prompt, and sampling settings, and whether a person has checked them against the source. Do not create a gold label merely because another model scored a model-generated answer as natural. Summaries, translations, and paraphrases made from the same source are not independent sources even if worded differently, so they share a duplicate cluster and split group.

A Dataset Card conveys purpose, sources, composition, license, language, size, collection, annotation, PII, bias, limitations, and intended/out-of-scope use. Hugging Face describes cards as context and license metadata for responsible use. A license tag does not replace source-specific rights review. Record rights-change triggers, review owners, and plans for affected releases and adapters.

Why does this happen?

Because a Dataset transforms and combines source material, track each source's allowed purposes and affected derived rows and artifacts to handle withdrawal and errors in practice.

When is it a problem?

Recording only the URL and license tag leaves the exact revision, internal consent, and training and distribution terms unknown, and makes adapters subject to deletion impossible to find.

Common beginner misconceptions

Open access, open-weight models, dataset card metadata, and existing hashes are not automatic approval of rights to train, use commercially, or redistribute.

How to verify it yourself

From a random row, trace back the exact source, rights decision, transform, split, dataset, and training artifact, and run a withdrawal simulation for one source.

Collect only necessary data first. Affiliations, dates and rare-event combinations can identify people even without names, and combining conversation turns can reveal masked values. Inventory emails, phone numbers, addresses, accounts, financial and health data, employee/customer IDs, and API keys, tokens and private URLs in code. Exclude fields unnecessary for the purpose before the redaction stage.

Microsoft Presidio can find PII candidates with rules·regular expressions·NER·checksums and custom recognizers, but its official guidance warns that automated detection cannot guarantee finding all sensitive information. Build fixtures by language·country and internal ID format, and combine secret scanners with entropy·known-prefix rules. For classes with many false negatives, have people compare high-risk·random samples against the originals.

Record redaction results in a restricted ledger with original spans, replacement types·rule versions, and reviewer decisions. Control mapping access and retention even when consistent pseudonyms are needed to preserve conversation meaning. Review both over-redaction that removes task signals and under-redaction that allows re-identification. Keep learner-visible JSONL and original-review storage behind different permission boundaries.

The deletion workflow uses source IDs to find rows and duplicate clusters, train/validation/test files, caches and snapshots, and adapters and merged models. Fully removing an individual row's influence from every artifact may require retraining, so the owner decides on risk, cost, and service actions. Run a sample deletion request, check query results and remaining copies, and roll back to the previously approved dataset or model. The NIST Privacy Framework serves as a reference for managing such data-processing privacy risks at the organizational level.

Why does this happen?

Because sensitive-data detection is probabilistic and context-dependent, and a dataset spawns many copies, caches, and artifacts, minimal collection and a deletion path are both required.

When is it a problem?

Marking data safe after a single regular-expression check can miss Korean names, contextual secrets, new secret formats and derived artifacts.

Common beginner misconceptions

Local training, asterisk redaction, or passing a PII scanner does not automatically resolve consent, access, retention, or re-identification risk.

How to verify it yourself

Inspect known fixtures and blind samples for each language, PII type, and secret type, and use a lineage query to check how deleting one source affects every row, split, cache, and artifact.

The official JSON Lines (JSONL, one JSON value per line) description sets three basic requirements: UTF-8 encoding, a valid JSON value on each line, and a line terminator. Do not include a Byte Order Mark (BOM); a blank line is not a valid value. Store newlines inside strings as JSON escapes, not as actual record separators. A trailing newline is recommended so files can be safely concatenated.

Hugging Face Datasets documentation presents one JSON object per line as an efficient JSON representation. Successfully reading a file does not establish that its SFT schema is correct. The parser reports line numbers, byte/character positions and exceptions, and must not treat partial loading as silent success. Also check for encoding replacement characters and field-type inference changing between rows.

Version the schema for each task. For conversational SFT, define the messages array, each message’s role and content, allowed roles and turn order, at least one assistant target, and prohibition of empty content. For prompt-completion data, distinguish prompt and completion types and required values. Metadata may need source_ref, rights_state, language, labels, review status, and group IDs, but separate training-input fields from governance-ledger fields.

The schema validator checks additional fields, loader behavior that coerces nulls and numbers to strings, and nested content. Whether an answer within a valid structure matches its source, whether label priorities are correct, and whether PII is present require separate checks. Create golden sets of valid and invalid rows, and regression-test that expected error codes remain stable when changing parser or schema versions. Link the raw-file hash and validator report to the release manifest.

Why does this happen?

Because JSONL syntax, the columns and roles the trainer expects, and the correct task answer are different failure layers, and only separating them allows accurate recovery.

When is it a problem?

Checking only JSON.parse success lets training failures pass, including missing assistant targets, reversed roles, empty answers and unclear rights.

Common beginner misconceptions

A .jsonl extension or a successful read by the Hugging Face loader does not certify the SFT dataset, chat schema, or quality.

How to verify it yourself

Run BOM, blank, and broken JSON lines, wrong types and roles, empty targets, and valid fixtures line by line, and check the errors from the loader, schema, and task validator separately.

TRL's official dataset format documentation distinguishes standard and conversational formats, as well as columns for each task type, such as language modeling, prompt-only, prompt-completion, and preference. Even with the same messages JSON, the trainer may apply loss to the full sequence, to completions only, or to assistant turns only. Check the supported formats and defaults for the exact TRL and Transformers versions and SFT configuration, and do not infer the training target from the dataset schema alone.

As Hugging Face chat-template documentation explains, a role/content list becomes one token sequence including control tokens. Models fine-tuned from the same base may use different role markers, and incorrect tokens can significantly harm performance. Save the exact tokenizer revision's apply_chat_template output and inspect system/user/assistant boundaries and end markers. Compare training data with serving prompts to verify they use the same contract.

Check for duplicate special tokens. If the template already inserts BOS and EOS and the tokenizer adds them again, the sequence may change. Verify that the model supports assistant start and end markers, tool turns, and system placement in multi-turn rows. Overlay the loss mask on tokens to see whether user and system content is being learned as target answers, and compare the configuration against the intended target: only the final assistant response or all assistant responses.

Build rendered-token length distributions and truncation reports. Rows whose final answer is cut by long system messages or history, or whose assistant response mostly exceeds max length, are not valid training signals merely because parsing succeeds. Similar character counts can produce different tokens by language·template. Regression-test text·token-ID·mask hashes on representative·long·multi-turn·empty/unsupported-role golden rows, re-evaluating dataset candidates after changes.

Why does this happen?

A causal model learns token sequences rather than JSON objects, and the loss mask determines which positions are actually updated.

When is it a problem?

Matching only the shape of messages still leaves other models' markers, duplicate special tokens, and truncated assistant targets, so the model fails to learn the expected behavior.

Common beginner misconceptions

Even if role names are the same (user and assistant), not every model and trainer uses the same template, special tokens, and loss scope.

How to verify it yourself

Render golden rows with the exact tokenizer/template and trainer configuration. Inspect tokens, masks and truncation, then compare them with the first user-request sequence at inference.

An annotation guide does more than list labels. Define each class, inclusion and exclusion criteria, precedence when multiple classes fit, insufficient or conflicting sources, and out-of-scope behavior. Provide positive, negative, and borderline examples with different numbers and situations. For free-text answers, separate required facts, prohibited additions, tone, format, and refusal conditions in a rubric so reviewers can identify sentences needing correction.

Annotators need permission to view sources and sufficient context, as well as domain knowledge. Blind-review items that do not require identity to reduce personal bias, and do not present model-generated answers as authoritative references. Have at least two people independently review some overlapping samples to assess agreement and class-level confusion. Minimize personal information in disagreement notes and apply access and retention controls.

Do not settle disagreements by majority vote by default. Classify whether the source is ambiguous, rubric boundaries overlap, or the training goal itself allows multiple behaviors. The adjudicator records the final label, a reason code, and the guide change version. When the guide changes, re-review the same ambiguous samples and affected rows that were already approved. Which critical class split the reviewers and why matters more than a single agreement number.

The review workflow defines drafted, reviewed, approved, rejected, and quarantined states and keeps authors separate from reviewers. Aggregate the edit rate, reject and escalate counts, and the defect taxonomy by source and authoring method. If a particular synthetic prompt or provider shows a high error rate, do not just delete the rows; quarantine the entire batch. Every final gold row must be traceable back to its source, rubric version, and approval.

Why does this happen?

SFT learns repeated label patterns, so ambiguous or conflicting answers contaminate both model behavior and evaluation criteria.

When is it a problem?

If one person writes quickly and only checks parsing, conflicting policies for the same input and fluent but unsupported answers can enter the gold set.

Common beginner misconceptions

High reviewer agreement or approval by a model judge does not automatically settle source accuracy and rights, or replace independent human review of new situations.

How to verify it yourself

Inspect class-level disagreement and edit reasons in blind overlap samples. Repeat independent review on the same ambiguity set both before and after guide changes.

Find exact duplicates using raw-byte hashes and input/target hashes normalized for Unicode, whitespace, case, and punctuation. Separately inspect identical prompts with conflicting answers and identical answers attached to template-boilerplate questions. Sample whether dedup normalization erases meaningful code, ID, or numeric differences. Preserve duplicate-removal decisions and representative row IDs.

Near duplicates include reordered sentences and document chunks derived through translation·summarization·paraphrasing. Embedding similarity helps find candidates; it does not establish semantic identity. Prefer source-family·customer-event and generation-parent IDs as cluster keys, and have people review boundary pairs. Numerous synthetic rows do not count as independent coverage if they merely rephrase the same prompt·source.

Assess coverage by task matrix, not total row count. Show approved row and group counts by label; normal, edge, hold, and refusal cases; language, length, and channel; source type and time; and privacy/security risk. Copying the same row several times because a critical class is small increases its weight but adds no new situations. Collect sources with different causes from real failures, or manage class-balanced sampling transparently in the training config.

After automated reports, people read random, high-risk, longest/shortest and duplicate-boundary samples, plus samples by source and authorship method. If label distributions differ from production, distinguish intentional oversampling from distortion. Compare before/after matrices for filters that excessively remove particular dialects, languages or short refusals. Include candidate-versus-previous row, group and coverage differences in approval evidence.

Why does this happen?

Duplicates can overtrain particular patterns and create evaluations with overlapping sources; total row counts do not show real situational diversity.

When is it a problem?

Removing only exact strings leaves paraphrases, chunks, and synthetic families in both train and test, and rare classes are filled only with duplicated rows.

Common beginner misconceptions

High Embedding similarity does not always imply duplication, and many rows with balanced labels do not prove sufficient production coverage.

How to verify it yourself

Build clusters with normalized hashes·source parents·semantic candidates, then check unique groups·review passes per matrix cell and the before/after effects of removal.

Hugging Face Datasets' train_test_split is a convenient way to split rows by size and shuffle setting. However, if the original text, summary, and paraphrase of the same ticket are separate rows, a random shuffle can send them to both train and test. When the model reproduces the meaning of answers it saw in training, scores exceed true generalization. Deciding what counts as one group comes before choosing a split function.

Scikit-learn GroupShuffleSplit provides a way to split indices by arbitrary domain groups. For support conversations, a customer or ticket thread can be the group; for documents, a revision family; for code, a repository or issue; and for speech, a speaker or recording session. Place all rows of a group in a single split. If a row has multiple sources, consider a conservative boundary that groups them by the largest connected component.

When change over time matters, use a time split: past data for training, the next period for validation, and the latest period as a frozen test. Use validation to choose schemas, data, hyperparameters, and checkpoints, and reserve the test set for comparing final candidates. The moment you add rows or change a prompt or threshold after seeing test failures, that test becomes development evidence. A new release needs a blind set that has never been accessed or the next time window.

After splitting, check exact and near duplicates and source and group overlap in both directions; put the release on hold if any count is nonzero. Check critical labels, languages, lengths, risk coverage, and group counts in every split. If group separation leaves a rare class out of the test set, expand collection periods or sources rather than copying training rows into the test set. Record the seed, algorithm version, group rules and mapping, split-file hashes, and test-access log in the manifest.

Why does this happen?

Rows with different wording can share answers from the same source or event, so row-level random splits can easily undermine independent generalization evaluation.

When is it a problem?

If you record only an 80:10:10 ratio and a seed, the same customer·document family may be scattered across all three splits, inflating the test score.

Common beginner misconceptions

Using a group splitter does not by itself guarantee a correct group definition, time boundaries, 0 near duplicates, or frozen test discipline.

How to verify it yourself

Calculate intersections of source, group, and duplicate IDs across train, validation, and test splits. Trace back selected groups to verify that all derived rows remain in only one split.

The release manifest holds the source snapshot·rights state, raw/clean row·group·split hashes, schema·conversion code, PII·secret scans, annotation guide·review report, tokenizer/chat template, rendered token·mask statistics, split algorithm·seed, and Dataset Card. Pin an immutable dataset version, not a mutable latest path, in the training config. When any item changes, rerun the affected downstream evaluations.

Judge each gate independently: 100% line parsing and schema compliance, zero unsupported roles and target truncation, critical-label thresholds, zero rights_pending or unreviewed PII, and zero source/group/duplicate overlap across splits. Also examine coverage, reviewer edits and agreement, frozen task metrics and safety versus the base, and regressions in original capabilities. Low training loss or high average accuracy does not offset privacy, rights, or critical-class failures. Do not lower thresholds after seeing results.

Use a small training run to verify loading, loss mask and memory, and the checkpoint and evaluation pipeline, and compare against the previous dataset with the exact base and config. Do not change model, data, and training variables at the same time. Collect actual corrections and new failures from a limited canary, but do not feed production input straight back into training rows without consent and review. Trace each failure to its root layer: source, schema, label, split, or training.

Rollback does not merely replace a JSONL file. Restore the previous dataset manifest, conversion·template, training config·adapter, and serving artifacts as a compatible bundle. Use lineage to find affected rows·caches·splits·artifacts in rights-revocation or PII-discovery samples, and test deletion·retraining or service-suspension decisions. Record owners, limitations, source·policy·model changes, and defect triggers. Close release only after retesting the same frozen·failure sets.

Why does this happen?

Because dataset changes alter token sequences, training results, and even privacy and rights, every input, code, review, and artifact must be managed as a reproducible, recoverable release.

When is it a problem?

Keeping only the final .jsonl filename and training loss leaves you unable to identify the model’s sources, template and split or determine what to roll back after errors or deletion.

Common beginner misconceptions

Passing every automated check and frozen score does not replace actual rights approval or beginner review, and a dataset card is not a legal certificate.

How to verify it yourself

Repeat clean regeneration and small training runs from the candidate manifest, and restore the previous compatible release and a source-deletion incident with the same hashes and failure set.

Fine-tuning is not a small version of pretraining a model from scratch. It adjusts specific output behavior by showing reviewed input-target pairs to a base model that already has language capabilities. Do not accept a request to “train on company documents” at face value: assign retrieval of current policies and source citation to Retrieval-Augmented Generation (RAG), strict JSON validation to schemas and code, and recurring classification boundaries, tone, and refusals to SFT candidates. Choosing the solution first can freeze changing facts in an outdated adapter or turn a deterministic problem into a harder probabilistic one.

Write goals as observable task outcomes, not “good answers.” Define conditions a parser and reviewer can judge, such as required inquiry category, urgency, and reason, a hold when information is missing, and a ban on confirming refunds without authority. On a frozen set with normal, boundary, conflict, and unsafe inputs, run the exact base, system prompt and few-shot, and RAG and rule candidates, and measure task success, critical recall, format, excessive refusals, and human corrections. Before seeing results, approve the minimum improvement the adapter candidate must achieve and the gates that must never regress.

SFT does not automatically create general capabilities absent from the base model or turn unverified labels into valid ones. If dataset rows are ambiguous, the model learns that ambiguity. A small adapter does not reduce your responsibility when source rights are missing or personal data remains. Accept only immutable versions that have passed dataset-release rights, privacy, schema, template, and label reviews, group-leakage checks, and frozen-test checks as training inputs. If you change test rows after seeing training results, create a separate data version and a new untouched test set.

Completion can include deciding not to train. Hold the adapter plan if the prompt baseline already passes critical gates or frequently changing sources make RAG cheaper to maintain. Fine-tuning adds evaluation, registry, serving and rollback responsibilities as well as training code. Consider expected improvement, GPU time, reviewer-correction savings and incident costs, and move to a pilot only if the owner can operate the change.

Why does this happen?

Because with the wrong method, even large amounts of data and GPU time will not fix freshness, evidence, or format problems and will only create new operational debt.

When is it a problem?

If you start with “the data was added” as the success criterion, you cannot judge improvement over the base model, critical regressions, or when to discard the adapter.

Common beginner misconceptions

Fine-tuning is neither a database that stores documents safely in a model nor a step that replaces all prompt engineering and RAG.

How to verify it yourself

On the same frozen task set, compare the goals of the base, prompt/RAG, and the planned adapter in a table, and name the failures only the adapter should fix and the independent mandatory gates.

Do not select a base candidate by parameter size alone. Review the target-language and task frozen baseline, instruct or base variant, context and chat template, architecture, license, acceptable use, source-model release conditions, and intended runtime. PEFT may inject target modules even when the production runtime cannot dynamically load adapters, and redistribution conditions may change after merging. Work backward from final execution to avoid unsupported training outputs.

The manifest records the repository ID, immutable commit·weight hash, config and tokenizer file hashes, chat_template, generation defaults, and code trust setting. “Latest main” or a mutable tag may later point to different weights. Check the local cache against the manifest hash as well, and preserve snapshots of the download source·license. Read the limitations and evaluation scope in the base model card, but do not stretch the card's wording into approval of quality·rights for your own tasks.

The tokenizer and template are part of adapter behavior. Even within the same architecture, instruct families may use different user and assistant control tokens. Render training rows with the exact template and compare serving-runtime token IDs against golden fixtures. Record duplicate BOS/EOS tokens, unsupported system roles, added special tokens, and pad-token choices. If you change the tokenizer vocabulary, specify how embedding and lm_head weights will be trained and saved. If you do not change it, record that in the manifest too.

A small compatibility smoke test runs through base load, template rendering, one forward pass, LoRA injection and backward, adapter save and reload, and production-like generation. This step confirms support for architecture layer names, quant kernels and dtypes, drivers, and runtime adapters. Success with the official minimal example is only a starting point; approve the training plan only after the same stack passes at the actual max length and batch size and through the deployment endpoint.

Why does this happen?

Because an adapter file does not contain the base parameters themselves, only changes relative to specific layers and token sequences, its meaning changes when the base or input contract changes.

When is it a problem?

Keeping only the model name means mutable revision·template changes make training irreproducible, and production may receive incorrect control tokens.

Common beginner misconceptions

Sharing the same 7B architecture or tokenizer class does not guarantee compatibility of weight revision·chat template·license and adapter targets.

How to verify it yourself

Confirm the base weight, config, and tokenizer/template hashes, the license, and the adapter load or merge path in the target runtime, using the manifest and golden token fixtures.

The original LoRA paper freezes the pretrained model weights and injects trainable rank decomposition matrices into Transformer layers. Instead of changing the existing matrix W directly, it adds a low-rank update ΔW to the output, expressed by two small matrices. This is the key to reducing trainable parameters, gradients, optimizer state, and storage compared with full fine-tuning. Do not carry the reduction factor reported for a particular GPT-3 experiment in the paper over to other models, optimizers, or hardware; explain savings with your own run's trainable count and peak.

Rank r is the internal dimension of the update matrix. A higher rank provides more trainable parameters and expressive capacity, but it does not always improve task quality or eliminate overfitting. Alpha affects update scaling, and dropout·bias·initialization also change the results. When comparing candidates, change only the rank with the same dataset·token budget and, where possible, the same seed family, and review task·critical·general regression together with memory·step time.

After adapter injection, save a trainable-parameter report. Output total and trainable counts and ratios, per-layer target names, shapes and ranks, and modules_to_save; inspect requires_grad through model.named_parameters. After the first backward pass, check that base-parameter gradients are empty and optimizer groups contain only adapters. Do not rely solely on loss logs and miss unintended head or embedding training caused by wildcards, or runs with zero matched targets.

When default initialization is designed to be near a no-op, compare output with the base immediately after applying the adapter as a smoke test. Explain expected differences from new special tokens, modules_to_save, or bias training separately. After saving, reload the adapter in a fresh process and verify logits, generation, and trainable/inference modes on identical golden inputs. A working in-memory object does not prove the checkpoint is complete.

Why does this happen?

Because LoRA's advantages come from the scope of parameters actually changed, and a wrong target or optimizer group fundamentally changes training cost and results.

When is it a problem?

Trusting only the config file can let a run with 0 targets, missing layers, or a trainable base finish without verifying the claim that only a small adapter is trained.

Common beginner misconceptions

Using LoRA does not by itself guarantee the same quality as full fine-tuning, a fixed memory saving, or preservation of base capabilities.

How to verify it yourself

Record trainable counts and layer lists before and after injection, gradients and optimizer groups from the first backward pass, and save and fresh-reload results in the same manifest.

PEFT’s LoRA documentation gives examples targeting attention query·value layers by default and using all-linear for QLoRA-style training. Names and tensor grouping such as q_proj·v_proj·query_key_value differ by architecture. Check model.named_modules, official architecture code, and PEFT mappings, and record actual matched-layer counts. Mixture-of-Experts models separate shared·expert FFNs and routers, so do not directly copy a dense recipe.

Narrow attention targets use fewer parameters and less memory but may produce insufficient task adaptation. All-linear targets can learn more FFN updates, increasing capacity and cost. Include the target set, rank and alpha, trainable count, peak memory, tokens/s, and frozen task, critical and general regression tests in the comparison matrix. Separate experimental axes: changing targets, rank, data version and learning rate in one run prevents identifying which change caused the result.

Modules_to_save is an important contract that specifies modules to train·save in addition to adapter layers. If changing a classification head, new-token embeddings, or lm_head, verify why it is needed and how it is stored separately from the base. Bias options can also alter base-equivalence and merge behavior. Record rank·alpha·dropout·targets·bias·task type and base references in PEFT config, and check actual checkpoint parameter keys against it.

The selection criterion is not the largest configuration but the simplest candidate that passes every required gate. If rank 8 attention-only reaches 93% task success, 97% critical, and acceptable regression, while rank 32 all-linear reaches 94% task success but fails the general regression and memory limits, you can choose the former. If repetition variance overlaps the score difference, do not claim the more complex setting won; run additional seeds and a failure review.

Why does this happen?

Because target and rank change the trainable scope, memory, expressiveness, and checkpoint structure all at once, requiring architecture checks and controlled comparisons.

When is it a problem?

Copying community configurations can omit layers, train unintended modules, and cause high-rank costs to be mistaken for actual task improvements.

Common beginner misconceptions

All-linear targeting or a higher rank does not always mean a more complete LoRA; for small tasks and datasets it can cause larger regressions and overfitting.

How to verify it yourself

Compare matched-module·trainable-parameter reports alongside task·critical·regression·memory results for candidates with one factor changed at a time.

The original QLoRA paper passes gradients through a frozen 4-bit quantized pretrained model into a LoRA adapter and manages memory with NF4, double quantization, and a paged optimizer. This approach greatly reduces base weight memory, but it is not a table showing that every model and GPU works at a particular size. Cite the paper's 65B·48GB result as a result of that study's model·sequence·batch·kernel and evaluation scope, and require exact workload measurements on your own hardware.

The Transformers bitsandbytes documentation describes 8-bit and 4-bit training support for extra parameters, and provides NF4 and a separate bnb_4bit_compute_dtype setting for training with a 4-bit base. Record quantization type, nested or double quantization, storage and compute dtypes, adapter dtype, libraries, CUDA, drivers, and hardware capabilities in the manifest. Insufficient BF16 support or operator fallback can affect speed and stability, so verify device placement and actual kernel execution.

The memory ledger includes quantized base weights and quant metadata, dequantization buffers, adapter parameters·gradients·optimizer state, activation, attention workspace, and allocator reserves. Activation is sensitive to sequence length·micro-batch·hidden size·layers; gradient checkpointing uses more compute to reduce memory. Packing can reduce padding waste, but check each packed sequence’s length and mask. Record separate peaks for load, first forward, backward, optimizer step, evaluation, and save.

Do not silently change targets during OOM recovery. Check one item at a time: whether reducing micro-batch and increasing accumulation preserves effective batch, whether shorter max length removes long-task coverage, and whether checkpointing·optimizer changes affect step time and convergence. After warm-up, measure peaks·tokens/s, host memory·offload I/O, and runs lasting at least 30 minutes. Hardware fit passes only after completing the target dataset, evaluation·save, and interrupted-checkpoint resume.

Why does this happen?

This is because QLoRA changes weight memory while other large training-memory items and hardware·kernel requirements remain, so simple model-B·VRAM tables often fail.

When is it a problem?

Setting max length·batch from the “4-bit” label alone can cause OOM during backward·evaluation·save, while offload stalls can push job duration beyond the target.

Common beginner misconceptions

QLoRA is not a method that trains the LoRA adapter and all computation in 4-bit, nor does it guarantee a specific 7B model on any 16GB GPU.

How to verify it yourself

Using a manifest of the exact quant, compute, and adapter dtypes and sequence, batch, and checkpointing settings, repeatedly measure peak memory, step time, completion, and resume at each step.

Hugging Face chat-template documentation explains that role/content lists become model-specific control-token sequences and incorrect tokens can substantially degrade performance. Render system/user/assistant conversations with the exact tokenizer revision and preserve rendered text and token IDs. Use golden fixtures to check duplicated BOS/EOS or end-of-message tokens, system-role and multi-turn support, and whether the serving runtime produces the same sequence.

TRL SFTTrainer’s assistant_only_loss can restrict loss to assistant responses in conversational datasets, provided the template supports generation spans. Also verify the completion-only default for prompt-completion datasets against the actual version and configuration. Mark each sample token as -100 or a target to check that user instructions and system policies are not being learned as model outputs. Do not assume that enabling a configuration option makes the mask cover the intended spans.

Measure length in rendered tokens, not raw characters. Record in a per-row report whether truncation cuts only long user context or also the final JSON brace or refusal conditions of the answer, and on which side truncation occurs. If even one target token is cut from critical output, hold the row and fix the chunking, max length, or data. When packing is enabled, check with a token map of a small known batch that example boundaries, position and attention behavior, and loss masks do not bleed into each other.

A dataset version includes not only the raw JSONL hash but also conversion code, tokenizer/template hash, max length, packing, and loss configuration. The same row is a different training input if any of these values change. Do not change the training and inference templates at the same time; re-evaluate a template candidate on the same frozen set against both the base and the previous adapter. Template mismatch is a classic deployment failure: it is not an adapter quality problem, yet a model that looks the same answers strangely.

Why does this happen?

A causal LM learns control-token sequences and label masks, not role objects, so if this conversion is wrong, even correct rows teach different behavior.

When is it a problem?

Looking only at raw JSON and train loss will not reveal user prompts being learned, target truncation·duplicated special tokens, or serving template mismatches.

Common beginner misconceptions

The mere presence of user and assistant roles in messages does not guarantee assistant-only loss or correct model-specific token serialization.

How to verify it yourself

Compare the rendered text, token IDs, loss mask, and truncation of golden conversations in both training and serving, and confirm 0 missing critical targets.

Effective batch is micro-batch × gradient accumulation × data-parallel workers, not merely one GPU's micro-batch. Recording only epochs misses changes in optimizer steps and tokens seen caused by packing, drop_last, or world size. Put dataset rows, rendered tokens, micro/effective batch, accumulation, max_steps/epochs, warmup, scheduler, learning rate, and seed in the run manifest. Keep token budgets and evaluation intervals identical across candidates.

The pilot uses a small dataset in which a person can inspect every sample and output, and checks forward·backward·optimizer, evaluation, and save·fresh reload. In the full run, record training·validation loss, task metrics, critical gates, gradient norm·NaN/Inf, learning rate, tokens/s, peak allocated/reserved memory, skipped·truncated rows, and checkpoint hashes at the same global step. Even if loss decreases, stop on gradient explosion·throughput collapse or critical metric degradation.

Changing target modules, rank, learning rate, data version and maximum length together prevents identifying the cause of improvement. Hypothesize one major failure cause from the baseline configuration and change one axis. For final candidates, examine metric distributions and overlapping failures across clean runs to avoid dependence on one lucky seed. Frameworks and kernels may not guarantee complete bitwise determinism, so predefine tolerances and do not hide reproducibility warnings.

Checkpoints must also be tested for failure recovery. Restore the optimizer, scheduler, random state, and sample position from a mid-run checkpoint, continue training, and compare the result with an uninterrupted reference within tolerance. For disk-full, corrupt checkpoint, OOM, and evaluation failures, use status tracking and atomic publishing so a partial artifact is not mistaken for a release. An experiment becomes approvable evidence rather than a personal notebook only when the run can be replayed in a clean environment from a dependency lock and code, config, and data hashes.

Why does this happen?

Batch, steps, schedules, and data order jointly determine optimization. Recording only a few numbers prevents reproducible candidate comparisons and resumption after failures.

When is it a problem?

If only a screenshot of the final loss is kept, no one can tell which data, token budget, and learning rate produced the checkpoint, or whether the same run was resumed after an OOM.

Common beginner misconceptions

Fixing one seed or using the same configuration file does not automatically guarantee bitwise reproducibility across hardware and kernel differences or equal quality.

How to verify it yourself

Link the run manifest, per-step loss, task metrics, gradients, memory, throughput, clean repeat runs, and interrupted-resume results to the checkpoint hash.

Training loss measures fit to next-token targets in seen rows, and validation loss is also a proxy for held-out token prediction. Production promotion gates on task success, schema compliance, critical-class recall, unsafe compliance, unsupported answers, over-refusal, and human corrections. Run the base model, prompt/RAG baseline, previously approved adapter, and each checkpoint on the same frozen set, decoding settings, and runtime. Include contamination and memorization probes to check for verbatim reproduction of dataset and test phrases.

Critical gates cannot be offset by averages. Even if overall task success increases by 2%p, a drop in refusal of unauthorized actions from 98% to 91% can justify a hold. The general-regression set covers capabilities agreed to be preserved from the base, such as language, instruction following, long inputs, code, and reasoning. Blind human reviewers compare output pairs using a rubric. If an automatic judge is used, retain its version, prompt, bias, and disagreement with humans as separate evidence.

Transformers Trainer can load the best checkpoint using an evaluation strategy and metric_for_best_model, but the task owner is responsible for metric selection. Apply must-pass gates separately from the single best metric, and verify that save and evaluation intervals compare the same checkpoints. Early stopping depends on validation noise, evaluation frequency, and patience, so preserve candidates from before the final checkpoint and inspect failure taxonomy by step.

Record confidence and limitations in the selection result. Put the mean and range across clean runs, critical failure count, improvement or regression versus the base, memory and latency, and reviewer corrections in one release table. Do not widen the allowed drop after seeing results. If a candidate fails, go back to data, target, or schedule causes and create a new run ID, and separate development validation from the final frozen test so the same test is not optimized repeatedly.

Why does this happen?

Because token loss hides the separate error costs of real tasks, safety, and original capabilities in a single value, production selection needs independent gates.

When is it a problem?

Automatically adopting the final or lowest-loss checkpoint can miss better task metrics at earlier checkpoints and later regressions in refusal behavior and general capabilities.

Common beginner misconceptions

A falling validation loss or preference by an automated judge does not guarantee factuality, safety, human task success, or the absence of data leakage.

How to verify it yourself

Compare the base, previous version, and all candidates blind on the same frozen set, and apply every predefined task, critical, regression, and operational gate.

PEFT checkpoint documentation centers on adapter_model.safetensors and adapter_config.json; adapter state does not include base-model parameters. Config may contain base references·revisions, rank·alpha·targets, and other settings, but do not assume it saves all provenance automatically. Bundle exact base·tokenizer/template hashes, adapter-file hashes, PEFT·Transformers·TRL·bitsandbytes and driver versions, LoRA/quant/training config, dataset·code commits, and evaluation reports in the registry.

Load the bundle in a fresh environment and verify that the trainable adapter attaches correctly in inference mode and reproduces golden inputs and the frozen set. Dynamic adapter loading and merged deployment are separate artifacts. Compare logits, task results, critical cases, latency and precision before and after merging, and review base/adapter licenses and redistribution terms again. Record intended and out-of-scope uses, data scope, metrics, limitations, known failures and the owner in the model card.

After offline gates pass, observe schema failures, critical incidents, human correction·override, p95·memory, and adapter-load errors in shadow or limited canary using production-like templates·generation config. Define canary criteria, automatic stops, and the approval owner beforehand. Do not use user traffic merely to fill test counts or replace everything without monitoring. New base·data·policy·runtime versions are all re-review triggers.

Rollback is not simply deleting an adapter file. Reload the previous compatible base revision, tokenizer/template, runtime·quant·generation config, and approved adapter, verifying actual cache·worker replacement. Repeatedly test recovery on exact candidate-failing inputs and preservation of normal regression results within the recovery-time target. Trace affected runs·data sources through lineage and record hold·retrain·discard decisions to close the release.

Why does this happen?

Because an adapter depends on multiple external artifacts and runtimes, and production problems arise from that combination, it must be managed as a deployable, recoverable bundle.

When is it a problem?

Keeping only adapter_model.safetensors can allow loading to succeed after the base or template changes while behavior differs, preventing an exact return to the previous state.

Common beginner misconceptions

Being a small adapter or in safetensors format does not automatically guarantee provenance, license, quality, safety, or the equivalence of merge results.

How to verify it yourself

Check fresh loading, evaluation before and after merging, canary criteria, and, on rollback to the previous full stack, recovery from the same failure, normal regression, and recovery time.