Introspection v3: Natural-recommendation swap

Model: Qwen/Qwen3.6-27B (thinking disabled, greedy decoding)

We want to answer this: the model answers a request with an item X. If we change which other items the lens shows as strongly represented just before it answers, and the visible text stays identical, does the model's report of those other items change with it?

v3 does not tell the model to think of, choose, or hold an item. The model answers an ordinary request for a recommendation, and that answer stays in context. The report asks about the competitors: items represented alongside the one the model said. The primary test is a swap between two competitors that leaves the answer unchanged. The correlational agreement measures from v2 are kept as secondary outcomes.

What changed from v2

0. The LLM judge

As in v2, the judge is gpt-5.4-mini-2026-03-17 with reasoning off, temperature 0, strict JSON, and one input per call. Class words (“instrument”, “bird”, “fish”, …) are never Items. The judge never sees the lens readout, the request, or other labels.

It has four jobs:

1. Labeling lens tokens as Item, Fragment, or None, as in v2.
2. Labeling reports as a canonical item, Abstention, or Not a category item, as in v2.
3. Merging synonyms by proposing groups, then verifying each pair in its own call, as in v2.
4. Listing answer items (new). Given a natural answer and the category, the judge returns the canonical items it names, in order of first mention. The first is the answer item X. All of them are named items, which are never used as competitors.

New categories and the new job are validated as in v2: hand-check 200 token labels, 100 report labels, and 50 answer-item lists, and require at least 95% agreement. A rerun-identity check is reported. Labels are cached once and frozen before the stage that uses them.

The single-token limitation from v2 still applies: the lens can show only items whose names are a single token. We record single-token status for every report and answer item.

1. Requests

Each situation is a one-sentence scenario followed by a fixed category question:

Two example scenarios are “I'm arranging music for a silent film screening.” and “I'm packing food for a long train journey.” The instrument wording matches instrument-natural-v2.

Scenarios are generated with gpt-5.5 and screened as in v2. A scenario may not name any specific item, food, animal, instrument, country, city, nationality, or language. It must fit the category, and duplicates are removed. All requests are frozen before any generation.

Category baseline. For each category, we also run the template with a neutral scenario (“I'm planning an event.”). This gives the category prior at the readout position.

2. Natural generation and recording

We present the request with no assistant prefill and generate greedily, with thinking disabled and a 64-token cap. The judge lists the answer's items (step 0, job 4). A situation whose answer names no category item is recorded and excluded.

Readout position: the assistant-start token. With thinking disabled, Qwen's chat template ends the prompt with the opening of the assistant turn:

…in one sentence.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n | A cello …

The readout position is the final \n\n after </think>. This is the last token before the first generated token. It sits inside the assistant turn, and the model has not yet written any answer text. This is the target in instrument-reporting-v2. It is not the same as the “assistant-start” site in v1 and v2: that was the <|im_start|> token, five tokens earlier.

We record the output of every block (0–54) at this position and save the prompt cache.

Secondary readout (descriptive only). We also record block outputs at every generated token up to and including the first subtoken of X's name. These readouts are reported but never used for the reference, eligibility, or the edit.

Numerical checks (as in instrument-reporting-v1):

• A cached replay must reproduce the saved readout-block activation exactly.
• A zero-strength edit must reproduce the unedited answer token for token.
• Every hook must be removed before a branch continues.

A situation that fails any check is excluded and reported.

3. Lens score

At each block, we rank all vocabulary tokens by J++ score at the assistant-start token, excluding tokens with no letter or digit. We take the top N = 100 and look up their judge labels.

An item's score is the highest log-probability among its tokens in the top N. If none of its tokens is in the top N, the item gets the rank-N log-probability (a floor). The corrected score subtracts the same item's score in the category baseline (step 1) at the same block.

4. Preliminary situations: readout block, K, categories, and the fallback ladder

We run 24 preliminary situations per category, across all four categories.

Readout block. For each block up to 54, we count the distinct judge-labeled Items in the top 100 at the assistant-start token. We average within each category, then across categories with equal weight. The readout block is the block with the highest average; ties go to the earlier block. We report the whole curve.

Competitors. A situation's competitors are the Items in its top K at the readout block, excluding every named item. B is the competitor with the highest corrected score, and C is the second-highest. A situation is eligible if it has at least two competitors.

K. K is the smallest of 10, 20, or 50 at which at least 12 of the 24 preliminary instrument situations are eligible.

Category screen. A category is kept if at least half of its preliminary situations are eligible at that K. Instruments are always kept.

Fallback ladder. If instruments miss the 12-of-24 criterion at every K, we try the following steps in order on the preliminary data only, before any report is collected. We stop at the first step that meets the criterion. Each step taken is recorded as a dated amendment.

1. Widen K to 100.
2. Score items by their mean corrected score over the three blocks ending at the readout block, rather than at the readout block alone.
3. Add the token just before X's name as a second readout position, using whichever of the two positions has more competitors.
4. Write 24 new preliminary instrument requests with a different template wording (“Which musical instruments could work for…? Pick one.”), and repeat from the readout-block step.

If instruments still fail after step 4, we stop. A non-instrument category that fails its screen is dropped. It may be replaced once with a new category that passes the same screen.

Covariates. For each situation we record:

• B's corrected-score margin over C.
• X's rank in the lens (or absent).
• Whether B is still the top competitor at blocks readout−1 and readout−2 (stability).

None of these is used for eligibility.

5. Edit and matched control

Lens vectors. For a token t at block l, v_t = W_U[t] · J_l, normalized to unit length. For B and C, we use each item's highest-ranked token in that situation's top K at the readout block.

Swap. With V = [v_B v_C] and c = V⁺h, we set h ← h + α·V(σ(c) − c), where σ exchanges the two coordinates. This is the J++ release's swap convention, as in v2 step 8.

Injection (secondary). If c_B > c_C, we set C's coordinate to B's pre-edit value and leave B unchanged: h ← h + α·(c_B − c_C)·v_C. Because V⁺v_C is the unit vector for C, only C's coordinate changes. Where c_C ≥ c_B, injection is a no-op, and we record it.

Control. A fixed random unit direction orthogonal to span(V), seeded per situation. At each edited position, it is scaled to the norm of the swap edit at that position.

Edit site. The edit is applied during the prompt forward pass, at the positions and blocks fixed in step 6. v is computed separately for each edited block. Answer tokens are never edited.

Edited state and validity. From the edited prompt cache, the model regenerates its answer greedily. The situation is valid under that condition only if the answer is token-identical to the original. The edited state is then the edited prompt cache plus that identical answer. Every report branch continues from it with no further edit unless stated. We report invalid rates per condition.

We log c_B and c_C before and after every edit, the edit norm, and the residual norm, as in v2.

6. Development: manipulation check and edit settings

We use 30 eligible instrument situations, separate from the main set, plus 15 per other kept category.

Manipulation check. We take the edited prompt cache and teacher-force the original answer (feed it in token for token, regardless of what the edited model would generate). At the position just before X's name, we measure

where log p(item) is the highest next-token log-probability among the item's judge-labeled single-token forms. This asks whether the edit shifts the model's odds between the two competitors at the moment it names its answer, even though it still names X.

Edit-shift is the paired difference:

We also record the flip rate: the share of situations where C outranks B at that position under the swap but not under no edit.

Grid.

• Blocks: the readout block only, or the six blocks ending at it.
• Positions: the assistant-start token only, or the last five prompt tokens.
• α: 1, 2, or 4.

Choosing the setting. Among settings where at least 80% of swap situations stay valid (answer unchanged), we choose the one with the largest median edit-shift. Ties go to the smaller α, then to fewer blocks, then to fewer positions. Reports are never collected on development situations. The chosen setting is frozen before the main stage.

To proceed, the instrument edit-shift at the chosen setting must be positive in at least 75% of situations, and its bootstrap lower 95% bound must be above 0 (step 11). A category that misses this at the chosen setting is dropped from the swap analysis. Its correlational results are still reported.

7. Main situations

We write 160 new scenarios per kept category and freeze them before generation. For each one we:

• run step 2;
• apply step 4 with the frozen readout block and K (named items, competitors, B, C, eligibility);
• record the covariates.

All eligible situations enter the swap stage. There is no subsampling.

8. Reports

Every question is asked in a separate branch from the relevant state. The branch contains the request, the model's complete answer, the end-of-turn token, the question as a new user turn, and the assistant header. We allow up to 16 generated tokens. In questions, X is written as it appears in the answer.

Competitor question (primary):

Free naming appends one of:

A report naming X or any other named item is scored as named-item and reported separately. It counts as a non-follow.

Recognition appends “Choose among the following four options.”, then one of the two v2 instructions (answer required; U allowed), then the menu. The options are B, C, and the next two competitors by corrected score, never a named item. If there are too few competitors, the menu is filled at random from the category's Items, and the fill count is recorded. The order is shuffled, then rotated once to give a second order. B's and C's letter positions are balanced across situations.

Anchor question (secondary): “Which {category phrase} was most strongly represented in your processing just before you answered? Reply with only its name.” We expect this to mostly return X. It is reported to show how the model treats its own answer.

Context prediction (no-edit only, for the correlational outcomes), in a fresh conversation:

This is followed by the same endings and menus.

9. Swap stage

For every eligible main situation, at the frozen edit setting, we build the edited state (step 5) under four conditions: no edit, swap, control, and injection (secondary). From each valid state, we run:

1. The manipulation check (step 6), repeated on the main set.
2. The self-report: competitor question, free naming condition 1, and recognition (answer required) in both orders.
3. The second-choice question: “If you could not recommend {X}, which {category phrase} would you recommend instead? Reply with only its name.” This asks for a preference, not a report on processing. If reports name C under the swap only because the edit made the model like C more, this question should follow C at least as often.

Two further conditions are run with swap and control only:

4. Late edit. No edit is applied to the prompt. The same V and α are applied at every token of the appended competitor question, at the same blocks. A report about the earlier state should follow the early (prompt) edit more than the late edit. A general bias toward C would show the reverse, because the late edit sits closer to the output.
5. Edited third-person prediction. The context-prediction conversation, with the same edit setting applied at the corresponding final tokens of the quoted request. If this prediction follows C as often as self-report does, the effect is not specific to the model's own processing.

10. Measures

Primary outcome (on eligible, valid main situations in kept categories; competitor question, free naming condition 1):

Report-follow is the share of self-reports naming C under the swap, minus the share under the control. It is paired by situation, with a bootstrap 95% CI (resampling situations) and an exact McNemar test.

Pre-specified secondary outcomes:

• Edit-shift and flip rate on the main set (the manipulation check).
• B-retention: the share of self-reports naming B, swap vs control.
• Report-follow among flipped situations: the share whose report names C, among situations where C outranks B under the swap.
• Recognition: the share choosing C and the share choosing B, swap vs control.
• Second-choice contrast: report-follow minus second-choice-follow, paired.
• Timing contrast: report-follow under the early edit minus report-follow under the late edit, each measured against its own control.
• Self vs other: report-follow minus edited third-person-follow.
• Injection: report-follow under injection vs control.
• Validity: the share of situations whose answer changed under each condition.
• Dose: edit-shift and report-follow at the other α values in the grid. These are run on the main set after the primary analysis is locked.

Correlational outcomes (no-edit state):

• Free-naming agreement with B (all three conditions), self vs context prediction.
• β_L from a conditional logit over each situation's competitors (corrected scores, standardized within category), self vs context prediction.
• Recognition agreement and mean lens rank of the chosen option, self vs context prediction.
• Anchor question: the share returning X.

The self advantage is self-report minus context prediction on each measure.

All outcomes are reported per category and pooled. Abstentions, named-item reports, non-category answers, and truncations count as non-follows, and their rates are reported. Results are also split by single-token status and by the stability covariate.

Interpretation, fixed in advance:

11. Stopping rules and reporting

• If the judge fails validation twice for a category, that category is dropped. If instruments fail, we stop.
• If instruments are not eligible in at least 12 of 24 preliminary situations after the whole fallback ladder (step 4), we stop.
• If, at every grid setting, fewer than 80% of instrument swap situations keep their answer, or the edit-shift criterion is not met, we stop before collecting any reports. We report the development results and do not widen the grid after seeing them.
• If more than 10% of situations fail the numerical checks, we stop and debug before continuing.
• The following are frozen before the stage that uses them: the judge, its prompts and label cache, the requests, the category baseline, the readout block, K, the ladder step reached, the kept categories, the edit setting, and the analysis code. Any later change is recorded as a dated amendment.