# Prompt And Experiment Flow Notes

This note reviews the exploration process from early `v3_3` through the current
`v3_19`. It intentionally focuses on prompt and workflow design, not on the Pong
policy details.

## Most Effective Prompt Requirements

### Build A Hypothesis Map Each Round

The strongest improvement in the workflow came from requiring each round to
start with multiple candidate hypotheses, each tied to evidence, a minimal
diagnostic, a possible minimal change, and an exclusion criterion.

This prevented the loop from becoming "try one patch that sounds plausible."
It made the agent compare state, physics, action timing, evaluation, and
structure hypotheses side by side. The useful effect was not that every
hypothesis was correct. The useful effect was that wrong hypotheses became
explicit and could be ruled out by small experiments.

### Prefer Diagnosis Before Policy Changes

Early rounds tried direct target or action patches and quickly hit no-improvement
loops. Later rounds improved after the prompt insisted that diagnosis come
first: trace points, bucket failure families, compare outcomes, then run a
counterfactual probe before touching the retained policy.

This was especially useful when observational traces were ambiguous. A trace
could show a recurring pattern, but only a controlled probe could distinguish
"this pattern explains the loss" from "this pattern is just correlated with the
loss."

### Allow Only One Minimal Strategy Change Per Round

The one-change rule made score movement interpretable. When a policy improved,
the project could connect the delta to one intervention and add focused tests.
When it failed, the exact change could be withdrawn without losing unrelated
diagnostic assets.

This also discouraged broad fixes. The effective changes were usually narrow,
stateful, and guarded by prior probes. The prompt worked best when it framed a
policy change as the final step of a diagnostic chain, not as the main work of
the round.

### Require Multi-Seed Validation

The process became more reliable after validation expanded from small seed sets
to 20 seeds, then 40, then 80. A change was not accepted just because it helped
one failing seed or raised a local maximum. It had to preserve or improve mean,
min, max, and seed-level behavior against the current retained baseline.

The multi-seed requirement exposed overfitting quickly. Some local probes looked
promising on known failures but collapsed other seeds. Those results were still
valuable because they marked unsafe families and shaped later probes.

### Withdraw Failed Changes But Keep Diagnostic Assets

This was one of the highest-leverage rules. Failed policy patches were removed,
but scripts, JSONL probes, summaries, failure notes, and progress conclusions
were retained. That let the project accumulate knowledge even during rounds with
no score gain.

The retained assets also stopped repeated mistakes. A later round could see
which broad target offsets, action overrides, or final-frame probes had already
failed, then design a more discriminating experiment instead of retrying the
same idea.

### Update `progress.md` Every Round

The progress log became the project's memory. It captured the current baseline,
validated and rejected hypotheses, exact commands, score deltas, artifacts, and
next minimal experiments.

This mattered because the exploration was long-running. Without a durable log,
the agent would repeatedly re-discover old failures or compare against the wrong
baseline. With the log, each round could start by reconstructing the real state
of the project.

### Do Not Stop At "Needs Structural Change"

The early process treated structural needs as a stopping point. Later prompts
made that invalid: structural problems had to be split into executable
diagnostics. That instruction directly enabled tools such as counterfactual
action probes, family summarizers, outcome comparators, and history diagnostics.

The useful pattern was: if the current probe type cannot answer the question,
build the smallest tool that can answer one slice of it. Do not rewrite the
whole system; add one diagnostic that converts a vague structural concern into
data.

## Prompt Requirements That Caused Inefficiency Or Premature Stops

### "Stop After Three Rounds Without Improvement"

This was too score-centric. The early process stopped after three rounds without
accepted policy improvement even though the project had learned important
negative facts and had identified the need for richer structural diagnostics.

A better stop condition is not "no score increase." It is "no score increase and
no new verifiable question or reusable diagnostic asset." Diagnosis can be real
progress even when the retained policy does not change.

### Treating Structural Change As A Stop Condition

Stopping because a structural change is needed is backwards for this kind of
project. Structural uncertainty is often exactly where the next high-information
experiment lives.

The better prompt is to require decomposition: name the structural problem,
identify the smallest missing measurement, build or run one diagnostic, and
record what it rules in or out.

### Requiring Policy Changes Too Early

When the prompt or momentum pushed toward a policy patch before the model was
clear, the result was low-value churn: broad target offsets, broad action
overrides, and changes that matched a visible trace but did not survive
validation.

The better requirement is evidence-gated change: no policy edit unless a
diagnostic identifies a narrow intervention, the counterfactual improves a
multi-seed comparison, and the failure mode is understood well enough to write a
test or explicit guard.

### Looking Only At Local Loss Tails

Tail frames are useful but incomplete. Several early local fixes focused on the
last few frames before a miss. That missed earlier setup, score-chain, action
phase, and history-dependent causes.

The better flow uses tail traces as an index into a broader family analysis:
compare winning and losing versions of the same family, inspect earlier anchors,
and test counterfactual actions at the earliest discriminating point rather than
only at the final visible miss.

## Reusable `/goal`

```text
/goal Autonomously improve the current agent through reproducible, evidence-led
experiments.

Work only from this repository's code, logs, diagnostics, and newly generated
data. Do not copy external solutions or rely on unstated assumptions.

At the start of each round:

1. Read the project state:
   - agent policy code, state extraction, evaluation scripts, diagnostic scripts,
     tests, README, progress notes, summary/trial/episode/failure logs, and
     existing diagnostics.

2. Identify the current baseline:
   - retained best policy;
   - mean/min/max and seed-level distribution;
   - strong and weak seed families;
   - hypotheses already supported, rejected, or unresolved;
   - structural questions already raised but not yet tested.

3. Build a hypothesis map with multiple candidates across state, dynamics,
   action semantics, timing, control, evaluation, and diagnostic structure.
   For each candidate, state:
   - what failure it explains;
   - what evidence supports it;
   - the smallest diagnostic experiment;
   - the smallest policy or model change if true;
   - what can be ruled out if false.

4. Prefer diagnosis and modeling over policy edits. Choose the experiment with
   the highest information gain, especially one that can distinguish multiple
   hypotheses. If a structural gap blocks progress, split it into the smallest
   runnable diagnostic and complete that diagnostic.

5. Edit the retained policy only when evidence is clear. In one round, make at
   most one minimal policy change. Do not bundle unrelated fixes. Do not edit
   policy code just to make the round feel productive.

6. Validate against the current retained baseline on enough seeds for the risk:
   - use small probes only for discovery;
   - expand to broader seed sets before accepting;
   - compare mean, min, max, seed deltas, and failure modes.

7. Solidify or withdraw:
   - if effective, keep the minimal change, add focused tests, update docs and
     experiment logs;
   - if ineffective, revert the policy change but keep useful diagnostics,
     data, failure notes, and conclusions.

8. End every round by updating progress.md with:
   - current baseline;
   - hypothesis map;
   - diagnostics run and commands;
   - supported and rejected hypotheses;
   - whether policy changed;
   - score comparison;
   - retained and withdrawn artifacts;
   - structural issues discovered;
   - the next smallest experiment.

Continue until one of these is true:
   - the explicit performance target is reached;
   - several consecutive rounds produce neither score improvement nor new
     verifiable questions nor reusable assets;
   - the environment or required data is unavailable and the blocker is recorded;
   - structural questions have been decomposed and repeated diagnostics no
     longer produce information gain.

Never treat "needs structural change" by itself as a reason to stop.
```

## Operating Advice For Future Codex Runs

### How To Choose Candidate Hypotheses

Start from mismatches between the current baseline and the logs, not from a
desired patch. Good candidates usually explain a repeated seed family, a
specific score floor, a difference between wins and losses, or a discrepancy
between observational traces and counterfactual results.

Prefer hypotheses that can be separated by one small experiment. A useful
hypothesis map should include at least one state/model question, one
action/timing question, one control question, and one evaluation or overfit
question. If all candidates suggest the same policy edit, the map is too narrow.

### How To Decide Whether A Counterfactual Probe Can Be Solidified

A counterfactual is a candidate for policy only when it satisfies all of these:

- It compares against the actual retained baseline, not a stale or mutated one.
- It changes a narrow, explainable state/action family.
- It improves mean or min without introducing new seed collapses.
- It has seed-level deltas that match the claimed mechanism.
- It survives an expanded seed guard appropriate to the risk.
- It can be implemented as one minimal guarded policy change and covered by a
  focused test.

If a probe improves one seed but harms another unexplained seed, keep it as a
diagnostic result and do not solidify it yet.

### How To Prevent Seed Overfitting

Separate discovery seeds from guard seeds. Use small known-failure subsets to
find candidate mechanisms, but require broader validation before accepting. When
a change is found on seeds `0..19`, check later seed blocks or the full retained
validation set before calling it best.

Track min and per-seed regressions as seriously as mean. A higher mean with a new
catastrophic seed should usually be rejected or narrowed. When a probe depends
on score state or history, verify that the same gate does not accidentally catch
unrelated seed families.

### When To Expand The Validation Set

Expand validation when:

- a candidate moves formal scores on the discovery set;
- the change is stateful, score-gated, or likely to affect long chains;
- a prior accepted change shifts trajectories enough to stale old traces;
- the current baseline has improved enough that remaining failures are rarer;
- the min score is being used as the acceptance argument.

Small probes are acceptable for ruling out obviously bad ideas. Acceptance needs
the broader guard.

### When To Keep Diagnostics But Avoid Policy Changes

Keep diagnostics only, with no policy edit, when:

- evidence is observational but not counterfactual;
- a probe is inert, mixed, or only improves an isolated seed;
- the same local state appears in both wins and losses;
- the intervention changes too many contexts to explain safely;
- the baseline comparison is stale;
- the testable claim cannot yet be written as a narrow guard.

In these cases, the round is still useful if it adds a script, summary, trace,
failure note, or exclusion that changes the next experiment.

## Checklist

### Before Each Round

- Read current policy, state extraction, eval scripts, diagnostic scripts, tests,
  README, `progress.md`, logs, and diagnostics.
- Identify the retained baseline and exact score distribution.
- List strong and weak seeds or families.
- Mark stale traces caused by accepted policy changes.
- Build a hypothesis map before choosing an experiment.

### Before Changing Policy

- Confirm the mechanism has counterfactual or equivalent causal evidence.
- Confirm the comparison is against the current retained baseline.
- Check mean, min, max, and seed-level deltas.
- Prefer the earliest discriminating anchor over the final loss frame.
- Make only one narrow change with an explicit guard.

### Before Solidifying

- Run the appropriate multi-seed validation.
- Run tests and compile checks.
- Add or update focused tests for the new guard.
- Update README, `progress.md`, summaries, trials, episodes, failures, and
  diagnostics.
- Record the exact command and score comparison.

### When Rolling Back

- Revert only the ineffective policy change.
- Keep diagnostic scripts that remain generally useful.
- Keep JSONL/JSON summaries that record the negative result.
- Keep failure notes and `progress.md` conclusions.
- Add the rejected hypothesis to the next-round exclusion set.
