Local work and the questions still open.

Can useful project knowledge become instructions an agent can act on? Small synthetic projects showed that source-linked context can deliver rules missing from a task description. A later comparison on real repository tasks did not establish a benefit from a compiled guide.

Jonathan Simone directed these Claude-based experiments with AI assistance. This is an author report from private local research, reviewed October 7. The published aggregate summary gives the counts, methods, corrections and identities of the retained source records. It is a curated summary, not raw logs or independent reproduction.

What worked on synthetic projects

The first task asked an agent to implement a fictional project’s rules. Each build faced 104 hidden checks, including 38 checks for rules recorded only in the project’s documents. Across five builds per arm, the task-only agent averaged 63.5% of all checks passed; an agent with library tools averaged 98.3%. Those percentages are overall check scores. On the house-rule subset, the task-only builds passed 0 of 38; library-assisted builds passed between 34 and 38 of 38.

A later method compiled the documents into a source-linked guide. All 11 tested builds passed all 104 checks, across four compilations of the guide. These were repeated builds of one synthetic task, not 11 different tasks. Here, “teaching” means supplying context and instructions; it does not mean changing model weights or establishing durable learning.

The simpler baseline mattered

In a second synthetic project, compiled guides passed all 86 checks in 11 builds across three compilations. Pasting the whole corpus into the prompt also passed all checks in five builds. The corpus fit in context. That comparison did not establish a unique advantage for the compiler.

The synthetic documents and checks were AI-authored. An early document selector also performed worse than the task-only agent. An initial run with unintended access to harvested material was set aside; the reported first-project scores use the repaired rerun. These development corrections remain in the aggregate summary.

What happened on real repository tasks

The corrected follow-up compared 37 tasks, with two builds per task in each of three arms: an ordinary agent, an agent with a compiled guide, and an agent with library tools. Across 222 builds, the mean fraction of previously failing checks that passed was 53.8% for the ordinary agent, 48.8% with the guide and 49.0% with library tools. These are partial-check scores, not task completion rates. Only one ordinary-agent build was fully resolved; neither other arm had a fully resolved build.

The earlier version of this evaluation had interface defects. These figures come from the repaired comparison. They do not establish a coding benefit from the guide, or a universal verdict on prepared context. Task selection, local evaluation and the private harness limit the result. Full preparation, engineering and review costs have not been compared.

The next question

The work now explores a narrower briefing: what is current, what was decided, what happens next and which pitfalls matter. A briefing prototype is built and unit-tested; its benefit evaluation remains unrun. The useful comparison is whether that briefing helps a new session more than ordinary source reading, with preparation and review effort included.

The GPT-assisted simulation feature is separate practical work. Its corrected ordinary baseline also succeeded. The public synthetic replay is a separate runnable correction-and-restore example. Neither reproduces the private studies reported here.

This author report describes local findings from October 6 and 7, 2026. Jonathan Simone directs the work with AI assistance. The private implementations and original receipts are not distributed; this is a public summary, not a runnable release or external reproduction.

The task, the preparation and the result

The task was to save a simulation session, reconstruct it in a fresh process and continue. The save file records a replay recipe under the current simulation rules. An agent built the feature using a manually prepared workflow and source map. A separate AI session ran the checks and continued a saved session.

• Preparation: selected project sources and a manually written workflow guided the agent.
• Output and check: a staged save/load feature passed checks for saving, restoring and continuing a session under the simulation rules.
• Comparison: a corrected ordinary workflow also passed. This useful output does not establish an advantage from the preparation.

The software and original check records remain private. The public synthetic replay is a separate example and does not reproduce this task.

The reported acceptance covered saving staged state, loading it and continuing the session. An initial ordinary-workflow version failed a malformed-input check; its corrected version was accepted. Keeping that correction matters: the completed feature does not show that preparing context caused a quality advantage.

The reviewer continued a saved session across fresh processes and checked it against the simulation rules. This supports the saved-session handoff on that machine, not general transfer of prepared context across new tasks. Saves replay against current rules; future rule migration was outside the checked slice. Preparation, execution, correction and review effort have not been compared in a qualified all-in cost study.

A later task in the same local project produced an accepted report explaining what saved actions changed and which recorded events supported that account. One AI role selected preparation from a small manually supplied library, and a second used it for the task. Initial role-selection and refusal controls were checked separately from the useful feature; these were development cases, not a general selection benchmark.

A recorded edit before the consumer acknowledged the handoff remains a process violation, even though the feature passed its acceptance checks. Existing initialization errors during development were corrected in the documented use sequence. This is useful assisted work in one project, not fully reliable handoff or generalized automatic selection.

These are separate staged outputs. Neither is the public synthetic fixture, a completed integrated platform or a completed real-repository pilot.

Separate synthetic studies

This update does not report new synthetic-study outcomes. A future quantitative report needs its exact scoring rules, repair history, repetitions, qualification status and a safe public method summary beside the figures. Percentages of specified task checks are not general accuracy; recorded turns do not measure full preparation and review cost. No compiler or speed advantage is claimed here.

These studies concern separate research implementations. They do not validate the local application or add up to one product success rate.

What has not cleared the bar

The earlier selection and maintenance studies retained unsuccessful comparisons. Later useful outputs do not change those recorded outcomes. A completed real-repository pilot and reliable benefit across new tasks remain unestablished.

We have not established general automatic teacher selection, reliable benefits across new tasks or lower total preparation, execution and review effort. The next milestone is a qualified held-out comparison against a capable ordinary workflow, with those costs included.

The public synthetic fixture is unchanged. It remains a separate runnable correction-and-restore contract. Its acceptance does not transfer to these private outputs. The current brief summarizes the product; the September 30 study report preserves earlier results and conditions.

Local ingestion and evidence delivery, and the procedure-study results.

Ingestion and evidence delivery, Procedure studies

One working day of the team’s decision ledger: 34 decisions, each with its bar stated first.

Decision ledger

Local demonstration with fictional documents.

Demonstration report