Prompt and tool evolution

Prompt-and-tool evolution means: a harness runs many iterations, scores their outcomes, and uses what it learned to mutate the system prompt or the tool set for the next iteration. This is the heart of the Stanford-style meta-harness (WORKSTREAMS/stanford-harness/EVOLUTION-ARCHITECTURE-v3.md).

Status (2026-05-05). The mechanism for swapping in alternate prompt templates exists today via PromptTemplateLoader. The mechanism for deriving new prompts/tool sets from past iteration scores does not. This page documents both: present state and target state.

Present state — `PromptTemplateLoader`

oracle/PromptTemplateLoader.ts loads prompt templates from two directories:

Directory	Purpose
`promptsBundledDir`	Templates shipped inside the agents package's `dist/prompts/`
`promptsUserDir`	Templates the operator has placed in `<host-home>/prompts/`

The user dir wins on conflict. AgentActorProps.promptFormat (legacy/edn/sexp/markdown) further selects which template variant is used at render time.

This means an operator (or a meta-harness) can swap templates between iterations by writing new files into promptsUserDir before spawning the next session. The agents package will pick them up. This is the mechanical hook on which evolution is built.

Present state — tool-set selection

Each iteration of a harness chooses a profile and a skill set. Because the resolved ResolvedToolContext is recomputed on every prompt (ToolContextResolver.resolve), iteration N+1 can use different tools from iteration N simply by selecting a different profile or skill bundle. This is "swap-in" tool evolution; nothing dynamically derives a new tool from past outcomes.

Target state — proposer-driven evolution

Per WORKSTREAMS/stanford-harness/EVOLUTION-ARCHITECTURE-v3.md:

A proposer agent observes iteration outcomes (scores, constraint results, traces) and proposes mutations to the prompt template or the tool set.
Mutations are candidate code — not rendered text. A new prompt template is a code change to prompts/; a new tool is a MatrixToolProjection entry.
A frontier ledger tracks which candidates passed evaluation; only frontier candidates are promoted to the live system.

None of that is wired today. The hook (template loader + per-iteration resolution) supports it.

What an evolving harness would record per iteration

HarnessIteration already has the necessary fields (harness/HarnessRun.ts:12-23):

promptHash — the hash of the prompt actually rendered.
score — the deterministic eval score.
constraintResults — per-named-check pass/fail.
trace — pointer to the activity-frame log.

A meta-harness builds on these: for each candidate prompt mutation, run N evaluations, record N iterations, average the score, decide whether to promote.

Cross-references

WORKSTREAMS/stanford-harness/EVOLUTION-ARCHITECTURE-v3.md — full target-state architecture (status: workstream design doc, no code yet).
The user-memory MEMORY.md § Evolution Architecture (CRITICAL, 2026-04-19) notes "Candidates are CODE. Three learning layers (routing, context, fine-tuning). Workstreams ARE domains. Interactive cold start. Skills evolvable. SSOT: WORKSTREAMS/stanford-harness/EVOLUTION-ARCHITECTURE-v3.md" — that's the design intent.

Prompt and tool evolution ​

Present state — PromptTemplateLoader ​

Present state — tool-set selection ​

Target state — proposer-driven evolution ​

What an evolving harness would record per iteration ​

Cross-references ​