Frozen context-recall baseline gate

The idea

A frozen offline set of 36 cases plants facts and records where each one shows in the next request and whether it can be read back. A change to context, memory or compaction must beat this baseline.

The problem
A context change could drop needed facts or leak forgotten ones without anyone seeing.
Why it matters
Each context change is compared with a frozen held-out baseline, with no model call.
Where it stands
Partly built as of

A step-by-step replay of this idea is not published yet. The library adds one when a recorded run shows it.

  • Hybrid memory recall

    Recall ranks memories with full-text and trigram search, plus vectors where embeddings exist. It says which arms ran and why vectors are off, and it gives the worker its readable memory as files.

  • Keep-or-discard harness loop

    A harness change is kept only if the declared capability metrics stay within tolerance, one efficiency metric improves by the minimum, and frozen held-out cases confirm it. Campaigns count every attempt.

Turn the numbers into shipped work.

Crewshal runs these choices for you: a persistent AI worker with memory and rules, on your Claude and Codex subscriptions.