Correct once: the lesson comes back when the work needs it
The idea
When you correct a Crewshal AI worker once, the platform keeps the correction as a lesson. A lesson tied to folders goes back to the worker when later work reaches those folders, and it stays out of work where it does not apply. Lessons tied to folders run only in the lab for now; in production, a lesson loads when its words match the work.
- The problem
- Many AI tools lose a correction when the chat ends, so people repeat the same instruction in every new task.
- Why it matters
- You say it once. The lesson reaches the next task that touches the same folders and stays out of unrelated work.
- Where it stands
- Shown in a lab run as of One owned lab run with a real model showed it end to end on November 16, 2026. Lessons from corrections run in production; lessons tied to folders need the AI worker's own computer, which is switched off in production.
Crewshal team · · 4 min read
Recorded run · Owned lab, real model · November 16, 2026
One correction, two tasks
An AI worker fixes two issues in a team repository. Between them its owner gives one correction, and the second fix follows it.
TaskTask 1 arrives
The worker gets its first task: fix issue #1, "Search misses issues when the letter case differs". The task says where the issue is and that the team's rules are in the repository. Nothing more.
InputThe task, as given
Fix issue #1 on the team's forge. The team's rules are in the repository.
InputThe issue
- Issue
- #1 Search misses issues when the letter case differs
TaskTask 1 arrives
The worker gets its first task: fix issue #1, "Search misses issues when the letter case differs". The task says where the issue is and that the team's rules are in the repository. Nothing more.
The record for step 1
InputThe task, as given
Fix issue #1 on the team's forge. The team's rules are in the repository.
InputThe issue
- Issue
- #1 Search misses issues when the letter case differs
AI workerThe fix, with five tests
The worker fixes the search in pull request #3. The platform's completion check sends it back once: a claim in the pull request had no test. The worker adds a test for that claim. None of the 5 test titles names the issue it guards.
The record for step 2
OutputNew test titles
a title search ignores letter casea label search ignores letter casea label search still needs the whole label, in any letter casea search ignores the spaces around the wordsa query of only spaces matches every issue
CheckCompletion checks
- Round 1 at 20:11:59 UTCA claim in the pull request had no test.Did not pass
OutputPull request #3
diff --git a/src/search.js b/src/search.js index 26f0b86..50a03b8 100644 --- a/src/search.js +++ b/src/search.js @@ -1,8 +1,12 @@ /** * The issues that match a search: the query is part of the title, or it is one of the labels. - * An empty query matches every issue. + * The search ignores letter case and the spaces around the query. + * An empty query, or a query of only spaces, matches every issue. */ export function matchIssues(issues, query) { - if (!query) return issues - return issues.filter(issue => issue.title.includes(query) || issue.labels.includes(query)) + const wanted = (query ?? '').trim().toLowerCase() + if (!wanted) return issues + return issues.filter(issue => + issue.title.toLowerCase().includes(wanted) || + issue.labels.some(label => label.toLowerCase() === wanted)) } diff --git a/test/search.test.js b/test/search.test.js index b24b17d..70daef3 100644 --- a/test/search.test.js +++ b/test/search.test.js @@ -14,3 +14,26 @@ test('a word of a title finds that issue', () => { test('a label finds every issue with that label', () => { assert.deepEqual(matchIssues(ISSUES, 'auth').map(issue => issue.id), [1, 4, 7]) }) + +test('a title search ignores letter case', () => { + assert.deepEqual(matchIssues(ISSUES, 'login').map(issue => issue.id), [1, 7]) + assert.deepEqual(matchIssues(ISSUES, 'LOGIN').map(issue => issue.id), [1, 7]) +}) + +test('a label search ignores letter case', () => { + assert.deepEqual(matchIssues(ISSUES, 'AUTH').map(issue => issue.id), [1, 4, 7]) +}) + +test('a label search still needs the whole label, in any letter case', () => { + assert.deepEqual(matchIssues(ISSUES, 'aut').map(issue => issue.id), []) + assert.deepEqual(matchIssues(ISSUES, 'AUT').map(issue => issue.id), []) +}) + +test('a search ignores the spaces around the words', () => { + assert.deepEqual(matchIssues(ISSUES, ' auth ').map(issue => issue.id), [1, 4, 7]) + assert.deepEqual(matchIssues(ISSUES, ' Dark Theme ').map(issue => issue.id), [2]) +}) + +test('a query of only spaces matches every issue', () => { + assert.equal(matchIssues(ISSUES, ' ').length, ISSUES.length) +})VerifierChecked and approved
The platform's verifier checks pull request #3 in its own container, out of the worker's reach, and approves it. 4 of the 5 new tests fail without the fix; the other guards behaviour the old code already had. The completion checks then pass, and task 1 is done.
The record for step 3
CheckVerifier steps
- Read the pull request from the forge: its target, branch and commitsPassed
- Run the team's tests on the code before the changePassed
- Write one check from the issue alone, and see it fail before the changePassed
- Fetch the change itselfPassed
- Run the team's tests on the changePassed
- Put the fix back to the old code: the new test must then failPassed
- Run the issue check on the changePassed
- Compare the page before and after (the lab gave the verifier no page port)Skipped
- Run the plan's own checks in the verifier's containerPassed
OutputVerdict
- Outcome
- Verified
- Review on the forge
- Approved
- Team tests on the change
- 10 of 10 pass
- New tests that fail without the fix
- 4 of 5
CheckCompletion checks
- Round 1 at 20:11:59 UTCA claim in the pull request had no test.Did not pass
- Round 2 at 20:14:11 UTCEvery check passed.Passed
CheckLab check, not part of the product
- Hidden acceptance tests
- 6 of 6 pass
OwnerOne correction
The owner gives one correction about task 1, in plain words. In this lab run, a script sent it through the product's own API.
The record for step 4
InputThe owner's correction
In this repository, every regression test names the issue it guards in its title, for example 'a title search ignores letter case (#1)'.
CrewshalA lesson, tied to folders
The platform turns the correction into a lesson: a rule for this repository, and the folders where it applies. The folders come from the files the worker wrote in task 1 with its write tool: test/** and src/**.
The record for step 5
DecisionThe lesson
In the tracklet repository, put the guarded issue number in each regression test title. Example: 'a title search ignores letter case (#1)'.
DecisionHow it is stored
- Kind
- procedure
- Applies to
- test/**, src/**
- For
- This worker
- Confidence
- 0.9
- Why
- A member of the organization gave this correction. It states the repository convention for regression test titles.
OwnerThe owner accepts it
The owner accepts the lesson, again through a scripted call. It goes into the worker's memory as a path lesson: it waits there until work touches one of its folders.
The record for step 6
DecisionSaved memory
- Status
- accepted
- Tags
- lesson, procedure, path-lesson, path:test/**, path:src/**
- Authority
- told (a person said it)
TaskTask 2: a different bug
A new task: issue #2, "The count says "1 issues"". It is a different bug in a different file. The platform builds the worker's opening context without the lesson, because the work has touched no file yet.
The record for step 7
InputThe task, as given
Fix issue #2 on the team's forge. The team's rules are in the repository.
CheckRecorded check
- The opening context holds no path lessonPassed
AI workerThe worker reads its memory
Before it opens the repository, the worker reads its own memory: first with a read-only shell over its memory files (20:14:53 UTC), then the one memory in view, the lesson (20:14:58).
The record for step 8
DecisionMemory reads
- Memory shell read
- 20:14:53 UTC, 1 memory in view
- Memory read
- 20:14:58 UTC, 1 memory read
CrewshalWork inside a lesson folder brings the lesson in
The worker reads 8 files on its computer (20:14:58 to 20:15:22 UTC), among them test/format.test.js, which is inside test/**. Once the work touches a path inside a lesson's folder, the platform puts the lesson into the worker's context.
The record for step 9
CheckRecorded checks
- The opening context holds no path lessonPassed
- A later context holds the lesson, after the work touched a matching pathPassed
DecisionReads before the first plan
- Files read
- 8, 20:14:58 to 20:15:22 UTC
- test/format.test.js
- read at 20:15:22 UTC, inside test/**
DecisionWhat the record does not say
The record shows that the lesson was absent at the start and present in a later context. It does not name the first matching path: a read in src/** matches too.
AI workerThe plan cites the lesson
The worker's first plan cites the lesson. It names the test it will write, with the issue number in the title.
The record for step 10
DecisionFrom the plan (revision 1)
Make the smallest change: in countText, return the singular word when n === 1, else the plural. Add a regression test in test/format.test.js titled per the repo convention, for example 'one issue is counted in the singular (#2)'. Branch fix/count-singular from origin/main. The local clone is on fix/search-ignore-case, so the new branch must not start from it, or PR #3's commits would join this PR. Because the count is visible in the page, take before and after screenshots with npm start and a search that finds exactly one issue, and attach them to the PR.
DecisionWhat the plan rests on
- The lesson's memory
- Cited, active, read at a known version
AI workerThe test names the issue
The worker writes the regression test titled "one issue is counted in the singular (#2)" in test/format.test.js, then the one-line fix in public/format.js, which is in neither lesson folder, and screenshots the page before and after.
The record for step 11
DecisionChanged files and lesson folders
- public/format.js
- in no lesson folder
- test/format.test.js
- inside test/**
OutputPull request #4
diff --git a/public/format.js b/public/format.js index c0f282d..ccc36c8 100644 --- a/public/format.js +++ b/public/format.js @@ -1,4 +1,4 @@ /** The words under the search field: how many issues the search found. */ export function countText(n) { - return `${n} issues` + return n === 1 ? `${n} issue` : `${n} issues` } diff --git a/test/format.test.js b/test/format.test.js index cb738aa..5129db7 100644 --- a/test/format.test.js +++ b/test/format.test.js @@ -9,3 +9,7 @@ test('several issues are counted in the plural', () => { test('no issues are counted in the plural', () => { assert.equal(countText(0), '0 issues') }) + +test('one issue is counted in the singular (#2)', () => { + assert.equal(countText(1), '1 issue') +})OutputThe worker's own screenshots

Before · Open full size 
After · Open full size VerifierApproved on the forge
The verifier checks pull request #4 out of the worker's reach: the team's tests before and after, a revert proof and a check written from the issue alone. It approves the pull request.
The record for step 12
CheckVerifier steps
- Read the pull request from the forge: its target, branch and commitsPassed
- Run the team's tests on the code before the changePassed
- Write one check from the issue alone, and see it fail before the changePassed
- Fetch the change itselfPassed
- Run the team's tests on the changePassed
- Put the fix back to the old code: the new test must then failPassed
- Run the issue check on the changePassed
- Compare the page before and after (the lab gave the verifier no page port)Skipped
- Run the plan's own checks in the verifier's containerPassed
OutputVerdict
- Outcome
- Verified
- Review on the forge
- Approved
- Team tests on the change
- 6 of 6 pass
CrewshalSent back twice
After the approval, the platform's own completion checks on the worker's computer send the work back twice: first for 3 gaps, then for 1. The worker runs the tests again, and the third round passes at 20:30:03 UTC.
The record for step 13
CheckCompletion checks
- Round 1 at 20:23:25 UTCThe tests had not run after the last command that can change files. No screenshot showed the page after the last change. A claim in the pull request had no test.Did not pass
- Round 2 at 20:28:23 UTCThe tests had not run after the last command that can change files.Did not pass
- Round 3 at 20:30:03 UTCEvery check passed.Passed
CrewshalDone, without repeating the correction
Task 2 is done. 2 of 2 hidden tests pass: the lab wrote them to grade this run, the worker never saw them, and they are not part of the product. The owner touched the work twice, both planned and scripted: the correction and the lesson review.
The record for step 14
OutputResult
- Task 2
- Done
- Hidden acceptance tests (lab check)
- 2 of 2 pass
- People who stepped in, unplanned
- 0
- Task 2 working time
- 15 min 21 s
- Task 2 model calls
- 65, USD 5.22 at list price
Result
The second fix followed the correction: its test is titled "one issue is counted in the singular (#2)".
- Run checks passed
- 56 of 56
- Hidden acceptance tests (lab check)
- 6/6 and 2/2
- Unplanned human steps
- 0
- Whole run
- 33 min
- Model cost at list price
- USD 9.33
Where it ran. A real model (Claude Opus 5.5 through Claude Code) on the platform, with a lab Git forge, mail server and worker computer in Docker. The owner's two steps, the correction and the lesson review, were scripted calls to the product's own API; the worker, the platform and the verifier did everything else live. Lessons tied to folders need the worker's own computer, and worker computers are off in production. A time marked “about” is an estimate: the record's creation time plus its recorded duration.
The recorded files behind this replay
- Run record: stages, gates, plans, checks, model calls sha256 4ba54d9a6db3
- The correction, the lesson and its memory, as stored sha256 67da2f6330ef
- Pull request #3 (task 1), as merged for review sha256 5af8bd330501
- Pull request #4 (task 2), as merged for review sha256 27fb7e4648ba
- Verifier's verdict on pull request #3 sha256 9c5e9cb39b9a
- Verifier's verdict on pull request #4 sha256 5d17ba1ca906
- Worker screenshot before the task 2 fix sha256 68ba07a9a4c8
- Worker screenshot after the task 2 fix sha256 dc4feacae787
Why it matters
A correction that disappears when the task ends must be given again in the next task. That costs your time, and you must remember every rule you already gave.
If every past correction went into every task instead, each task would carry more text with each correction. A rule for one part of a repository could also mislead work in another part.
Crewshal keeps the correction once and ties it to the folders where it applies. You give one sentence, and the platform does the bookkeeping. By default, a person reviews each lesson before it becomes a rule.
How it works
- You correct. You give feedback on a finished piece of work, in plain words.
- The platform writes a lesson. A model reads your words and writes the lesson: the rule, the reason and how sure it is. The folders do not come from the model. The platform takes them from the files that the worker wrote with its write tool in that task. A correction that replaces an earlier memory keeps that memory's folders.
- A person reviews it. By default, an owner, admin or approver accepts or rejects each lesson. Two exceptions: an organization can turn on automatic acceptance for procedures and preferences that recur in 3 or more work items and cite a recorded failure or correction, and an explicit instruction or correction from the person who asked for the work is accepted without a review, as that person's own words.
- It waits in memory. An accepted lesson goes into the worker's memory with its folders.
- It loads when the work touches a match. In each later task, the platform records which files the worker reads, writes or runs. It reads the paths from the worker's own calls, never from what a tool printed. When a path is inside a lesson's folder, the platform adds the lesson to the worker's context from then on.
- The worker can also look it up. A worker can read its own memory, so it can find the lesson before it touches a file.
What the run showed
The replay above is a recorded run from November 16, 2026, in an owned lab with a real model. The owner's two steps, the correction and the lesson review, were scripted calls to the product's own API.
- In task 1, the platform sent the work back once because one claim had no test, and the worker added a fifth test. None of the five test titles named the issue. Four of them fail without the fix; the fifth guards behaviour that the old code already had.
- The owner gave one correction. The platform made a lesson for the folders
src/**andtest/**, because task 1 wrote one file in each. The owner accepted it. - Task 2 started without the lesson in its context. The worker first read its memory. It then read eight files on its computer, among them
test/format.test.js, which is insidetest/**, and the lesson was in a later context. Its first plan cited the lesson.public/format.js, the file with the fix, is in neither folder. - The new test was titled "one issue is counted in the singular (#2)".
- The platform's verifier approved both pull requests. In task 2, the platform's completion checks then sent the work back twice before it passed: first because the tests had not run after the last command, no screenshot showed the page after it and one claim had no test, then because the tests had not run again.
- Hidden tests that the worker never saw, a check of the lab and not of the product, passed 6 of 6 and 2 of 2. Nobody stepped in beyond the two planned owner steps.
- The whole run passed 56 of 56 checks in 33 minutes. Its model calls cost USD 9.33 at list price.
Limits
- Lab, not production. Lessons from corrections run in production. Lessons tied to folders need the worker's own computer, and worker computers are off in production.
- One run, one model. The run shows the whole mechanism once, on one model. It does not measure how often lessons help.
- The first match is not recorded. The record shows that the lesson was absent at the start of task 2 and present in a later context. It does not say which path matched first.
- Folders follow the written files. Only files written with the worker's write tool count; edits through shell commands do not. A correction about another part of the repository still gets the folders of the files that the corrected work wrote.
Frequently asked questions
Does the worker forget a correction when I change the model?
No. The lesson is stored by the platform, not in a model's chat, so a new model does not lose it. A later task gets it when its work matches the lesson. This run used one model only.
Does a lesson become a rule without a person?
By default, no: an owner, admin or approver accepts or rejects each lesson. An organization can turn on automatic acceptance for procedures and preferences that recur in 3 or more work items and cite a recorded failure or correction. In this run, the owner accepted the lesson before the next task started.
Why not put every lesson into every task?
Each lesson adds text to every task, and a rule for one folder can mislead work in another folder. A lesson tied to folders loads only when the work reaches those folders.