Feature · Admin
Benchmarks
Compare providers and inspect benchmark evidence.
How it works
Admin overview
Admin overview links every administrator tool: benchmarks and evaluation, people and access, models and routing, governance and spend, and, for platform operators, platform operations.
Benchmarks
Benchmarks has two tabs: Provider benchmarks compares model providers and CLIs on the same cases; Crewshal harness shows harness runs. Paid runs need consent from an owner or admin.
Who can open it
Only owners and admins. Others see "Admin access required".
From the help article The Admin area, checked against the product on .
What you can do
The operations this feature's screens call. Each one is POST /api/<name>, and the badge says who may call it.
AdminNo API tokensharnessBenchmarks/listCasesHarness benchmarks: lists cases.
Types:
HarnessBenchmarkListCasesRequestreturnsHarnessBenchmarkListCasesResponse
AdminNo API tokensharnessBenchmarks/listRunsHarness benchmarks: lists runs.
Types:
HarnessBenchmarkListRunsRequestreturnsHarnessBenchmarkListRunsResponse
AdminNo API tokensharnessBenchmarks/saveCaseHarness benchmarks: saves the case.
Types:
HarnessBenchmarkSaveCaseRequestreturnsHarnessBenchmarkCase
AdminNo API tokensharnessBenchmarks/compareRunsHarness benchmarks: compares runs.
Types:
HarnessBenchmarkCompareRunsRequestreturnsHarnessBenchmarkComparison
AdminNo API tokensharnessBenchmarks/startRunHarness benchmarks: starts the run.
Types:
HarnessBenchmarkStartRunRequestreturnsHarnessBenchmarkStartRunResponse
AdminNo API tokensharnessBenchmarks/recordRunHarness benchmarks: records a run.
Types:
HarnessBenchmarkRecordRunRequestreturnsHarnessBenchmarkRun
AdminNo API tokensharnessBenchmarks/readOfflineCampaignHarness benchmarks: reads the offline campaign.
Types:
{ id: string }returnsHarnessOfflineCampaignReceipt
AdminNo API tokensharnessBenchmarks/runOfflineCampaignPhaseHarness benchmarks: runs the offline campaign phase.
Types:
{ id: string; phase: 'development' | 'held_out' }returnsHarnessBenchmarkStartRunResponse
AdminNo API tokensharnessBenchmarks/proposeOfflineCampaignHarness benchmarks: proposes an offline campaign.
Types:
{ seed: HarnessOfflineCampaignProposalSeed; wasteRunIds: string[] }returnsHarnessOfflineCampaignProposed
AdminNo API tokensharnessBenchmarks/declareOfflineCampaignHarness benchmarks: declare offline campaign.
Types:
HarnessOfflineCampaignProposalreturnsHarnessOfflineCampaignReceipt
AdminNo API tokensharnessBenchmarks/listOfflineCampaignsHarness benchmarks: lists offline campaigns.
Types:
{ limit?: number }returns{ items: Array<{ id: string; goal: string; createdAt: string }>; truncated: boolean }
AdminNo API tokensharnessBenchmarks/freezeOfflineCampaignHarness benchmarks: freeze offline campaign.
Types:
{ id: string }returnsHarnessOfflineCampaignReceipt
AdminNo API tokensharnessBenchmarks/stopOfflineCampaignHarness benchmarks: stops the offline campaign.
Types:
{ id: string; reason: string }returnsHarnessOfflineCampaignReceipt
Screens
/admin/benchmarks/*/admin/benchmarks/admin/benchmarks/provider/*/admin/benchmarks/harness
What it records
The feature writes these events to the audit log, the permanent record of what happened.
- Harness benchmark case version created
- Harness benchmark run recorded