Recorded run
Blank start, first rubric after 21 stories
Same setup as the first recording, but the first rubric is proposed after 6 label transitions instead of 20. The loop writes its first rubric at story 21 and gets about 13 optimizer calls instead of 4.
What to notice
- The cyclotron scored 77.1%, 94.3%, 92.9%, 95.7% across the 4 windows of 70 stories.
- Rejecting everything would have scored 84.3%, 80.0%, 78.6%, 84.3%. Read each accuracy beside that number.
- In the last window the cyclotron beat that baseline by 11.4 points.
- The first rubric proposal arrived at cycle 21. Until then the decision model worked from the blank start.
- In the last window the decision model alone scored 94.3%, so the cyclotron's added work helped there.
- The first window is much higher because the first rubric arrives about 60 stories sooner.
- The later windows end in the same range as the default trigger, at the cost of more optimizer calls.
Setup
| Decision model | Jev 1.13.0 (jev) |
|---|---|
| LLM optimizer | gpt-6-luna |
| Objective | Accuracy |
| Starting rubric | Blank (no written rubric) |
| Stories reviewed | Every story |
| Rubric changes considered | Every 6 cycles |
| Idea screening | Off |
| Max rubric optimizations | null |
| Rubric trigger basis | label_transitions |
| Selection policy | {"aggregation":"positive","max_secondary_regression":0,"minimum_secondary":null,"positive_class":null,"primary":"accuracy","secondary":null} |
| First rubric | Cycle 21 |
| Corpus | editorial-relabel-v3-1, 287 stories |
| Window size | 70 stories |
| Recorded | 2026-10-11 |
| Engine commit | c060abd |
Accuracy by window
Each window is 70 stories. The last column is what rejecting everything would have scored.
| Window | Cyclotron | Decision model alone | Rejecting everything |
|---|---|---|---|
| Stories 1–70 | 77.1% | 71.4% | 84.3% |
| Stories 71–140 | 94.3% | 92.9% | 80.0% |
| Stories 141–210 | 92.9% | 92.9% | 78.6% |
| Stories 211–280 | 95.7% | 94.3% | 84.3% |
How the answers were wrong
Accuracy says how often an answer was wrong. These tables say which way. They count the 280 stories whose right answer was revealed.
| Published | Rejected | |
|---|---|---|
| Should be published51 stories | 3976.5% of those to publishPublished: right | 1223.5% of those to publishRejected: wrong |
| Should be rejected229 stories | 167.0% of those to rejectPublished: wrong | 21393.0% of those to rejectRejected: right |
| Published | Rejected | |
|---|---|---|
| Should be published51 stories | 4588.2% of those to publishPublished: right | 611.8% of those to publishRejected: wrong |
| Should be rejected229 stories | 2812.2% of those to rejectPublished: wrong | 20187.8% of those to rejectRejected: right |
Of 51 stories that should be published, the final decision found 39 and the decision model alone found 45. Of 229 stories that should be rejected, the final decision published 16 and the decision model alone published 28.
Cost
About $0.25 in model calls at list prices: 2,313 decision-model calls ($0.15) and 13 optimizer requests ($0.10).
Price basis: list prices, USD per million tokens, applied to the provider-reported usage on every recorded call; decision model jev (TypeSafe's published price); optimizer gpt-6-luna (LiteLLM model price table (standard tier), read 2026-10-11).
Where the labels came from
Labels are simulated. Original labels came from a single labeling model run under a written rubric; an offline audit found it over-published against that rubric. These v3 labels come from two independent blind relabeling passes under an explicit written policy; only stories where both passes agreed are included.
Replay the run
One correction can improve the next tested decision.
Cycle 280 of 280 · 280 revealed · recorded agreement
A standalone model repeats its answer. cyclotron records the correction, tests guidance and configuration changes, then uses only accepted evidence to align later decisions.
This recorded decision
SpaceX successfully launches its first crewed spaceflight
Model: publish · 82%
Reviewer: publish
SpaceX launched NASA astronauts Hurley and Behnken on Crew Dragon, the first private company to send astronauts to orbit and the first US crewed launch since the shuttle.
Recorded feedback policy: All. Realized feedback is shown in the playback.
Recorded editorial run · Run
Prediction, before the label
publish · 82% confidence
Reviewer label
publish · prediction matched
What the revealed feedback says so far
- Recall
- 77%
- Of the items that should be a yes, how many it found.
- 30 of 39 · 95% interval 62%–87%
- Precision
- 94%
- Of the items it said yes to, how many were right.
- 30 of 32 flagged
- Accuracy
- 95%
- Of all items, how many it got right.
- 189 of 200; always “reject” would score 81%
Accuracy is the share of decisions that matched revealed labels. Precision asks how often a “publish” call was right; recall asks how many actual “publish” stories were found. These are running totals, not future results.
Simulated reviewer replay
Visual replay only — clicking these controls never changes the recorded run.
Editorial decision
Recorded agreement
Recorded simulated reviewer explanation, replayed after the prediction. Ground-truth source evidence remains separate from optimizer feedback. The run records 280 explanations; the last optimizer proposal came at cycle 269, and the explanations after it were recorded but followed by no optimizer request. Only recorded correction events appear below.
Curated public source
SpaceX successfully launches its first crewed spaceflight
Wikinews reports: SpaceX successfully launches its first crewed spaceflight. The original linked article provides the complete reporting context.
Brief adapted and shortened for this demo; see the original Wikinews article for the complete work. Read original · CC BY 2.5 (Wikinews policy; articles dated through 2024-12-15).
What happened after the label
No optimization was due. cyclotron recorded the prediction and the label.
Recorded reliability
Observed accuracy versus the logged pre-review confidence, using only 280 revealed labels. Hidden labels are not used.
Recorded rolling metrics at this cycle: Brier 0.092 · ECE 0.033.
Matched evaluator-only windows
First 70 vs last 70 items, each scored against frozen simulated reference labels, evaluator-only; never learner feedback.
Accuracy 77% → 96% (rejecting everything would score 84% → 84%) · Brier 0.273 → 0.085 · ECE 0.101 → 0.034.
This permits equal-window comparison across modes; it was not learner-visible feedback.