Recorded run
Review less, starting with the rejects
Same blank start and early trigger, but the review schedule is lighter: for the first 50 stories every story the model calls publish and half of those it calls reject are reviewed, then half of the publishes and a quarter of the rejects.
What to notice
- Only 113 of 280 stories (40.4%) were reviewed: for the first 50 stories, every story the model calls publish and half of those it calls reject; after that, half of the publishes and a quarter of the rejects. The other stories gave the cyclotron no feedback, so read these accuracies as coming from less review than a run that reviews every story.
- The cyclotron scored 78.6%, 95.7%, 97.1%, 91.4% across the 4 windows of 70 stories.
- Rejecting everything would have scored 84.3%, 80.0%, 78.6%, 84.3%. Read each accuracy beside that number.
- In the last window the cyclotron beat that baseline by 7.1 points.
- The first rubric proposal arrived at cycle 21. Until then the decision model worked from the blank start.
- In the last window the decision model alone scored 87.1%, so the cyclotron's added work helped there.
- The run reviews far fewer stories than a full-review run.
- Accuracy stays in the same range, with a dip in the last window.
Setup
| Decision model | Jev 1.13.0 (jev) |
|---|---|
| LLM optimizer | gpt-6-luna |
| Objective | Accuracy |
| Starting rubric | Blank (no written rubric) |
| Stories reviewed | 113 of 280 stories (for the first 50 stories, every story the model calls publish and half of those it calls reject; after that, half of the publishes and a quarter of the rejects) |
| Rubric changes considered | Every 6 cycles |
| Idea screening | Off |
| Max rubric optimizations | null |
| Rubric trigger basis | label_transitions |
| Selection policy | {"aggregation":"positive","max_secondary_regression":0,"minimum_secondary":null,"positive_class":null,"primary":"accuracy","secondary":null} |
| First rubric | Cycle 21 |
| Corpus | editorial-relabel-v3-1, 287 stories |
| Window size | 70 stories |
| Recorded | 2026-10-11 |
| Engine commit | c060abd |
Accuracy by window
Each window is 70 stories. The last column is what rejecting everything would have scored.
| Window | Cyclotron | Decision model alone | Rejecting everything |
|---|---|---|---|
| Stories 1–70 | 78.6% | 72.9% | 84.3% |
| Stories 71–140 | 95.7% | 91.4% | 80.0% |
| Stories 141–210 | 97.1% | 95.7% | 78.6% |
| Stories 211–280 | 91.4% | 87.1% | 84.3% |
How the answers were wrong
We left this run mostly unreviewed, so the right answer was revealed for only 113 of its 280 stories. A table over so few would not show how the answers were wrong, so we do not draw one.
Cost
About $0.05 in model calls at list prices: 637 decision-model calls ($0.03) and 5 optimizer requests ($0.02).
Price basis: list prices, USD per million tokens, applied to the provider-reported usage on every recorded call; decision model jev (TypeSafe's published price); optimizer gpt-6-luna (LiteLLM model price table (standard tier), read 2026-10-11).
Where the labels came from
Labels are simulated. Original labels came from a single labeling model run under a written rubric; an offline audit found it over-published against that rubric. These v3 labels come from two independent blind relabeling passes under an explicit written policy; only stories where both passes agreed are included.
Replay the run
One correction can improve the next tested decision.
Cycle 280 of 280 · 280 revealed · recorded agreement
A standalone model repeats its answer. cyclotron records the correction, tests guidance and configuration changes, then uses only accepted evidence to align later decisions.
This recorded decision
SpaceX successfully launches its first crewed spaceflight
Model: publish · 98%
Reviewer: not revealed
No feedback was recorded on this cycle.
Phased review schedule: cycles 1–50, publish 100%, reject 50%; cycles 51–end, publish 50%, reject 25%. Feedback selection occurs after prediction, before hidden label reveal.
Recorded editorial run · Run
Prediction, before the label
publish · 98% confidence
Reviewer label
not revealed · no feedback on this cycle
What the revealed feedback says so far
- Recall
- 95%
- Of the items that should be a yes, how many it found.
- 20 of 21 · 95% interval 77%–99%
- Precision
- 53%
- Of the items it said yes to, how many were right.
- 20 of 38 flagged
- Accuracy
- 83%
- Of all items, how many it got right.
- 94 of 113; always “reject” would score 81%
Accuracy is the share of decisions that matched revealed labels. Precision asks how often a “publish” call was right; recall asks how many actual “publish” stories were found. These are running totals, not future results.
Curated public source
SpaceX successfully launches its first crewed spaceflight
Wikinews reports: SpaceX successfully launches its first crewed spaceflight. The original linked article provides the complete reporting context.
Brief adapted and shortened for this demo; see the original Wikinews article for the complete work. Read original · CC BY 2.5 (Wikinews policy; articles dated through 2024-12-15).
What happened after the label
No optimization was due. cyclotron recorded the prediction and the label.
Recorded reliability
Observed accuracy versus the logged pre-review confidence, using only 113 revealed labels. Hidden labels are not used.
Recorded rolling metrics at this cycle: Brier 0.200 · ECE 0.078.
Matched evaluator-only windows
First 70 vs last 70 items, each scored against frozen simulated reference labels, evaluator-only; never learner feedback.
Accuracy 79% → 91% (rejecting everything would score 50% → 24%) · Brier 0.272 → 0.109 · ECE 0.104 → 0.054.
This permits equal-window comparison across modes; it was not learner-visible feedback.