Recorded run
Review everything first, then half
Same blank start and early trigger as the full-review run, but after the first 50 stories only half of the stories are reviewed.
What to notice
- Only 167 of 280 stories (59.6%) were reviewed: every story for the first 50 stories, then half of the stories. The other stories gave the cyclotron no feedback, so read these accuracies as coming from less review than a run that reviews every story.
- The cyclotron scored 78.6%, 97.1%, 92.9%, 94.3% across the 4 windows of 70 stories.
- Rejecting everything would have scored 84.3%, 80.0%, 78.6%, 84.3%. Read each accuracy beside that number.
- In the last window the cyclotron beat that baseline by 10.0 points.
- The first rubric proposal arrived at cycle 21. Until then the decision model worked from the blank start.
- In the last window the decision model alone scored 94.3%, so the cyclotron's added work did not help there.
- Accuracy after the first rubric stays in the same range as the full-review run.
- The run needs about 40 percent fewer reviewed stories and fewer optimizer calls.
Setup
| Decision model | Jev 1.13.0 (jev) |
|---|---|
| LLM optimizer | gpt-6-luna |
| Objective | Accuracy |
| Starting rubric | Blank (no written rubric) |
| Stories reviewed | 167 of 280 stories (every story for the first 50 stories, then half of the stories) |
| Rubric changes considered | Every 6 cycles |
| Idea screening | Off |
| Max rubric optimizations | null |
| Rubric trigger basis | label_transitions |
| Selection policy | {"aggregation":"positive","max_secondary_regression":0,"minimum_secondary":null,"positive_class":null,"primary":"accuracy","secondary":null} |
| First rubric | Cycle 21 |
| Corpus | editorial-relabel-v3-1, 287 stories |
| Window size | 70 stories |
| Recorded | 2026-10-11 |
| Engine commit | c060abd |
Accuracy by window
Each window is 70 stories. The last column is what rejecting everything would have scored.
| Window | Cyclotron | Decision model alone | Rejecting everything |
|---|---|---|---|
| Stories 1–70 | 78.6% | 78.6% | 84.3% |
| Stories 71–140 | 97.1% | 95.7% | 80.0% |
| Stories 141–210 | 92.9% | 94.3% | 78.6% |
| Stories 211–280 | 94.3% | 94.3% | 84.3% |
How the answers were wrong
Accuracy says how often an answer was wrong. These tables say which way. They count the 167 stories whose right answer was revealed.
| Published | Rejected | |
|---|---|---|
| Should be published22 stories | 1672.7% of those to publishPublished: right | 627.3% of those to publishRejected: wrong |
| Should be rejected145 stories | 149.7% of those to rejectPublished: wrong | 13190.3% of those to rejectRejected: right |
| Published | Rejected | |
|---|---|---|
| Should be published22 stories | 1881.8% of those to publishPublished: right | 418.2% of those to publishRejected: wrong |
| Should be rejected145 stories | 1510.3% of those to rejectPublished: wrong | 13089.7% of those to rejectRejected: right |
Of 22 stories that should be published, the final decision found 16 and the decision model alone found 18. Of 145 stories that should be rejected, the final decision published 14 and the decision model alone published 15.
Cost
About $0.08 in model calls at list prices: 904 decision-model calls ($0.04) and 6 optimizer requests ($0.03).
Price basis: list prices, USD per million tokens, applied to the provider-reported usage on every recorded call; decision model jev (TypeSafe's published price); optimizer gpt-6-luna (LiteLLM model price table (standard tier), read 2026-10-11).
Where the labels came from
Labels are simulated. Original labels came from a single labeling model run under a written rubric; an offline audit found it over-published against that rubric. These v3 labels come from two independent blind relabeling passes under an explicit written policy; only stories where both passes agreed are included.
Replay the run
One correction can improve the next tested decision.
Cycle 280 of 280 · 280 revealed · recorded agreement
A standalone model repeats its answer. cyclotron records the correction, tests guidance and configuration changes, then uses only accepted evidence to align later decisions.
This recorded decision
SpaceX successfully launches its first crewed spaceflight
Model: publish · 99%
Reviewer: not revealed
No feedback was recorded on this cycle.
Phased review schedule: cycles 1–50, publish 100%, reject 100%; cycles 51–end, publish 50%, reject 50%. Feedback selection occurs after prediction, before hidden label reveal.
Recorded editorial run · Run
Prediction, before the label
publish · 99% confidence
Reviewer label
not revealed · no feedback on this cycle
What the revealed feedback says so far
- Recall
- 73%
- Of the items that should be a yes, how many it found.
- 16 of 22 · 95% interval 52%–87%
- Precision
- 53%
- Of the items it said yes to, how many were right.
- 16 of 30 flagged
- Accuracy
- 88%
- Of all items, how many it got right.
- 147 of 167; always “reject” would score 87%
Accuracy is the share of decisions that matched revealed labels. Precision asks how often a “publish” call was right; recall asks how many actual “publish” stories were found. These are running totals, not future results.
Curated public source
SpaceX successfully launches its first crewed spaceflight
Wikinews reports: SpaceX successfully launches its first crewed spaceflight. The original linked article provides the complete reporting context.
Brief adapted and shortened for this demo; see the original Wikinews article for the complete work. Read original · CC BY 2.5 (Wikinews policy; articles dated through 2024-12-15).
What happened after the label
No optimization was due. cyclotron recorded the prediction and the label.
Recorded reliability
Observed accuracy versus the logged pre-review confidence, using only 167 revealed labels. Hidden labels are not used.
Recorded rolling metrics at this cycle: Brier 0.166 · ECE 0.057.
Matched evaluator-only windows
First 70 vs last 70 items, each scored against frozen simulated reference labels, evaluator-only; never learner feedback.
Accuracy 79% → 94% (rejecting everything would score 77% → 37%) · Brier 0.271 → 0.106 · ECE 0.090 → 0.056.
This permits equal-window comparison across modes; it was not learner-visible feedback.