Recorded run
Blank start on 853 stories
Cyclotron starts with no rubric and learns one while deciding 853 stories in order, with every label revealed after each decision.
What to notice
- The cyclotron scored 86.6%, 94.4%, 93.0%, 95.1%, 97.9%, 95.8% across the 6 windows of 142 stories.
- Rejecting everything would have scored 83.8%, 81.0%, 81.7%, 78.9%, 76.8%, 76.8%. Read each accuracy beside that number.
- In the last window the cyclotron beat that baseline by 19.0 points.
- The first rubric proposal arrived at cycle 24. Until then the decision model worked from the blank start.
- In the last window the decision model alone scored 95.8%, so the cyclotron's added work did not help there.
- Accuracy is above the most-common-label baseline in every window, starting with the first, and stays at 93 percent or higher after the first window.
- All twenty rubric changes happened in the first 410 of the 853 stories; the last 440 ran on the final rubric.
Setup
| Decision model | Jev 1.13.0 (jev) |
|---|---|
| LLM optimizer | gpt-6-luna |
| Objective | Accuracy |
| Starting rubric | Blank (no written rubric) |
| Stories reviewed | Every story |
| Rubric changes considered | Every 6 cycles |
| Idea screening | Off |
| Max rubric optimizations | null |
| Rubric trigger basis | label_transitions |
| Selection policy | {"aggregation":"positive","max_secondary_regression":0,"minimum_secondary":null,"positive_class":null,"primary":"accuracy","secondary":null} |
| First rubric | Cycle 24 |
| Corpus | editorial-relabel-v4, 853 stories |
| Window size | 142 stories |
| Recorded | 2026-10-11 |
| Engine commit | 41608e3081081492044fda1d9a46fc5327472620 |
Accuracy by window
Each window is 142 stories. The last column is what rejecting everything would have scored.
| Window | Cyclotron | Decision model alone | Rejecting everything |
|---|---|---|---|
| Stories 1–142 | 86.6% | 84.5% | 83.8% |
| Stories 143–284 | 94.4% | 94.4% | 81.0% |
| Stories 285–426 | 93.0% | 93.0% | 81.7% |
| Stories 427–568 | 95.1% | 94.4% | 78.9% |
| Stories 569–710 | 97.9% | 97.2% | 76.8% |
| Stories 711–852 | 95.8% | 95.8% | 76.8% |
How the answers were wrong
Accuracy says how often an answer was wrong. These tables say which way. They count the 852 stories whose right answer was revealed.
| Published | Rejected | |
|---|---|---|
| Should be published172 stories | 14785.5% of those to publishPublished: right | 2514.5% of those to publishRejected: wrong |
| Should be rejected680 stories | 284.1% of those to rejectPublished: wrong | 65295.9% of those to rejectRejected: right |
| Published | Rejected | |
|---|---|---|
| Should be published172 stories | 15187.8% of those to publishPublished: right | 2112.2% of those to publishRejected: wrong |
| Should be rejected680 stories | 375.4% of those to rejectPublished: wrong | 64394.6% of those to rejectRejected: right |
Of 172 stories that should be published, the final decision found 147 and the decision model alone found 151. Of 680 stories that should be rejected, the final decision published 28 and the decision model alone published 37.
Cost
About $0.64 in model calls at list prices: 5,477 decision-model calls ($0.38) and 20 optimizer requests ($0.26).
Price basis: list prices, USD per million tokens, applied to the provider-reported usage on every recorded call; decision model jev (TypeSafe's published price); optimizer gpt-6-luna (LiteLLM model price table (standard tier), read 2026-10-11).
Where the labels came from
Labels are simulated. Original labels came from a single labeling model run under a written rubric. These labels come from two independent blind relabeling passes under an explicit written policy; where the passes disagreed (6 percent of the new stories), the label was settled by hand.
Replay the run
One correction can improve the next tested decision.
Cycle 852 of 852 · 852 revealed · recorded agreement
A standalone model repeats its answer. cyclotron records the correction, tests guidance and configuration changes, then uses only accepted evidence to align later decisions.
This recorded decision
Podolski announces retirement from international football
Model: reject · 100%
Reviewer: reject
A player retirement is routine sports news.
Recorded feedback policy: All. Realized feedback is shown in the playback.
Recorded editorial run · Run
Prediction, before the label
reject · >99% confidence
Reviewer label
reject · prediction matched
What the revealed feedback says so far
- Recall
- 87%
- Of the items that should be a yes, how many it found.
- 39 of 45 · 95% interval 74%–94%
- Precision
- 95%
- Of the items it said yes to, how many were right.
- 39 of 41 flagged
- Accuracy
- 96%
- Of all items, how many it got right.
- 192 of 200; always “reject” would score 78%
Accuracy is the share of decisions that matched revealed labels. Precision asks how often a “publish” call was right; recall asks how many actual “publish” stories were found. These are running totals, not future results.
Simulated reviewer replay
Visual replay only — clicking these controls never changes the recorded run.
Editorial decision
Recorded agreement
Recorded simulated reviewer explanation, replayed after the prediction. Ground-truth source evidence remains separate from optimizer feedback. The run records 852 explanations; the last optimizer proposal came at cycle 410, and the explanations after it were recorded but followed by no optimizer request. Only recorded correction events appear below.
Curated public source
Podolski announces retirement from international football
Wikinews reports: Podolski announces retirement from international football. The original linked article provides the complete reporting context.
Brief adapted and shortened for this demo; see the original Wikinews article for the complete work. Read original · CC BY 2.5 (Wikinews policy; articles dated through 2024-12-15).
What happened after the label
No optimization was due. cyclotron recorded the prediction and the label.
Recorded reliability
Observed accuracy versus the logged pre-review confidence, using only 852 revealed labels. Hidden labels are not used.
Recorded rolling metrics at this cycle: Brier 0.078 · ECE 0.036.
Matched evaluator-only windows
First 142 vs last 142 items, each scored against frozen simulated reference labels, evaluator-only; never learner feedback.
Accuracy 87% → 96% (rejecting everything would score 84% → 77%) · Brier 0.168 → 0.079 · ECE 0.067 → 0.034.
This permits equal-window comparison across modes; it was not learner-visible feedback.