Recorded run
Blank start: the first rubric lifts accuracy
A cyclotron starts with no written rubric and learns one from reviewer labels as 280 stories arrive. The first rubric proposal waits for 20 label transitions, so it arrives after about 80 stories.
What to notice
- The cyclotron scored 44.3%, 84.3%, 90.0%, 95.7% across the 4 windows of 70 stories.
- Rejecting everything would have scored 84.3%, 80.0%, 78.6%, 84.3%. Read each accuracy beside that number.
- In the last window the cyclotron beat that baseline by 11.4 points.
- The first rubric proposal arrived at cycle 82. Until then the decision model worked from the blank start.
- In the last window the decision model alone scored 88.6%, so the cyclotron's added work helped there.
- Before the first rubric, accuracy is below what always answering the most common label would score.
- Once the first rubric takes effect the next windows climb and finish well above that baseline.
- The whole run costs about 6 cents at list prices.
Setup
| Decision model | Jev 1.13.0 (jev) |
|---|---|
| LLM optimizer | gpt-6-luna |
| Objective | Accuracy |
| Starting rubric | Blank (no written rubric) |
| Stories reviewed | Every story |
| Rubric changes considered | Every 20 cycles |
| Idea screening | Off |
| Max rubric optimizations | null |
| Rubric trigger basis | label_transitions |
| Selection policy | {"aggregation":"positive","max_secondary_regression":0,"minimum_secondary":null,"positive_class":null,"primary":"accuracy","secondary":null} |
| First rubric | Cycle 82 |
| Corpus | editorial-relabel-v3-1, 287 stories |
| Window size | 70 stories |
| Recorded | 2026-10-11 |
| Engine commit | c060abd |
Accuracy by window
Each window is 70 stories. The last column is what rejecting everything would have scored.
| Window | Cyclotron | Decision model alone | Rejecting everything |
|---|---|---|---|
| Stories 1–70 | 44.3% | 42.9% | 84.3% |
| Stories 71–140 | 84.3% | 88.6% | 80.0% |
| Stories 141–210 | 90.0% | 94.3% | 78.6% |
| Stories 211–280 | 95.7% | 88.6% | 84.3% |
How the answers were wrong
Accuracy says how often an answer was wrong. These tables say which way. They count the 280 stories whose right answer was revealed.
| Published | Rejected | |
|---|---|---|
| Should be published51 stories | 3772.5% of those to publishPublished: right | 1427.5% of those to publishRejected: wrong |
| Should be rejected229 stories | 4620.1% of those to rejectPublished: wrong | 18379.9% of those to rejectRejected: right |
| Published | Rejected | |
|---|---|---|
| Should be published51 stories | 4690.2% of those to publishPublished: right | 59.8% of those to publishRejected: wrong |
| Should be rejected229 stories | 5524.0% of those to rejectPublished: wrong | 17476.0% of those to rejectRejected: right |
Of 51 stories that should be published, the final decision found 37 and the decision model alone found 46. Of 229 stories that should be rejected, the final decision published 46 and the decision model alone published 55.
Cost
About $0.06 in model calls at list prices: 738 decision-model calls ($0.04) and 3 optimizer requests ($0.02).
Price basis: list prices, USD per million tokens, applied to the provider-reported usage on every recorded call; decision model jev (TypeSafe's published price); optimizer gpt-6-luna (LiteLLM model price table (standard tier), read 2026-10-11).
Where the labels came from
Labels are simulated. Original labels came from a single labeling model run under a written rubric; an offline audit found it over-published against that rubric. These v3 labels come from two independent blind relabeling passes under an explicit written policy; only stories where both passes agreed are included.
Replay the run
One correction can improve the next tested decision.
Cycle 280 of 280 · 280 revealed · recorded agreement
A standalone model repeats its answer. cyclotron records the correction, tests guidance and configuration changes, then uses only accepted evidence to align later decisions.
This recorded decision
SpaceX successfully launches its first crewed spaceflight
Model: publish · 95%
Reviewer: publish
SpaceX launched NASA astronauts Hurley and Behnken on Crew Dragon, the first private company to send astronauts to orbit and the first US crewed launch since the shuttle.
Recorded feedback policy: All. Realized feedback is shown in the playback.
Recorded editorial run · Run
Prediction, before the label
publish · 95% confidence
Reviewer label
publish · prediction matched
What the revealed feedback says so far
- Recall
- 64%
- Of the items that should be a yes, how many it found.
- 25 of 39 · 95% interval 48%–77%
- Precision
- 93%
- Of the items it said yes to, how many were right.
- 25 of 27 flagged
- Accuracy
- 92%
- Of all items, how many it got right.
- 184 of 200; always “reject” would score 81%
Accuracy is the share of decisions that matched revealed labels. Precision asks how often a “publish” call was right; recall asks how many actual “publish” stories were found. These are running totals, not future results.
Simulated reviewer replay
Visual replay only — clicking these controls never changes the recorded run.
Editorial decision
Recorded agreement
Recorded simulated reviewer explanation, replayed after the prediction. Ground-truth source evidence remains separate from optimizer feedback. The run records 280 explanations; the last optimizer proposal came at cycle 202, and the explanations after it were recorded but followed by no optimizer request. Only recorded correction events appear below.
Curated public source
SpaceX successfully launches its first crewed spaceflight
Wikinews reports: SpaceX successfully launches its first crewed spaceflight. The original linked article provides the complete reporting context.
Brief adapted and shortened for this demo; see the original Wikinews article for the complete work. Read original · CC BY 2.5 (Wikinews policy; articles dated through 2024-12-15).
What happened after the label
No optimization was due. cyclotron recorded the prediction and the label.
Recorded reliability
Observed accuracy versus the logged pre-review confidence, using only 280 revealed labels. Hidden labels are not used.
Recorded rolling metrics at this cycle: Brier 0.128 · ECE 0.044.
Matched evaluator-only windows
First 70 vs last 70 items, each scored against frozen simulated reference labels, evaluator-only; never learner feedback.
Accuracy 44% → 96% (rejecting everything would score 84% → 84%) · Brier 0.658 → 0.090 · ECE 0.269 → 0.036.
This permits equal-window comparison across modes; it was not learner-visible feedback.