How it works

Your team's corrections become the rules it decides by.

Every cyclotron runs a decision flywheel: it decides, you correct, it aligns, and each correction makes the next decision a little better. The four lessons below build on each other, and the animations replay a recorded editorial example.

1
Start with one decision.
A decision model can answer the question for each item without a task-specific training set or new data engineering.
2
You correct as you go.
When a reviewer sees a wrong answer, they fix it and can say why. Each correction becomes evidence.
3
It tests every change.
A small ML model trained on your corrections gives the decision and a confidence estimate. An LLM optimizer proposes clearer rules and examples. Cyclotron tests each proposal against your labels; an early change can start as a provisional one that it keeps checking.

1 · Start with a decision model

Off-the-shelf AI is almost right. Cyclotron helps it learn the rest.

Everything starts with a decision model: an AI that answers one question about each item. In this recorded example, it decides whether a Wikinews story belongs in a constructive editorial feed. With a blank rubric it does not know this feed's editorial rule: in the first 70 stories it was wrong about as often as right, and far too sure of itself. That is why a general AI should not run unchecked on your decisions.

“Bangladesh President-elect meets outgoing president Hamid at Bangabhaban”

How often it is right

Recall and precision are for articles to publish.

In this window, rejecting everything would score 84% accuracy.

Recall: 100%
Precision: 22%
Accuracy: 43%

2 · Prompt engineering

A better rubric and examples help.

The usual prompt-engineering tools still apply. A clearer rubric describes the task. In this recording the LLM optimizer wrote one from the corrections, and the decision model got right more often. Few-shot examples, which show the editor's choices, are another tool; this run changed only the rubric. Only recorded review can show whether confidence matches results.

“Greek prime minister reaffirms EastMed pipeline project is open for other countries to join”

How often it is right

Recall and precision are for articles to publish.

In this window, rejecting everything would score 84% accuracy.

Recall: 100%
Precision: 22%
Accuracy: 43%

3 · One cyclotron, several decisions

One item, several decisions, each with its own rubric.

One cyclotron can make several decisions about the same item. Each decision has its own rubric. Inside the cyclotron, supporting questions add detail that your team never reviews, and a question two decisions need is asked once. Cyclotron passes every answer on to an ML model: a small model trained on your corrections. It combines the answers into the decision and its confidence. A new cyclotron starts at full review. Once it is ready, only the unsure decisions, plus a random sample, go to your team, and one review can label several. A correction goes to that decision's own labels. By default the optimizer learns inside the review; the animation ends on deferred learning, which is opt-in.

Illustrative example. The questions and decisions are made up. A new cyclotron starts at full review; deferred learning is opt-in, and the default learns inside the review.

4 · Tested before it goes live

The LLM optimizer proposes. The evidence decides.

Cyclotron puts two separate learners around each decision-model request. Then evaluation decides. No part grades its own work.

Cyclotron tests a proposal before it keeps it. It sets some of your corrected decisions aside, where the optimizer does not read them, and scores the proposal against the current rubric on those. Decisions in one cyclotron share one decision-model request, so a change to one is part of every sibling's request. Before a change ships, each sibling decision is checked on its own development labels. A proposal that scores better, loses no recall on any label and regresses no sibling is promoted. By default a regression is refused, and the sibling's result is recorded either way. One that scores worse is dropped, and the attempt stays on record. Early on, before enough decisions are set aside, a proposal can run provisionally. In the recorded example in lesson 2, none of the three proposals reached this test: they all ran provisionally.

Illustrative example. The set-aside decisions are a development check, not a final independent test.

Language

The LLM optimizer proposes

It reads labels and explanations. Then it proposes a clearer rubric, better few-shot examples, or extra questions. Code checks each proposal before Cyclotron tests it.

Numbers

The ML fitter learns

The decision model's answers become features, the inputs to the ML model. The ML fitter trains the ML model: it learns their weights and the calibration from trusted labels only. The LLM optimizer cannot change these numbers.

Decisions

Evaluation decides

A change counts as an improvement only if it passes a declared rule on development labels. Those labels stay out of optimization and training. Cyclotron marks early changes as provisional and records every attempt.

Start now, steer as you go

Start with a decision. Learn from corrections.

A decision model can make initial decisions without a task-specific training set or new data engineering. Review its answers before you expand automation: each correction becomes a label for the ML model. The recorded run reviews every decision; later schedules can lighten review, but their results are not shown. See the recorded run, its figures and its limits →

Going deeper

What ‘sure’ means, on a curve.

Group the stories by how sure it said it was. For each group, count how often it was right. A system whose confidence matches its results sits on the diagonal. Confidence that matches results is what tells you which decisions need a person.

decision model alone
with the cyclotron

Decision model alone: it said 95%, and was right 100%.

With the cyclotron: it said 99%, and was right 97%.

Going deeper

Choose the measure before you automate.

Accuracy, precision, and recall answer different questions. A team agrees which error trade-off matters for the decision, tests a proposed change on representative examples, and keeps reviewers involved where the evidence requires it.

Accuracy

Of all stories, how many it got right.

Precision

Of the stories it published, how many were worth it.

Recall

Of the stories worth publishing, how many it found.

The recorded replay on the recordings page shows these measures moving in different directions. A goal guides how a team evaluates a proposed change; it does not mean every measure will improve. Review the recorded figures and limits →

See it run.

The recorded run Ways to work with us