Why Jeff

AI primer

How Jeff produces decisions and calibrates their probabilities.

Decisions, not conversation

Automated workflows need decisions that code can act on, a measure of uncertainty and consistent results for the same input. Generating and parsing text adds latency, cost and validation work.

How Jeff reads out an answer

Jeff starts with a small open language model and trains it specifically for decisions. For each question, it reads the state, the question and the options labelled A, B, C … then reads the model's probability for each label in one step. Nothing is generated, so:

• the answer is always one of your options;
• there is no parsing and no unexpected prose;
• each decision takes one pass over the input, with no time spent generating text;
• the same input on the same checkpoint gives the same probabilities, up to tiny rounding differences on GPUs.

Calibration

Jeff is trained for calibration, and its probabilities are scaled using a temperature stored in the checkpoint. Across many answers, those assigned a probability of 0.2 should be correct about 20% of the time, and those assigned 0.8 about 80%. These measured frequencies provide a basis for confidence thresholds. Calibration applies to a set of answers, not to an individual prediction.

An untrained base model is not calibrated: in our tests the plain Qwen3.5-0.8B picked the last, longest option at 68–92% confidence regardless of the question. No threshold catches confident wrong answers, which is why Jeff's training targets calibration directly.

Where the data comes from

Jeff is trained only on data that is real, public, or generated by models and code running on our own machines. When you build an adapter, the same rule applies to your data: it stays on your hardware.