27 October 2026. We’ve published a new build of cox-1-3b, the open-weight decision model we released on 21 October. Build 1.2 replaces build 1.1 on Hugging Face, keeps the same size, speed and training data, and gives probabilities you can rely on a little more: its calibration error falls from 0.082 to 0.069 on our 15-dataset comparison, and its skill rises slightly.
What changed
Cox answers three kinds of question: yes or no, a choice among options, and a rating on a scale. In build 1.1, yes/no questions went through a separate readout from the other two, a choice we made when we built the first versions of Cox on smaller models, where the separate readout tested better on yes/no tasks. This month we went back over the design choices we had made on smaller models and re-tested the ones that still shape the model we ship at the size we ship. When we trained the same recipe with one readout for every question type, so that a yes/no question is read like a choice between “yes” and “no”, it scored higher than build 1.1’s recipe on both of the training runs we compared, with lower calibration error on the broader half of our test tasks in both, and we built 1.2 from those two runs in the same way as 1.1.
Nothing changes in how you call the model: the same questions return the same kinds of answers, and coxlm handles the new build without an update.
How it compares
We scored the new build exactly as we scored the first one: the same 15 public datasets that none of the compared models is known to have trained on, about 18,500 decisions, the same corrected labels and the same code for every model.
| model | size | skill (95% interval) | accuracy | calibration error |
|---|---|---|---|---|
| cox-1-3b, build 1.2 | 2.9B | +0.653 (+0.640 to +0.665) | 80.7% | 0.069 |
| cox-1-3b, build 1.1 | 2.9B | +0.648 (+0.635 to +0.660) | 80.3% | 0.082 |
| Kev 4B | 4B | +0.615 (+0.601 to +0.628) | 78.7% | 0.082 |
| Kev 9B | 9B | +0.652 (+0.639 to +0.664) | 80.7% | 0.082 |
| Jev 1.14.0 | hosted | +0.669 (+0.655 to +0.682) | 82.0% | 0.064 |
The skill change from build 1.1 is small (+0.005, with an interval from −0.002 to +0.013), and build 1.2 now matches Kev 9B, a model three times its size, on both skill and accuracy, while Jev 1.14.0 still scores higher. The larger change is in calibration. Averaged over these tasks, build 1.2’s stated confidence is within a tenth of a point of its actual accuracy, where build 1.1’s was about a point below, and more decisions clear a given bar: if you automate every answer above the threshold that keeps accuracy at 95%, build 1.2 answers 57% of these decisions on its own, against 46% for build 1.1.
On JevBench, the independent public benchmark for decision models maintained by Benchmark Heaven, build 1.2 answers 179 of the 231 scenarios correctly (77.5%) with a Brier score of 0.299. That is level with build 1.1, whose runs scored between 77.1% and 77.9%, and against the build 1.1 run we compared it with in detail (77.9%, Brier 0.306), the lower Brier score comes from the new build being less confident when it’s wrong: an average confidence of 0.59 on its wrong answers, against 0.61.
Other decision models
As in our first release, we score each other decision model only on the tasks its makers don’t list as training data, with cox-1-3b on the same tasks beside it.
| model | open weights | size | tasks | its skill | cox-1-3b build 1.2, same tasks | what we know about its training data |
|---|---|---|---|---|---|---|
| Liquid d1-ultra | no | not disclosed | 21 | +0.750 | +0.665 | not published |
| Perplexity Decider V1.2 | yes | 27B | 14 | +0.747 | +0.635 | partly published; also trains on benchmarks’ official training splits |
| Perplexity Decider Flash V1.2 | yes | 4B | 14 | +0.705 | +0.635 | as above |
| Jev 1.14.0 | no | not disclosed | 21 | +0.701 | +0.665 | not published |
| Strands Decider 2B | yes | 2B | 9 | +0.536 | +0.670 | published |
| Jeff 2B | yes | 2B | 12 | +0.432 | +0.690 | published |
On each model’s own task set, build 1.2 scores 0.007 to 0.011 higher than build 1.1 did, but the models that scored higher than build 1.1 still score higher than build 1.2, and Perplexity’s Decider Flash leads by about the same margin as before on the five tasks that no model with a published data list trained on (+0.657 against +0.594 for build 1.2 and +0.596 for build 1.1). The model card has build 1.2’s accuracy on every dataset.
Speed
Build 1.2 runs at the same speed as build 1.1: on an RTX 4090, about 35 ms for a short text with a couple of questions, 61 ms for a 1,000-token text and 401 ms for an 8,000-token document, one request at a time, in about 6 GB of GPU memory. An INT8 build, which needs less GPU memory, will follow.
If you need build 1.1
We tagged the previous build on Hugging Face, so you can keep using it while you check the new one against your own data:
model = coxlm.load("Vitund/cox-1-3b", revision="cox-1.1")Adapters made with coxlm record the build they were trained on, so an adapter trained on build 1.1 should be loaded on build 1.1.
Try it
- Model: huggingface.co/Vitund/cox-1-3b
- Package: github.com/Vitund-AI/coxlm
- Worked examples: real inputs and outputs
If you’ve been using build 1.1, we’d be glad to hear whether the new build changes anything on your own decisions, for better or worse. You can reach us at [email protected].