Practice · Layer V / Practitioner

Recheck your model choices each release

When a new model family ships, treat your routing as expired. Re-measure which size and effort give the most score per dollar, spot the size you can skip, and drop the speed modes that do not pay.

Nº 022 · Vol. I·6 min read· Updated July 2026

Read first Rent the model, own the method (Assumes you already keep a routing table of model, effort, and cost. This guide is how you maintain it across releases.)

“a new model family turns last month’s routing into a guess”

When a new model family ships, treat your routing as expired. Which model you reach for, and how hard you let it think, were tuned against a score-versus-cost curve that just moved, and the rules you wrote against the old one are now guesses. This guide is a short habit for re-measuring at each release, and it extends the routing table in Rent the model, own the method. The specifics come from Matthew Berman’s notes after a thousand hours in one coding agent, with the product walkthrough left out.

01Every release resets the map

A model release redraws the score-versus-cost curve. You get different scores at different prices, and often a different number of effort levels to choose from. The size-and-effort pairing you settled on last month may no longer sit on the best part of it.

So make re-measuring part of adopting any new family. Before you trust it with real work, run a handful of your own representative tasks across the sizes and effort levels, and note where score per dollar actually peaks. Vendor benchmarks tell you the shape of the curve. Your own tasks tell you the point on it.

02Smaller can win per dollar

The trap is assuming a bigger model is always a better model. Count the price and it is not. In Berman’s numbers for one recent family, the middle size, thinking as hard as it could, scored lower than the smallest size doing the same, and charged more for the worse answer. There was never a reason to send it any work.

“For your hardest problems, go with Soul. For everything else, go with Luna.”

Matthew Berman, “You aren't using Codex like me...” · 2:05

That is the shape to look for. Plot score against cost, then pick the point that is highest and furthest left. A size that never lands on that upper-left edge is a size you can skip. Rent the model, own the method makes the matching case for effort: the top model set to think lightly can score about the same as the next size down thinking hard, so what you are really choosing is a size-and-effort pairing, not a model name. When the pressure is a subscription cap rather than a bill, Stop hitting your token limit turns the same choice into budget relief.

03Price the speed premium

Speed modes deserve the same arithmetic. A fast variant that costs 2.5 times as much for 1.5 times the speed, the ratio Berman reports, is worth paying only when a deadline is the thing you are short of. You are paying for speed alone, and the answer is no better. When you have time to spare, the slower run is the cheaper way to the same answer.

04Sweep the stale rules

Standing instructions pile up. The config files and skills that pin a model or an effort level, a rule that spins up a certain size on deploy, a skill that hard-codes the top model, were written against an older version of that curve and quietly outlive it. A rule that made sense two releases ago can route work to a size you would now skip.

So when you re-measure, re-read. Have the agent list every place your setup pins a model or an effort, and check each line against the curve you just measured. This is the same reason Every new repo starts with your lessons treats standing rules as a layer you maintain, not one you set once.

“those get loaded very frequently into the context window of every agent.”

Matthew Berman, “You aren't using Codex like me...” · 5:31

FAQ

Why treat your routing as expired when a new model family ships?

Because the release moves the score-versus-cost curve and often changes the effort levels on offer. The model-and-effort pairing you tuned against the old curve may no longer sit on the best part of it, so last month's choice is a guess until you re-measure.

What makes a model size worth skipping, and how do you spot it?

Another size beats it on both score and price. For example, a middle size thinking hard scores below the smallest size at its own hardest setting, and costs more doing it. It never lands on the upper-left edge of the score-versus-cost curve, so you send it no work.

A fast mode costs 2.5 times as much for 1.5 times the speed. When is it worth paying?

Only when a deadline is the thing you are short of. You pay for speed alone, not a better answer, so with time to spare the slower run gives the same quality for less. Judge every speed mode by that cost-to-speed ratio, not by its label.

Sources

Matthew Berman · “You aren't using Codex like me...”
cited at 1:04 · 2:05 · 2:31 · 5:05 · 5:31