Research
Stop renting your judgement.
What do the companies actually getting returns from AI have in common? They stopped paying per call for a model that has never seen inside their business.
The whole argument in one sentence. On an identical review workflow, with the same tools, images and scorer, a fine-tuned small open model scored higher than every frontier configuration we tested, at a small fraction of the price per thousand decisions.
Part I
Everyone asked the same question
For four years the question in every boardroom has been the same one: what can this do for us? The first answers were modest: summarise the document, draft the email, produce something a human then fixes. From there it moved into work that actually costs money: code, research, customer conversations, and eventually the ambition of a system wired into internal knowledge that could run parts of the business on its own.
An enormous amount of budget and attention has gone in. Measurable outcomes have been rarer. A small group of companies restructured themselves around the technology and pulled away on productivity, revenue and cost. Most of the rest ran pilots that never left the pilot stage.
Across the deployments we have worked on, the difference between the two groups keeps landing in the same five places.
01
Redesign the process, not the task
Dropping a model into a workflow that was shaped around people leaves every old bottleneck in place, and the bottleneck absorbs the gain before it reaches the P&L. The teams that see EBIT impact rewrote who approves what and which handoffs still need a human.
02
Pay people to run experiments
Tooling and best practice move week to week, so last quarter's setup is rarely still correct. That only surfaces if reporting a failed attempt is rewarded as clearly as shipping one. Engineering is the natural starting point, because they see which problems recur everywhere.
03
Feed it your context properly
Retrieval can carry business context into a call, but doing it well is an engineering programme of its own: reaching the data, enforcing what each request is allowed to see, surfacing the right evidence, and managing a window that models read unevenly as it fills.
04
Measure usage and impact
Every AI line item eventually meets the CFO's question: what changed, and was it worth it? Most deployments cannot answer, because nothing tracks decision quality or cost. Without a scored evaluation on your own data, a vibe is the ceiling of what you can claim.
05
Put a goal inside the budget
Paying per token instead of per seat makes cost scale with usage, which makes an annual plan very hard to hold. Teams routinely burn a year of budget in a quarter, and today's prices are widely subsidised, so the direction of travel is up.
What follows is the technique the leading group keeps converging on for the last three of those problems: take an open-weight model, and train it on your own workflow until the judgement lives in the weights instead of the prompt.
In short
2.2×
Revenue growth over three years among the heaviest AI adopters in one large spend dataset, against roughly 15% for companies spending nothing, in the same economy over the same window.
87%
Share of the achievable score our trained 9B open model reached on catalogue review, against 77% for the best frontier configuration, and 64% for its own untrained base.
68×
Cost gap per reviewed listing against the strongest frontier setup. At forty million decisions a day that is single-digit millions a year instead of hundreds.
Part II
What the leaders do differently
The companies pulling ahead arrived at the same discovery: owning the model wins on quality and on cost at the same time. A model trained to complete your workflows, in your environment, tends to beat a general one that has never been inside your company. And because it doesn't need to hold the whole world in its head, it can be small enough to run for cents.
This is not an argument for cancelling your frontier subscriptions. Prototyping on the best available model is the right first move: it establishes what is technically possible, and every call it makes generates the inputs, decisions and corrections a specialist later trains on. The priority flips once a workflow leaves the prototype stage and starts running at volume. That is where a trained open model takes over, and where the two can call each other: the specialist for anything that depends on internal knowledge, the frontier model for anything that needs breadth.
The loop is unglamorous: the model acts on your tools and data, a rubric scores the outcome, and the reward nudges the weights. Repeat it across tens of thousands of episodes and the workflow's judgement settles into the model, while the facts that keep changing, such as prices, inventory and policy text, stay in tools where they belong.
Finance
Relevance is a house judgement
A large fund's analysts sift filings and news for what matters to their thesis. No prompt reliably transferred that taste; a model trained on their own analysts' labels made materially fewer mistakes than the best rented option, at a fraction of the cost per document.
Legal
Long jobs punish small errors
Diligence and memo drafting run for hundreds of steps across huge document sets, where mistakes compound. Frontier models at maximum effort kept missing the bar; reinforcement learning on an open base cleared it on the firm's own rubrics.
Support
At two million tickets, unit cost is the product
One support platform post-trained a vertical model on years of resolved conversations. It resolves a higher share of issues than the frontier models it replaced and costs less per resolution. At that volume, both matter equally.
Part III
Case study: keeping a marketplace catalogue honest
Every listing on a marketplace has to land in the right category, and the attributes that drive search, filters and recommendations have to be pulled correctly out of its images and copy. Platforms staff this with catalogue integrity analysts, and the team grows with the catalogue: more listings, more categories to know, more ambiguous edge cases to rule on.
Inconsistency spreads. Products get harder to find, recommendations degrade, and violations slip through. Miss a counterfeit and you expose customers and brands; over-flag and you build an expensive queue that punishes honest sellers. Reviewed call-by-call on a frontier API, a mid-size marketplace's daily volume runs into the hundreds of millions of dollars a year. That is not an expensive architecture. It is one that fails the budget outright.
So we built the agent the way the platforms running this at scale build it: examine the listing, search the taxonomy, check the brand, retrieve the attribute schema for the chosen category, commit a structured decision, and escalate to a human when the evidence is thin or the downside is large.
One episode, end to end
A clean listing ends in an approval; a suspicious one has to end in a flag. The scorer weights those two mistakes very differently, because a missed violation costs the business several times what a false alarm does, and the model learns that asymmetry rather than being told it.
Part IV
A place to practise, fail, and retry
A model can only learn a workflow it can actually perform badly. So the first build is not the model. It is a simulator: the same listings, the same tools, the same stakes as production, wired to a scorer that can grade any decision the moment it is made. From a public catalogue of real product images and copy we generated roughly 180,000 review episodes, each with an image, title, description, claimed brand and region.
Into that stream we planted controlled cases: policy violations, mismatched images, conflicting brand claims, and deliberately legitimate claims as hard negatives, so every episode has a known right answer to score against. Inside the simulator the model searches a taxonomy of some thirteen thousand categories, checks whether a brand is registered and protected, and retrieves the required attributes before committing. The scorer rewards correct output and penalises missed violations, unsupported attributes, invalid categories and wasted tool calls. Business priorities are encoded directly in those penalties.
Before training anything we measured the ceiling: five frontier models on the same validation episodes, identical tools, images, scorer and turn budget, run both with a plain prompt and with a heavily engineered one. The engineered instructions helped, then stopped helping: the optimised configurations bunched within a fraction of a point of one another, and one of the strongest zero-shot models got worse with them. They were not free either: the extra instructions inflated input tokens on every call, forever. Prompted task knowledge is rented per call. Trained task knowledge is bought once.
The gap isn't intelligence. A rented model starts every episode from nothing: it has never seen this store's taxonomy, doesn't know which attribute values count as supported, and has to rebuild all of that from the prompt on every single call. The corner cases that decide the score are exactly the ones no instruction list can enumerate.
Part V
The training run was the cheap part
Two rented workstation GPUs, one generating rollouts and one applying updates, plus an open-source RL framework and a thousand optimiser steps. About three and a half days of wall clock and a few hundred dollars of compute. The model crossed the frontier band after roughly a quarter of that; everything after was squeezing.
What it gains is the semantics of its own environment: how this taxonomy, these tools and these policies relate to the reward. A general model spreads its capacity across everything; the specialist spends all of it here, and gives up breadth to do so. Nor is it a dead end. When a stronger open base ships, the recipe transfers, and the deployed model produces its own training data as it works, so each retraining is cheaper and better informed than the last.
The budget takeaway is the simple one. Everywhere else, quality and cost trade against each other. The trained specialist ends that trade: the best score we measured, at close to the lowest cost per thousand decisions in the table below.
Part VI
Where this is the right tool, and where it isn't
The recipe transfers to anything your business repeats at volume: routing tickets, extracting fields from documents, checking submissions against policy, classifying products, approving or flagging transactions. What unites them is that each decision can be checked, by a rule, a schema, or an expert who can say whether it was right. That check is the whole trick. If a decision can be scored, a model can practise it. If it can only be argued about, it can't.
Knowing when to walk away matters just as much. Two questions settle most cases: how often does the task run, and can you verify its outcome? Frequent and verifiable is where training pays. Rare but verifiable is better served by a well-prompted rented model. Unverifiable keeps a human in the loop at any volume. And when the problem is changing facts rather than judgement, retrieval is the fix, not fine-tuning.
Your workflow is a candidate if any of these hold
That last point is also the security argument: a model you trained runs inside your own boundary, so neither the prompts nor the records ever reach a vendor.
If one of your workflows fits, the open question is no longer whether this works. It is which of your decisions you should stop renting. The advantage compounds in a way a subscription never will: the model, the evaluation and the data stay yours, and they get better together.
See these numbers on one of your own workflows.
Bring a decision your team makes thousands of times a week. We'll work through what it costs today, whether it can be scored, and what owning it would be worth.
Appendix
Trained specialists against prompted frontier models, elsewhere
A sample of publicly reported deployments, each measured by the company itself against the rented model it was competing with. Sectors differ; the shape does not.
Figures are as reported by each company in its own write-up and are not independently verified. Our own numbers come from the simulator described in Part IV and are specific to that workflow.
How to read these numbers
This is one internal benchmark on one client workflow, not a general claim about model quality. Every score is the share of an achievable total defined by a rubric we wrote with that client's reviewers, on validation episodes drawn from their own catalogue. Change the rubric or the workflow and the ordering can change with it.
What we cannot publish: the client, the catalogue, the episodes, and the rubric itself, all of which are covered by the engagement. That means the run is not independently reproducible from this article, and you should read it as evidence of a method rather than a benchmark you can cite.
Frontier configurations are unnamed deliberately. Model versions move faster than a published article does, and naming a version that has since been replaced would mislead more than it informs. Cost per decision is measured at the prices in effect during the run and excludes our engineering time.
If you want to test the same claim on your own work, the honest version is to build the rubric and the validation set first and let them decide the answer. We are glad to walk through the method with your team.