Everyone's Watching the Wrong Benchmark

Why the gap between open and closed models is about to widen, and what it means for the data you thought was yours.

Picture this: it’s 11pm and a former Goldman analyst is sitting at her kitchen table, laptop open, working through a stack of forms. She’s not building a pitch or a model of a leveraged buyout. She’s writing out, in careful detail, how she’d actually reason through one: the judgment calls, the shortcuts, the things you only learn after a decade on a desk. She’s getting paid $200 an hour for it. On the other end is a frontier lab, and the thing she is patiently teaching will, if it works, be very good at her old job.

Multiply that scene by tens of thousands of experts (doctors annotating clinical reasoning, litigators marking up briefs, senior engineers reviewing architecture decisions) and paying out more than $1.5 million a day at a single vendor, and you have the most under-reported story in AI. A brand-new industry has materialized in about 24 months, almost entirely to sell one thing to a dozen buyers. And it tells you something the leaderboards don’t.

Is the open-source gap really closing?

The consensus right now is comforting if you like open source: the gap is closing, maybe closed. And on the public benchmarks, that’s largely true. Epoch AI’s aggregate capability index puts open-weight models only about four months behind the closed frontier. Stanford’s 2026 AI Index tells the same story from another angle: the top U.S. model’s lead over China’s fast-rising — and largely open-weight — competitors has narrowed to 2.7% as of March 2026, after DeepSeek-R1 briefly matched the best American model a year earlier. Four months. Measured against valuations that price in a durable moat, that’s a rounding error.

Credit: Stanford’s 2026 AI Index Report

If that were the whole picture, this post would be an argument for open source, and I’d be writing it.

But look closer at what these models can and can’t actually do. The same 2026 AI Index that shows convergence also shows frontier models now matching or beating human experts on PhD-level science and competition mathematics — while still failing at, among other things, conducting financial analysis. Sit with that. The benchmarks that are saturated are the public, gameable ones. The capabilities that would actually let a model replace an incumbent (the messy, high-context expert work) aren’t there yet. Epoch’s own footnotes say the quiet part: open-weight models do worse on private benchmarks than public ones, the polite way of saying they’re trained to the test, and the leading labs don’t always release their most capable models. The frontier you can measure isn’t the frontier that exists.

That’s the whole thesis in two sentences from the people who build the charts. The benchmarks are saturated, so everyone can climb them. What they can’t do is climb the things nobody publishes.

Where is the future gap being created?

For most of the deep-learning era, everyone trained on more or less the same raw material: the public internet. That’s why open source kept pace — the ingredients were a commons, and the rest was compute and technique, both of which diffuse fast. Compute gets cheaper. Papers get published. Weights leak or get released.

The internet is now largely used up as a source of new signal, and the labs have quietly moved the fight to ground open source can’t stand on: data they pay to create. Not the old crowd-labeling of stop signs, but expert reasoning, the tacit, hard-won knowledge that lives in people’s heads and in the workflows of companies that would never hand it over. The vendors selling this are growing at rates that don’t look real. One crossed $1.2 billion in revenue while bootstrapped. Another went from a $1 million to a $500 million run rate in 17 months. A third went from $7 million to $100 million in a single year. Meta paid nearly $15 billion for a stake in one of them.

Add it up and the labs are spending somewhere around $10–15 billion a year creating training data today, and at least one operator close to it thinks that number passes $100 billion within two years. To put that in context: the entire global market for enterprise application software — every CRM, ERP, and HR system on earth — is ~$300 billion a year. We are watching a data-creation industry grow, from a standing start, toward the scale of the software industry it aims to automate in a few years (relative to 20+ years of SaaS development to reach this scale).

Almost all of this is private, undisclosed opex. Unlike the compute capex that gets guided and dissected every quarter, nobody has to report what they spend on data. When a category is this hard to see and growing this fast, the reported figures are a floor, not a ceiling. Our view at Gradient is that the true size of the data-provision business is materially understated today, likely three to five times larger than the headline run-rate numbers suggest, or growing to that over the coming year.

This is the part that doesn’t diffuse, and the companies with the resources to invest heavily in it seem to be a fairly limited set. Compute you can rent. A technique you can read about. But a million hours of a surgeon’s or a litigator’s judgment, bought exclusively and never published, is an asset that compounds and stays put. The frontier labs figured out that the moat was never the algorithm. It’s the people, and the data only they paid for.

Why can’t open source just distill the frontier?

There’s one more reason the gap should widen rather than close, and it’s the newest. For years the fastest way to approximate a frontier model was to learn from its outputs — distillation. That door is closing. OpenAI now enforces its terms against it, and in February 2026 Anthropic disclosed industrial-scale distillation campaigns by three Chinese labs DeepSeek, Moonshot, and MiniMax) that generated more than 16 million exchanges through roughly 24,000 fraudulent accounts to extract Claude’s reasoning, coding, and agentic capabilities. Both Anthropic and OpenAI have since framed this kind of extraction as a national-security concern, and OpenAI has taken its case to Congress.

We don’t think anti-distillation is the moat.t’s leaky, and it always will be. But it raises the cost and slows the copy, and it’s arriving alongside a broader retreat into opacity. Stanford’s Foundation Model Transparency Index fell to 40 from 58 in a single year, and the most capable models now disclose the least. When access to the true frontier is gated by contracts, export controls, safety reviews, and simple silence, the measurable frontier and the real one drift further apart. The recent constraints around Mythos-tier access are a preview, not an exception.

Enterprise Exposure: The proprietary capability you’re quietly giving away

Here’s the uncomfortable turn, and it’s the reason I care about this beyond the leaderboards.

The same mechanism that’s building the labs’ moat runs straight through the enterprise. The reason those former analysts and consultants are so valuable is that their employers won’t sell the labs their data, so the labs hire the people who carry it in their heads instead. Brendan Foody, who runs one of these marketplaces, says it plainly: Goldman Sachs doesn’t love the idea of a model that can automate its own value chain. An industry’s knowledge, he admits, can slip out the back door through its former employees, and get used to automate the work of the people still there.

Now extend that from ex-employees to current systems. Every time an enterprise runs its proprietary workflow — its actual way of doing things, the trade secret that makes it worth more than its competitors — through a closed model it doesn’t control, it’s handing over exactly the signal the labs are spending billions to acquire elsewhere. The feedback, the corrections, the edge cases: that’s the good stuff. You are, slowly and for a monthly fee, teaching a system owned by someone else to do the thing you’re paid to do. And that someone else may not stay in their lane. Anthropic just launched Claude Science and stood up its own internal drug-discovery program, a reminder that today’s model provider can be tomorrow’s competitor in your own market.

The convergence data, read this way, is a trap. It tells enterprises that models are commoditizing, so it doesn’t matter which one you pick or where your data flows. But if the real gap is widening and it’s made of proprietary data, then the choice of where your “way of doing things” accumulates is one of the most important decisions a company will make this decade.

A Call for Builders: Where we’re investing

This is where it stops being a diagnosis and starts being a thesis. If proprietary data is the moat, then the most valuable place to sit is between the enterprise and the frontier, helping companies get the full benefit of AI without giving away the thing that makes them special. Three kinds of companies stand out to us, and we’re actively looking to back all four.

US-based and US-funded open-source model labs built around security and guardrails. If open weights are going to underpin serious enterprise work, someone trustworthy has to make them. Today the leading open-weight model comes out of China, and there’s a wide gap between “open” and “safe to run your business on.” This may change given recent news of Chinese labs considering going closed source. Google and NVIDIA seem dedicated to continuing to publish leading open source models, and we bet more will follow. We want to fund the teams closing the safety and security gap, enabling enterprises to deploy models that are self-hostable, but hardened, safety-reviewed, and accountable in a way a weights file alone never will be. This is table stakes for everything that follows: the enterprise can’t own its stack if there’s no trustworthy stack to own.

Companies that bring open-weight model adoption into the enterprise, and hand the enterprise the controls. The tooling, data management, and fine-tuning layer that lets a company encode its own workflows and proprietary approaches into a model it controls, and manage the full model lifecycle in-house. This is the picks-and-shovels of data sovereignty: everything a company needs to capture its way of doing things in a model whose weights and training data stay home, rather than renting a black box and feeding it the crown-jewels one query at a time. These companies may support fine-tuning, reinforcement learning, data synthesis, observability and monitoring, or other important aspects of full model ownership.

Orchestration and routing layers that let the enterprise mix models without surrendering to any one of them. Many deployments will not be all-open or all-closed; they’ll be hybrid: a frontier closed model for the hardest reasoning, open-weight models for everything that can run in-house, and a fleet of agents in between. We want to back the routing and orchestration layer that makes that choice dynamic and the enterprise’s own: sending each task to the right model on cost, latency, sensitivity, and capability, keeping the data and high-value workloads on models the company controls while still reaching for the frontier when it’s worth it. Done well, this is also where the enterprise’s leverage lives. The router sees which tasks actually need the frontier and which don’t, turning what looks like lock-in into a portfolio the customer manages, not a dependency a lab manages for them.

Agentic and application vendors whose products get better with usage,but keep the compounding with the customer. The application, the model, and the agentic workflows all improve as the enterprise adopts them. The difference is architectural: they deploy federated models and let the enterprise retain all of the feedback, data, and resulting model weights. The customer gets a system that sharpens with every interaction; the proprietary capability it builds along the way belongs to the customer, not to a lab three abstraction layers away.

The common thread is ownership. The pure wrapper on a closed model — no proprietary data, no compounding, nothing the customer keeps — gets squeezed from both sides. The companies we’re most excited about understand what the benchmarks are hiding: the gap isn’t closing, it’s moving somewhere you can’t see it. And it’s made of the very thing every enterprise already has and is quietly giving away.

If you’re building in these spaces or curious to do so, we’d love to talk.