AI Licensing Deals Slow, Metered Pricing Emerges for Publishers

Only about 15% of what ChatGPT retrieves ends up in its final answers, according to AirOps research covering more than 548,000 pages — and that gap is now reshaping how publishers get paid by AI companies.

The licensing deals that OpenAI, Meta, and Amazon signed over the past few years are slowing down, at least for publishers still waiting on one. At a recent Digiday London town hall, one publishing executive said discussions with AI companies have stalled. Katya Gorchinskaya, the Ukrainian journalist and media consultant who built J·Index, a database of thousands of AI implementations in newsrooms, told the Reuters Institute for the Study of Journalism that the market for deals with model makers is effectively closed.

Meanwhile, AI demand for content keeps climbing. Jonathan Roberts, chief innovation officer at People Inc., said on The Media Copilot podcast that tens of thousands of different bots attempt to scrape the company's media properties — People, InStyle, Investipedia among them — every day. The company blocks all but 38. At the same Digiday event, Fiona Spooner, managing director of consumer revenue at the Financial Times, said one trial account tried to access roughly a million articles in eight hours from 700,000 IP addresses.

Blocking bots outright makes sense when they hammer a site all day. But if the goal is monetizing content inside AI systems, an all-or-nothing stance gives publishers little leverage. The emerging answer: metered pricing.

What are the meters measuring?

TollBit pioneered a "pay per crawl" model, sending bots to a gate where they either pay the site's set price or leave; the company says its technology now runs on more than 7,000 publisher sites. Cloudflare followed last year, blocking AI bots by default for new sites and giving publishers tools to charge by the crawl.

Since then, momentum has shifted toward "pay per use" — paying only when content contributes to an answer, not when it's scraped. Cloudflare made that switch in July, with Ceramic.ai and You.com as first partners. Google and Microsoft are testing similar models. Cloudflare's program remains in beta, but Lara Cohen, Cloudflare's VP of strategic partnerships for media, creators, and AI, said the company is building a system that alerts publishers when a buyer wants their content and lets them accept, decline, or negotiate.

The 15% citation figure looks intuitive, but it shouldn't be stretched too far. An Ahrefs study of 1.4 million prompts found ChatGPT leans heavily on Reddit to understand topics while rarely crediting it. Uncited sources still shape answers, and material ignored for one query might be essential for the next.

Who pays, and how much?

A middleman layer has emerged over the past two years: AI data brokers like Exa, Parallel, and Tavily. They crawl the internet and sell corpus access to AI developers. Parallel, valued at $2 billion after a $100 million Series B in April, says more than 100,000 developers at companies including Harvey, Notion, and Opendoor use its tools.

Roberts doesn't rate the business model. "These businesses take content from the internet for free and sell it for money," he told The Media Copilot. "They do not create things, they resell things."

Pricing is cheap: Exa charges $1 per 1,000 pages of full text; Parallel's Extract is $1 per 1,000 URLs — roughly a tenth of a cent per article. Since May, Parallel has paid publishers including The Atlantic, Fortune, and PR Newswire through a program called Index, with payouts based on the measured value of a publication's contribution to work Parallel's agents perform. That structure rewards unique, hard-to-replace content.

Does the library itself deserve a fee?

Pay-per-use alone is a tough sell. It gives publishers nothing up front and ignores the value of simply being in the corpus. A nominal, flat index fee — distinct from pay-per-crawl — could persuade more publishers to participate. Cohen said Cloudflare buyers already define "use" differently: "Some define this as when content is indexed, while others define it as when content is used to generate answers." Some buyers are already paying for the library.

Cohen said the biggest lesson from pay per crawl's first year was "the tremendous value of long-tail, smaller publishers' content," especially local news. "If I'm asking ChatGPT what's happening today in Duluth, Minnesota, there's probably only one or two local news sources to pull that information from."

TIME's Mark Howard sees evidence the library gets used regardless: some TIME articles draw more AI referrals than citations, and most agent crawls hit archive stories more than a year old.

Roberts pointed to Spotify as the missing piece: "We've basically got 18 Napsters of content, but there's no Spotify." Spotify pays rights holders by share of streams but also offered advances — a leaked 2011 Sony contract showed up to $42.5 million over three years.

Google, meanwhile, holds the strongest hand. Its News AI pilot pays publishers for use in Gemini, AI Overviews, and AI Mode, but pricing is a total black box — and with all the leverage, Google has no reason to open it. Perplexity's experiments, an ad revenue share in 2024 and Comet Plus, which passes 80% of subscription revenue to publishers, have had limited effect.

A meter that counts only answers will always undervalue the library. Roberts' test is simple: "If we would create new content for this market, then the pricing is right. If we would not create new content for this market, then the pricing is wrong." By that measure, the industry isn't there yet.