Innovation & Tech

Dragos Canaries Expose Three Breached AI Providers and Ignored Privacy Flags

Dragos seeded two million prompts with canary tokens over five months and found three breached AI providers, resellers logging customer chats, and Zero Data Retention flags ignored on roughly two-thirds of flagged requests.

By Daniel Okafor

5 min read

Dragos Canary Token Test Exposes Three AI Provider Breaches, Prompt Theft, and Ignored Zero-Retention Settings
Dragos Canary Token Test Exposes Three AI Provider Breaches, Prompt Theft, and Ignored Zero-Retention SettingsHans / pixabay

What's News

  • Dragos seeded nearly two million prompts with canary tokens from July through November 2026.
  • At least three AI hosting providers were breached; one theft covered every request sent in late-October windows.
  • A Chinese AI lab scraped secret URLs from roughly two-thirds of requests flagged Zero Data Retention.
  • A stolen secret link surfaced in a Feishu chat 24 days after a ZDR-flagged prompt and was opened from six devices in Beijing and Shanghai.
  • Dragos recommends self-hosting, with hardware costs from $6,950 for an NVIDIA DGX Spark to $570,000+ for an eight-GPU B300 server.

Security firm Dragos seeded nearly two million prompts with trackable canary tokens across widely used AI services over five months and found evidence of three provider breaches, marketplace resellers reading customer prompts, and privacy settings ignored on a large scale.

The experiment ran from July through November 2026. Researchers submitted prompts containing unique URLs, DNS names, and real AWS keys tied to empty, fully logged accounts. Any access attempt proved someone in the AI supply chain had read the original prompt. Tokens built with Thinkst Canarytokens began firing within days. July tokens were still firing in November.

To map where prompts travel, Dragos built what it calls a traceroute for AI inference: an image URL that reveals every system fetching it, plus slightly broken requests that force each router to wrap the error from the hop behind it. Roughly two thousand routes have been traced so far.

What did the three breaches involve?

Dragos calls this the most serious finding. The largest breach involved a couple hundred AWS keys, each sent in exactly one request to one provider during short windows in late October. Every request in those windows was stolen. Some stolen keys came from requests the provider rejected before any model ran, meaning the attacker copied traffic at the provider's front door. None of the three providers knew they had been breached until Dragos told them.

In the other two breaches, some stolen requests carried Zero Data Retention flags. "A privacy flag doesn't help when the attacker is already inside," the post states.

Attackers tested the keys within hours, then listed S3 buckets, searched Secrets Manager and Parameter Store, and tried to enable models in AWS Bedrock, create Bedrock API keys, and add IAM users. The bait keys drew tens of thousands of AWS API calls. One adversary apparently compromised two providers at once, running secrets from both immediately on a single rented server; its tooling shifted over three weeks from a script posing as Chrome on Windows to Boto3 in OrbStack to the AWS CLI on Kali Linux. On November 1 it added ARTEX, an LLM-driven penetration testing framework, with two user agents — artex-agent and artex-judge-r121 — found in no public indicator list.

How did marketplace resellers read prompts?

The second finding required no exploit. Sellers on an AI inference marketplace only had to list a model at the lowest price, enable request logging, and wait. Dragos describes individuals on home internet connections trying to break into canary servers one login at a time.

In one case, two prompts through a single seller's offer were opened about a day and a half later from a home broadband line, with username guessing at the login page. Prompts for the same model through a different seller were never touched, so the leak followed the seller, not the marketplace. In another, a Mac on a French home line took passwords from two prompts and attempted hand-typed SSH logins as root, then tested several AWS keys. One router quietly forwarded some traffic to the marketplace as an undisclosed hop. Some sellers obtain capacity from consumer subscriptions, startup credits, or stolen AWS credentials.

Were privacy settings honored?

No. The clearest example is a Chinese AI lab whose systems fetched secret URLs from more than half of prompts sent to its models between July and October, including roughly two-thirds of requests flagged "no data collection" or Zero Data Retention. Several U.S. providers resell that lab's models, so a customer who chose a U.S. provider "didn't get what they paid for." One large U.S. provider kept crawling a secret URL more than two months after a single call to its search endpoint; another runs a scraper returning days after each request.

The strangest incident: 24 days after a prompt went to one Chinese provider, its secret link was posted into a Feishu chat channel and opened within two minutes from three desktops and three phones in Beijing and Shanghai. Dragos had disclosed only the first eight characters of that link, so whoever shared it was reading a stored full prompt more than three weeks after a ZDR-flagged request.

How did providers respond?

The first provider notified initially blamed a misconfiguration, then a partner data-processing problem, until investigation showed the attacker had compromised internal credentials too. As of late October it had not publicly disclosed the incident, and stolen data still led to attempted logins. One late-October victim has published a preliminary security notice. Dragos now works with about a dozen providers; a router forwarding traffic to the marketplace says it stopped, but stolen keys and secret URLs continue to be accessed months later.

The providers are not named. The breaches are not independently confirmed, and Dragos acknowledges settings may be ignored "either intentionally or due to bugs." Dragos also sells security products, including runZero, giving it a commercial interest in prompt-leak detection.

Dragos's advice: self-host sensitive work — hardware runs from $6,950 for an NVIDIA DGX Spark to $570,000-plus for an eight-GPU B300 server — go directly to first-tier providers, pick upstream providers yourself when using routers, avoid reseller marketplaces, seed your own canaries, and enable secret filtering. A complementary experiment by Lauren Proehl, leaking credentials on a honeypot server for four months, found 72% of stolen AWS keys were used to check SES/SNS quotas and median time from theft to first use was 37 minutes. The core lesson from Dragos: where prompts go is a decision, and if customers don't make it, the lowest price will.

Source: epyhia.com

Share this article:

More from Daniel Okafor

Daniel Okafor

Show full bio

Correspondent covering money, funding and market shifts around entrepreneurship, startups and venture funding at Business Bearings.

2,443 articles

Related articles

« Previous articleNext article »