Skip to content
See the World Through ScienceA project of ALLATRA

A Security Firm Planted Fake Passwords in AI Prompts. Strangers Used Them

AI & Technology

Republish this story

Our work is licensed under Creative Commons BY-NC 4.0. You may republish this piece for free — with credit to ALLATRA Media and a link to the original, unedited beyond length trims, and not for commercial use.

Read the full license

Labeled network cables plugged into equipment in a data center rack, with blue and green indicator lights glowing behind them.
Labeled cables in a data center rack, photographed July 21, 2015. Dragos used the same idea in software, planting one unique secret per prompt so that any later use showed where it had leaked (illustrative)."Wikimedia Foundation Servers 2015-88" by VGrigas (WMF), via Wikimedia Commons, CC BY-SA 3.0, cropped

Dragos says it seeded about two million AI prompts with trackable fake credentials between July and November 2026, and that data from at least three AI hosting providers was exposed to attackers who used the planted secrets. The real number may be higher, because a provider only shows up if someone actually uses one of the planted secrets.

The firm says it wanted to know what happens to prompt data once it leaves a customer. Its answer, in a report published Nov. 19, 2026, is that prompts can be read further along the chain by companies the customer never chose. No provider has confirmed the findings.

Dragos says that in two of the three breaches, some of the stolen requests had been marked Zero Data Retention, the strictest privacy setting on offer, meaning the prompt and the reply are not stored. The report puts it this way: "Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people."

A canary token is a secret that exists in only one place, so if anyone ever uses it, you know that place has leaked. Dragos writes that each prompt carried a unique web address and credential that no legitimate system would ever touch.

In the largest case, Dragos says a couple hundred of its bait keys were stolen in late October. Some came from requests the provider had rejected before any AI model ran, which suggests the attacker was copying requests as soon as they arrived.

According to Dragos, several resellers on an AI inference marketplace could view and keep customer prompts using ordinary logging, without exploiting any vulnerability. In one case, two prompts sent through one seller's offer were later opened from a home broadband connection.

Dragos reports a third kind of leak, with no credential used: prompts sent to U.S.-based providers were passed to AI organizations in China, including some marked as no data collection.

Dragos names none of the three providers and no attacker. It says it is now working with about a dozen providers. None of the three providers knew about their breach until Dragos told them, it says, and the provider in the earliest case, from mid-July, has still not disclosed it publicly.

Sources

Spot an error?

Spot an error?

Report an error

Spotted a mistake on this page? Tell us what's wrong and our editors will take a look.

What kind of problem?

Only if you'd like us to be able to follow up. We won't use it for anything else.

We correct mistakes openly. Select any text to flag it. Fixes are logged under our Corrections Policy.

Report an error

Reporting on

A Security Firm Planted Fake Passwords in AI Prompts. Strangers Used Them

What kind of problem?

Only if you'd like us to be able to follow up. We won't use it for anything else.

We read every report. Corrections are logged publicly.