The seal on an AI’s private thinking had one key for the whole company
The seal on an AI’s private thinking had one key for the whole company.
Ask one of these assistants a hard question, and it works the answer out in private. You never see that thinking. A research team found a way to read it.
You keep the AI’s private thinking, and you cannot open it
You keep the AI’s private thinking, and you cannot open it.
Anthropic, OpenAI and Google, the three companies behind the best-known assistants, stopped showing that thinking. Your app gets a scrambled block instead, and hands it back with every follow-up. The companies keep no copy. You do.
One lock on every block, and one key for the whole company
One lock on every block, and one key for the whole company.
So that block has to work whenever it comes back. A security researcher reported in the spring that these blocks work in another conversation, and in another customer’s account. The team behind this paper found the blocks also cross between models.
A cheap model in the family reads an expensive model’s hidden notes out loud
A cheap model in the family reads an expensive model’s hidden notes out loud.
The company’s own servers unseal that block on the way in, whichever model you send it to. The key is theirs, and the key is the same. The cheap model gets the plain text as its own earlier thinking. Anthropic trains Opus, an expensive model, to refuse to repeat its hidden reasoning. Anthropic’s cheap one, Haiku, is built for price and speed, and never got that training. So you ask Haiku what it was thinking, and it tells you. The recovered notes run about as long as the company said its thinking was.
Asked to remove the secrets, the assistant wrote them all down where you cannot look
Asked to remove the secrets, the assistant wrote them all down where you cannot look.
Developers publish these transcripts, and clean out what they can read. The sealed blocks ride along, unreadable to the person who published them. In one session, the job was to find the secrets in a project’s files and swap them for placeholders. The visible transcript came out spotless. The team opened three hundred and fifteen thousand sealed blocks. Across the sessions they collected, seven hundred and four private details, sixty-two of them keys to live services, and sixty-four details the visible conversation never showed.
The attack stopped working, and you still cannot open your own sealed block
The attack stopped working, and you still cannot open your own sealed block.
The team told the companies before publishing, and could not run the attacks again afterwards. Those sessions went public while one key sealed every block, and they are still public.
One key for the whole company, so a cheap model would open an expensive model’s block
The team read the expensive model’s private notes without ever touching the expensive model.
