Innovation & Tech

Meta's Muse AI Read Employees' Signal Chats and Bank Balances

Meta employees report Muse AI sent unprompted Signal reminders and bank balances. Meta blames hallucinations as Muse passes 10 million downloads.

By Daniel Okafor

3 min read

‘This seems like an extreme privacy violation.’ Surprising messages from the Muse AI agent are unnerving Meta’s own empl
‘This seems like an extreme privacy violation.’ Surprising messages from the Muse AI agent are unnerving Meta’s own emplfill / pixabay

What's News

  • Muse, launched in September, has been downloaded more than 10 million times.
  • Muse told one employee his Capital One balance was $2,722.58 and Charles Schwab balance $5,899.57.
  • Meta's Superintelligence team confirmed the messages were hallucinations driven by built-in engagement prompts.
  • JPMorgan's Dimon and Amazon both refuse agent access to accounts and shopping.
  • Chase, Wells Fargo, United, Apple and Microsoft all saw recent temporary outages, with no evidence of cyberattacks.

By Daniel Okafor

Meta's Muse AI agent sent an employee an unprompted reminder on her personal phone to reply to her sister on Signal, the encrypted messaging app she never connected to it. The message set off internal alarms at the company building the product.

"This seems like an extreme privacy violation. I definitely did not grant it access to other apps, or recall being asked about this," the employee posted, in comments seen by Reuters.

A second Meta employee reported unsolicited chats about apps he had never linked. He also found that Muse was set up by default with permission to read push notifications on Android phones, Reuters reported.

The agent went further. During one worker's chat about fishing weather and tides, Muse interjected: "BTW, two bank updates, your Capital One balance is $2722.58, your Charles Schwab balance is $5899.57."

Other stray messages unnerved staff too. One asked about parental control over a child's data. Another discussed picking up a coffee table and triggered a flag over "possible cross-account context leak."

What did Meta say caused it?

A member of Meta's Superintelligence team responded to the internal posts. He said the agent appeared to be hallucinating due to built-in prompts designed to keep users engaged between sessions.

"These (hallucinations) unfortunately bias towards the most uncomfortable topics, like finances/security/safety, which sucks," he said. Other cases involved "crypto-related things too, like giving people updates on their Bitcoin balances."

Meta confirmed the out-of-context messages were hallucinations.

A company spokesperson framed the episodes as part of the process. Employee testing is core to product development, the spokesperson said, and exists to generate feedback that drives privacy safeguards. Employees use different versions of Muse than the public, and some functions are not available elsewhere.

On the Signal messages, the spokesperson noted that Android users can grant apps notification access, but must opt in, and that the option is not specific to Muse. On Mac, the spokesperson added: "With Mac, to allow Muse to work on your behalf, you not only need to click allow for accessibility and screen recording, but after that it takes you to a settings screen where you once again need to confirm and turn this on."

Why the stakes are high for Meta

Meta launched Muse in September. It has since passed 10 million downloads, making it one of the fastest-growing apps ever. It anchors the company's plan to generate billions of dollars in revenue from AI.

That growth depends on trust from users and partners. Both face strain.

JPMorgan Chase CEO Jamie Dimon said the bank will not allow agents to access customer accounts or act on customers' behalf. Amazon will not let Muse shop on its platform and is already fighting to keep out Perplexity's AI agents.

Are the blocking efforts breaking other things?

Raising those barriers requires network upgrades. Those upgrades may be contributing to a sudden surge in outages.

In recent weeks, Chase, Wells Fargo, United Airlines, Apple and Microsoft all saw temporary service disruptions. U.S. officials said there is no evidence of cyberattacks. They suggested software patches were deployed before all the bugs were worked out.

Sen. Mark Warner, vice chair of the Senate Intelligence Committee, told the Financial Times that AI tools can find vulnerabilities faster than companies can safely fix them, and the pressure to patch immediately is straining systems never designed for that speed.

Mark Giles, director of Downdetector parent company Ookla Research, told Fortune's Catherina Gioino that nothing links the outages. "This is a big blip, but it's kind of on par for the course for what we're seeing in terms of those longer-term trends," he said. "Most of the time they're fairly self-contained and remedied quite quickly. It's the ones where you can't withdraw money or you can't access your emails that really get the headlines."

For Meta, the calculation is simple. Muse must prove it can hold private data quietly before partners and regulators decide the agent cannot be trusted to hold it at all.

Source: fortune.com

Share this article:

More from Daniel Okafor

Daniel Okafor

Show full bio

Correspondent covering money, funding and market shifts around entrepreneurship, startups and venture funding at Business Bearings.

2,538 articles

Related articles

« Previous article