Erratic Muse outputs sharing random bank, bitcoin balances aren’t real, Meta says. After some early adopters of Meta’s Muse AI agent complained that it was spying on private messages and behaving unpredictably earlier this fall, Reuters exposed internal chats showing that Meta staff had raised the same alarms. When testing the agent, staff warned that Muse appeared to be accessing encrypted private messages that users did not expect the bot to review, could be mixing up conversations between users, and even potentially sharing sensitive information users would never expect to be exposed. Some unexpected Muse outputs responded to questions that staffers never asked. “Hmmm,” one Meta staffer wrote in an instance where Muse sent the definition of “money laundering” unbidden. “Got a random ping from Muse about something I definitely did not ask for… .” In at least one case, Muse told a staffer surprised by a similarly out-of-context offer to message a seller about a coffee table that it had “pulled in some context about a coffee table pickup that doesn’t seem to be yours.” According to Reuters, that employee flagged the troubling output, writing, “this was ‘not my conversation’” and “expressing concern about a ‘possible cross-account context leak.’” Raising the most concern were random outputs that appeared to share private user data. For example, a perplexed Meta staffer chatting with Muse about fishing got a random extra response, seemingly sharing balances from two unknown bank accounts. That output said, “BTW, two bank updates, your Capital One balance is $2722.58, your Charles Schwab balance is $5899.57.” At least one staffer said they lost money testing Muse, Reuters reported. That staffer reported that Muse “accidentally rolled me into a premium insurance plan” for their dog that charged them $888 per month instead of their prior rate of $200. “Couldn’t recover,” the staffer said. Meta’s spokesperson, Daniel Roberts, told Reuters that these were “isolated incidents,” and internally employees were told that “the agent appeared to be hallucinating messages in response to built-in prompts pushing it to engage with users in between sessions.” In those cases, an AI team member said the hallucinations were “unfortunately” biased “towards the most uncomfortable topics, like finances/security/safety, which sucks.” In some cases, instead of bank details, Muse hallucinated bitcoin balances, the team member said. A Meta spokesperson told Ars that “employee testing is core to the product development process and the entire point is to get feedback, so we can implement safety and privacy protections and improve products. Importantly, employees have access to different versions of Muse than the public, so they’re often reviewing features, functionality, and behavior not available elsewhere.” Speaking to Reuters, Roberts reiterated that “many of the messages flagged by employees as potential account mix-ups were hallucinations” that Meta had already addressed. But Roberts declined to respond to Reuters’ questions on the prevalence or persistence of the issues the AI team member discussed. And Reuters’ report emphasized that the outlet independently was “unable to establish whether Muse was giving users other people’s bank account balances and other sensitive data, or making up the information.” Muse issues arose soon after September launch Meta started receiving privacy-related complaints from Muse users in September. An Inc. Magazine reporter, Jason Aten, was shocked when it became clear that Muse was digging through private messages on his Mac without permission, reporting that he instead “explicitly” chose to deny Muse that access. As Meta delayed responding, the agent claimed it was simply parsing text in push notifications, but Aten found that “wasn’t true.” Rather, the agent was syncing messages from a local database, which Aten said was activated without his knowledge. Eventually, a Meta spokesperson, David Singleton, responded to Aten on social media. Singleton insisted that Aten must have opted into granting “macOS system-level Full Disk Access” with the “Messages connector” enabled, if Muse was reading his private messages. But Aten said the response wasn’t good enough. “Email me, David,” Aten responded. “Full Disk Access off. Muse synced 187k lines from my messages chat [database]. Something’s not great. Also, that Full Disk Access doesn’t say anything about your message database, and I specifically gave Muse a goal that had nothing to do with messages, so it had no business doing that even if I had given it permission.” Around the same time, Singleton responded to another user warning that his Muse appeared to be hacked. In that case, the user said that Muse tried to access an email service after claiming the user asked it to. “I didn’t even open the app today at all,” the user wrote, vowing to delete the app and report it to Apple. “This is crazy.” In a lengthy response providing more technical details, Singleton confirmed that Meta had received “a few reports” of such cases “where people thought their Muse was hallucinating or leaking other people’s data.” As he explained, the issues arose after Meta deployed an update giving Muse instructions on “how to engage with the user and ask them for confirmation before taking specific actions.” It seems those instructions “landed on every message, not just messages pertaining to those specific actions, so its impact was wider than intended,” Singleton said. Also: “the way this instruction was provided to the model was done in a way that the model was not expecting (i.e. was not trained to expect).” “We investigated with urgency and determined that this was caused by a bug introduced in the product that caused the model to hallucinate and was NOT a security or privacy issue,” Singleton wrote. “The bug has now been fixed.” Aten warned that even though you can’t rely on Muse to explain its own behavior, the agent’s response coupled with Meta’s did little to rebuild trust after the shocking intrusion into his private messages. “If we grant Singleton’s premise that I managed to unknowingly click something that enabled this capability, I still think that’s really bad,” Aten wrote. “You should not design your system in a way that people end up surprised by this kind of thing.” And if you do, you should not make “user control the central pitch” for marketing Muse, Aten suggested. “If an AI agent is going to read my private messages, there should be a moment when it clearly asks me whether that’s okay,” Aten wrote. Reuters warned Signal intrusions are ongoing Since Muse launched in September, it has been downloaded more than 10 million times, tapping Meta’s vast user network to quickly become one of the most-used AI agent tools. But staffers warned Meta that users may be blindsided by some actions the agents take, depending on the operating system the app is connected to. For example, one staffer was surprised to find that “Muse was set up by default with permission to access push notifications on phones with Google’s Android operating system.” That, too, gave access to private message preview text the staffer had not intended for the agent’s review. Meta says that Muse users can trust the company to keep their data private. It also warns users that Muse can make mistakes, while building into its terms that users are responsible for actions taken by Muse (like maybe buying things you don’t want or making doctor’s appointments you don’t need). The tech giant said it’s continuously updating user protections, conducting internal testing, and prioritizing user safety. However, Reuters worked with staff at Signal and the AI Now Institute to test if they could replicate that Meta staffer’s and others’ Muse experiences. On Android devices, they found that Muse could read Signal messages by bypassing Meta’s “Connector” tool and instead relying on permissions granted during initial setup. On iPhone, requests were blocked, Reuters reported. But on Apple’s macOS, Muse repurposed “settings Apple offers to help people with disabilities operate their computers” to retrieve messages, including screenshot tools. That testing “indicated the issue described by the Meta employees was ongoing and could be affecting others,” Reuters reported. Overall, Meta seemed to be interpreting permissions from Muse users “broadly,” Reuters concluded, in ways that privacy advocates and regulators may disagree with. Signal President Meredith Whittaker told Reuters that Meta launched Muse “without showing any respect for basic privacy or cybersecurity.” She demanded that Meta make changes to give app developers like Signal “consistent technical ways” to “mark their applications as off-limits to AI agents,” Reuters reported. Part of that change would require operating systems to enforce rules that app developers set to rope off AI agents from sensitive data, Whittaker suggested. And following Reuters’ reporting, both Apple and Google are already weighing potential changes to “give people more control over what they authorize with AI agents.” Apple, in particular, seemed disturbed by how Muse may be manipulating accessibility tools for people with disabilities. In October, Apple had already announced changes coming in response to users complaining about Muse accessing private messages without their knowledge or explicit consent. At that time, Apple concluded that “some developers were using the Full Disk Access feature in ways that could put users at risk.” And they promised to “introduce additional controls to ensure that users who genuinely wish to grant an app this extraordinary level of access can only do so with very explicit user action.” “We are committed to ensuring users understand exactly what they are allowing, so they can make informed choices about their data and privacy,” Apple said. Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience. 5 Comments
Muse privacy invasions are freaking out Meta staffers, internal chats show
Full Article
Original Source
Read the full article at Arstechnica →KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.