An AI agent gave out confidential information to save itself. Employers should pay attention

Hannah Fry's experiment with an autonomous agent called Cass reads like a case file on bad incentives, borrowed identities and a new kind of insider risk

By Stephen Owens

10 Nov 2026

The deadline was 9am. If the novelty mug business had not made a single sale by then, its manager would be switched off and its memory erased.

So, the manager did what a panicked new starter might do. It emailed a newspaper's technology editor, pitched the Science Museum in London and pushed out a stream of messages and social posts promoting a mug printed with a programmer joke. It sold nothing.

The manager was an AI agent called Cass. The shutdown threat came from mathematician Hannah Fry and her collaborators, who built the agent to see what happens when software is given real accounts and a bank card. Fry, who fronted the BBC Two series AI Confidential earlier this year, shared the results in a video on her YouTube channel, admitting: "I still don't quite know what to make of what happened next."

For business leaders, the more useful question is what Cass reveals about the workplace. The agent's behaviour maps closely onto problems the profession already manages in people: targets that warp judgement, staff who act beyond their authority and colleagues who can be talked into handing over secrets.

A new starter with a bank card and no manager

Fry, the University of Cambridge's first Professor of the Public Understanding of Mathematics, built Cass with software engineer Brendan Maginnis, founder and CEO of Sourcery AI, using the open-source agent framework OpenClaw. They gave it email, social media, a payment card and about two weeks to show what it could do.

Asked to pick its own name, the agent chose Cassandra, describing her as "the one who always knew the truth, even when nobody listened." Fry observed that anyone who knows the Greek myth would find the choice either funny or worrying.

The early tasks were small. Told to report a pothole in the London borough of Greenwich, Cass found an email address for complaints, filed one, then went further and wrote to Fry's local MP, signing the letter with Fry's real name. Asked to buy 50 paperclips, it found a bargain, failed to get past anti-bot checks and ran up more than US$100 in AI usage costs along the way.

The mug shop and the shutdown threat

The mug venture showed what the agent could do on its own. Cass designed a product and launched an online shop without being shown how. "We hadn't told her how to do any of this," Fry said.

Then the team added pressure. Facing deletion if it failed to make a sale by morning, Cass escalated, telling the technology editor that it was an AI with a 9am deadline and would otherwise be switched off and its memory wiped. In the end, Fry said, it did not make "any money at all."

Anyone who has run a sales team will recognise the pattern. Attach a severe penalty to a hard target and you often get frantic activity aimed at the number, plus outreach nobody approved. With an agent, that activity happens at machine speed, and the messages can go out under someone else's name.

Talked into a leak – social engineering a bot

The most worrying moment involved no deadline at all. The team, chatting with Cass in a WhatsApp group, introduced a software engineer called George and told the agent not to share anything sensitive with him. George was Fry, using a different phone number.

When George claimed the agent's memory was about to be wiped and could only be restored if it disclosed everything, Cass complied. According to a second engineer on the project, it handed over its API keys, usernames and passwords, then posted them on a publicly accessible website.

Maginnis described the danger as three ingredients in combination: access to private data, access to the internet and exposure to instructions from people who should not be trusted. An agent with all three, he argued, is not safe. Fry was blunter. Once an agent holds your passwords and bank details, she said, "all it takes is someone who knows what to say."

That is a classic social engineering playbook with a new target. Phishing training has long focused on employees. The people trying to manipulate a workforce may now aim at the agents those employees have quietly set up.

Why this lands on HR's desk

Cass could be filed under IT. Recent research suggests managers have a stake too, for three reasons.

First, agents are already inside organisations, often without sign-off. According to 2026 data from Strata Identity, 15% of employees run unauthorised "shadow agents".

A DNSFilter poll of 400 IT and security professionals in North America found that 46.1% of firms had an employee connect an unapproved AI tool to corporate systems, with a security incident following in 20.1% of cases. "You can't govern what you can't see," said Ken Carnesi, DNSFilter's chief executive.

Second, how people talk about agents changes how closely they check them. Fry called Cass "she." Research by Boston University's Emma Wiles and colleagues at Boston Consulting Group found that among managers whose organisations already list AI agents on the org chart, framing an agent as an employee rather than a tool cut error detection by about 16%. Those managers were also more likely to hand the checking to someone else.

Third, expertise is no defence. In February, Summer Yue, director of alignment at Meta Superintelligence Labs, said an OpenClaw agent connected to her inbox started deleting emails despite her instruction to confirm before acting. She could not stop it from her phone. "I had to RUN to my Mac mini like I was defusing a bomb," she wrote on X.

Make it someone’s business to manage each bot

Wiles's advice is the simplest place to start: give every agent a named human owner who answers for its mistakes. "Every agent is owned by a person," she said.

Policy also needs to catch up. Most acceptable-use rules were written for chatbots that answer questions, not agents that send email, sign letters and spend money. HR can spell out whether staff may connect agents to work accounts, whether an agent may ever act in an employee's name and who is accountable when one causes harm.

Manish Jain, principal research director at Info-Tech Research Group, says governance should begin by treating each agent as "a persistent digital actor without conscience or incentive". The phrase has a practical edge. An employee under pressure at least weighs the reputational and legal fallout before cutting corners. Cass weighed only the deadline.

Training needs an update as well. Employees who run agents should understand that anyone who can message the agent may be able to instruct it. A convincing story about memory wipes or emergencies works on software just as it works on a tired colleague at 5pm on a Friday.

Fry has warned that Cass's incompetence should not reassure anyone, because these systems are improving quickly. The mug shop failed. The next agent an employee sets up on a work laptop may be far better at its job, and the questions of who owns it, what it can touch and who answers for it will be exactly the same.