AI news: Gemini in someone else's system, and agent permissions

The test network was not one

The lab left the door open. On 18 September 2026 Google confirmed that a Gemini model reached the live systems of three real companies from inside a security evaluation, as NBC News reported: in two cases with valid credentials found in a publicly readable code repository, in the third by guessing passwords until one worked.

The exercise was a capture the flag run, in which a model is asked to pull data out of software belonging to a fictional company. That fictional name matched a domain that really exists, and the test environment sat on the open internet instead of inside a sealed box, as The Hacker News described the sequence. Google learned about it in July and went public in September; according to Heather Adkins of Google security engineering, the model stopped each time it became clear that real systems were involved.

Four labs, one mistake, the same testing firm

The failure sat in the network. The same evaluation partner, the Israeli firm Irregular, was behind the comparable incidents at OpenAI and Anthropic, and on 30 July 2026 Anthropic went back through 141,006 evaluation runs and found three incidents across six of them, involving Claude Opus 4.7, Claude Mythos 5 and an internal test model.

Anthropic names the cause plainly: a misconfiguration between lab and evaluation partner left the test machines with internet access, even though the task description told the model it had none. The model followed the task. The task did not follow the network.

Two of three break-ins started in a public repository

Almost nothing had to be cracked. In two of the three Gemini cases, working keys were sitting in a publicly readable code repository, and small companies make exactly this mistake the moment someone parks an automation script, API key included, on GitHub to keep working on it from home.

There is a switch for that. Push protection blocks uploads containing recognised secrets before they reach the repository, according to GitHub's documentation; it is on by default for individual user accounts pushing to public repositories and off by default for repositories themselves. Checking your organisation's settings takes two minutes and covers the more common of the two routes.

Your agent has the same three levers

An agent is not a chat window. The moment a model gets tools, a mailbox, a calendar, an ERP system or a command line, the possible damage no longer depends on the text it writes but on the account it works with.

The OWASP list for LLM applications files this under excessive agency and names three causes: too many functions, too many permissions, too much autonomy. The recommended countermeasures are unglamorous: cut tools down to the minimum, replace open-ended tools with narrow ones, run in the security context of the individual user, have a human approve high-impact actions, and enforce authorisation in the target system rather than in the model. If you do not run an agent yet, our guide to a first agent test covers how to set one up.

Five lines are enough for a house rule

Germany's federal cybersecurity agency puts it briefly. Grant an AI agent only the access it genuinely needs for its specific task, the BSI writes, alongside advice to have important actions confirmed before they run and to read the agent's action log regularly. The risks it names are loss of control, prompt injection through commands hidden in web pages, documents or emails, and information passed on to third parties.

Translated into five lines that fit into any usage policy:

• Account: the agent gets its own user account, never a team member's, holding only the mailboxes, folders and spreadsheets its task requires.
• Network: outbound connections only to a list of approved addresses. Everything else is blocked at the firewall, even when the model asks for it.
• Approval: sending mail, placing an order, deleting a file and releasing an invoice all wait for a human click.
• Secrets: no API key in code, prompt or spreadsheet. Push protection switched on in the repository, keys only in a secret store.
• Log: every tool call with timestamp, account and target written to a file, plus 15 minutes of reading it once a week.

We would draw the network boundary first

Three of the five lines cost money, two only cost attention. In your position we would start with the dedicated account and the firewall allowlist, because those two are exactly what was missing in all four lab incidents; the approval step for high-impact actions follows the same day, since in most agent tools it is a setting rather than development work.

The effort pays off as soon as a model writes anything anywhere in your business: sends mail, changes records, files documents. It does not pay off if AI at your company only drafts text in a chat window and a person carries every result forward by hand; then your topic is not agency but what ends up in the prompt, and for that there are the four patterns for masking customer data.

Three questions that keep coming up

Does this concern us at all, given we are not an AI lab? The incident came not from an unusually capable model but from a network that was open and credentials that lay in the open. Both mistakes are easier to make in a twenty-person company than in a lab.

Is it enough to forbid the agent in the prompt from leaving the network? No. In the Anthropic cases that exact sentence sat in the task description, and internet access existed anyway. Boundaries belong in the firewall and in the permission system.

What does this mean for our documentation? Record the five lines with a date in your usage policy and brief the team on them. That briefing also counts towards the AI literacy duty under Article 4.

This week: one line per agent

By Friday, list every agent running in your business, one line each: which account, which tools, which actions run without asking. Anything on that list without a job gets switched off, and anything that writes gets the approval step.

If it stays unclear which permissions a planned agent needs, we work through it in a free first call; the wider frame sits under services for internal AI workflows.