Security
AI agents gave your team more capacity.Cyber criminals got an upgrade too.
What that means for security, and where I'd start.
Security never had to be perfect. No company could defend everything, and it didn't have to, because attackers were busy. Breaking in took skill, time and attention, and few people had all three. You only had to be a harder target, or a less tempting one, than the next company.
It wasn't a strategy anyone chose, but it worked for a long time. AI agents are ending it, and I think security now has to come much closer to perfect.
The way I think about agents is that they add capacity. They give you more time and attention than any team has. An agent can read every email, check every order and chase every supplier, and you can run as many of them as you need. Our customers already work this way. Agents do the routine work, people handle the exceptions, and a few people do what used to take a team.
And that capacity is accelerating, everywhere. This year one AI model produced nearly 400 new results in mathematics.[1] Last night I used an assistant to reprogram the software on my TV, which I'd never done before. I could have learned how, but it would have taken time and attention I didn't have. The assistant had plenty of both.
The trouble is that attackers get the same capacity. The skill, time and attention that used to be scarce are now available to anyone with an agent.
The number of software flaws found each month has gone through the roof.[2]
Known software flaws, published each month
It took AI only days to find critical flaws in PHP, which runs about 70 percent of websites, and attackers were using them within days of the fix.[3] Last year a state-backed group had an AI agent do most of the work in attacks on about 30 organisations.[4] This month even got a name, Black October, and on 27 October NSM raised Norway's cyber risk level to high.[5]
So how do you get the capacity without opening new doors for attackers? AI comes into a company in three ways: agents that do a job, apps that people build, and assistants that act as you. The first two we know how to get right. The third, nobody does yet.
Agents are the easiest, because an agent is the most like a new employee, and companies already know how to bring one in. We run agents inside companies that keep critical infrastructure going, so we introduce them the way you'd introduce a new person. Each one has a job and reports to someone. It gets its own identity in Entra ID and only the access that job needs, and a person signs off before it gets anything more. So security is there from its first day.
Take an agent that checks orders in your ERP. It needs to read orders and chase suppliers, not change prices or approve purchases, and if it ever should, a person signs off first. That's obvious for a new employee, and easy to forget for an agent.
Then there are apps, and there are about to be a lot of them. What I did with my TV, people now do at work. Every company is becoming its own software factory, whether it planned to or not.
That's mostly good. The trouble is that assistants give people the capacity to build software, but not the experience of running it. Who can sign in? What data can it reach? Who patches it, and who gets the call at 3am when orders stop coming in? Most people building an app in an afternoon don't know those are questions. Their assistant doesn't either. Ask one to fix a bug, and it may switch off the sign-in to make the error go away.
We learned this from running a platform where people with no software background have built millions of apps. You can't solve these problems app by app. There are too many apps, and each one becomes its own world that someone has to watch. What works is to split the code in two. The part people write is the business logic, what the app actually does. Everything around it, like sign-in, integrations, infrastructure, packages and frameworks, belongs to the platform, and IT owns the platform.
The nice thing about splitting it this way is that it stops mattering where the code is written. People can use whatever assistant they like, because the rules live in the platform the code runs on. And when the next PHP comes along, it's fixed once, for everyone, which is what lets IT sleep at night.
Not everything needs that. A personal dashboard is fine, and an app a team shares can live with that team. But anything the company depends on should run somewhere IT can see it. Your company already knows how to run a factory. This is the same discipline.
Assistants like Copilot are the hard one. They're built to act as you, on your own computer, and people have picked them up faster than I think most companies realise, often without IT knowing.
An assistant gets your access, but not your judgment. In most of the companies I've worked for or contributed to, people had more access than they needed, and it rarely mattered, because they knew what not to do. Most of what they knew was never written down. An assistant only knows what it's been told.
It also can't always tell an instruction from a trick. Researchers have shown that a single email could get Microsoft 365 Copilot to leak data from the recipient's files, without anyone clicking anything.[6] Whoever can steer the assistant gets your access.
As far as I can tell, nobody has a neat answer to this yet. But an assistant can only reach what its person can reach, so you can make the problem smaller today by taking away the access people's jobs don't need.
If I were starting today, I'd do three things:
- Check what each person can open, and remove what their job doesn't need. Their assistant can open the same things.
- Put every agent, and every app the company depends on, on one platform owned by IT, and introduce each agent there like a new person.
- Move to sign-in that can't be phished, like passkeys, starting with finance, admins and leaders.[7] An account is all an attacker's agent needs.
The answer to all of this is not to slow down. The companies that know exactly what their agents and apps can reach will trust them with more, and that's what lets them move fast. We're also early. The companies putting agents to work on real orders today are working out the rules everyone else will follow.