TLDR: Generative AI gave attackers the ability to produce flawless, personalized phishing at scale, but it also gave defenders faster detection and triage. Agentic AI raises the stakes on both sides by removing the human from the loop. When an autonomous agent reads and acts on an email without a person reviewing it, cryptographic verification becomes the only check left standing.
Security teams have spent two decades training people to spot bad email. Look for the typos. Check if the greeting feels generic. Trust your gut when something reads wrong.
Generative AI erased all of that in about eighteen months.
The typos are gone. The greeting uses your name, title, and a reference to the project you posted about on LinkedIn last week. Your gut has nothing to work with because the message reads exactly like the hundred legitimate ones you got this week.
Now, agentic AI is starting to remove the human from the loop entirely. When an autonomous agent processes an email and acts on it, there’s no gut to trust and no eyes to catch anything. It’s all robots identifying robots.
Well, there’s a bit more nuance to it than that.
Below, we break down what generative and agentic AI changes in cybersecurity (on both sides of the fight) and what still works when content can’t be used as evidence.
What generative AI changed for attackers
Generative AI didn’t invent any new attack types. Phishing, business email compromise, and social engineering all predate large language models. What AI changed is the economics.
- Personalization. Spear phishing used to require research: an hour or more per target, digging through LinkedIn and company pages to build a convincing pretext. That cost limited targeted attacks to high-value victims. An LLM does that research and drafts the message in seconds, which means every employee is now worth targeting individually.
- Quality. The broken English and formatting errors that once flagged a phishing attempt were artifacts of attackers working in a second language at volume. Generative AI writes clean, native-fluency copy in any language, matched to the tone of whatever organization it’s impersonating.
- Scale. Traditional phishing forced a choice between generic messages to thousands or tailored messages to a handful. AI removes the tradeoff. Attackers now send individually personalized messages at the same volume that generic spam used to operate.
- Voice and video. Deepfake audio has moved from novelty to operational. A finance employee who receives a suspicious wire request, then gets a follow-up voice message that sounds exactly like the CFO, is facing a much harder judgment call than one working from text alone.
Check out our guide to AI phishing attacks to go deeper into how these attacks are built and what defenses hold up.
How generative AI helps cybersecurity defenders
The picture isn’t one-sided, though. Generative AI has given security teams new capabilities:
- Triage and summarization. SOC analysts drown in alerts. LLMs are good at reading a pile of logs, alerts, and telemetry and producing a readable summary of what likely happened. That compresses investigation time, especially for tier-one triage.
- Detection engineering. Writing detection rules and threat-hunting queries is skilled, slow work. AI assistance speeds up rule authoring and helps translate a described behavior into a working query, which lets smaller teams cover more ground.
- Threat intelligence synthesis. Reading and correlating threat reports across dozens of sources is a full-time job. AI does the first pass, surfacing what’s relevant to your environment.
- Simulation and training. The same capability that lets attackers write convincing phishing lets security teams generate realistic training scenarios, so awareness programs test people against the quality of attack they’ll really encounter.
These are real efficiency gains. They help defenders do existing work faster. Still, what they don’t do is solve the underlying problem, and that’s because the underlying problem isn’t speed. It’s that the signal defenders relied on has stopped being reliable.
What agentic AI changes for cybersecurity
Generative AI produces content. Agentic AI takes actions.
An AI agent reads the inbox, decides what needs a response, and sends one. It doesn’t just summarize an invoice—it routes it for payment. Organizations are deploying these agents now, and the security implications are starting to land.
A human who receives a suspicious request might sit on it for an hour, mention it to a colleague, or notice something’s off on second reading. But an agent doesn’t hesitate. The window where a mistake could be caught by hesitation closes completely.
Every piece of human phishing training assumes a person applying judgment. Agents apply logic to inputs. If the input is well-formed and appears to come from a legitimate source, the agent proceeds.
Sure, it’s quick, but that’s not always a good thing.
An agent that reads email can be targeted through email. Prompt injection (where an attacker embeds instructions inside content the agent will process) turns a routine message into a set of commands. The agent is just following instructions it was designed to follow but from a source it had no way to verify.
You can see how quickly that becomes a major cybersecurity issue. Jerry in HR at least had gut instincts, but the robots you put in charge—they don’t. Right or wrong, they often just do what they’re told.
Agent-to-agent communication is coming fast. As organizations deploy agents that transact with other organizations’ agents, the volume of machine-to-machine communication will dwarf human email. And none of it involves a person looking at a message and deciding whether it feels right.
Ultimately, generative AI made content unreliable as evidence. Agentic AI removes the human who was the last line of defense when content couldn’t be trusted.
The trust problem that nobody’s solving
Most security tooling asks a version of the same question: Does this look legitimate?
Secure email gateways analyze content and behavior for signs of malice. Anti-phishing tools score messages against known patterns. However, these are just inferences from appearance.
That model worked when producing convincing fakes was expensive. It works considerably less well now that generating a flawless, contextually accurate impersonation costs pennies.
Yes, secure email gateways are still a necessary layer, and Valimail partners with leading SEG providers for exactly that reason. Inbound content analysis catches things authentication can’t. But a defense built on evaluating appearance is in a losing arms race with a technology optimized to control appearance.
Now, you shouldn’t care if an email looks legitimate. That’s easy. You really want to know if a sender can prove they are who they say they are?
That’s where cryptographic verification comes in.
What cybersecurity holds up when AI writes everything
Cryptographic verification doesn’t care how good the content is.
DMARC (Domain-based Message Authentication, Reporting, and Conformance) at enforcement stops any message claiming to come from your domain that can’t prove it. Not because the message looks suspicious, but because the sender can’t produce a valid SPF or DKIM result aligned to the domain in the From address.
An LLM can write a perfect impersonation of your CEO, but it can’t forge a DKIM signature for a domain it doesn’t control. That’s a mathematical constraint (not a detection heuristic), and it doesn’t degrade as the models improve.
This matters more in an agentic environment.
- When a human reads an email, authentication is one signal among several, alongside judgment and context.
- When an agent processes an email, authentication may be the only signal available that an attacker can’t manufacture.
Machine-to-machine trust has to be established cryptographically because there’s no other mechanism left. The organizations that will handle agentic AI security well are the ones that establish verifiable sender identity before they deploy agents that act on unverified input.
What does this mean for your organization?
Fortunately, you can take steps today (right now) to start protecting your organization from AI-based threats. And it’s easier than you’d think:
- Get to DMARC enforcement. Monitoring isn’t protection. A policy of p=none gives you visibility while leaving your domain fully spoofable. Enforcement (p=reject) is what stops exact-domain impersonation, and it’s the foundation everything else sits on.
- Audit what your agents can act on. If you’re deploying AI agents that process email, know what actions they can take autonomously and what requires human approval. Financial transactions, credential changes, and data access should never be in the autonomous column.
- Treat prompt injection as an email security problem. If an agent reads your inbox, your inbox is now an input channel to a system that executes instructions. The same authentication that keeps impersonated email out protects the agent from acting on attacker-supplied instructions.
- Shift training from content to process. Teach employees to verify unusual requests through a separate channel, regardless of how legitimate the message looks. This still works and always will.
- Layer authentication with content analysis. DMARC handles identity. SEGs handle content. You need both.
Start protecting your business from AI
You can’t verify sender identity you can’t see. Most organizations are surprised by how many services send email as their domain, and the unknown ones are where the real risk sits.
Valimail Monitor is free and identifies every service sending on your behalf by name. Check your domain to see where your authentication stands right now.
Whatever AI does next, the domains that can prove who they are will be in a better position than the ones that can’t.
Frequently asked questions
How can generative AI be used in cybersecurity?
Defenders use generative AI for alert triage, log summarization, detection rule authoring, threat intelligence synthesis, and generating realistic phishing simulations for training. Attackers use the same capability to produce personalized phishing at scale, write malware variants, and generate deepfake audio and video for social engineering.
What is agentic AI in cybersecurity?
Agentic AI refers to systems that take autonomous action rather than just producing output. In security, that means agents which investigate alerts, query systems, and apply fixes without human input. The risk is that agents acting on email or external data can be manipulated through prompt injection, and they act at machine speed with no human judgment in the loop.
Can AI replace human security analysts?
No, not currently. Generative AI compresses investigation and triage time, which lets smaller teams cover more ground, but it produces confident errors and lacks the organizational context analysts use to judge whether something matters. The realistic near-term picture is augmentation with agents handling routine work under human oversight.
