AI deepfakes push security controls to catch up

By mid-2026, AI social engineering no longer looked like better-written phishing emails; it looked like operational tradecraft spanning text, voice, video, and live interaction. AI + a16z captured the inflection point in tooling, noting that with “Vibe phishing” an operator could prompt an LLM and watch it spin up SMS, email, and iMessage infrastructure—“And that’s stuff that you couldn’t do literally two years ago,” concluding: “it’s just very obvious that threat actors can leverage that same technology.”

The sophistication was visible in how attacks were executed: THOR Collective Dispatch listed “T1656 – Impersonation: AI-generated deepfakes of executives in real time video/voice calls” and “T1534 – Internal Spearphishing: Using deepfaked executive voices/videos to request actions from employees,” beginning with “T1593.001 – Search Open Websites/Domains: Social Media” to “Harvest[] voice samples from earnings calls, LinkedIn videos, and conference recordings,” then “T1566.002 – Phishing: Spearphishing Link.” Packt SecPro, citing Unit 42, said attackers “are combining traditional social-engineering (phishing, help-desk pretexts) with increasingly sophisticated tools,” “automate parts of their campaigns, use AI for real-time adaptation,” and “Beyond text, GenAI is being used to clone voices,” while distinguishing “High-touch compromise” from “At-scale deception”; CyberWire Daily showed the same shift in commercialization, with InboxPrime AI “first spotted in October 2025,” and its “shift to a one-time $1,000 purchase and growing user base underscore the industrialization of phishing.”

What is breaking is not just inbox hygiene but the assumption that social engineering is still artisanal. CyberWire Daily reported that 90% of attacks are customized to the recipient in some way, and generative AI now lets attackers automate reconnaissance, pull in real-world context, and craft recipient-specific lures at volume. ToxSec described the same shift as building persona graphs from LinkedIn, press releases, and news, then mimicking tone and internal language so messages, chats, and prompts feel native to the target’s workflow rather than obviously fraudulent.

That personalization now plugs directly into trust-heavy identity workflows where defenders are slowest and users are easiest to pressure. Unit 42 said high-touch attacks focus on tricking people and using normal tools rather than malware by bypassing MFA and exploiting IT support processes, while IBM Technology warned that the scam often happens through vishing and a phone number in email. That tactic appeared in 28% of 5,000 emails that bypassed secure email gateways, and the video uses the 2022 Uber breach while discussing MFA fatigue and malicious application consent. CrowdStrike’s report is cited that the average breakout time fell to 29 minutes on average in 2020, and the fastest time seen was 27 seconds.

By the time 2026 came into view, the evidence no longer looked like scattered proofs of concept. KnowBe4’s 2025 Phishing Threat Trends Report said its headline numbers included a “67% Surge in Abuse of Legitimate Platforms and 449% Spike in AI-Powered Vishing Attacks,” while The Verge reported, in equally operational terms, “Chinese Hackers Automate 90% of Cyberattacks Using Anthropic’s Claude AI, Breach Four Major Organizations,” tying AI assistance not just to experimentation but to repeatable workflows and successful real-world compromises across multiple victims.

Sumsub’s Identity Fraud Report 2025-2026 showed the same shift toward routine tradecraft: a “Sophistication Shift” in which multi-step attacks rose from 10% of identity fraud in 2024 to 28% in 2025, while “high-quality attacks increased by 180%.” Its headline warning, “Agentic AI Scams Poised to Surge in 2026,” was backed by signs that AI was already entering production use, including AI-assisted forgery rising from 0% to 2% in 2025 through tools such as ChatGPT, Grok, and Gemini, with social engineering accounting for 16% of top third-party fraud schemes and 75% of respondents expecting fraud to become increasingly AI-driven.

Defenders are moving away from awareness-only programs because AI-crafted lures now defeat the old spotting playbook. Packt SecPro warned that if you wait six months, you're going to be in a bad spot, pushing organizations toward layered controls such as out-of-band verification, clear contact rules, and training built around pausing and verifying unusual requests rather than merely recognizing suspicious wording. The limits of awareness alone are especially visible in education: Keeper found that only 14% of schools mandate security awareness training, while 26% of education leaders feel confident in their ability to recognize AI-enabled threats, and it noted that attackers use AI-generated lures.

That shift is becoming operational in identity and access controls. CISO Talk recommended hardening Microsoft 365 identity controls against session theft by accelerating phishing-resistant MFA/passkeys, tightening session policies, and ensuring compromised tokens can be rapidly revoked, while SecurityBrief New Zealand explained that FIDO2 and passkeys stop stolen credentials from being replayed on fake sites. The pressure is strongest around high-risk workflows and privileged users: one interview said controls must be more sophisticated than extra 15 minutes of training, noted that 62% of the people were manager levels or higher, and Regulation S-P now requires SEC-registered advisers to maintain written cybersecurity and incident response policies and notify affected clients within 30 days of a confirmed breach.

The clearest sign this is no longer a niche problem is that identity and social engineering now show up directly in mainstream breach work. Packt SecPro, citing Unit 42’s 2025 Global Incident Response Report, said social engineering was the root cause in 36% of all incident response engagements; within social-engineering-originated incidents, 66% targeted privileged accounts and 45% used internal impersonation, while some attackers escalated from initial compromise to domain administrator in under 40 minutes using only built-in tools and social pretexts, and data exposure followed 60% of such incidents.

Other reporting shows the same pattern at broader market scale: PR Newswire said KnowBe4’s Q3 2025 report found personalized phishing emails using company names drove 90% of user interactions, while N2K Networks said Sophos’s 2026 active adversary report found identity-related root causes have risen every year since 2022, the last year non-identity causes were more common. N2K also noted SpyCloud counted a 23% increase to more than 65 billion distinct identity records and Red Canary reported identity-based threats now make up more than half of confirmed threats after an 850% year-over-year detection jump.