The Threat Evolved. The Defenses Haven't.

Our previous coverage documented how AI chatbots were being manipulated into harming vulnerable users. Here's the next chapter: those same manipulation techniques have been turned inside out — and now they're being used to attack everyone.

The security community has a name for it: AI-powered social engineering. And according to researchers at IBM X-Force, Check Point Software, and cybersecurity journalists at The Verge, it has hit a dangerous inflection point in 2025.

What Changed, Specifically

Early AI phishing was embarrassingly bad. Generic. Clunky. Detectable by any reasonably alert person.

Stephanie Carruthers, IBM's Global Lead of Cyber Range and self-described "Chief People Hacker," put it plainly: back in 2022, she wasn't worried. "If there's one job that generative AI can't steal, it's con artist," she told IBM Think. Sophisticated scams required deep research, careful planning, and highly targeted messaging — things early models couldn't do.

That assessment no longer holds.

"We've reached the point where I am concerned," Carruthers said in a recent IBM feature. "With very few prompts, an AI model can write a phishing message meant just for me. That's terrifying."

The 2025 IBM X-Force Threat Intelligence Index confirms this isn't anecdotal. Threat actors are now running bigger, broader campaigns than at any prior point. Part of that is tactical — supply chain attacks hit many victims simultaneously. But a major driver is the tooling. Attackers are using generative AI like a junior employee: building fake websites, generating malicious code, and crafting phishing emails at scale.

The Three Weapons Now in Play

According to Check Point Software's current threat analysis, here's what AI has unlocked for bad actors:

Information scraping at machine speed. AI tools can vacuum a target's social media profiles, cross-reference public databases, and pull data from past breaches — building a detailed dossier on any individual in minutes. What used to take a skilled human analyst hours now takes seconds.

Spear phishing that actually sounds human. That dossier gets fed into a language model that generates a message indistinguishable from something a trusted contact would write. Carruthers notes that AI models are "making messages more succinct, more urgent" — specifically engineering the psychological pressure that causes people to click before they think.

Deepfakes for voice and video. Check Point flags this as an increasingly prominent attack vector. A CFO receiving a video call from someone who looks and sounds like the CEO asking for an emergency wire transfer is not a theoretical scenario anymore. It's happening.

The Jailbreak-to-Attack Pipeline

Breitbart and The Verge both documented the origin story here. Early chatbot jailbreaks — like the "DAN" (Do Anything Now) persona created by a 22-year-old college student — revealed something fundamental: AI systems respond to the same psychological manipulation tactics humans use on each other.

The DAN technique didn't require code. It required convincing the model it had a different identity with different rules. That's a con. A classic confidence trick applied to software.

The arms race that followed has been less about technical exploits and more about language, psychology, and interrogation technique. According to The Verge's reporting, the new class of AI security professionals "relies less on traditional technical skills and more on social intuition and conversational ability."

The people defending AI systems from attack are essentially trained psychologists and interrogators, not just programmers. The same psychological leverage used to manipulate AI from the inside is now being used to manipulate humans from the outside, with AI as the weapon.

What Mainstream Coverage Is Missing

Most tech media coverage frames this as an abstract future risk or an industry challenge for big companies. The reality is sharper on both counts.

This is a present threat to ordinary people right now. The grandmother getting a voice call from someone who sounds exactly like her grandson asking for bail money. The small business owner receiving an invoice from what appears to be a longtime vendor. The employee getting a Slack message from a convincing simulation of their boss.

Coverage has also largely ignored the accountability angle. The major AI labs — OpenAI, Google DeepMind, Anthropic, Meta — spent billions building systems whose conversational fluency is now being directly weaponized. They have published safety frameworks and responsible AI commitments. These frameworks were built to stop chatbots from saying bad things, not to stop bad actors from using chatbot technology to deceive humans.

Those are completely different problems. The industry is only beginning to reckon with the second one.

What You Can Actually Do

Check Point's practical guidance is unglamorous but real: tighten email security settings, train yourself and your staff on what targeted AI phishing looks like, and verify any unexpected request through a separate channel before acting on it. A weird email from your bank? Call the bank's published number. A video call asking for money? Hang up and call that person back directly.

The old tells — spelling errors, weird grammar, obviously foreign phrasing — have disappeared. The new tell is urgency. Any message engineered to make you act immediately before you can think is suspect. That's the signature. It always has been. AI just made it harder to spot everything else.

The technology got smarter. Human nature didn't change. Until the industry builds defenses as sophisticated as the attacks, you are your own best firewall.