Margin of Safety #32: Email Security & Awareness Training

Can email security and security awareness training vendors stop phishing turbocharged with AI?

Phishing is not a secondary concern; it is a consistent, costly, and persistent failure point for enterprise security teams. Decades of investment in perimeter defenses have done little to stop the fundamental attack: compromising the human inbox. For example, the Verizon 2025 DBIR (here) shows “human element breaches from social actions” (e.g., phishing or pretexting/spear-fishing) at the core of 23% of breaches and having been a top 3 problem since 2019.

We believe the persistance of this problem is a reflection of tooling inadequacy. Enterprises rely heavily on static, signature-based filtering for inboxes and on annual, compliance-driven training for employees. This model was clearly strained even before the arrival of Gen AI – after all, 2019 significantly predates the arrival of LLMs – but GenAI will only exacerbate these problems. This will translate to market need for reinvented solutions that move beyond compliance and deliver tangible, quantitative improvements in security hardening.

The Gen AI problem

Historically, savvy users could sniff out the tells of a phishing email: poor grammar, awkward phrasing, or even just generic, somewhat irrelevant content and requests. These errors were the result of attackers’ manpower constraints, a necessary trade-off for executing a high-volume, low-effort campaign.

Generative AI (in particular, LLMs) has eliminated much of this manpower constraint and substantially eased the rest. An attacker no longer needs to be a native speaker or spend hours researching employees’ backgrounds. Instead, AI enables the rapid production of highly contextualized, large-scale campaigns that can be virtually indistinguishable from legitimate internal or vendor correspondence. This capability manifests in a few key ways.

First, automated, personalized spear-phishing is now possible at scale. Automated scripts, armed with public search APIs and LLMs, can now be used to collect an organization’s publicly available data (job titles, employee social profiles, location information, etc) and to craft messages that reference known projects or specific team functions. These highly-personalized emails can then be launched at mass scale to exploit employee assumptions that personalized communications involve an actual person, and turning previously capacity-constrained attacks into a widespread threat.

Second, the attacks achieve excellent impersonation. Most leaders have extensive public footprints in corporate communications, LinkedIn, and even personal social media. Ingesting this data enables creation of messages that closely mimic the writing style and tone of specific C-suite executives, legal counsel, or trusted vendors. By matching tone while removing grammatical and stylistic anomalies, the attacks bypass the signature-based filters and human instincts that traditional security was built on.

Third, the challenge extends beyond the text. AI facilitates new, difficult-to-detect threats like deepfake audio and video used for voice phishing (the driver of our conviction around GetReal Security!) and complex Business Email Compromise (BEC) fraud. An attacker can now clone a CEO’s voice from a single public recording to authorize an emergency wire transfer. Some video deepfakes can still be spotted by expert users, but we expect this to change in the near future. These attacks bypass text-centric email defenses entirely, forcing a re-evaluation of every communication channel.

The Implications for Legacy Vendors

We expect these updated attacks to further stress existing problems with the legacy security technologies and user training programs that enterprises today rely on.

The first systemic problem lies with legacy Email Security Gateways (SEGs). These systems enjoy a multi-billion-dollar TAM due to their status as a front-line defense against phishing and malicious emails, but they rely heavily on classic pattern recognition and static signatures, known blacklists, and established linguistic patterns to identify and quarantine threats. Many or all of these approaches will struggle when confronted with attacks that can generate a unique, high-quality message for every recipient in a massive campaign. The legacy systems are designed to match known bad inputs, not to identify novel, contextually legitimate-seeming communication flows.

The second major systemic weakness is the over-reliance on Security Awareness Training (SAT). For many organizations, this remains a prevalent, often regulatory-mandated, control. However, research findings, such as a recent joint study by UC San Diego and the University of Chicago (link), often question whether training drives a quantifiable reduction in employee phishing susceptibility – even when the training uses simulated phishing campaigns and targeted interventions based on employee behavior. The previously referenced Verizon report shows that training can drive greater portions of employees to report suspected phishing, but also that a small, core fraction of apparently untrainable employees appears to click on attacks no matter what you do to them (previous link, pages 48/49). Given this, we question how far training can be pushed in the best of cases, but we also observe that technologies serve the need of their primary purchase criteria; namely, compliance. When training is primarily purchased from compliance budgets, should we be surprised that it does not strongly move the security needle?

Reinventing Legacy Tools for the Future

We believe both SEGs and Employee Training are ready for reinvention.

For email, next-gen tooling should be able to move beyond pattern matching to contextual analysis on a per-email basis. Tooling will need to be capable of discovering system attacks, even when the common element of the attack is significantly reduced; for example, a common theme in the call-to-action versus a wholesale repeated email. More sophisticated tooling will be able to understand emails in the context of an individual organization. Should someone be receiving messages with an invoice for printer services?

Similarly, the focus of human risk management must shift away from mandatory annual compliance and awareness tracking toward demonstrable and measurable behavior change. This may be as much a question of psychological research as anything else, but moving the needle on the category will require addressing the persistent research findings that some employees are highly resistant to training, either by finding a training strategy that works on such personnel or by allowing the organization to effectively ringfence the security risk they pose – even when they are an otherwise senior member of the organization.

Driving True Phishing Resilience

The true measure of a next-generation security strategy needs to be efficacy rather than just compliance checkboxes. Otherwise, it’s a race to the bottom on cost and we shouldn’t expect the threat report findings to change. Assuming we escape that race, the success and longevity of new market security solutions will depend entirely on their capacity to deliver — and prove — quantitative security outcomes against an increasingly sophisticated and adaptive adversary (and, we hope, unlock CISO budgets beyond compliance in doing so). If you’re building or testing solutions in this space and believe you have the numbers to prove your results, we’d love to talk to you.

Reach out to us if you are building in the space of AI & security. We have some thoughts!
Kathryn Shih – kshih@forgepointcap.com
Jimmy Park – jpark@forgepointcap.com

This blog is also published on Margin of Safety, Jimmy and Kathryn’s Substack, as they research the practical sides of security + AI so you don’t have to.