TIPS #32: The Fear of False Positives

Issue: Many companies overfocus on reducing false positives and inadvertently create false negatives.

Every alert in cybersecurity, identity, and fraud monitoring is defined across two dimensions- True/False (correct/incorrect classification) and Positive/Negative (alert/no alert):

• True Positive (TP):A real breach, fraudulent transaction, or compromised identity is detected. An alert is generated.
• True Negative (TN):Legitimate, non-criminal activity is correctly identified. No alert is generated.
• False Positive (FP):Normal, legitimate behavior is flagged as malicious or fraudulent. An alert is generated.
• False Negative (FN):A breach, fraud event, or identity compromise occurs but is not detected. No alert is generated.

True positives and true negatives mean that a detection system correctly identifies an event or behavior as legitimate (true negative) or dangerous (true positive). The challenge for any organization is in managing and mitigating false positives and false negatives.

False positives

False positives are incredibly common across security, identity management, and financial crime detection contexts. For example, recent research on public cloud storage monitoring found that default S3 detection rules generated more than 80% false positives.

False positives are highly visible and disruptive and can create alert fatigue, wasted analysis cycles, and customer friction. In identity management, false positives block legitimate users and erode trust. In financial systems, they frustrate customers with declined transactions. In Security Operations Centers (SOCs), they often overwhelm analysts due to their sheer volume.

A high volume of false positives can also cause analysts to miss subtle but genuine indicators of real threats. There’s empirical evidence that false positives significantly decrease analyst performance in detection scenarios. A recent controlled intrusion detection experiment found that analysts working under high false alarm conditions showed 47% lower precision and 40% slower response times when analyzing alerts.

False negatives

False negatives, on the other hand, are silent and costly blind spots where adversaries thrive. They give attackers a window of time to silently compromise accounts and systems, facilitate fraud, and commit identity theft.

The tradeoff

Companies face an inherent tradeoff between false positives and false negatives: friction versus threat detection. Higher false positives and lower false negatives mean better detection but more friction; lower false positives and higher false negatives mean lower friction but more successful threats.

In a recent podcast interview with Qevlar AI founder and CEO Ahmed Achchak, I explained how these tradeoffs define modern security operations and why the industry’s obsession with reducing false positives has backfired. From a company’s perspective, false positives lead to a measurable expenditure of limited resources, particularly in the SOC, where SOC analysts must sort through many alerts to determine if they indicate a legitimate threat or not. Given the visibility of this cost, most companies end up suppressing noisy alerts to provide operational relief and avoid too much customer friction. However, this removes the early warning signals of breaches, fraud, and identity abuse. In other words, the fear of false positives creates false negatives.

Impact: False negatives allow attacks, breaches, and compromises to go undetected for long periods of time, leading to unmitigated risk and significant losses.

Overfocusing on reducing false positives allows attackers to operate undetected. In cybersecurity, this means more breaches that go unnoticed for longer periods of time. This trend plays out in the data: the 2024 IBM Cost of a Data Breach report found that it took companies an average of 204 days to identify that a breach had occurred.

In financial services and identity management contexts, false negatives mean more fraud. Attackers increasingly pivot toward identity compromises and financial fraud with tactics including AI-enabled synthetic identities, account takeover attacks, and payment redirection schemes. The impacts are significant. Global scam losses exceeded $1 trillion in 2024, according to the Global Anti-Scam Alliance and in the U.S. alone, the FTC’s Consumer Sentinel Network received 2.6 million reports of fraud in 2024 with $12.5 billion in fraud losses.

The following case studies demonstrate the impact of overfocusing on false positive reduction:

Increasing Business Email Compromises (2013-present)

Business Email Compromises (BECs) provide a clear insight into the impacts of overfocusing on false positives. In BEC scams, criminals target businesses or individuals and convince them to perform legitimate fund transfers or share sensitive data. In many cases, attackers compromise legitimate business or personal email accounts (via phishing or other methods) and use the email addresses to perpetrate the scam; or, attackers may use a similar-looking email address that appears to come from a trusted person. The Internet Crime Complaint Center (IC3) reports that there have been over 300,000 reports of BEC scams between 2013 and 2023, amounting to over $55 billion in losses to victims globally.

What makes BECs so effective is the operational balance between security sensitivity and business continuity. Email gateways must minimize disruption to legitimate business communications. Filtering messages too aggressively for wording that suggests urgency or payment requests would result in unacceptable false positives that delay or block legitimate transactions. For that reason, most email systems are tuned to focus on known technical indicators such as suspicious links, attachments, and domain reputation rather than the intent or tone of a message.

BEC attacks exploit this limitation by using plain text messages that appear normal and contain no malware or suspicious URLs. They depend on social engineering to persuade recipients to perform legitimate actions such as approving invoices or initiating wire transfers. Because these transactions are executed by authenticated users within trusted systems, they often evade both email filters and downstream financial controls.

Operation High Roller (2012)

Operation High Roller was a large-scale financial fraud campaign where attackers made automated, stealthy transfers from victims’ bank accounts. The campaign targeted high net worth individuals and businesses, starting in Europe before spreading globally.

At the time, the operation marked a clear evolution in online financial fraud tactics. The attackers moved beyond traditional man-in-the-browser methods to coordinated automation originating from purpose-built, cloud-hosted servers. These systems were used to take over accounts, initiate unauthorized transactions, and bypass multifactor authentication. The campaign ultimately targeted accounts at approximately 60 financial institutions, with attempted thefts totaling at least $78 million dollars.

Operation High Roller was effective because it exploited the design of institutional fraud detection systems. Financial institutions faced the same challenge then as they do today: maintaining the trust and continuity of legitimate high-value transactions while minimizing false positives. The attackers engineered their automation to take advantage of this constraint. Rather than attempting to drain entire accounts, which would have triggered a high-confidence alert, the system transferred a small and consistent percentage of available balances that remained below typical detection thresholds.

By tuning fraud detection systems to reduce the number of legitimate transactions flagged for review, institutions inadvertently created the conditions that allowed these automated transfers to occur without immediate detection. The operational bias toward avoiding false positives created a blind spot that the attackers systematically exploited.

Action: Focus on causation (not correlation), efficient detection, and finding balance across detection systems.

1) Implement strong detection based on causation, not just correlation

Today, correlation (not causation) generates most false positives. Often, false positives are prevalent because SIEM, SOAR, and XDR tools rely on third-party intelligence from past activities which may not have relevance to the company using the tools. For example, a real estate company using an XDR may find that it generates a large number of false positive alerts related to banking cyber threats because the solution is correlating OVH network IPs to time server lookups misclassified as botnet addresses by cyber threat intelligence.

2) Augment the SOC with AI

Alert fatigue is the norm in most SOCs, but it doesn’t have to be that way. Qevlar AI addresses the analyst fatigue and false positive problem at its source by using AI to automate investigation and alert triage, empowering human analysts to focus on complex, high-stakes threats that would otherwise be missed- while maintaining high detection accuracy.

3) Focus on holistic network detection and visibility

It’s critical to correlate network metadata and map alerts to definitive causes to identify and resolve compromises across the network. Lumu Technologies helps you streamline network incident management, analysis, and response to continuously identify breaches and compromises.

4) Strengthen identity proofing to reduce ambiguity

Today, attackers compromise credentials, use sophisticated AI deepfakes, and generate synthetic identities to bypass identity security systems, even spoofing biometric data. 1Kosmos provides identity proofing and passwordless authentication to reduce ambiguity and ensure strong user verification, reducing false positives in access control while minimizing false negatives that could allow fraudulent or unauthorized access.

5) Implement intelligent detection and verification in financial systems and payments

The false positive/false negative tradeoff has direct monetary consequences when it comes to payments. VERITUITY uses intelligent verification to ensure legitimate transactions proceed with low friction while stopping fraud and errors that legacy systems miss.

Fraud patterns and fraudster tactics are constantly changing- often faster than financial institutions can keep up with. Lynx Tech uses adaptive AI models that update daily to help your organization learn from, detect, and stay ahead of the latest financial crime trends, reducing customer friction without increasing false negatives.

6) Unify your detection and response

In the security context, it’s critical to unify detection and response capabilities including EDR, XDR, CDR, ITDR, NDR, and DDR to ensure aligned detection across your environment. SolCyber offers a managed security program that filters through the noise to reduce false positives while escalating genuine compromise signals, alleviating alert fatigue that leads to over-tuning and higher false negatives.

7) Analyze and prioritize vulnerabilities from an attacker’s perspective

Prioritizing vulnerabilities, excessive false positives, and overall detection efficacy across security, identity management, and financial systems can be a challenge. Bishop Fox helps you cut through the noise with offensive security services including penetration testing, simulating real-world attacks to bolster your security based on the threats you are most likely to face.

8) Additional practical tips

In addition to the above action items, I recommend taking the following general points into consideration across any detection context:

1. Treat anomalies as potential breach or fraud indicators. Do not suppress outbound traffic anomalies, failed logins, or unusual transactions without investigation.
2. Review tuning decisions regularly.Evaluate whether suppressed rules are silencing useful signals and reintroduce them if they provide compromise visibility.
3. Correlate across domains.Connect endpoint, identity, and payment telemetry so that weak signals in one area can be reinforced by context from another.
4. Measure detection effectiveness.Track how often true incidents are caught versus missed and use these metrics to calibrate thresholds and priorities.
5. Prioritize based on potential impact.Focus analyst and IT remediation resources on alerts tied to high-value assets, privileged accounts, and financial processes most likely to be exploited.