TIPS #43: Build, Ship, Run

Issue: AI software security vendors often market nearly identical promises, but their tools protect different stages of the software lifecycle. Many companies have coverage gaps they don’t know about, especially in production.

In the past eighteen months, nearly every application security, software supply chain, attack surface management, and penetration testing vendor has added AI to its platform. Their pitches are almost identical: faster detection, fewer false positives, and expert-level triage at machine speed. But the tools address fundamentally different parts of software security, which makes them hard for CISOs and business unit buyers to compare.

Demystifying AI software security categories

It’s first critical to delineate where AI security tools operate within the software development lifecycle (SDLC):

• In development (Build): AI-assisted static, dynamic, and interactive testing embedded in the SDLC to catch insecure code, hardcoded secrets, and risky dependencies before a build is cut. These tools enable companies to secure their own code during development.
• Pre-deployment (Ship): AI-driven package, container, and binary analysis that inspects what is about to ship, including open-source and vendor components the organization did not write. These tools help companies block, replace, or strip third-party code to prevent supply chain compromises.
• Post-deployment/operations (Run): AI-augmented attack surface management and continuous testing that evaluates software that is already running, including systems that companies can’t modify directly, to find vulnerabilities and assist in remediation.

Each phase has a different security problem and requires a different solution.

The Run gap

Shift left, which has been the industry’s focus for the past decade, has increased focus and spending on Build and Ship tooling. It’s for good reason: moving security earlier in the SDLC prevents more vulnerabilities from reaching production in the first place.

However, companies are increasingly experiencing material losses rooted in tech debt already in production, and shifting left doesn’t address code that is already deployed (Run). Only post-deployment tooling can help here. Since that tooling is frequently marketed with the same language as Build and Ship tooling, a gap in Run-level AI-enabled software security is easy to miss.

Breach data shows the visible edge of the operational tech debt problem. The 2026 Verizon Data Breach Investigations Report found that vulnerability exploitation is now the leading initial access vector and occurs in 31% of breaches, a 20% increase over the prior year, and that organizations fully remediated only 26% of Known Exploited Vulnerabilities (KEVs) with remediation taking a median of 43 days.

The conditions enabling vulnerability exploitation are widespread. CISA’s Cyber Hygiene Vulnerability Scanning program, which covers more than 10,000 U.S. critical infrastructure entities, reports that 26% of organizations exposed vulnerable network services using protocols threat actors routinely exploit, 51% ran unsupported software linked to more than half of all KEVs, and 91% relied on deprecated SSL/TLS protocols left unsecured for a median of 459 days.

The public sector faces the same issue. The Government Accountability Office reported in July 2025 that the U.S. federal government spends about 80% of its over $100 billion annual IT budget on operating and maintaining existing systems. Of the 11 legacy systems the GAO judged most critical, eight use outdated languages, four run unsupported hardware or software, and seven operate with known vulnerabilities, including one whose flaws cannot be remediated without full modernization.

Tech debt and vulnerabilities in production

Closing the Run gap is highly contextual because vulnerabilities rooted in production-level tech debt* aren’t uniform and require different types of security interventions:

• Systems with patchable vulnerabilities that are operationally straightforward can be fixed with routine security workflows. A patch exists, and implementing it requires scheduling a change window and a failover plan for a load balancer or a hypervisor cluster.
• Systems with patchable vulnerabilities that are operationally fragile require careful consideration before anything is patched because the fix may break a dependency, disrupt a business process, or touch a system that can’t be taken offline. Tech debt in identity federation, domain controllers, and core database platforms often falls in this category.
• Unpatchable systems can’t be directly fixed. This includes software, appliances, and platforms that are at end of life, are no longer supported by vendors, are from defunct vendors, or can’t be patched on the security team’s timeline. Identity infrastructure with years-old trust relationship configurations and un-inventoried shadow infrastructure also fall in this category. The only option to secure an unpatchable system is a compensating control.

*Read TIPS #29 and TIPS #30 for a deeper dive into tech debt (including long- and short-term tech debt), patch management, and compensating controls.

Impact: When software security tooling doesn’t match the stage, exposures persist, and operational tech debt makes Run-level exposures the hardest to remediate.

The following case studies demonstrate how security failures that occur at different points in the SDLC require tooling specifically aligned to Build, Ship, or Run, and how Run failures can persist even after vulnerabilities are known due to operational tech debt and patching complexities.

Hardcoded cloud credentials in shipped mobile apps (2024)

In October 2024, Symantec’s Threat Hunter team at Broadcom published an analysis of widely distributed Android and iOS apps whose compiled binaries contained unencrypted credentials for the publishers’ production cloud infrastructure. Impacted applications included Pic Stitch and Meru Cabs, both of which had more than 5 million downloads.

This was initially an in-development problem (a Build gap) where the publishing companies’ cloud infrastructure was put at risk of compromise. In each case, application developers introduced the keys during ordinary build work. The keys then shipped through normal release cycles and persisted across versions because nothing in the publishing companies’ Ship pipelines inspected the compiled binary the way an attacker would (a Ship gap). A Ship-stage binary analysis tool capable of detecting exposed credentials could have blocked these releases, and a well-calibrated Build tool could have removed the exposed credentials from the code in the first place.

Axios npm package compromise (March 2026)

On March 31, 2026, an attacker compromised the npm account of the lead maintainer of the popular Axios package (over 100 million weekly downloads) and published two malicious versions of the library. Each malicious version carried a hidden dependency that installed a cross-platform remote access trojan and was available for roughly three hours. Google Threat Intelligence attributed the attack to UNC1069, a financially motivated North Korean group, and advised companies that installed the versions to treat affected systems as fully compromised and rotate every credential.

This was a pre-deployment problem (a Ship gap). The malicious code arrived through a dependency impacted companies trusted and pulled automatically, so scanning their own source code with Build tooling would not have found it. A software composition analysis (SCA) tool- common Ship-level tooling- checks names and versions against vulnerability databases and would have missed it because a package compromised just hours earlier wouldn’t match a database entry yet. Catching this malicious code required targeted Ship controls that inspect the packages that enter the CI/CD pipeline deeply enough to detect tampering, not just known vulnerabilities.

F5 BIG-IP source code theft and patching delay (October 2025 to April 2026)

On October 15, 2025, technology provider F5 disclosed that a nation-state actor had maintained long-term access to its BIG-IP development environment and exfiltrated source code and details of undisclosed vulnerabilities. F5 shipped patches the same day and CISA issued Emergency Directive 26-01 ordered U.S. federal agencies to mitigate F5 vulnerabilities and patch systems within a week.

In April 2026, F5 reclassified CVE-2025-53521, one of the vulnerabilities identified in October, from a denial-of-service flaw to a 9.8-severity unauthenticated remote code execution flaw and confirmed active exploitation. CISA added it to the Known Exploited Vulnerabilities catalog and non-profit group Shadowserver found more than 14,000 BIG-IP Access Policy Manager instances still exposed at the time.

For impacted companies, this was an operations problem (a Run gap) with a patchable solution. BIG-IP is a perimeter device, the easiest kind of tech debt to see and fix, and F5 clustering meant patching only required a change window and a failover plan. However, the exposure persisted because the vulnerability was initially labeled denial-of-service and went to the back of the patch queue.

In this case, pre-release (Ship) testing of the next BIG-IP version wouldn’t address the live appliance on the perimeter and Build tooling does not apply since BIG-IP is third-party technology. To patch more effectively here, security teams needed Run tooling that coordinates patch order based on real risk, with attacker-informed validation of what exposed instances are reachable and exploitable rather than just a CVSS label. Compensating controls should also have been applied and enforced through the existing security stack while patching caught up.

SolarWinds campaign and follow-on ADFS compromise (2020-2021)

In December 2020, CISA Alert AA20-352A outlined a campaign, later attributed to Russia’s SVR and threat group APT29, that used compromised SolarWinds Orion software as an initial access vector into U.S. federal agencies and private networks. CISA Alert AA21-008A, published in January 2021, detailed a follow-on technique observed across victims. After escalating to administrative rights, threat actors enumerated the on-premises Active Directory Federation Services (ADFS) token-signing capability, forged authentication tokens for identities including global administrators, and used those tokens to enter Microsoft 365 and Azure without a password or an MFA challenge.

This was an operations problem (a Run gap) for impacted companies, spanning both patchable but operationally fragile systems and unpatchable systems. The underlying exposure is a design property, not a defect: by default, the ADFS token-signing key is protected by Windows Distributed Key Manager (DKM), a software mechanism rather than a hardware security module, so anyone who can read the DKM container in Active Directory can decrypt the key. Mandiant’s post-campaign analysis documented the threat actor doing exactly that, using ADFind to query the container.

Hardening ADFS requires careful planning because every federated application depends on it, and where it can’t be replaced, a compensating control is the only option. ADFS also sits inside the network, so no external Run-level scan would have found the exposure. Securing it required Run tooling that discovers internal identity infrastructure and continuously verifies its configuration.

Action: Match AI security investments to SDLC stages, implement tooling according to your needs at each stage, and set patching and compensating control priorities based on the remediation path.

1) Classify every AI security tool by SDLC stage

Before adding another AI-powered software security tool, map your current coverage against the Build, Ship, and Run stages. Most organizations will find dense and overlapping coverage in development (Build) and pre-deployment (Ship) testing and thin, reactive coverage in production (Run). Make budget decisions based on the gaps in your security posture. It’s also important to ask vendors to clarify which SDLC stage their products cover and which they don’t.

2) Identify insecure code and secrets before they leave development

Development teams need security testing that runs continuously as they write code, not just at release. Automated static, dynamic, and API testing should be integrated into the development workflow to catch hardcoded secrets and insecure data handling while you can still change the code.

NowSecure delivers mobile application security testing built into the SDLC. Its platform analyzes every build on real devices to see what attackers see, flagging hardcoded secrets, risky SDKs, and embedded AI components before release. NowSecure’s findings create tickets that can be sent to AI-enabled developer tools for remediation.

3) Reduce unused packages before shipping and verify what’s left for signs of tampering

A typical container image carries thousands of open-source packages, and a large share of that code never executes in production. Every unpatched CVE in those packages becomes production tech debt the day the image ships. Reducing that volume before release is critical.

RapidFort addresses tech debt and vulnerabilities at the package level before production, reducing the attack surface by as much as 90%. Its platform generates a software bill of materials for each container in the CI/CD pipeline, replaces vulnerable base images with curated near-zero-CVE versions, remediates open-source packages without code changes, and uses runtime intelligence to remove packages the application never uses. Less debt ships and Run-level tools have a smaller estate to defend.

Volume reduction addresses known vulnerabilities in packages that are what they claim to be but doesn’t address artifacts that have been tampered with. It’s also important to identify signs of tampering, embedded malware, or behavior that never appears in a dependency list, which is precisely what attackers optimize for.

ReversingLabs analyzes and deconstructs packages, containers, and release artifacts before they ship. Their Spectra technology breaks them down to the component level, evaluates what they actually do, and flags malware, tampering, and exposed secrets.

4) Discover, validate, and close Run-level exposures

For production systems that can be patched, the question is what to patch first. For those that can’t be patched, the question is which controls close the gap. Both start from knowing what an attacker can actually reach.

Finding every vulnerability is not the priority here. Instead, focus on identifying the specific discontinuity an attacker could use, confirming it is real, and closing it with controls already in the stack. When the exposure is already public, validation should come first. When the exposure is latent, discovery should come first.

Discern Security’s agentic Security Loop platform ingests data across your existing security and IT stack to discover and articulate Run-level trust relationships and configuration discontinuities. Discern maps each finding to the asset, control, and compliance obligation behind it. Once an exposure is confirmed, the Discern platform enforces a fix with security tools your organization already owns and produces framework-mapped evidence that the change happened.

Bishop Fox’s Cosmos platform combines continuous attack surface discovery with expert-led testing to confirm whether an exposure in your systems is exploitable with current attacker tradecraft. Their validation capabilities help you set patch order across exposed appliances, turning a configuration observation into a board-defensible risk.

5) Define tech debt-based vulnerabilities by the remediation path, not just severity

CVSS scores tell you how bad a flaw is, but not the remediation steps you need to take. Create an inventory that tags every asset and dependency by one of four actionable remediation paths:

• The asset can be fixed now with operational effort.
• The asset can be fixed only after careful planning because of dependencies or downtime.
• The asset can’t be fixed and requires a compensating control.
• The asset is still in development and requires either Build or Ship controls before it reaches production.

Featured in this edition: Bishop Fox, Discern Security, NowSecure, RAPIDFORT, and ReversingLabs.