Margin of Safety #64: Your AI Indemnity Has an Engineering Dependency

Lab-provided usage indemnities come with more technical requirements than you might assume.

Disclaimer: We are not lawyers, LLMs are not lawyers, and if you trust any of us for legal advice you are getting exactly what you paid for.

A few days ago, at Black Hat, we heard a new rumble: buyers and users starting to question the IP issues around LLMs. Especially if you think that some model providers are being opaque about their training data partly so they can use or distill content to which they lack rights, this is an interesting topic[1]. In the extreme form, these concerns can manifest as demand from cautious executive teams for models from US frontier labs, to operate under the IP indemnification those labs offer. The need for such an indemnity was occasionally described as must-have for any usage, overriding performance or cost efficiency. But as we’ll get into, this channel of demand pins a lot of expectation on some relatively complex requirements.

The initial topic of choosing a provider based on an indemnification caught our attention for two reasons. The first is the concentration risk sitting on the labs. Copyright law as applied to model training and output remains somewhat unsettled, and a judgment could land that triggers simultaneous liability across many customers. CFOs and insurance companies typically dislike such things. The second is that Kathryn has worked on ToS at hyperscalers and knows the standard response to correlated exposure is to reduce risk by bounding the payout conditions until it’s unlikely a single problem results in a large provider liability. So if the labs are offering an uncapped-looking indemnity in a domain with concentrated and legally unsettled tail risk, we should suspect that they’re using terms to reduce that concentration.

This brings us back to a recurring point: before you get too worried or too excited about what your AI(or cloud) provider is doing, read the terms of service. It will often tip their hand to what they’re thinking about, including when it comes to your data or their IP. We went down a Claude powered rabbit hole with the public terms[2] for OpenAI, Anthropic, Google Cloud, and xAI[3]. To our non-lawyer eyes, there’s enough complexity in these terms that anyone counting on the indemnity better have their legal and engineering functions working *very* closely together.

At a high level, there was one pleasant surprise: the indemnity seems to be uncapped, because it’s consistently excluded from language that otherwise limits payouts to a function of trailing customer spend.

But the rest of the surprises probably aren’t so pleasant. Providers have varying thresholds for excluding coverage when output is sufficiently modified or combined with non-provider services (but notably, they all have a line: if you use their model in conjunction with your own services and output tweaking, you will lose some or all coverage[4]). By our read, this means you should be cautious if you want coverage when the lab’s model is used in your own products, and you should recognize that switching providers may move you from one side of the coverage line to the other, even without changing a workload. The risk of modification is very real: many engineers make some changes, however marginal, to the output of their code models, to say nothing of changes like dependency version bumps that can happen automatically in build or release pipelines — Google and OpenAI require that output be fully unmodified! Worse, beta services (and in Google/Anthropic’s case, free or credit-based usage, a startup staple) are often excluded. And finally some providers introduce another mode of risk: OpenAI and Google both void coverage if you disregard or fail to use their citation and safety features. But both vend their citation content(when it’s available) as separate portions of API response[5], and developers may not appreciate the ToS implications of ignoring them. Combined, this is a lot of requirements for an engineering team if you want to maintain compliance.

On top of all those technical gotchas, there’s the question of converting the indemnity into a payout. Google seems particularly restrictive, and it’s not clear to us that a claim which goes to arbitration would ever trigger their terms. And both Google and Anthropic require that they approve any non-court-awarded resolution to make a payout. Especially if you’re at a large company, there’s some non-zero chance that you already have a commercial relationship and an arbitration clause with the party that would bring an IP claim.

After going through all this, the picture is at best mixed. It’s easy to imagine cases where the indemnity turns out to be highly valuable for an enterprise user, but it’s also very easy to imagine cases where a customer was expecting coverage which never materializes. If you’re accepting higher inference costs to get one of these indemnifications, reading the fine print — hopefully alongside a licensed attorney — is just the start. You also need to know the exact conditions under which coverage attaches, and to work with the engineers and users to keep usage inside them. That implies ongoing technical attention and guard railing, but without it your odds of continued compliance seem low.

If you’re building in this space, we’d like to hear from you.

Feel free to reach out to jpark@forgepointcap.com and kshih@forgepointcap.com.

This blog is also published on Margin of Safety, Jimmy and Kathryn’s Substack, as they research the practical sides of security + AI so you don’t have to.