Quantification Uncertainty and Discounting

Reflections on the biggest near-term risk to the maturation and flourishing of open-system pathways

Note: This is the second in a multi-part series about big looming risks and flaws in the way we are collectively pursuing CDR pathway diversity—revolving around “open-system” pathways, quantification uncertainty, and scale—that Dai Ellis and I are writing together and publishing simultaneously on our respective blogs. 

In our first post, we made the case for open-system pathways playing an especially important role in gigaton-scale carbon removal since they allow humans and our machines to take on a smaller overall share of the thermodynamic work involved in CDR. To enable that future we argued for an explicit and expansive portfolio mentality in the way we pursue different CDR pathways. We also highlighted the evidence that we are tolerating a rapid drift—without much scrutiny or public debate—toward a tilted playing field that favors closed-system pathways with their relative “tidiness.”  

The rest of this series revolves around quantification uncertainty because we see it as the biggest near-term risk to a level playing field that enables the best open-system pathways to flourish and the CDR industry to achieve gigaton scale. Why? Any pathway will have some inherent uncertainty in trying to precisely quantify the amount of carbon removed, yet the uncertainty is greater when we intervene in open systems. And if corporate and government purchasers are scared off by that uncertainty, they won’t buy—at all, or nearly as much—from open-system suppliers. Right now this appears to be a very real possibility. 

Other “sources of messiness” attached to open-system approaches—notably ecosystem risks, social fears, governance questions, and (for some pathways) durability uncertainties—also represent big hurdles. But we are narrowly focusing on quantification uncertainty here because it is so foundational to the CDR enterprise, and because it would be analytically misguided to lump together all the sources of messiness as if they were cut from the same cloth. We will need to take a different approach to mitigating and handling each in a principled way, and a rich treatment of all of them goes beyond the scope of this series. 

Today’s post lays the groundwork for the rest of the series, diving into the nuances of quantification uncertainty and the reasons why it need not put the brakes on progress for open-system pathways. In particular, we highlight how effective MRV and discounting can create extremely high-confidence tons even in the context of uncertainty. 

Background: Frontier and CarbonPlan work on quantification uncertainty

Last year Frontier and CarbonPlan published some important analysis of the quantification uncertainty associated with different CDR pathways. The output of that impressive work—including a mental model for uncertainty based on “Verification Confidence Levels” (VCLs) and a more detailed mapping of uncertainty sources for various CDR pathways—clearly underscores the difference in uncertainty profiles outlined in our previous post. Pathways with greater closed-system character tack toward the higher quantification confidence end of the VCL spectrum, while open-system approaches occupy low and middling spots on the spectrum. 

We want to highlight two of the most core principles Frontier suggested in its writing about VCLs, both of which we strongly endorse [note: these are all copy/paste excerpts]:

• Systematically incorporate uncertainty by applying an uncertainty discount to tons

  • We propose that buyers can systematically incorporate uncertainty into purchasing by asking suppliers to apply an uncertainty discount to tons: Delivered tons = Net tons * (1-Uncertainty discount)¹

• Drive down uncertainty for high potential pathways by acting as an early customer

  • As buyers we can help improve VCLs by purchasing from suppliers that will increase the VCL for their pathway or shrink the VCL range.

Frontier also started to tackle the much trickier question of how much uncertainty is tolerable in different circumstances. We start unpacking that in this post and will go deeper in our next post that takes a market-shaping lens on quantification uncertainty and open-system pathways.

Extracting real tons from uncertainty

Handling quantification uncertainty from a systems or market-shaping perspective will get very complex, but it’s important to start from a simple premise: you can still have very high-certainty tons created from hard-to-quantify CDR pathways. What does that mean? 

Let’s consider a simple deployment removing 100 net tons through a higher-uncertainty, open-system CDR pathway. Given the current state of the science and the existing measurements done on the project site, imagine we conclude that the deployment actually removed something between 40 and 100 tons. Based on the Frontier/CarbonPlan framework, a deployment with this level of uncertainty would fall in the VCL 2 range. 

So what if the supplier just sells 40 tons here? While this deployment involves substantial quantification uncertainty, what this probability distribution literally means is that we feel very confident that the number of actually-removed tons is at least 40—it’s the remaining 60 tons that we’re less sure about (even though the chances that the true number is north of 40 are extremely high). If our uncertainty bounds are scientifically rigorous, and take into account our best understanding of all the major biogeochemical fluxes and uncertainty sources that contribute to the pathway, then these first 40 tons should be just as high quality as direct air capture tons. 

More broadly, crediting down to the worst-case scenario, given existing science and measurements, should make open-system tons just as credible as any other type of removed ton. In some deployments for some currently extremely uncertain pathways, the worst-case scenario may well be that zero tons were removed, or even that the deployment was net positive. In such cases it’s time to do more science to confirm the effectiveness of the pathway before selling credits that can compensate for emissions. But this isn’t always (or even usually) the case. The entire field of statistics exists to help us work with on-site data and geophysical science to constrain the probabilities of different outcomes and to create believable confidence intervals and error bounds. Discounting to the lower bound produces rigorous, high-confidence tons from higher-uncertainty pathways.

Of course, doing this lower-bound discounting makes the deployment’s unit economics less competitive for the open-system CDR supplier in question—it now can sell fewer tons for the same deployment cost, and so its price per ton goes up commensurately. To illustrate, let’s assign some hypothetical numbers to the example taken above. Imagine that the supplier spent $16,000 on the 100-net-ton deployment that gets discounted down to 40 delivered tons. Where the supplier’s per-ton cost would have been $160/ton, yielding say a $200/ton would-be price including margin, the supplier’s price will instead have to jump to $500/ton to preserve the same margin—a 2.5X multiple.² 

This impact on pricing is all a question of degree and scale. As discussed further below, suppliers—and the broader CDR ecosystem of actors—can take steps to drive down uncertainty and discounting over time. For smaller uncertainties and discounts, or when we’re just considering pricing in the early phases of a supplier’s scale-up, the price impact of discounting may not weaken a supplier’s competitive positioning too much. And some open-system carbon removal pathways may prove to have a low cost structure as they scale—given the ways they piggyback on thermodynamic work done by natural systems, as discussed in our previous post—so even with substantial discounting applied they may often still be cost-competitive.

Practical discounting challenges

The first question this analysis begs is whether discounting and finding a reasonable lower bound is tractable, and whether we can construct a functioning system for doing this in a wide variety of contexts. For this to happen, the durable CDR market will need to move from today’s bespoke and informal discounting approaches toward scientifically robust discounting machinery. As we explore below and in later posts, this is no easy task, and faces a range of potential pitfalls that we must consider and overcome.

Constructing uncertainty ranges and lower bounds is challenging in part because we can’t just set a generally-applicable uncertainty range for a given pathway or even a given supplier. Quantification error bars are a function of several things, including our current knowledge of a pathway’s geophysical science, the specific measurement and modeling approaches that a supplier is using for any particular delivery, and the deployment strategy used in that delivery. Different suppliers using roughly the same pathway may choose to deploy differently or measure their removals with more or less granularity, and thus should be subject to different discount factors. Discounting also needs to be dynamic; as suppliers (and other ecosystem actors) develop improved tools and approaches that chip away at key sources of uncertainty, we will need to reflect these improvements in updated discount factors.

To get discounting right we will also need more consistency in who makes discounting determinations and how these determinations get made. We are far from any consensus answers. What core principles or rules should be centrally mandated, and by whom? How much can we standardize certain building blocks of discounting by having standards bodies weigh in on and quantify the maximum uncertainty level for particular carbon fluxes? How much should be left up to the particular supplier, buyer, and verifier who are party to any given transaction? Building alignment around these kinds of questions is critical to making the system work at scale.

Even so, a perfect and universal system is unlikely to materialize, which means we need to concern ourselves with the prospect of discount shopping. By discount shopping we mean a scenario in which suppliers seek to work with verifiers (or buyers) who agree to assign smaller discount factors and thereby improve the supplier’s economics and competitiveness. In an unregulated and supply-constrained CDR market, one can easily imagine a race to the bottom where buyers and verifiers chasing scarce supply are incentivized to be less rigorous in their discounting so that suppliers will opt to work with them. We’ll have more to say about this in the last post in this series.

Weighing risks

Some might question whether even discounting to the lower bound is conservative enough. For starters, does the risk of “unknown unknowns” make the lower bound unreliable as the basis for high-confidence tons? Indeed, our quantification of uncertainty is itself subject to a deeper uncertainty: we might compute our error bars and uncertainty ranges incorrectly because our current understanding of the earth-system science is incomplete. What if (e.g.) a new biogeochemical feedback is discovered that we hadn’t previously considered, such that even our lower-bound number turned out to be an overestimate? 

Another potential response from skeptics might be to draw distinctions between purchasing contexts in deciding whether lower-bound discounting is sufficient. For example, some might say we should be fine with doing this in the context of the voluntary market or maybe even government procurement; but that in compliance markets—where we legally require the purchase of removed tons to offset emitted tons—we should only allow supply from low-uncertainty pathways. 

These are understandable lines of questioning, and the “unknown unknowns” conundrum in particular is worth lots of unpacking and scientific discussion in the context of each open-system pathway. But if we indulge these anxieties too much we can easily miss the forest for the trees. 

In weighing the risks associated with quantification uncertainty, there are several things we need to bear in mind:

1. Discounting to the lower bound already puts us in a very conservative and risk-minimizing stance.

2. Since carbon removal is fundamentally a public good, the risk that matters is not primarily the risk of mis-quantifying an individual deployment but rather the aggregate, diversified portfolio-level quantification risk we are taking—and that diversification can do a lot to reduce portfolio risk. 

3. If we are too stringent in minimizing quantification risk, and thereby crowd out or hinder the maturation of uncertainty-laden open-system pathways with great scale potential, we raise the risk of failing to achieve gigaton scale. 

We will make things worse if our understandable quantification anxieties lead us into endless debates about tolerable magnitudes of uncertainty ranges and discount factors—or worse still, to pick arbitrary ceilings. Instead, we should be focusing our scrutiny on the scientific credibility of the removed-tons probability distribution for any deployment, starting with its specified lower bound. 

If we feel confident about that floor and a CDR supplier is able to deliver millions of tons with an 80% discount but for $95/ton post-discounting, we should want this supplier to rapidly deploy. As an analytical frame, lower-bound discounting is robust enough that it should extend even to compliance markets. As one of the largest sources of future purchase volume, compliance-market demand will be central to scaling CDR and we should lean into integrating open-system tons into these markets as much as possible. 

The risk-weighing principles above make it just as reasonable to ask whether discounting to the lower bound is tooconservative. We only have a path to gigaton scale if suppliers can thrive. If we discount to the lower bound across the board, then at the portfolio level—essentially by definition—we are systematically under-crediting total removals. This drives up effective prices, reducing demand and making the market less attractive to emerging suppliers. 

It may be appropriate in some contexts to allow discounting to something north of the lower bound, whether to the middle of the delivered-tons probability distribution or otherwise. It’s particularly easy to make this case for the earliest exploratory purchases from a given supplier or pathway, and there are colorable arguments to allow this for voluntary purchasing more broadly—perhaps even direct government procurement, in the context of a diversified portfolio—since these forms of buying do not involve compensatory claims underpinned by the force of law. 

There are several different ways you could imagine architecting market arrangements to selectively allow for crediting above the lower bound. At the same time, the more that discount rates differ across contexts, verifiers or buyers, the greater the risk that discount shopping causes dysfunction and erodes trust. And irrespective of the degree to which we sometimes discount to a level above the lower bound, our core thesis here is to recognize that discounting to the lower bound gives us a floor that we can align on and build upon to establish a high-confidence market for uncertainty-laden pathways.

The silver linings of quantification uncertainty

Quantification uncertainty is a significant risk to the carbon removal enterprise, but also has important silver linings. These silver linings revolve around the fact that uncertainty forces us out of a system of binary yes/no crediting and into a system of sliding-scale discounting where pricing is directly tethered to the uncertainty level. 

How does this help? The first silver lining lies in the incentives that uncertainty-driven discounting creates. Because discounting directly affects a supplier’s pricing and competitive positioning, each supplier under this system has a strong incentive to reduce the uncertainty of its own removals.  Whether by modifying their deployment strategy or taking additional measurements or advancing scientific understanding to bound sources of uncertainty, suppliers can earn themselves smaller discount factors and can sell more tons for the same amount of deployment work and cost. They can prioritize these efforts based on the particular carbon fluxes that represent the largest drivers of uncertainty. Meanwhile, they can avoid worrying too much about uncertainty sources that would cost them a lot to better quantify, but don’t actually contribute much to their aggregate uncertainty and discount factor.

These incentives aren’t just private incentives to suppliers, but also public incentives to “the system”—to anyone who’s invested in the health and growth of the CDR industry and driving down prices for a promising pathway over time. That includes market shapers like Frontier, philanthropy, academia and government. To inform these ecosystem actors’ decisions, we’d ideally have a system that not only quantifies current uncertainty and discounts accordingly, but that qualitatively describes how much potential there is for uncertainty reduction for any given pathway with its component carbon fluxes and sources of uncertainty. If we put ourselves in the shoes of a Frontier or a government constructing a portfolio of what to buy—or in the shoes of a philanthropy or government agency deciding where to invest my R&D dollars—we should care not just about current uncertainty, but about future potential for uncertainty reduction as well.

Non-binary discount-based crediting also helps at least moderately with verifier incentive problems. In a binary system, verifiers are very strongly incentivized to issue credits for a ton even where there are big reasons to be skeptical; they either issue the credits, or they don’t—and if they don’t, everyone makes less money. A sliding-scale discounting regime doesn’t eliminate this problem but at least shrinks it somewhat, because verifiers (and buyers) can simply increase or decrease the size of a discount factor rather than being forced into all-or-nothing determinations.³ We are still left with the problem of discount-shopping risk discussed above, but this will be a preferable problem to have. 

Finally, quite apart from incentives, sliding-scale discounting also facilitates market entry much better than a binary yes/no crediting regime. In a binary world—at least a rigorous one—a supplier has to reach a higher level of readiness and sophistication (optimized MRV arrangements etc.) to attain the certainty level associated with the yes/no threshold. In a discounting world, a supplier can enter the market and begin selling even with a higher uncertainty level and therefore higher discount factor, and from there “climb the ladder” to become more competitive. This will be especially important for suppliers pioneering new open-system pathways, and likely also often for suppliers in new geographies, especially ones outside the Global North who do not enjoy the same access to VC funding to optimize their MRV arrangements and deployment strategies in the early going.

* * * * * *

All these elements we’ve been discussing—uncertainty, discounting, scale, and a level playing field for open-system pathways—make for quite a complex cocktail. In our next post, we will zoom back out a bit and consider how we should be thinking about this cocktail from a market shaping perspective as we work to build a healthy and high-quality CDR market at gigaton scale. 

Wish John luck in his star performance tomorrow night

1. Though we agree with the broader principle here, as the "market machinery" for rigorous discounting gets built over the coming years, the goal should be to move quickly away from a system that has suppliers making their own decisions about discount factors.

2. Here we are consciously taking round and illustrative numbers, assuming for the purposes of the example that the $16,000 deployment cost includes all allocated overheads etc.

3. Verifiers in today's mainstream voluntary carbon market sometimes use discount factors and buffer pools to avoid simple binary determinations; but methodologically, the way this discounting and buffering is handled is often deeply flawed.