open-climate.aiDocs

How wrong are spend-based GHG estimates? We measured it

We tested EXIOBASE, Open CEDA and USEEIO against 7,040 EPDs. Half the spend-based estimates were off by more than a factor of 2.

Robin Dierckx10 min read

Key finding. In our test, half of the spend-based estimates were off by more than a factor of 2, and one in ten by a factor of 6 or more. We checked three widely used spend libraries against 7,040 product footprints, almost all EPDs, of construction products, furniture and lighting.

What is a spend-based estimate? It turns money into emissions: what you paid, times an average emission factor per euro for the sector you bought from. EUR 10,000 of cement times the cement industry's kg CO2e per euro gives the cement's emissions. It needs no quantities, which is why so many scope 3 inventories start there. Its blind spot: the factor is an average over a whole industry, so it knows what you paid, not what you bought. Spend-based vs activity-based factors, explained.

The better route is often already in your system. Many procurement exports carry quantities next to the amounts. Match them on activity data instead: send your raw export, no cleaning needed.

The test: 7,040 products, three spend libraries#

A spend-based estimate is fast. But how far is it from the footprint of the product you actually bought? An EPD (Environmental Product Declaration) answers that for one product, so we used EPDs as the yardstick.

  1. We took 7,040 product footprints (6,873 EPDs and 167 generic product datasets) in 24 product classes: cement, concrete, steel, aluminium, insulation, plasterboard, glass, bricks, tiles, timber, wood panels, paint, mortar, asphalt, cables, luminaires, furniture and carpet. We gave each a price, in one of two ways:
    • Method 1: the official EU average factory-gate price of its product class, for all 7,040.
    • Method 2: the real price of that exact product, found by searching shops and price lists, for 54 products.
  2. We computed each product's emissions with a spend-based factor: price times the factor of the matching sector, in three widely used libraries (EXIOBASE, Open CEDA and USEEIO).
  3. We compared that estimate with the EPD for the same quantity, cradle-to-gate (modules A1 to A3) on both sides, and summarised the error in two numbers:
    • k50, the median error: half of the estimates fall within a factor k50 of the EPD.
    • k90, the 90th percentile: 9 estimates in 10 fall within a factor k90; the last 1 in 10 is further off.

A k of 2 means "no more than twice too high and no less than half". The factor works the same in both directions, so being 2 times too high and 2 times too low count as the same error.

The results: a factor of 2, and a long tail#

On average, a spend-based estimate has a k50 of 2.1 and a k90 of 6.0: half of your lines are off by more than a factor of 2, and 1 in 10 by a factor of 6 or more. Each library is tested with the price it was built for (see Method 1 and Method 2 below).

Median error factor (k50) and 90th percentile (k90) of spend-based estimates against EPDs: EXIOBASE 2.5 and 6.4, Open CEDA 2.3 and 7.2, USEEIO 1.6 and 4.5, average 2.1 and 6.0. Half the estimates 9 in 10 estimates ×1 ×2 ×5 ×10 EXIOBASE 7,040 products ×2.5 ×6.4 Open CEDA 7,040 products ×2.3 ×7.2 USEEIO 42 products ×1.6 ×4.5 Average of the three ×2.1 ×6.0 Error factor against the EPD, log scale
How far a spend estimate lands from the EPD. The solid bar reaches the median error factor (k50), the light band the 90th percentile (k90). EXIOBASE and Open CEDA: 7,040 products at the EU factory-gate price of their class. USEEIO: 42 products at their own shop price.
Library Price used Products k50 k90 Half of your lines are at least 1 line in 10 is more than
EXIOBASE Method 1: EU factory-gate price 7,040 2.5 6.4 146% too high or 59% too low 6.4 times too high, or 84% too low
Open CEDA Method 1: EU factory-gate price 7,040 2.3 7.2 125% too high or 56% too low 7.2 times too high, or 86% too low
USEEIO Method 2: real shop price 42 1.6 4.5 64% too high or 39% too low 4.5 times too high, or 78% too low
Average 2.1 6.0 112% too high or 53% too low 6.0 times too high, or 83% too low

Worse, the errors lean one way. With factory-gate prices, EXIOBASE comes out below the EPD for 71% of products and Open CEDA for 61%: on these materials, both libraries tend to underestimate. USEEIO with shop prices shows no lean (52% below, 48% above). A lean matters more than the spread: random errors partly cancel when you add up many lines, a lean does not.

Share of spend-based estimates below and above the EPD: EXIOBASE 71% below, Open CEDA 61% below, USEEIO 52% below. Below the EPD Above the EPD EXIOBASE factory-gate 71% 29% Open CEDA factory-gate 61% 39% USEEIO shop price 52% 48% 50%, no lean
The errors lean low. Share of estimates below and above the EPD, each library fed the price basis it was built for. Same samples as above: 7,040 products for EXIOBASE and Open CEDA, 42 for USEEIO.

Method 1: the EU average factory-gate price (EXIOBASE and Open CEDA)#

With one official average price per product class, EXIOBASE has a k50 of 2.5 and a k90 of 6.4 on 7,040 products, and Open CEDA 2.3 and 7.2.

How it works. Eurostat's PRODCOM survey reports, for every industrial product code, the value and the quantity of what EU factories sold. Value divided by quantity is the average factory-gate price. We used the EU27 figures for 2023 and pooled the codes of each class, weighted by volume. Every product in a class gets the same price:

Product class Average factory-gate price, EU27 2023
Cement EUR 0.12 per kg
Ready-mix concrete EUR 0.036 per kg
Steel rebar EUR 0.64 per kg
Paint and coatings EUR 2.04 per kg
Plasterboard EUR 2.68 per m2
Sawn timber EUR 252 per m3
Luminaires EUR 46 per piece
Furniture EUR 108 per piece

Why these two libraries. EXIOBASE and Open CEDA are built on producer prices, the price at the factory gate before transport and trade margins. A factory-gate price is the matching input. EXIOBASE factors are in 2019 euros, so we brought the 2023 price back to 2019 with Eurostat's producer price index (how to deflate a spend factor). Open CEDA factors are per 2023 US dollar, converted to euros.

An example. A CEM III/B cement EPD declares 0.353 kg CO2e per kg. One kg costs EUR 0.120 in 2023, or EUR 0.076 in 2019 euros. The EXIOBASE cement sector emits 1.81 kg CO2e per euro, so the estimate is 1.81 × 0.076 = 0.138 kg CO2e per kg. That is 2.6 times below the EPD.

What we found.

  • Heavy materials are underestimated. Cement, steel, flat glass and mortar carry more carbon per euro than the sector average, so a spend estimate comes out low. Flat glass is the extreme: the typical flat glass product comes out 12 times too low.
  • Some light or cheap products are overestimated. Asphalt, aggregates, bricks and luminaires come out high.
  • One price per class hides the price spread. A premium product and a basic one get the same price here, so Method 1 measures how much the carbon varies between products, not how much the price does. Method 2 tests that.

Method 2: the product's own shop price (USEEIO)#

With the price a buyer actually pays for that exact product, USEEIO has a k50 of 1.6 and a k90 of 4.5 on 42 products, with no lean either way.

How it works. For 385 EPD products, balanced across product classes, we searched the web for a public price of that manufacturer's product: DIY chains, builders' merchants, manufacturers' web shops and published price lists. We accepted only the same manufacturer and product line, never a competitor's. We kept 54 products with a usable public price, and USEEIO has a sector for 42 of them. Each price is converted to euros, VAT is removed, and the price is divided by the quantity it buys (a 25 kg bag, a 2.6 m2 board, one lamp).

Why this library. USEEIO is built on purchaser prices, the price a buyer pays including transport and trade margins. A shop price is the matching input. A spend line in your ledger is also a purchaser price.

Two examples.

  • An OSB board, close. An 18 mm SmartPly MAX board costs EUR 32.48 before VAT at an Irish builders' merchant, or EUR 606 per m3. The USEEIO wood panel sector emits 0.241 kg CO2e per euro. For one m3 the estimate is 146 kg CO2e, against 156 kg in the EPD: 7% too low.
  • A lamp, far off. A Steinel RS 16 ceiling light costs EUR 38.35 at a German DIY chain, or EUR 32.23 before VAT. The USEEIO lighting sector gives 0.132 × 32.23 = 4.3 kg CO2e. The EPD declares 40.3 kg, so the estimate is 9.5 times too low. Lighting is one of the most scattered classes: in Method 1 as well, one luminaire in ten is off by more than 16 times with EXIOBASE.

What we found.

  • Shop prices are far above factory prices. The median product sells at 3.6 times its class's factory-gate price. One product in ten sells at more than 8 times.
  • Expensive products carry only slightly more carbon. Within a class, price and carbon move together only weakly (rank correlation 0.29). A product's own price helps, but much less than you might expect.
  • The price basis matters more than the price. USEEIO fed the factory-gate class price instead of the shop price falls to a k50 of 3.6 and comes out too low for 98% of products.
Share of spend-based estimates above the EPD with the matched and the wrong price basis: EXIOBASE 29% then 80%, Open CEDA 39% then 70%, USEEIO 48% then 2%. Matched price basis Wrong price basis 0% 50% 100% EXIOBASE factory-gate 29% 80% Open CEDA factory-gate 39% 70% USEEIO shop price 48% 2% Share of estimates above the EPD
The wrong price basis flips the lean. Share of estimates above the EPD; an unbiased library sits near 50%. Matched: EXIOBASE and Open CEDA at the factory-gate price (7,040 products), USEEIO at the shop price (42). Wrong: EXIOBASE and Open CEDA at the shop price (the 54 shop-priced products), USEEIO at the factory-gate class price.

Watch out. Do not feed shop prices to a factory-price library. On the same 54 products with their shop prices, EXIOBASE comes out too high for 80% of them (k90 8.9), and Open CEDA for 70% (k90 19). Your invoice amounts are shop prices. With EXIOBASE or Open CEDA, remove trade and transport margins first, or expect a systematic overestimate. Price basis, explained.

What to do about it#

A spend-based line is an order of magnitude, not a measurement: usually within a factor of 2, and one line in ten off by 5 to 7 times or more.

  • Match the price basis to the library. Your invoices are purchaser prices. USEEIO takes them as they are. EXIOBASE and Open CEDA expect factory-gate prices, so remove trade and transport margins first.
  • Replace your largest lines with physical data. The mass, volume or count of what you bought, matched to a physical factor or to the supplier's EPD, does better, and our matching agent does it on your raw export (see below).

The limits of this test#

  • Scope. 24 classes of construction products, furniture, lighting and carpet. Services, food, energy and transport were not tested.
  • The EPD is treated as the truth. EPDs have their own uncertainty, and their scope and method differ between programmes.
  • Method 1 uses one price per class. It cannot tell a premium product from a basic one. This means real errors are probably higher: on the Method 2 products, giving EXIOBASE each product's own price, with the typical shop margin removed, raised its k90 from 4.0 to 4.6.
  • Method 2 is a small sample. 42 products for USEEIO, tilted toward products sold in shops, since most industrial products have no public price. 23 of the 54 prices come from search-result snippets. VAT rates and exchange rates are approximate, which moves a price by a few percent against errors of a factor of 2 or more.
  • Large classes weigh more. Giving every class the same weight gives EXIOBASE 2.5 and 7.0, Open CEDA 2.4 and 7.8: the same picture.

Use the quantities you already have#

A spend-based estimate is the fallback for when you know nothing but the amount. Most procurement exports know more: tonnes of cement, square metres of plasterboard, numbers of chairs, litres of diesel, in a column next to the amount or buried in the line description.

Our matching agent reads that export as it is:

  • No cleaning, no template. Send the raw file from your ERP or accounting tool, with its own column names, languages and abbreviations.
  • Activity data first. Each line is matched on what was bought, with its quantity and unit, to an activity-based factor. A spend factor is used only when a line carries nothing but an amount.
  • Every match cited. Each line comes back with its factor, library, year, unit and licence, and the reason for the choice.

Send us your raw procurement export and get every line back with a cited emission factor. No account needed.