Data quality ratings and the normalised tier

Numbers checked against the live data on

Every factor carries one data quality tier, data_quality_normalized: high, medium, low or very_low, best first. It is the only quality measure you can compare across libraries. A publisher's own rating is kept too, but it only means something on that publisher's scale.

Two kinds of rating#

  • The normalised tier (data_quality_normalized) is ours. It puts every library on one four-step scale, so you can sort or filter factors from different publishers together.
  • The native rating (data_quality_rating_native) is the publisher's number, on the publisher's scale. It never travels without its scheme (data_quality_scheme_native), because the scale, its direction and the number of criteria behind it all differ between publishers. A 2 is good on one scale and poor on another.

The native rating and its scheme are kept on the underlying value record, with a breakdown by criterion where the publisher gives one. They are not columns of the rows factor search and the API serve by default, precisely so that nobody compares them across libraries by accident.

How the tier is set#

An import can set a tier outright, and that tier wins. Where it does not, the first signal a value carries decides:

  1. A rating on a known scheme, read against that scheme's cut-offs.
  2. Else a stated uncertainty, as a percentage of the value: below 25 % is high, below 50 % medium, below 100 % low, and 100 % or more very_low.
  3. Else a reviewed default for the library, set once for publishers that rate nothing value by value.
  4. Else low, so an unrated factor never outranks a rated one.

Two adjustments apply on top. A value that counts CO2 only, and none of the other gases, is capped at medium. And a value that should not be used as it stands, such as a number whose denominator is close to zero, is marked problematic and not served at all.

The ADEME and AGRIBALYSE scale#

ADEME and AGRIBALYSE share one scheme, the data quality ratio (DQR) of the European Product Environmental Footprint method. It scores from 1 (best) to 5 (worst) on technological, geographical and temporal representativeness and on precision, and ADEME adds completeness and methodological consistency. ADEME's own advice is to treat a rating above 3 with caution.

DQR Tier
2 or less high
Above 2, up to 3 medium
Above 3, up to 4 low
Above 4 very_low

Three AGRIBALYSE foods, per kg at the consumer:

Food DQR Tier
Almond cake 1.34 high
Almond drink, not sweetened 2.35 medium
Amaranth, raw 4.1 very_low

For amaranth, the breakdown shows why: geographical, technological and temporal representativeness all score above 4, so the underlying data is a poor match for this crop's technology, origin and period.

Most ADEME factors carry an uncertainty percentage instead. Anthracite, with an uncertainty of 20 %, is high.

The uncertainty percentage#

uncertainty_pct is the field behind step 2 of how the tier is set: Anthracite's 20 %. Few publishers state one at all, and the Source of the tier column says which of the example libraries do. A leg open-climate.ai estimated carries one too, the error measured for the method behind it.

It is a bare percentage, and that is all it is. Nothing travels with it: no distribution, no interval convention, no confidence level, and no shared definition between two publishers. So it is a tier input and a rough sense of scale, not a statistic:

  • It is not a 95 % interval, a standard deviation or a bound. Reading 20 % as "plus or minus 20 % at 95 % confidence" invents a convention the publisher never stated.
  • Do not pool it, average it or compare it across libraries, for the same reason a native rating never travels without its scheme.
  • No uncertainty means no signal, not zero uncertainty. A rating is read before it, so a value with no rating either takes the library's reviewed default, and the low floor where the library has none: steps 3 and 4 above.

If you need an error bar for a sensitivity analysis or an ISO 14064 uncertainty assessment, the percentage is where to start on the libraries that state one, and the step from a percentage to an interval is your assumption. Record it as one.

Note. The percentage is not served beside the value. It is not a column of factors_flat, so it is in neither factor search, the export nor the API. It sits on the value record: join open_ef.emission_factor_value on the row's value_id, the way Trace a factor to its source does, and read the field on the emission_factor_value reference.

Why a tier is what it is#

data_quality_reason_normalized explains a tier that did not come from a rating or an uncertainty, and every zero value on a served top-level factor carries one. It is usually empty when the publisher's own rating or uncertainty set the tier. The reasons fall in two groups: those that explain a tier, and those that explain a zero.

Reasons that explain a tier#

Reason Meaning
library_prior The tier is the library's reviewed default: the publisher rates nothing value by value.
no_native_signal No rating, no uncertainty and no library default: the low floor.
context_dependent_basis The value is stated on a basis a plain unit does not carry, such as a kilogram of nitrogen or a square metre of net floor area. A quantity fits it only when it is measured on that same basis, so the factor is kept below any directly usable one.
generic_spend_category A spend category too broad to price sharply.
mixed_gwp_vintage A derived total whose legs were published on different IPCC GWP tables: a faithful sum of what the publisher printed, standing on no single vintage, so it sits one rung below the tier its legs would otherwise give it.
other_form_proxy The publisher prints, as this row's own value, its figure for another form of the product or a neighbouring one, such as a recycled resin printed with the figure of another recycled resin. The value is kept as printed, with the tier set no higher than low.

Those six explain a tier set from what the publisher printed. Four more mark a leg open-climate.ai estimated rather than transcribed, on electricity factors whose publisher prints generation only and on fuel factors whose publisher prints combustion only. Each caps the tier the leg can reach. Estimated electricity legs and estimated fuel legs say what each rung is and how far its measured error pulls the tier below the cap.

Reason Cap Meaning
estimated_from_regional_statistic medium Estimated from published figures for the region itself: for electricity, the publisher's own figures or a statistic such as the region's grid loss rate; for a fuel's supply chain (well-to-tank), another publisher's figures for the same country.
estimated_from_regional_model low Estimated from a model of the region rather than a measurement of it: for electricity, the region's own generation mix and per-fuel upstream factors; for a fuel's supply chain, a published model of that supply chain in the region.
estimated_from_broader_region very_low The region has no figures of its own for this leg, so those of a larger region that contains it are used: for electricity, the nearest one with a statistic; for a fuel's supply chain, the European Union's average.
estimated_from_proxy_region very_low A fuel's supply-chain leg estimated from another country's supply chain, such as the United Kingdom's, standing in for the region's own. Used by fuel legs only.

The total open-climate.ai sums on an estimated leg is graded by its parts, each weighted by its share of the total. Each part's error, read from its tier or from the error measured for its method, is scaled by its share; the scaled errors combine as a root sum of squares, read back onto the cut-offs of step 2. The result never rises above the tier of the largest part and never falls below that of the weakest, so a small estimated leg no longer pulls a well-known total down to its own tier: an AIB total whose medium generation value is four fifths of the sum stays medium over its low legs. A total with a withheld part, or a total of zero, takes its weakest part's tier. It carries the reason the parts at its own tier agree on, none when they disagree, and the weakest part's reason when the shares put it between the largest part's tier and the weakest's. is_estimated marks every one of them.

One more token marks an estimated leg that is kept but not served: implausible_estimated_loss_share. It is a grid loss share read from the publisher's own consumption and generation figures that comes out above 20 %, higher than any loss statistic open-climate.ai holds, because the gap between the two series is mostly electricity traded with neighbouring regions. The leg is rated problematic, so no query of the served rows returns it (the ceiling).

Reasons that explain a zero#

A zero you can read is usually a stated zero, not a gap: a zero that is an artefact, an empty cell in the source or a leg the publisher never printed, is meant to be marked problematic, and a problematic value is not served, so a query returns no row for it rather than a 0. Read that as the intent rather than a guarantee. The marking is a judgement each import makes value by value, not a filter applied to the served rows, so the reason token beside a 0 is the test, not the fact that you can see the row at all. Every served top-level zero carries one, and it says whose zero it is. Where the token describes how the number was set, a regulation's mandated default for instance, rather than what was measured, the 0 is one to leave alone even though it reached you.

Reason Meaning
green_electricity_zero A zero from a green electricity contract: correct for its buyer, not for anyone else.
outside_kyoto_basket_zero A refrigerant whose gases are controlled by the Montreal Protocol and not counted in the Kyoto basket, so the Kyoto total is 0; their warming effect is given beside it as the non-Kyoto value.
rounded_to_zero The publisher itself prints 0 at the precision it works to: a real figure too small to show, not an absence of one.
statutory_zero A zero that a mandatory reporting programme prescribes as the figure its reporters must use, such as heat recovered from waste incineration under Japan's reporting law. A real 0 for a reporter under that programme, not a measurement of the activity.
structural_zero A source that emits none of the gases the factor counts, at the boundary on the row: a carbon-free fuel burned, an inert material landfilled, a refrigerant with no warming potential, a vehicle running on a certified green tariff. A real 0, usable when that boundary and that contract are yours.

structural_zero is the token the API offers as its own example in the description of the column, and no row of the ten example libraries carries it: it is stamped elsewhere in factor search, which serves many more libraries. It names a genuine zero and nothing else. The zeros that are withheld as problematic, and counted by Rows not imported, carry one of two other tokens. implausible_printed_zero is a 0 that cannot be true of what it prices: a zero intensity in a computed spend matrix, which is what EXIOBASE's are, or a single-gas zero in a national-inventory default. biogenic_excluded_zero is a 0 that describes the accounting convention rather than the fuel: the biogenic CO2 released at the burn is left out, and the publisher prints no biogenic figure beside it. Both are zeros the publisher printed. An empty cell never becomes one: a blank leaves through the loader's skip path long before a reason is stamped. So "the source cell was empty" is not the test for a withheld zero, and a structural_zero you can read is a printed 0 at a boundary where it is true.

Watch out. A zero at one boundary is not a zero overall. Burning hydrogen emits nothing at the burn, and making it may have emitted a great deal; a green tariff is a market-based claim about supply, not a property of the vehicle. Read the boundary and the calculation approach beside the 0.

An empty value, a zero and a missing row are three different answers. An empty cell means the publisher gives no figure (reading a factor page, what the export holds); a 0 means the number is zero; and no row at all can mean the publisher never printed the value, that it was withheld, or that the year you asked for falls outside the factor's window (a value for a past year).

The full registry of reasons, including those that only ever sit on a value nobody is served, is in the code lists.

By library#

Default top-level factors, by tier:

Library Source of the tier high medium low very_low
ADEME DQR or uncertainty, estimated totals graded by their parts' shares 466 1,634 771 277
AGRIBALYSE DQR, library default for the farm products of the agriculture stream 662 1,775 413 29
AIB Library default, estimated totals graded by their parts' shares 34
DESNZ Library default, one rung down for a mixed vintage, estimated totals graded by their parts' shares 332 716 63
U.S. EPA Library default, estimated totals graded by their parts' shares 64 204 37 55
EXIOBASE Library default, very_low where the spend category is too broad to price 12,810 1,995
MITECO Library default, estimated totals graded by their parts' shares 124 534 72
ÖKOBAUDAT Library default 1,496 2,634
Open CEDA Library default 119,076
USEEIO Library default 392

An empty cell means no factor at that tier. AGRIBALYSE's farm products, the 428 factors of its agriculture stream, carry no DQR value by value and take the library's medium default. DESNZ's 63 low factors are the exceptions to its library default: 47 are derived totals whose legs were published on different IPCC GWP tables, so no single vintage stands behind the sum, and 16 are plug-in hybrid totals whose electricity share adds an estimated supply-chain leg for the grid. The low and very_low factors of the U.S. EPA and MITECO are totals open-climate.ai sums where the publisher prints generation only or combustion only: each adds an estimated leg capped by the method behind it (estimated electricity legs, estimated fuel legs), and the published factor under it keeps its library default. AIB's 34 totals add the same kind of legs and stay medium, the tier of the generation value that makes up most of each. The spend-based models sit at low or below by design: a sector average says little about the product you bought. Filter factor search with data_quality_normalized, for example high-quality factors.

Watch out. A tier says how well the number is known, not whether it fits your purchase. A high factor for the wrong country or boundary is still the wrong factor.