How TSK works

TSK turns a supplier's uploaded evidence into structured truth once, then renders it two ways. The extraction, confidence, factor, scope, verification, and chain-of-custody methods described below apply to both modes — they describe how a value becomes trustworthy, regardless of where it is surfaced.

Provenance discipline applies to both modes

Whether a number becomes a questionnaire answer (Mode 1) or an exported disclosure field (Mode 2), it traces back to a source document and page, or is explicitly marked as supplier-attested / self-declared. No value is invented by the AI — deterministic logic owns every number, and anything we cannot evidence is shown as needing review rather than silently asserted.

TSK processes uploaded supplier documents in two passes. The first pass uses regex patterns to extract structured values — energy readings, invoice totals, meter references — from plain text and native-PDF text layers. A pre-validation step rejects documents that contain no parseable text before any extraction begins, so garbage documents are discarded early. When an API key is configured (via OPENAI_API_KEY or TSK_LLM_PROVIDER), a second LLM extraction pass runs in parallel: the provider chain tries Ollama (local, free) first, then OpenAI (cloud), then falls back to regex if both fail. The best result from any provider is selected for the evidence record.

Honest limits

LLM extraction is OFF by default. Without an API key the system uses regex only. Regex covers the most common UK utility bill formats; documents with non-standard layouts or heavily scanned content may produce partial extractions, which are flagged for review rather than silently accepted.

Each extracted evidence item carries a source_status signal (unconfirmed → confirmed) that reflects how the value was obtained and whether a human reviewer has validated it. A user-confirmed flag is set when a supplier explicitly accepts a value in the review interface. Before any item reaches the supplier pack, a dimensional sanity gate checks that extracted quantities are unit-consistent — for example, that a reading expressed in kWh is not paired with a price denominated in cubic metres. Items that fail this gate are quarantined into 3_AUDIT/dimensional_quarantine_v1.json and marked for manual review rather than being silently dropped or promoted.

Honest limits

The dual-LLM verification signal — produced when two independent AI models are run on the same extraction — is not yet surfaced in supplier-facing confidence badges. The field dual_llm_verified exists in the evidence schema but is never populated in the current output. When this wiring ships it will add a second layer of machine-validation to confidence scoring; until then, badges reflect regex and single-LLM extraction quality plus the dimensional sanity gate only.

TSK ships every DEFRA conversion-factor vintage it needs and selects one per activity period for all emissions calculations — electricity, natural gas, liquid fuels, water, and waste. The factor set is pinned in contracts/emission_factors/uk_defra_<year>_v1.json, where each vintage is declared in its own "year" field and the calculation method as "method": "location-based". The calculate_emissions() function selects this factor set as its default in tools/calc/emissions_calculator_v1.py.

Every factor names the published row it came from. Each entry carries the DEFRA flat-file row ID, its category path, and the unit of measure the row is published in — so natural_gas_kwh, for example, records that it is the gross calorific value row, which is the basis UK gas bills are metered on (the net-CV row is ~11% higher and would be the wrong one to apply). The recorded SHA-256 of the source spreadsheet identifies the exact published file. A test re-checks every value against that ground truth on each CI run, so a factor cannot drift from its source unnoticed.

Global warming potentials: IPCC Fifth Assessment Report (AR5) 100-year values — CH₄ = 28, N₂O = 265 — as used by the DEFRA methodology and consistent with UNFCCC reporting.

Where DEFRA publishes no factor, we do not invent one. Waste sent to recycling is the clearest case: DEFRA lists the row and leaves it empty, because emissions from reprocessing belong to the organisation that reprocesses the material, not to the business that sorted it. We report the tonnage and say so, rather than converting it.

Honest limits

The factor library is DEFRA, UK only. Non-UK geographies fall back to the UK factor today — this is a known limitation. Calculations applied to non-UK consumption data using the UK factor will misrepresent location-based emissions and should be re-run when a regional factor set ships. DEFRA republishes its factors every June and, by its own stated policy, does not revise editions it has already published — so TSK ships every vintage it needs and selects one from each reading’s activity period. A figure for energy used in 2025 uses the DEFRA 2025 factors and will keep doing so after the 2026 set ships; a 2026 bill uses DEFRA 2026. This is what makes a historical calculation reproducible, and it matters: the 2026 release revised how UK electricity is calculated and the grid factor fell from 0.177 to 0.13096 kgCO₂e/kWh. Each figure states the vintage that produced it and the date it was calculated. For purchased-electricity Scope 2, TSK reports both GHG Protocol methods: a location-based figure (DEFRA grid-average factor) as the headline total, and a parallel market-based figure (UK residual fuel mix, reflecting a contractual instrument such as a renewable tariff where the evidence shows one). The two are shown side by side and never summed. Both are UK-only, and the market-based figure is CO₂-only — it excludes CH₄/N₂O and transmission & distribution.

TSK follows GHG Protocol scope boundaries for all emissions derivation. Scope 1 covers direct combustion from owned or controlled sources — natural gas, diesel, petrol, LPG, and fuel oil are all supported with live factor lookups. Scope 2 uses a location-based approach for purchased electricity: the grid emission factor is applied to metered kWh consumption, following the "method": "location-based" declaration in contracts/emission_factors/uk_defra_<year>_v1.json. For that same electricity, TSK also derives a parallel market-based Scope 2 figure — using the UK residual fuel mix, and reflecting a contractual instrument such as a renewable tariff where the evidence shows one — and reports the two side by side under GHG Protocol dual reporting, never summing them. For Scope 3, activity keys are mapped from supplier document data using KEYWORD_METRIC_MAP in tools/ingestion/mapping_candidates_v1.py, and five of those categories now derive emissions with DEFRA factors (see honest limits).

Honest limits

The mapping schema addresses 9 of the 15 GHG Protocol Scope 3 categories, and five of those now derive emissions with DEFRA factors: Cat 4 (upstream transport), Cat 5 (operational waste — which includes wastewater treatment, per GHG Protocol Scope 3 Technical Guidance chapter 5), Cat 6 (business travel), Cat 7 (employee commuting), and Cat 9 (downstream transport) — using documented estimation methods where an activity figure has to be derived (for example, flight count to passenger-km). Cat 1 (purchased goods & services) is calculated for purchased water supply only, which is the single Cat 1 input the document pipeline can currently evidence; the rest of Cat 1 is not calculated. The other mapped categories — Cat 2 (capital goods), Cat 3 (fuel & energy-related), and Cat 12 (end-of-life treatment) — are schema-ready but not yet calculated. The remaining 6 categories — Cat 8 (upstream leased assets), Cat 10 (processing of sold products), Cat 11 (use of sold products), Cat 13 (downstream leased assets), Cat 14 (franchises), and Cat 15 (investments) — are not addressed in the current schema. Market-based Scope 2 is UK-only and CO₂-only (it uses the UK residual fuel mix and excludes CH₄/N₂O and transmission & distribution). Neither Scope 2 figure includes well-to-tank fuel emissions or electricity transmission & distribution losses: DEFRA publishes both, but they belong in Scope 3 Category 3, which TSK does not calculate. Scope 1 coverage is limited to the five fuels listed above — refrigerants, fugitive emissions, and process-combustion sources are not yet in scope.

TSK's production profile runs two independent AI models on extractions where the dual-LLM adjudication module is engaged. Today, model disagreements are logged internally for operations review; the verification result is not yet surfaced in supplier-facing confidence badges. Surfacing the signal in badges is the next step in dual-LLM visibility. This module is implemented in tools/ingestion/dual_llm_adjudication_v1.py and is enabled in production via the FF_DUAL_LLM_ADJUDICATION feature flag.

Honest limits

The dual_llm_verified field exists in tools/evidence/evidence_summary_v1.py but is never populated in the current supplier output. When this wiring ships, Section 2 (Confidence scoring) will also be updated to reflect the added signal.

Every supplier pack produced by TSK is tamper-evident. When the pipeline finalises a run, it writes a manifest to 3_AUDIT/manifest.json inside the supplier_pack.zip archive. Each file in the pack receives a SHA256 hash entry in this manifest, so any post-delivery modification to an individual evidence file is detectable by recomputing and comparing hashes. The manifest is written by tools/pack/supplier_pack_zip_v1.py.

Honest limits

The manifest provides integrity (SHA256 hashes per file). Cryptographic signing — auditor-verifiable authenticity with a private key — is on the post-launch roadmap. Your completed pack benefits from terminal-state durability (H2 Item 2.5): once a run is finalised, it survives a server restart. Mid-pipeline resume is not promised — if a run is interrupted before reaching the 5-artifact threshold, it may be left in an incomplete state and will not be automatically re-completed. The orphan-recovery mechanism is a heuristic, not a guarantee — runs with fewer than 5 artifacts are not recovered.