Methodology

Every check is published. Every regex is open. Every score is reproducible.

scroll↓

The substance behind the surface.

Every score in this product, every comparable record, every recommendation is produced under five rules. None are secret; all are demonstrable; each can be verified against the source data on demand.

Section 01

Calibrated language

Output describes what the data shows. It does not assign verdicts. It does not aggregate intent. It does not direct the reader.

Forbidden patterns

Hover any line for a description of what the pattern catches.

/\b(fraud|fraudulent|corrupt(?:ion)?|crime|criminal|illegal(?:ly)?)\b/i
/\b(should have|ought to have|was supposed to)\b/i
/\b(proves?|proven|definitely|certainly|clearly shows?)\b/i
/\b(failed to|refused to|deliberately|intentionally|knowingly)\b/i
/\b(coverups?|cover[- ]ups?|schemes?|scams?)\b/i
/\b(stunning|shocking|egregious|alarming|massive amounts? of public money|astronomical)\b/i
/\b(allegedly|reportedly|sources say|many believe|it is widely known)\b/i
/\bbecause of\b[^.]{0,80}\b(grant|contract|funding|donation|lobbying)\b/i
/\b(caused|led to|resulted in)\b[^.]{0,80}\b(grant|contract|funding)\b/i
/\bin (?:exchange|return) for\b/i
/\b(this raises serious questions|warrants investigation|warrants further investigation|the public deserves answers|officials should explain)\b/i

Rejected → calibrated

  • ✗ The government failed to deliver the program.
    ✓ The dataset does not show recorded outcomes for this program.
  • ✗ Evidence of fraud in the contribution.
    ✓ Pattern consistent with sole-source amendment growth across the agreement chain.
  • ✗ This grant should have stated its outcomes.
    ✓ Comparable filings in this program typically state expected outcomes.
  • ✗ The donations were in exchange for grants.
    ✓ Lobbying registrations were filed 47 days before grant award dates.
  • ✗ This raises serious questions about oversight.
    ✓ The dataset shows 31 amendments to the underlying contribution agreement.

Try it yourself

Type any sentence below. PYTH-GOV runs the calibration sweep against it in real time, the same way every output is gated before it ships.

Type any sentence — the calibration sweep runs live as you type. No submit button, no API call. Try one of the seeded examples below to see how PYTH-GOV catches uncalibrated language (causal claims, hedging absent, directives, “clearly shows”, and so on).

PYTH-GOV idle — type a sentence(Status updates with every keystroke — try an example below.)
Seeded examples — click to load

Accepted phrasings

  • · The dataset shows federal contributions totalling $134M between fiscal 2017 and fiscal 2024.
  • · Pattern consistent with sole-source amendment growth across the agreement chain.
  • · Records indicate the contribution agreement was terminated on July 3, 2020.
  • · Comparable filings typically state the program stream and the recipient's expected outcomes.
  • · Public records do not contain a final disposition statement for this contribution.
Section 02

Citation discipline

Every prose claim ships with at least one source pointer. No quote may exceed 15 words. No source may be quoted directly more than once.

  • Tier 1 — Auditor General reports. Highest authority.
  • Tier 2 — Departmental Results Reports, Hansard, parliamentary committee evidence.
  • Tier 3 — Major-outlet news with editorial review (Globe, La Presse, CBC investigative).
  • Tier 4 — Other news. Requires multiple corroborating sources before any claim ships.
Section 03

The Proof token

Every output of an accountability AI must itself be accountable. The token records what evaluated the input, what model produced the analysis, what gates passed, and what audit was issued.

Section 04

Structural correspondence with the federal AIA

The audit token is structurally equivalent to a Treasury Board Algorithmic Impact Assessment, applied at every output rather than once at deployment.

TierAudit token (Glassbox)Federal AIA
Tier 1 (Input)Filters applied + landmines respected (F-3, A-13, A-10)Project Description + scope of automated decisions
Tier 2 (Context)Embedding model + synthesis prompt version + temperatureRisk Assessment — algorithm class, training data, error modes
Tier 3 (Output)Citation count, max-quote length, calibration verdictMitigation Measures — peer review, monitoring, recourse
Tier 4 (Audit)Operator agent, token hash, previousTokenHash chainApproval & Review — sign-off, periodic re-assessment
Section 05

Data landmines

The corpus carries documented defects. Every canonical query guards against them explicitly. The Proof token's Tier 1 records which guards applied.

  • F-1
    ref_number collisions across distinct recipients (~41K rows)
    Guard: Partition by (ref_number, COALESCE(bn, legal_name, _id::text))
  • F-3
    agreement_value is cumulative — each amendment row carries the running total, so a naive SUM triple-counts amendments
    Guard: WITH agreement_current AS (DISTINCT ON … ORDER BY amendment_number DESC); every growth ratio is latest ÷ original for the (ref_number, BN|legal_name) chain per F-1, never a sum across rows
  • A-13
    AB exact duplicates + 951 reversal pairs
    Guard: Dedupe on (ministry, business_unit_name, recipient, program, amount, payment_date); pair-collapse opposite-sign matches
  • A-10
    AB recipient-NULL roll-up rows (~$25B in FY24+25)
    Guard: Filter recipient IS NOT NULL; disclose the omitted aggregate
  • C-7
    CRA name history is mostly missing (1.4% of BNs)
    Guard: Treat cra.cra_identification.legal_name as current-state, not historical
Section 06

Evidence strength and thresholds (v2)

Methodology v2 moved pattern detection from request-time queries to precomputed match tables. In doing so several thresholds became explicit Glassbox choices rather than inherited constants. They are listed here so no reader mistakes them for upstream or Ministry definitions; the data lineage is on /trace and the operator description is in docs/METHODOLOGY-V2.md.

  • Evidence strength (0–1) is separate from severity.
    Every stored match carries an evidence-strength score: how much of the published record supports the match — row volume, recency, corroborating rows across sources. Severity says how far the match sits from its peers; strength says how well-documented it is. Labels: weak < 0.40, moderate < 0.70, strong ≥ 0.70. Formula reference: src/lib/patterns/strength.ts.
  • Thresholds are relative percentiles over rolling windows.
    v1 used fixed cutoffs chosen in April 2026. v2 detectors compute cutoffs as percentiles of the comparable population — same department, program or fiscal-year window — and recompute them at every refresh. A match means 'top decile of its peers in this window', not 'above a number that stopped being reviewed'.
  • The loop-score attention threshold of 12 is a Glassbox choice.
    The upstream scoring script reports the score distribution at ≥ 15 and ≥ 10 and defines no attention cut. The 12 / 15 / 18 bands behind the observation / attention / flag pills are Glassbox calibration choices, and are documented as such on /trace.
  • Zombie recipients use a rolling 36-month window, not a fixed date.
    The upstream zombie script keys on last agreement start before 2022-01-01. Glassbox measures silence relative to the refresh date (last agreement start < today − 36 months), so the set advances with each refresh instead of growing against a stale cutoff.
  • Amendment values are cumulative; growth is always latest ÷ original.
    agreement_value on a federal amendment row is the running total (F-3). Sole-source creep, amendment growth on /transparency/risk and every ratio on a match card divide the latest-amendment value by the amendment-0 value for the chain keyed by (ref_number, BN | legal name) per F-1.
  • Data quality is scored, not just listed.
    The /transparency/data-quality scorecard records, per documented landmine, how many rows and dollars the guard touches at refresh time and whether the property is active, mitigated by a canonical table, or resolved by the publisher. These are properties of the published data, not findings about any entity.
Section 07

The five agents

Each agent operates within bounded perception. The strata are not decorative — they are how the architecture is scoped.

  • PYTH-LEAD · Task agent (descends along provenance strands)
    Orchestrator, schedule, scope cuts
  • PYTH-DATA · S4 source-linked
    Canonical SQL, landmine guards, hybrid retrieval, embedding job
  • PYTH-SYN · S3 semantic
    Similarity scoring, awardee patterns, calibrated recommendation text
  • PYTH-FE · S2 procedural
    Every UI component, every page, every motion. Reads DESIGN-SYSTEM.md as authoritative.
  • PYTH-GOV · S3 semantic
    Calibrated-language gate, Proof token completeness, route-existence + visual smoke gates. Veto authority.