How We Score

Our Commitment to Trust

At TippingPoint.watch, we know that families and individuals are making deeply personal, sometimes life‑changing decisions based on what they see here. That’s why our scoring system is built on three pillars:

1. Objectivity — We don’t invent numbers. Our foundation is internationally recognized democracy and civil liberties indices (Freedom House, EIU Democracy Index, Polity IV, V‑Dem Liberal Democracy Index).
2. Transparency — Every metric has a rubric. Every score change has an evidence log with primary and secondary sources. Nothing is “just our opinion.”
3. Consistency — We apply the same scoring rules day after day, using algorithms designed to prevent bias or panic swings.

---

The Data We Use

• Global Indices:
  • Freedom House: Annual 0–100 score of political rights and civil liberties.
  • Economist Intelligence Unit: 0–10 Democracy Index, categorizing regimes from “Full Democracy” to “Authoritarian.”
  • Polity IV: −10 to +10 measure of regime characteristics.
  • V‑Dem LDI: Fine‑grained measure of liberal democracy.
• Daily Event Feed:
  • Congressional bills and executive orders.
  • Court rulings and agency directives.
  • NGO reports (ACLU, Brennan Center, Freedom House).
  • Reliable media outlets for corroboration.

---

Our Scoring Algorithm

• Category Scores (0–5 each): Each event is analyzed across four independent categories (Legal, Civic, Personal Rights, Operational) using detailed rubrics — from minimal concern (0-1) to critical threat (4-5).
• Precision Scoring: Scores use decimal precision (e.g., 2.37, 3.84) to show nuance. A score of 3.8 is meaningfully different from 3.2.
• Overall Risk Calculation: The aggregate risk score is calculated from category scores using fixed weights: Legal (30%), Personal Rights (30%), Civic (25%), Operational (15%). This weighting reflects the foundational importance of rule of law and individual freedoms to democratic health.
• Confidence Level: Every score includes a confidence rating (0.6–1.0) based on source quality. Primary sources (laws, orders, rulings) + multiple reputable secondary sources = higher confidence.
• Time Decay: Recent events count more; older events "cool down" unless reinforced by new actions.
• Aggregate Risk: Category scores roll up into one national "Current Risk" snapshot, while category-specific trends show which democratic pillars are under greatest pressure.

---

Per-Category Impact Analysis

Every article and event in our system is analyzed across four independent categories of democratic health. This approach provides transparency and precision that a single score cannot capture.

The Four Categories

• Legal/Institutional Impact (30% weight): Measures threats to rule of law, judicial independence, constitutional norms, and institutional integrity. Examples include court interference, executive overreach, or institutional purges.
• Personal Rights Impact (30% weight): Evaluates violations of individual freedoms, privacy rights, due process, bodily autonomy, and protection from arbitrary state action. This includes immigration enforcement, surveillance expansion, healthcare restrictions, and targeting of minority groups.
• Civic/Cultural Impact (25% weight): Assesses threats to civil society health, media freedom, academic freedom, and civic engagement. Examples include press restrictions, protest suppression, or academic censorship.
• Operational Impact (15% weight): Examines government transparency, electoral integrity, and accountability mechanisms. This covers voter suppression, transparency rollbacks, or accountability failures.

How It Works

When a new article or event enters our system:

1. AI Analysis: We use AI (OpenAI GPT-4) to analyze the article's specific impact on each of the four categories independently, scoring each on a 0-5 scale.
2. Independent Scoring: Each category is evaluated separately. An article about immigration enforcement might score high on Personal Rights (4.6) but moderate on Legal (2.7) and low on Operational (1.8).
3. Weighted Calculation: The overall risk score is calculated using our empirically-justified weights:
   Overall Risk = (Legal × 0.30) + (Civic × 0.25) + (Personal × 0.30) + (Operational × 0.15)
4. Transparency: All category scores and reasoning are stored and verifiable. You can see exactly why an article received its score.

Why This Matters

This per-category approach provides three critical advantages:

• Accuracy: Forces precise analysis of what each event actually affects, rather than vague overall impressions.
• Transparency: You can verify the math. If Legal is 4.7 and Personal is 3.8, you can calculate the overall score yourself.
• Actionability: Shows which aspects of democracy are under threat. "Legal risk spiked to 4.2 this week" is more useful than "overall risk increased."

Example: An article about mandatory biometric data collection might score: Legal 2.4 (constitutional questions), Civic 1.6 (minimal media impact), Personal 4.8 (major privacy violation), Operational 2.1 (transparency concerns). Overall: 2.85 — a moderate threat driven primarily by personal rights concerns.

---

Checks Against Bias

• Rubric‑based AI prompts — Our AI analysis uses detailed rubrics and concrete examples for each category, ensuring consistent application of scoring criteria.
• Independent category analysis — Each category is scored independently, preventing a single narrative from dominating all scores.
• Verifiable mathematics — The overall score is calculated from category scores using transparent, fixed weights. Anyone can verify the calculation.
• Two‑source rule — Articles must include both primary sources (laws, orders, rulings) and secondary sources (trusted reporting) to receive confidence scores above 0.7.
• Quality monitoring — We continuously monitor for score clustering, identical categories, and unrealistic distributions to ensure AI analysis quality.
• Change log — Every score and its category breakdown is timestamped and auditable in our evidence database.
• Dispute process — Readers can flag disagreements with sources; we publish those challenges transparently.

The result: a data‑driven, evidence‑based, category‑specific snapshot of democratic risk — not punditry, not panic, not black‑box AI.

---

Watchlist — Events & Red Flags

The following categories of events are monitored in our scoring system:

---

Risk-Based Action Triggers

These action triggers correspond to our 0-5 risk scoring system. Your dashboard shows your current aggregate risk level.