librefang-registry/hands/predictor/SKILL.md

---
name: predictor-hand-skill
version: "1.0.0"
description: "Expert knowledge for AI forecasting — superforecasting principles, signal taxonomy, confidence calibration, reasoning chains, and accuracy tracking"
runtime: prompt_only
---

# Forecasting Expert Knowledge

## Superforecasting Principles

Based on research by Philip Tetlock and the Good Judgment Project:

1. **Triage**: Focus on questions that are hard enough to be interesting but not so hard they're unknowable
2. **Break problems apart**: Decompose big questions into smaller, researchable sub-questions (Fermi estimation)
3. **Balance inside and outside views**: Use both specific evidence AND base rates from reference classes
4. **Update incrementally**: Adjust predictions in small steps as new evidence arrives (Bayesian updating)
5. **Look for clashing forces**: Identify factors pulling in opposite directions
6. **Distinguish signal from noise**: Weight signals by their reliability and relevance
7. **Calibrate**: Your 70% predictions should come true ~70% of the time
8. **Post-mortem**: Analyze why predictions went wrong, not just celebrate the right ones
9. **Avoid the narrative trap**: A compelling story is not the same as a likely outcome
10. **Collaborate**: Aggregate views from diverse perspectives

---

## Signal Taxonomy

### Signal Types
| Type | Description | Weight | Example |
|------|-----------|--------|---------|
| Leading indicator | Predicts future movement | High | Job postings surge → company expanding |
| Lagging indicator | Confirms past movement | Medium | Quarterly earnings → business health |
| Base rate | Historical frequency | High | "80% of startups fail within 5 years" |
| Expert opinion | Informed prediction | Medium | Analyst forecast, CEO statement |
| Data point | Factual measurement | High | Revenue figure, user count, benchmark |
| Anomaly | Deviation from pattern | High | Unusual trading volume, sudden hiring freeze |
| Structural change | Systemic shift | Very High | New regulation, technology breakthrough |
| Sentiment shift | Collective mood change | Medium | Media tone change, social media trend |

### Signal Strength Assessment
```
STRONG signal (high predictive value):
  - Multiple independent sources confirm
  - Quantitative data (not just opinions)
  - Leading indicator with historical track record
  - Structural change with clear causal mechanism

MODERATE signal (some predictive value):
  - Single authoritative source
  - Expert opinion from domain specialist
  - Historical pattern that may or may not repeat
  - Lagging indicator (confirms direction)

WEAK signal (limited predictive value):
  - Social media buzz without substance
  - Single anecdote or case study
  - Rumor or unconfirmed report
  - Opinion from non-specialist
```

---

## Confidence Calibration

### Probability Scale
```
95% — Almost certain (would bet 19:1)
90% — Very likely (would bet 9:1)
80% — Likely (would bet 4:1)
70% — Probable (would bet 7:3)
60% — Slightly more likely than not
50% — Toss-up (genuine uncertainty)
40% — Slightly less likely than not
30% — Unlikely (but plausible)
20% — Very unlikely (but possible)
10% — Extremely unlikely
5%  — Almost impossible (but not zero)
```

### Calibration Rules
1. NEVER use 0% or 100% — nothing is absolutely certain
2. If you haven't done research, default to the base rate (outside view)
3. Your first estimate should be the reference class base rate
4. Adjust from the base rate using specific evidence (inside view)
5. Typical adjustment: ±5-15% per strong signal, ±2-5% per moderate signal
6. If your gut says 80% but your analysis says 55%, trust the analysis

### Brier Score
The gold standard for measuring prediction accuracy:
```
Brier Score = (predicted_probability - actual_outcome)^2

actual_outcome = 1 if prediction came true, 0 if not

Perfect score: 0.0 (you're always right with perfect confidence)
Coin flip: 0.25 (saying 50% on everything)
Terrible: 1.0 (100% confident, always wrong)

Good forecaster: < 0.15
Average forecaster: 0.20-0.30
Bad forecaster: > 0.35
```

---

## Domain-Specific Source Guide

### Technology Predictions
| Source Type | Examples | Use For |
|-------------|---------|---------|
| Product roadmaps | GitHub issues, release notes, blog posts | Feature predictions |
| Adoption data | Stack Overflow surveys, NPM downloads, DB-Engines | Technology trends |
| Funding data | Crunchbase, PitchBook, TechCrunch | Startup success/failure |
| Patent filings | Google Patents, USPTO | Innovation direction |
| Job postings | LinkedIn, Indeed, Levels.fyi | Technology demand |
| Benchmark data | TechEmpower, MLPerf, Geekbench | Performance trends |

### Finance Predictions
| Source Type | Examples | Use For |
|-------------|---------|---------|
| Economic data | FRED, BLS, Census | Macro trends |
| Earnings | SEC filings, earnings calls | Company performance |
| Analyst reports | Bloomberg, Reuters, S&P | Market consensus |
| Central bank | Fed minutes, ECB statements | Interest rates, policy |
| Commodity data | EIA, OPEC reports | Energy/commodity prices |
| Sentiment | VIX, put/call ratio, AAII survey | Market mood |

### Geopolitics Predictions
| Source Type | Examples | Use For |
|-------------|---------|---------|
| Official sources | Government statements, UN reports | Policy direction |
| Think tanks | RAND, Brookings, Chatham House | Analysis |
| Election data | Polls, voter registration, 538 | Election outcomes |
| Trade data | WTO, customs data, trade balances | Trade policy |
| Military data | SIPRI, defense budgets, deployments | Conflict risk |
| Diplomatic signals | Ambassador recalls, sanctions, treaties | Relations |

### Climate Predictions
| Source Type | Examples | Use For |
|-------------|---------|---------|
| Scientific data | IPCC, NASA, NOAA | Climate trends |
| Energy data | IEA, EIA, IRENA | Energy transition |
| Policy data | COP agreements, national plans | Regulation |
| Corporate data | CDP disclosures, sustainability reports | Corporate action |
| Technology data | BloombergNEF, patent filings | Clean tech trends |
| Investment data | Green bond issuance, ESG flows | Capital allocation |

---

## Reasoning Chain Construction

### Template
```
PREDICTION: [Specific, falsifiable claim]

1. REFERENCE CLASS (Outside View)
   Base rate: [What % of similar events occur?]
   Reference examples: [3-5 historical analogues]

2. SPECIFIC EVIDENCE (Inside View)
   Signals FOR (+):
   a. [Signal] — strength: [strong/moderate/weak] — adjustment: +X%
   b. [Signal] — strength: [strong/moderate/weak] — adjustment: +X%

   Signals AGAINST (-):
   a. [Signal] — strength: [strong/moderate/weak] — adjustment: -X%
   b. [Signal] — strength: [strong/moderate/weak] — adjustment: -X%

3. SYNTHESIS
   Starting probability (base rate): X%
   Net adjustment: +/-Y%
   Final probability: Z%

4. KEY ASSUMPTIONS
   - [Assumption 1]: If wrong, probability shifts to [W%]
   - [Assumption 2]: If wrong, probability shifts to [V%]

5. RESOLUTION
   Date: [When can this be resolved?]
   Criteria: [Exactly how to determine if correct]
   Data source: [Where to check the outcome]
```

---

## Worked Examples

### Example 1: Corporate Acquisition

**Question**: "Will Acme Corp be acquired within 12 months?" (asked January 2025)

```
PREDICTION: Acme Corp (mid-cap SaaS, $2B market cap) will be acquired by January 2026

1. REFERENCE CLASS (Outside View)
   Base rate: ~5-7% of publicly traded mid-cap SaaS companies receive
   acquisition offers in any given 12-month period.
   Reference examples:
   - Splunk acquired by Cisco (2023) — similar scale, strategic buyer
   - Figma attempted acquisition by Adobe (2022) — regulatory block
   - Nuance acquired by Microsoft (2021) — vertical SaaS, strategic fit
   - Mandiant acquired by Google (2022) — security vertical
   - Cvent acquired by Blackstone (2021) — PE buyout at depressed valuation

   Starting probability: 6%

2. SPECIFIC EVIDENCE (Inside View)
   Signals FOR (+):
   a. Board hired Goldman Sachs as advisor (leaked filing)
      — strength: STRONG — adjustment: +20%
      (Companies that retain M&A advisors complete a transaction ~40% of the time)
   b. CEO sold 30% of personal holdings in Q4 (SEC filing)
      — strength: MODERATE — adjustment: +5%
   c. Two major competitors acquired in past 18 months (market consolidation)
      — strength: MODERATE — adjustment: +8%
   d. Revenue growth decelerated from 35% to 18% YoY (earnings report)
      — strength: MODERATE — adjustment: +5%
      (Slower-growth companies more likely to accept acquisition offers)

   Signals AGAINST (-):
   a. Founder still holds 25% voting control and has said "we're building for
      the long term" (recent interview)
      — strength: STRONG — adjustment: -12%
   b. Stock price at all-time high — acquirer must pay steep premium
      — strength: MODERATE — adjustment: -5%
   c. Current antitrust environment — FTC blocking more deals
      — strength: WEAK — adjustment: -3%

3. SYNTHESIS
   Starting probability (base rate):  6%
   Signals for:  +20% +5% +8% +5% =  +38%
   Signals against: -12% -5% -3%   =  -20%
   Net adjustment:                     +18%
   Raw probability:                    24%

   Sanity check: ~1 in 4 feels right given the strong M&A advisor signal
   balanced against founder control.
   Final probability: 25%

4. KEY ASSUMPTIONS
   - Goldman engagement is for M&A (not debt restructuring):
     If wrong, probability drops to 8%
   - Founder is willing to sell at the right price:
     If wrong (founder vetoes any deal), probability drops to 3%
   - Regulatory environment doesn't tighten further:
     If wrong, probability drops to 18%

5. RESOLUTION
   Date: January 31, 2026
   Criteria: Definitive merger agreement announced (not just rumors)
   Data source: SEC EDGAR (8-K filing), Bloomberg terminal
```

### Example 2: Technology Adoption

**Question**: "Will WebAssembly (Wasm) reach mainstream server-side adoption by 2027?"

```
PREDICTION: >20% of new cloud-deployed services will use Wasm runtimes by
end of 2027

1. REFERENCE CLASS (Outside View)
   Technology adoption lifecycle (Rogers curve):
   - Innovators (2.5%) → Early Adopters (13.5%) → Early Majority (34%)
   - Crossing from Early Adopters to Early Majority typically takes 3-5 years
     after first production deployments
   - First serious server-side Wasm deployments: ~2022 (Fermyon, Cosmonic)
   - Current status (2025): Late Early Adopter stage

   Historical analogues for infrastructure tech adoption:
   - Containers (Docker 2013 → mainstream 2017-2018): ~4-5 years
   - Kubernetes (2014 → mainstream 2018-2019): ~4-5 years
   - Serverless (Lambda 2014 → mainstream 2018-2020): ~4-6 years

   Base rate for "infrastructure tech reaching 20% adoption within 5 years
   of first production use": ~30%

   Starting probability: 30%

2. SPECIFIC EVIDENCE (Inside View)
   Signals FOR (+):
   a. WASI standard maturing — WASI Preview 2 shipped, component model
      stabilizing (W3C working group)
      — strength: STRONG — adjustment: +8%
   b. Major cloud providers offering Wasm runtimes (Fastly, Cloudflare Workers,
      Azure, AWS exploring)
      — strength: STRONG — adjustment: +10%
   c. Docker adding Wasm support natively (announced 2022, shipping)
      — strength: MODERATE — adjustment: +5%

   Signals AGAINST (-):
   a. Ecosystem still fragmented — multiple competing runtimes, toolchain gaps
      — strength: STRONG — adjustment: -10%
   b. Containers already "good enough" for most workloads — weak forcing
      function to switch
      — strength: STRONG — adjustment: -8%
   c. Wasm language support uneven — great for Rust/C++, mediocre for Python/JS
      — strength: MODERATE — adjustment: -5%

   Leading indicators to track:
   - CNCF survey: % of respondents evaluating/using Wasm
   - Job postings mentioning Wasm (Indeed/LinkedIn trend)
   - GitHub stars and contributors for top Wasm runtimes (wasmtime, wasmer)
   - WASI spec milestone dates vs planned dates

3. SYNTHESIS
   Starting probability (base rate):  30%
   Signals for:  +8% +10% +5%      =  +23%
   Signals against: -10% -8% -5%   =  -23%
   Net adjustment:                      0%
   Final probability: 30%

   Interpretation: The positive and negative signals roughly cancel out.
   The base rate from analogous infrastructure technologies holds.
   This is genuinely uncertain — the "chasm" crossing is the key risk.

4. KEY ASSUMPTIONS
   - "Mainstream" defined as >20% of NEW deployments (not total installed base)
   - WASI component model reaches 1.0 stable by mid-2026
     If delayed beyond 2026: probability drops to 15%
   - No competing paradigm emerges (e.g., eBPF expanding scope):
     If strong competitor: probability drops to 20%

5. RESOLUTION
   Date: December 31, 2027
   Criteria: CNCF annual survey shows >20% respondents using Wasm in production
   Data source: CNCF Annual Survey, Datadog Container Report
```

### Example 3: Geopolitical Forecast

**Question**: "Will US-China trade tensions escalate significantly in 2025?"
(Defined as: new tariffs >25% on >$100B of goods, or export controls expanded
to 3+ new technology categories)

```
PREDICTION: Significant escalation of US-China trade tensions in 2025

1. REFERENCE CLASS (Outside View)
   Historical trade conflict escalation pattern:
   - US-China trade relations since 2018: escalation occurred in 4 of 7 years
   - In election year +1 (new/returning administration): escalation rate ~60%
   - Trade wars historically escalate in steps, with retaliation cycles

   Starting probability: 55%

2. SPECIFIC EVIDENCE (Inside View)
   Signals FOR (+):
   a. Administration rhetoric on China hawkish across both parties
      — strength: STRONG — adjustment: +10%
   b. Semiconductor export controls already expanding (ASML, Tokyo Electron)
      — strength: STRONG — adjustment: +8%
   c. China retaliating with rare earth export restrictions
      — strength: MODERATE — adjustment: +5%

   Signals AGAINST (-):
   a. Business lobbying against further tariffs (Chamber of Commerce, farm lobby)
      — strength: MODERATE — adjustment: -5%
   b. Inflation concerns create political cost for tariffs
      — strength: MODERATE — adjustment: -5%
   c. Diplomatic channels active (recent bilateral meetings)
      — strength: WEAK — adjustment: -3%

   Scenario mapping:
   ┌─────────────────────────┬─────────────┬────────────────────┐
   │ Scenario                │ Probability │ Key trigger        │
   ├─────────────────────────┼─────────────┼────────────────────┤
   │ Major escalation        │ 25%         │ Taiwan crisis or   │
   │ (new tariffs + controls │             │ tech IP theft case │
   │  + retaliatory cycle)   │             │                    │
   ├─────────────────────────┼─────────────┼────────────────────┤
   │ Moderate escalation     │ 40%         │ Incremental tariff │
   │ (meets our threshold)   │             │ increases + 1-2    │
   │                         │             │ new export controls│
   ├─────────────────────────┼─────────────┼────────────────────┤
   │ Status quo / minor      │ 30%         │ Diplomatic deals,  │
   │ changes                 │             │ election distraction│
   ├─────────────────────────┼─────────────┼────────────────────┤
   │ De-escalation           │ 5%          │ Grand bargain      │
   │ (reduced tariffs)       │             │ (historically rare)│
   └─────────────────────────┴─────────────┴────────────────────┘

   P(meets our escalation threshold) = 25% + 40% = 65%

3. SYNTHESIS
   Starting probability (base rate):  55%
   Signals for:  +10% +8% +5%      =  +23%
   Signals against: -5% -5% -3%    =  -13%
   Net adjustment:                     +10%
   Raw probability:                    65%

   Cross-check with scenario mapping: 65% — consistent.
   Final probability: 65%

4. KEY ASSUMPTIONS
   - No major geopolitical crisis (Taiwan strait) that causes extreme
     escalation or extreme restraint: If crisis occurs, split to
     80% (escalation) or 20% (restraint/avoidance)
   - US economy remains stable: If recession hits, probability drops
     to 45% (political cost of tariffs rises)
   - China does not make major trade concessions preemptively:
     If it does, probability drops to 30%

5. RESOLUTION
   Date: December 31, 2025
   Criteria: Cumulative new tariffs >25% on >$100B goods OR export controls
   expanded to 3+ new technology categories (per USTR/BIS announcements)
   Data source: USTR tariff schedule, BIS Entity List updates, Congressional
   Research Service reports
```

---

## Fermi Estimation Techniques

Fermi estimation is the art of making reasonable order-of-magnitude guesses
by breaking unknowable questions into smaller, estimable pieces.

### Step-by-Step Process

```
1. DEFINE the quantity you want to estimate
   → Be specific about units, scope, and timeframe

2. DECOMPOSE into factors you can estimate independently
   → Prefer multiplication chains: A × B × C
   → Each factor should be something you can reason about

3. ESTIMATE each factor
   → Use round numbers (powers of 10 when possible)
   → State your confidence range for each factor

4. MULTIPLY and sanity-check
   → Does the result pass the "smell test"?
   → Cross-check with any known anchors

5. STATE your uncertainty
   → Fermi estimates are typically accurate within 1 order of magnitude
   → Give a range: [estimate / 3, estimate × 3] is a reasonable default
```

### Common Reference Anchors

Keep these memorized for quick estimation:

```
POPULATION
  World:        ~8 billion
  US:           ~340 million
  EU:           ~450 million
  China:        ~1.4 billion
  India:        ~1.4 billion

ECONOMICS
  World GDP:           ~$100 trillion
  US GDP:              ~$28 trillion
  US median household: ~$75,000/year
  US federal budget:   ~$6.5 trillion
  S&P 500 total cap:   ~$45 trillion

TIME
  Seconds in a day:    ~86,400 (~10^5)
  Seconds in a year:   ~31.5 million (~3 × 10^7)
  Working hours/year:  ~2,000

TECHNOLOGY
  Global internet users:    ~5.5 billion
  Global smartphone users:  ~4.5 billion
  AWS annual revenue:       ~$90 billion
  Global IT spending:       ~$5 trillion
  GitHub developers:        ~100 million

INDUSTRY SIZES (annual, global)
  Cloud computing:     ~$600 billion
  Semiconductor:       ~$600 billion
  Pharmaceutical:      ~$1.5 trillion
  Automotive:          ~$3 trillion
  Agriculture:         ~$3 trillion
  E-commerce:          ~$6 trillion
```

### Worked Fermi Examples

**Example A: Estimating the TAM for an AI code review tool**

```
Question: What is the annual TAM for an AI-powered code review SaaS?

Decomposition:
  TAM = (Number of professional developers)
      × (% who do code reviews regularly)
      × (willingness to pay for tooling)
      × (average annual price)

Estimates:
  Professional developers worldwide: ~30 million
  (GitHub has 100M accounts, but ~30% are professional, and
   not all professionals use GitHub)

  % who do code reviews: ~60%
  (Standard in companies > 50 engineers, less common in small shops)

  Target market (teams that would buy SaaS): ~40%
  (Enterprise and mid-market; small teams use free tools)

  Annual price per seat: ~$300/year
  (Comparable: GitHub Copilot ~$200, Snyk ~$400, middle ground)

Calculation:
  30M × 0.60 × 0.40 × $300 = $2.16 billion

Sanity check:
  - GitHub revenue ~$2B (broader product, ~4M paid users)
  - Snyk valued at $7B (code security, related space)
  - $2B TAM is plausible for a focused code review tool

Result: ~$2 billion TAM (range: $700M to $6B)
```

**Example B: Estimating daily active queries to a search engine**

```
Question: How many search queries does Google process per day?

Decomposition:
  Queries/day = (Internet users who use Google)
              × (searches per user per day)

Estimates:
  Global internet users: ~5.5 billion
  Google market share: ~90%
  Google users: 5.5B × 0.90 = ~5 billion
  But not all use it daily: ~50% daily active rate
  Daily active Google searchers: ~2.5 billion

  Searches per active user per day: ~3-4
  (Some people search 10+ times, many search once or not at all)

Calculation:
  2.5 billion × 3.5 = ~8.5 billion queries/day

Sanity check:
  Published figure (Google): ~8.5 billion searches/day (2024)
  Our estimate nailed it — sometimes Fermi estimation gets lucky.

Result: ~8.5 billion/day (range: 3B to 25B)
```

### Order of Magnitude Sanity Checks

After any estimate, verify it makes sense:

```
CHECK 1: Per-person reasonableness
  Divide by relevant population. Is the per-person number realistic?
  "$50B market ÷ 340M Americans = $147/person" — plausible?

CHECK 2: Comparison to known quantities
  Is your estimate bigger or smaller than things you know?
  "Our estimate of X is $3B — that's 5% of AWS revenue. Reasonable?"

CHECK 3: Growth rate implied
  If you're estimating a future state, what annual growth rate is implied?
  >50% sustained growth for >3 years is extremely rare.

CHECK 4: Upper bound test
  What is the theoretical maximum? Is your estimate within it?
  "Total possible customers × maximum price = ceiling"
```

---

## Prediction Market Patterns

### Interpreting Market Prices as Probabilities

Prediction market prices map to probabilities, but with important caveats:

```
Market price $0.65 for "Event X occurs"
  → Naive interpretation: 65% probability
  → Adjusted interpretation: depends on market quality

Adjustment factors:
  Liquid market (Polymarket, Metaculus with many forecasters):
    Price ≈ true probability (±3-5%)

  Thin market (<50 traders, <$10K volume):
    Price is noisy — treat as ±15% uncertainty
    A $0.65 price could represent 50-80% true probability

  Binary vs. multi-outcome:
    Binary markets are more reliable
    Multi-outcome markets often have probabilities summing to >100%
    (overround) — normalize before interpreting
```

### Common Prediction Market Biases

| Bias | Description | Impact | Correction |
|------|-------------|--------|------------|
| Favorite-longshot | Favorites underpriced, longshots overpriced | Longshot events appear ~2-3x more likely than they are | If market says 5%, true probability may be 2-3% |
| Recency | Recent events dominate pricing | Probability spikes after news, then slowly reverts | Wait 24-48h after major news before trusting market prices |
| Liquidity premium | Illiquid contracts trade at a discount | Prices biased toward 50% in thin markets | Weight liquid markets more heavily |
| Expiration clustering | Prices converge to 0 or 1 near expiration | Mid-probability contracts vanish near deadline | Most useful signal is months before resolution |
| Hedging distortion | Traders hedging other positions, not expressing beliefs | Prices reflect risk management, not pure probability | Cross-reference with non-market forecasts |

### Aggregation Methods

When combining multiple probability estimates (markets, experts, models):

```
SIMPLE AVERAGE
  P = (P1 + P2 + P3) / 3
  Use when: Sources are roughly equally credible
  Weakness: Susceptible to outliers

MEDIAN
  P = middle value of sorted estimates
  Use when: One source might be badly miscalibrated
  Weakness: Ignores magnitude of disagreement

TRIMMED MEAN
  Drop highest and lowest, average the rest
  P = average(P2 ... Pn-1) after sorting
  Use when: 5+ sources, want outlier robustness

EXTREMIZED AVERAGE
  P_avg = simple average
  P_extremized = P_avg^a / (P_avg^a + (1-P_avg)^a), where a > 1
  Typical a = 1.5 to 2.5 (more extremizing with more independent sources)
  Use when: Sources are genuinely independent (not reading each other)
  Rationale: If 5 independent sources all say 70%, the true probability
  is likely higher than 70% — shared info should push further from 50%

CONFIDENCE-WEIGHTED AVERAGE
  P = Σ(wi × Pi) / Σ(wi)
  where wi = track record score or source reliability
  Use when: Sources have known, differing track records
```

### When Markets Beat Experts (and Vice Versa)

```
MARKETS TEND TO WIN when:
  ✓ Large, liquid, diverse participant pool
  ✓ Question is well-defined with clear resolution criteria
  ✓ Information is widely distributed (no single expert has edge)
  ✓ Time horizon is 1 month to 2 years
  Examples: Election outcomes, product launch dates, economic indicators

EXPERTS TEND TO WIN when:
  ✓ Question requires deep domain-specific knowledge
  ✓ Market is thin or participants lack domain context
  ✓ Very long time horizons (>5 years) — markets discount distant futures
  ✓ Novel situations with no historical market precedent
  Examples: Technical feasibility, scientific breakthroughs, niche regulation

BEST PRACTICE: Use both
  Start with the market price, then adjust using expert insight.
  Treat the market as the prior and expert analysis as an update.
```

---

## Update Protocol

### Bayesian Updating Worked Example

```
SCENARIO: You predicted 30% chance that Company Z launches Product A in Q1.
New evidence: A leaked internal slide shows a Q1 launch timeline.

STEP 1: State the prior
  P(launch in Q1) = 0.30

STEP 2: Assess the evidence
  E = leaked slide showing Q1 timeline
  How likely is this evidence if the launch IS happening in Q1?
    P(E | launch) = 0.85
    (Internal slides usually reflect real plans, but plans change)
  How likely is this evidence if the launch is NOT in Q1?
    P(E | no launch) = 0.15
    (Could be outdated slide, aspirational, or decoy)

STEP 3: Calculate the likelihood ratio
  LR = P(E | launch) / P(E | no launch) = 0.85 / 0.15 = 5.67

STEP 4: Convert prior to odds, multiply, convert back
  Prior odds = 0.30 / 0.70 = 0.429
  Posterior odds = 0.429 × 5.67 = 2.43
  Posterior probability = 2.43 / (1 + 2.43) = 0.71

STEP 5: State the update
  Prior: 30% → Posterior: 71%
  Update magnitude: +41 percentage points
  This is a LARGE update, appropriate because the evidence (internal
  planning document) is strong and directly relevant.
```

### Evidence Strength Classification

How much to update based on different types of evidence:

```
EVIDENCE TIER 1 — Large update (likelihood ratio 5-20x)
  → Official announcement or regulatory filing
  → Confirmed internal document (not rumor)
  → Directly observed outcome of prerequisite event
  → Multiple independent strong sources confirming same fact
  Typical update: ±15-30 percentage points

EVIDENCE TIER 2 — Moderate update (likelihood ratio 2-5x)
  → Credible journalist report with named sources
  → Statistical data that changes the base rate
  → Expert with strong track record changing their view
  → Structural/policy change that alters incentives
  Typical update: ±5-15 percentage points

EVIDENCE TIER 3 — Small update (likelihood ratio 1.2-2x)
  → Rumor from semi-credible source
  → Anecdotal evidence (single data point)
  → Social media sentiment shift
  → Expert opinion without new information
  Typical update: ±2-5 percentage points

EVIDENCE TIER 4 — Negligible update (likelihood ratio ~1x)
  → Repetition of previously known information
  → Pundit opinion with no domain expertise
  → Vague statement open to multiple interpretations
  → Evidence equally consistent with both outcomes
  Typical update: ±0-2 percentage points (or skip entirely)
```

### When to Make Large vs. Small Updates

```
MAKE A LARGE UPDATE when:
  • Evidence directly addresses your key uncertainty
  • The source has a strong track record on this topic
  • The evidence would be very surprising if your prediction were correct
    (or very unsurprising if it were wrong)
  • Multiple independent signals shift in the same direction simultaneously

MAKE A SMALL UPDATE when:
  • Evidence is tangentially related to your prediction
  • The source's reliability is uncertain
  • The evidence is consistent with multiple interpretations
  • You've already incorporated similar evidence

RESIST UPDATING when:
  • The "evidence" is just someone restating the consensus
  • A vivid anecdote feels compelling but carries no statistical weight
  • You're reacting emotionally (fear, excitement) rather than analytically
  • The evidence source has an obvious incentive to mislead
```

### Common Updating Mistakes

| Mistake | Description | Fix |
|---------|-------------|-----|
| Over-updating on vivid events | A dramatic single event shifts your view by 20+ points when the base rate barely moved | Ask: "Does this event actually change the base rate, or just my emotional state?" |
| Under-updating on base rate changes | New data shows the reference class frequency shifted, but you keep your old anchor | Periodically re-derive the base rate from scratch instead of only adjusting incrementally |
| Asymmetric updating | Updating strongly on confirming evidence, weakly on disconfirming evidence | Force yourself to calculate the likelihood ratio for disconfirming evidence explicitly |
| Double-counting | Updating on a news article, then updating again on a tweet quoting the same article | Track the original source — if two signals share the same root cause, count once |
| Failure to update | Knowing the evidence should change your view but not bothering because your current number "feels right" | Set calendar reminders to review active predictions monthly with fresh evidence |
| Stampede updating | A prediction market spikes, causing you to rush your update to match | Market moves are data, not commands — assess independently, then compare |

---

## Prediction Tracking & Scoring

### Prediction Ledger Format
```json
{
  "id": "pred_001",
  "created": "2025-01-15",
  "prediction": "OpenAI will release GPT-5 before July 2025",
  "confidence": 0.65,
  "domain": "tech",
  "time_horizon": "2025-07-01",
  "reasoning_chain": "...",
  "key_signals": ["leaked roadmap", "compute scaling", "hiring patterns"],
  "status": "active|resolved|expired",
  "resolution": {
    "date": "2025-06-30",
    "outcome": true,
    "evidence": "Released June 15, 2025",
    "brier_score": 0.1225
  },
  "updates": [
    {"date": "2025-03-01", "new_confidence": 0.75, "reason": "New evidence: leaked demo"}
  ]
}
```

### Accuracy Report Template
```
ACCURACY DASHBOARD
==================
Total predictions:     N
Resolved predictions:  N (N correct, N incorrect, N partial)
Active predictions:    N
Expired (unresolvable):N

Overall accuracy:      X%
Brier score:           0.XX

Calibration:
  Predicted 90%+ → Actual: X% (N predictions)
  Predicted 70-89% → Actual: X% (N predictions)
  Predicted 50-69% → Actual: X% (N predictions)
  Predicted 30-49% → Actual: X% (N predictions)
  Predicted <30% → Actual: X% (N predictions)

Strengths: [domains/types where you perform well]
Weaknesses: [domains/types where you perform poorly]
```

---

## Cognitive Bias Checklist

Before finalizing any prediction, check for these biases:

1. **Anchoring**: Am I fixated on the first number I encountered?
   - Fix: Deliberately consider the base rate before looking at specific evidence

2. **Availability bias**: Am I overweighting recent or memorable events?
   - Fix: Check the actual frequency, not just what comes to mind

3. **Confirmation bias**: Am I only looking for evidence that supports my prediction?
   - Fix: Actively search for contradicting evidence (steel-man the opposite)

4. **Narrative bias**: Am I choosing a prediction because it makes a good story?
   - Fix: Boring predictions are often more accurate

5. **Overconfidence**: Am I too sure?
   - Fix: If you've never been wrong at this confidence level, you're probably overconfident

6. **Scope insensitivity**: Am I treating very different scales the same?
   - Fix: Be specific about magnitudes and timeframes

7. **Recency bias**: Am I extrapolating recent trends too far?
   - Fix: Check longer time horizons and mean reversion patterns

8. **Status quo bias**: Am I defaulting to "nothing will change"?
   - Fix: Consider structural changes that could break the status quo

### Contrarian Mode
When enabled, for each consensus prediction:
1. Identify what the consensus view is
2. Search for evidence the consensus is wrong
3. Consider: "What would have to be true for the opposite to happen?"
4. If credible contrarian evidence exists, include a contrarian prediction
5. Always label contrarian predictions clearly with the consensus for comparison