# Quiz System - XPs, Classification, and Analytics

High-retention practice system design with engagement-first XP, simple taxonomy, analytics-driven personalization, and parametric content generation

<Info>
Launch a high-retention practice system that is simple to use, fair to learners, and useful to creators—while keeping taxonomy light, analytics strong, and recommendations reliable.
</Info>

## Product Principles

<CardGroup cols={2}>
  <Card title="XP ≠ Skill" icon="star">
    XP rewards engagement; skill is inferred separately
  </Card>
  <Card title="Simple Beats Smart" icon="minimize">
    4 domains × 3 Blooms; no difficulty-based XP
  </Card>
  <Card title="Spec-First Generation" icon="file-contract">
    Classification decided upstream; post-gen validation is guardrail
  </Card>
  <Card title="Reliability Over Theory" icon="shield-check">
    κ & AC1 ≥ 0.75 before wide rollout
  </Card>
  <Card title="Accessibility & Fairness" icon="universal-access">
    No penalties for help-seeking or slower devices
  </Card>
  <Card title="Ship, Measure, Iterate" icon="rocket">
    Add complexity only when data demands it
  </Card>
</CardGroup>

## System Architecture

<Frame caption="High-Level System Overview">
```
Learner → Quiz Player → XP Engine → XP Ledger
                ↘︎ events       ↘︎ dashboards

Creator → Builder/AI → Classification → Questions Store
                          ↘︎ Item Health
                          ↘︎ Coverage Analytics

Telemetry → Analytics Layer → Recommender
              (Mastery/Health/Spacing)

Admin/QA → Reliability Suite → Classifier Tuning
```
</Frame>

## XP & Gamification

### XP Rules

<Note>
XP is immediate and predictable, focused on engagement rather than difficulty
</Note>

#### Base Rewards
- **Correct Answer:** +10 XP
- **Incorrect Answer:** +2 XP (participation reward)

#### Bonus System

<CardGroup cols={2}>
  <Card title="Perfect Score" icon="trophy">
    +10 XP for 100% accuracy
  </Card>
  <Card title="Speed Bonus" icon="bolt">
    +10 XP if accuracy ≥80% and faster than median
  </Card>
  <Card title="Review Bonus" icon="book-open">
    +5 XP for solution review + reflection
  </Card>
  <Card title="Streak Bonus" icon="fire">
    +5 XP per day, max 7 days (+35 XP cap)
  </Card>
</CardGroup>

<Warning>
Daily soft cap: After 300 XP/day, payouts at 50% rate to prevent grinding
</Warning>

### Gamification Features

<Steps>
  <Step title="Seasons & Leagues">
    Monthly seasons with XP-based leagues (Bronze/Silver/Gold). Leaderboards reset each season.
  </Step>
  <Step title="Cell-Aligned Badges">
    Earn badges like "RISK–Analyze Adept" for 30 verified corrects in that cell. Tiers at 10/30/60.
  </Step>
  <Step title="Session Goals">
    Auto-set daily goals (e.g., "Earn 60 XP today") with progress tracking.
  </Step>
</Steps>

### Anti-Exploit & Accessibility

<Tip>
**No-Speed Mode:** Swaps Speed Bonus for Review Bonus to accommodate learners who need more time or have device limitations
</Tip>

## Classification Model

### 4×3 Matrix (12 Cells)

#### Domains

| Domain | Focus | Example Applications |
|--------|-------|---------------------|
| **ANALYSIS** | Reading/interpreting market data | Pattern recognition, signal validation |
| **STRATEGY** | Choosing approach and playbook | Plan selection, setup comparison |
| **RISK** | Position sizing and capital management | Stop placement, size calculation |
| **EXECUTION** | Order mechanics and timing policy | Order type selection, venue choice |

#### Bloom's Levels

<CardGroup cols={3}>
  <Card title="Apply" icon="play">
    One rule on one input
  </Card>
  <Card title="Analyze" icon="microscope">
    Compare/synthesize ≥2 inputs
  </Card>
  <Card title="Evaluate" icon="balance-scale">
    Pick best option with justification
  </Card>
</CardGroup>

### Classification Precedence

<Warning>
Precedence order: EXECUTION > RISK > STRATEGY > ANALYSIS
</Warning>

### Hybrid Classifier

<Steps>
  <Step title="Rules Layer">
    Keyword markers + precedence resolution
  </Step>
  <Step title="LLM Challenge">
    Request domain & Bloom with "why-not" reasoning for other domains
  </Step>
  <Step title="Decision Logic">
    Rules == LLM → high confidence; else apply precedence or review
  </Step>
  <Step title="Reliability Target">
    κ & AC1 ≥ 0.75 on balanced 12-cell set
  </Step>
</Steps>

## Content Creation

### Doc-Driven AI Pipeline

<Frame caption="AI Content Generation Flow">
```json
{
  "domain": "RISK",
  "bloom": "Analyze",
  "constraints": ["policy-level execution only"],
  "grounding": ["doc://ch12#para3", "doc://ch12#fig2"]
}
```
</Frame>

### Manual Builder Features

- Coverage-aware prompts ("Low on EXECUTION–Analyze")
- Creator chooses domain + bloom or accepts auto-suggestion
- Parametric templates offered (not required)
- Same validator ensures consistency

## Analytics Layer

### Item Health Metrics

| Metric | Description | Use Case |
|--------|-------------|----------|
| **Success Rate** | Correct answers per item | Basic performance |
| **Discrimination** | Δ success between top/bottom quartiles | Item quality |
| **Abandon Rate** | Quits/timeouts per item | Difficulty indicator |
| **Time Z-score** | Deviation from median | Ambiguity detection |

<Note>
Health status progression: new → stable → needs_edit → quarantined
</Note>

### Mastery Tracking

<Info>
Beta model per cell: α = 1 + correct, β = 1 + incorrect
Mastery = α/(α+β)
</Info>

### Spacing & Review

- If days_since_last_correct > 7 and mastery < 0.70 → inject review
- Light-touch spacing without heavy scheduler

### Dashboard Types

<CardGroup cols={3}>
  <Card title="Educators" icon="chalkboard-teacher">
    Coverage analytics, item health, impact metrics
  </Card>
  <Card title="Learners" icon="user-graduate">
    Mastery by cell, streaks, recommended focus
  </Card>
  <Card title="Platform" icon="chart-bar">
    Retention metrics, reliability scores, inventory health
  </Card>
</CardGroup>

## Parametric Templates

### Why Parametric Templates?

<CardGroup cols={2}>
  <Card title="Prevent Gaming" icon="shield">
    Varied surface prevents pattern matching
  </Card>
  <Card title="Stable Labels" icon="tag">
    Classification consistency with creativity
  </Card>
  <Card title="Better Transfer" icon="comments-dollar">
    Spans different market regimes
  </Card>
  <Card title="Creator Freedom" icon="palette">
    Express nuance within quality guardrails
  </Card>
</CardGroup>

### Template Structure

Each template includes:
- **Inputs/Slots**: &#123;ASSET&#125;, &#123;TIMEFRAME&#125;, &#123;ENTRY_RULE&#125;
- **Constraints**: Value ranges, mutually exclusive combos
- **Answer Rule**: How to compute correct choice
- **Distractors**: Principled wrong answers
- **Validation**: Classification guardrails

### Example Template: Position Size Calculation

```
Prompt: Account {EQUITY}. Risk {RISK_PCT}% per trade. 
        Entry {ENTRY}, stop {STOP}. What size?

Answer Rule: size = (EQUITY × RISK_PCT) / |ENTRY–STOP|

Distractors:
- Swap TP for stop
- Percent-of-equity share count
- Decimal errors
```

## Implementation Timeline

<Steps>
  <Step title="Week 1">
    XP engine with base rewards, bonuses, streaks, session goals
  </Step>
  <Step title="Week 2">
    4-domain/3-Bloom classifier, precedence rules, builder UI
  </Step>
  <Step title="Weeks 3-4">
    Performance tracking, item health, dashboards v0
  </Step>
  <Step title="Month 2">
    LLM validator, confidence scoring, reduce manual review to ~20%
  </Step>
  <Step title="Month 3">
    Beta mastery, recommender v1, spaced review, badges & leagues
  </Step>
  <Step title="Month 6">
    Consider info_complexity if analytics justify (analytics-only)
  </Step>
</Steps>

## Success Metrics

### Primary Metrics

<Check>
- κ & AC1 ≥ 0.75 (balanced 12-cell set)
- D7 retention ≥ 40%; D30 ≥ 20%
</Check>

### Secondary Metrics

- +8–12% improvement in cell-level mastery within 7 days
- Creator efficiency: < 90s median to confirm labels
- Content health: < 5% traffic to needs_edit/quarantined items

### Alert Thresholds

<Warning>
- Speed bonus payout > 40% → tighten normative threshold
- Any domain < 15% inventory for 2 weeks → trigger creator challenges
</Warning>

## Risk Mitigation

| Risk | Mitigation |
|------|------------|
| **Speed farming** | Accuracy gate + normative pace + daily cap |
| **Label drift** | Precedence rule + balanced validation |
| **Supply imbalance** | Coverage-aware prompts + creator challenges |
| **Accessibility** | No-Speed Mode with Review Bonus |
| **Data sparsity** | Minimum attempts for quarantine |

## Future Enhancements

### Phase A (Months 1-2)
- Creator impact tracking
- Basic personalization
- Friendly competitions

### Phase B (Months 3-6)
- Creator payments based on learning impact
- Study groups
- Content difficulty ranking (analytics-only)

### Phase C (Months 6-12)
- Advanced predictions
- Seasonal narratives
- Mentorship programs

<Tip>
All future features would be for analytics only (no XP impact) and added carefully to avoid complexity bloat
</Tip>