6.18 ReplaySimulator

1. Bot Identity

Operational profile

2. Purpose

Re-runs any historical pipeline trace against the current bot revisions to verify that the outcome would be the same (or to surface changes). Used for regression testing, incident post-mortems, and 'what would have happened' reviews. Runs only on recorded ReportEnvelope streams — never against live state.

3. Why This Bot Matters

• Regression detection

  When a single bot is bumped, the simplest correctness check is to replay yesterday's traffic and diff the outcomes.

• Postmortem reproducibility

  An incident review that cannot reproduce the exact decision is just speculation.

• Promotion gate

  Templates require a passing replay against the canonical fixture set before promoting from shadow to advisory.

No worked examples on this bot yet. Worked examples are optional but strongly recommended — they turn an abstract failure mode into something a developer can verify in a fixture.

4. Required Polymarket Inputs

— not yet authored —

5. Required Internal Inputs

6. Parameter Guide

7. Detailed Parameter Instructions

{ "max_replay_minutes": 60 }

if (replay_window > p.max_replay_minutes) chunk();

{ "tolerance_bps": 5 }

if (abs(now - then) > p.tolerance_bps) flag('REGRESSION');

8. Default Configuration

{
  "max_replay_minutes": 60,
  "tolerance_bps": 5
}

9. Implementation Flow

— not yet authored —

10. Reference Implementation

Pseudocode is language-agnostic. FETCH = read input. EMIT = produce output. IF/THEN/ELSE = decision. Translate directly to TypeScript, Python, Go, or Rust.

for env in archive.window(job.start_ms, job.end_ms):
  out = sandbox.run(current_bots, env.input)
  if differs(out, env.recorded_output, p.tolerance_bps):
    regressions.append(diff_record(env, out))
  else:
    matches += 1
emit('ReplayDigest', job, matches, regressions[:10])

11. Wire Examples

Input — what arrives on the wire

{
  "job_id": "replay_001",
  "window_start_ms": 1715260000000,
  "window_end_ms": 1715263600000
}

Output — what the bot emits

{
  "kind": "ReplayDigest",
  "matches": 1042,
  "regressions": 3
}

12. Decision Logic

13. Standard Decision Output

This bot returns a RiskVote object. See RiskVote schema.

{
  "kind": "ReplayDigest",
  "window_start_ms": 1715260000000,
  "window_end_ms": 1715263600000,
  "matches": 1042,
  "regressions": 3,
  "first_regressions": [
    {
      "intent_id": "intent_001",
      "field": "cost_estimate.slippage_bps",
      "then": 35,
      "now": 41
    }
  ]
}

14. Reason Codes

15. Metrics & Logs

Metrics emitted

Dashboards

• 6.18 overview dashboard

16. Developer Reporting

"Per replay: job_id, window, total_inputs, matches, regressions, runtime_ms."

17. Plain-English Reporting

18. Failure-Mode Block

19. Failure-Injection Recipes

20. State & Persistence

Replay archive index. Job history. No live state.

State stores

Cold-start recovery

Cold-start hydrates from fast KV; missing keys default to safe fallback.

On restart

All in-flight decisions are re-evaluated; no bot decision is trusted across restart without re-emit.

21. Concurrency & Idempotency

22. Dependencies

23. Security Surfaces

Sandbox network is fully blocked. Only replay archive read access.

Signing surface

None — bot does not sign or submit.

Mitigations

• Rate-limit per source
• Audit-log every override
• Require role-based authz on admin paths

24. Polymarket V2 Compatibility

25. Versioning & Migration

26. Acceptance Tests

Unit Tests

Integration Tests

Property Tests

27. Operational Runbook

If replays fail with NO_NETWORK_VIOLATION, the offending bot leaks an external call — file a P2 issue.

On-call actions

Manual overrides

• polytraders bot pause 6.18 — Disables the bot's enforcement layer; downstream consumers fall back to safe defaults.

Healthcheck

GET /healthz/replay_simulator → 200 if last successful evaluation < 60s ago.

28. Promotion Gates

A bot does not advance to the next readiness state until every gate below is green. Gates are observable from production data — no subjective sign-off.

Promote to Shadow

Promote to Limited live

Promote to General live

29. Developer Checklist

Ready-to-ship score: 27/27 sections complete · 100%