Test pack
What every bot must ship. Without this pack, the bot does not promote past stub.
Required test classes
| Class | Purpose | Minimum coverage |
|---|---|---|
| Unit | Cover every branch of decide(). | Every reason code listed in the spec must be exercised by at least one unit test. |
| Integration | Run the bot inside a real BotContext with a real (mocked) ReportEnvelope sink. | The bot's full spec — input schema → decide → emit → ReportEnvelope round-trip — must run end-to-end. |
| Property | Catch assumptions the unit tests forgot to assert. | For numeric bots: monotonicity, idempotency on identical input, boundedness against the hard threshold. |
| Failure-injection | Confirm the documented safe_fallback actually fires. | Every failure mode in the spec must have a recipe in the test pack and a passing assertion that the safe fallback was taken. |
Required fixtures
Every bot's repo folder ships with:
bots/<layer>/<slug>/
config.default.json
input.normal.json
input.warning.json
input.hard_reject.json
output.expected.normal.json
output.expected.warning.json
output.expected.hard_reject.json
tests.spec.ts
The expected outputs are diff-checked against decide(input) output by the integration test. Any change to expected output is a config-version bump.
Forbidden in tests
- Reading the wall clock. All time goes through
context.now(). - Hitting the live Polymarket API. Use the recorded fixture set under
/fixtures/clob_v2/. - Asserting on free-text log output. Assert on the
reason_codeof the emitted envelope. - Sleeping. Use the test scheduler.
Promotion gate checks
The promotion-gate page on each bot lists the automated checks that must pass before promotion. The minimum is:
- Unit + integration suites pass on a clean checkout.
- Property tests run for at least 1,000 random inputs without a counterexample.
- Every documented failure mode has a passing failure-injection test.
- Shadow run for at least 24 hours produces ReportEnvelopes that decode against the schema.