1. Bot Identity
| Layer | Governance Governance |
|---|
| Bot class | Governance Service |
|---|
| Authority | Explain |
|---|
| Status | LIVE |
|---|
| Readiness | General live |
|---|
| Runs before | Every scheduled bot task — CronRunner fires the trigger |
|---|
| Runs after | System startup |
|---|
| Applies to | All cron-scheduled tasks across all 97 production bots |
|---|
| Default mode | general_live |
|---|
| User-visible | Advanced details only |
|---|
| Developer owner | Polytraders core — Governance pod |
|---|
2. Purpose
CronRunner maintains the authoritative cron schedule for the entire Polytraders system. It reads a declarative cron manifest (cron_expression per task, UTC), fires each task at the correct wall-clock moment, serialises concurrent firings into an ordered queue, and emits an OperationsReport heartbeat after each dispatch. CronRunner never executes strategy logic or touches the CLOB directly; it is purely a scheduling and dispatch service. Internal-only — no API surface beyond the internal bus.
3. Why This Bot Matters
Cron task not fired on schedule
Downstream bots that rely on scheduled triggers (e.g. daily P&L reconciliation, pre-event quoting cycles) silently miss their window. Delayed reconciliation compounds drift.
Two instances of CronRunner fire the same task simultaneously
Duplicate fills or double-counted reconciliation records corrupt ledgers and inflate reported P&L.
CronRunner crashes and does not recover
All scheduled tasks across all 97 bots stop firing. The system drifts into an uncontrolled state without time-based governance.
Missed quiet-hours suppression
Strategies fire during configured quiet hours (e.g. market-open volatility windows), creating unintended exposure.
No worked examples on this bot yet. Worked examples are optional but strongly recommended — they turn an abstract failure mode into something a developer can verify in a fixture.
6. Parameter Guide
| Parameter | Default | Warning | Hard | What it controls |
|---|
| cron_expression | 0 * * * * | None | None | Standard POSIX cron expression (UTC) defining when a scheduled task fires. |
| enabled_strategies | [] | None | None | List of strategy or bot IDs that this cron entry should trigger. |
| disable_during_quiet_hours | False | None | None | When true, suppresses this task during configured system quiet hours. |
| max_concurrent_tasks | 20 | 40 | 60 | Maximum number of cron tasks that may be dispatched concurrently in a single tick. |
7. Detailed Parameter Instructions
cron_expression
What it means
Standard POSIX cron expression (UTC) defining when a scheduled task fires.
Default
{ "cron_expression": "0 * * * *" }
Why this default matters
Hourly default prevents unintentional high-frequency dispatch. Each task must explicitly set its own expression.
Threshold logic
| Condition | Action |
|---|
| valid cron expression | Schedule task |
| invalid expression | Reject at config load; emit CRON_RUNNER_INVALID_EXPRESSION |
Developer check
parseCron(p.cron_expression) || throw ConfigError('CRON_RUNNER_INVALID_EXPRESSION')
User-facing English
The system runs certain maintenance and reporting tasks on a fixed schedule.
enabled_strategies
What it means
List of strategy or bot IDs that this cron entry should trigger.
Default
{ "enabled_strategies": [] }
Why this default matters
Empty list by default — tasks must be explicitly enabled to prevent accidental firing of inactive strategies.
Threshold logic
| Condition | Action |
|---|
| non-empty list | Dispatch trigger to each listed bot ID |
| empty list | Log warning CRON_RUNNER_NO_TARGETS and skip |
Developer check
if (!p.enabled_strategies.length) log.warn('CRON_RUNNER_NO_TARGETS')
User-facing English
Scheduled tasks run only for active components.
disable_during_quiet_hours
What it means
When true, suppresses this task during configured system quiet hours.
Default
{ "disable_during_quiet_hours": false }
Why this default matters
Off by default so critical governance tasks (health checks, reconciliation) always fire. Enable only for trading-adjacent tasks.
Threshold logic
| Condition | Action |
|---|
| disable_during_quiet_hours=true AND now in quiet window | Skip task; emit CRON_RUNNER_QUIET_HOURS_SKIP |
| disable_during_quiet_hours=false | Fire regardless of quiet hours |
Developer check
if (p.disable_during_quiet_hours && isQuietHour(now)) { log.info('CRON_RUNNER_QUIET_HOURS_SKIP'); return; }
User-facing English
Some automated tasks are paused during quiet periods.
max_concurrent_tasks
What it means
Maximum number of cron tasks that may be dispatched concurrently in a single tick.
Default
{ "max_concurrent_tasks": 20 }
Why this default matters
At 20 concurrent tasks, the internal bus stays within throughput limits. Above 40, dispatch latency degrades.
Threshold logic
| Condition | Action |
|---|
| max_concurrent_tasks <= 20 | Normal dispatch |
| 20–60 | WARN — bus throughput may degrade |
| > 60 | Reject config change — PARAMETER_CHANGE_REQUIRES_APPROVAL |
Developer check
if (p.max_concurrent_tasks > p.hard) throw ConfigError("PARAMETER_CHANGE_REQUIRES_APPROVAL")
User-facing English
— not yet authored —
8. Default Configuration
{
"bot_id": "gov.cron_runner",
"version": "2.0.0",
"mode": "general_live",
"defaults": {
"cron_expression": "0 * * * *",
"enabled_strategies": [],
"disable_during_quiet_hours": false,
"max_concurrent_tasks": 20
}
}
9. Implementation Flow
- On startup, load the cron manifest from the config store; parse each entry's cron_expression (UTC) and validate.
- Register a ticker per unique cron expression; at each tick, collect all tasks whose expression matches the current UTC time.
- Before dispatching any task, check KillSwitch active flag; if active, suppress trading-adjacent tasks but allow health and governance tasks.
- For each eligible task, check disable_during_quiet_hours against the current UTC time window.
- Dispatch the task trigger to each enabled_strategies entry via the internal message bus; log dispatch with a trace_id.
- After dispatch, emit an OperationsReport heartbeat: { task_id, fired_at_ms, targets, suppressed_count, trace_id }.
- Detect and deduplicate concurrent firings using a distributed lock keyed on (task_id, scheduled_at_ms); if lock is held, skip and emit CRON_RUNNER_DUPLICATE_FIRE.
- Retain dispatch logs for 1 year for operational audit.
10. Reference Implementation
Loads a cron manifest, registers tickers, dispatches task triggers at UTC boundaries, deduplicates via distributed lock, and emits an OperationsReport per dispatch.
Pseudocode is language-agnostic. FETCH = read input. EMIT = produce output. Translate to TS/Python/Go/Rust.
// ---- STARTUP ----
FUNCTION init():
manifest = FETCH config_store.GET('/cron-manifest')
FOR entry IN manifest:
validateCronExpression(entry.cron_expression)
registerTicker(entry) // fires onTick() when expression matches UTC now
// ---- PER-TICK HANDLER ----
FUNCTION onTick(entry, scheduled_at_ms):
lock_key = entry.task_id + ':' + scheduled_at_ms
IF NOT redis.SET(lock_key, NX=true, PX=5000): // 5s TTL
log.warn('CRON_RUNNER_DUPLICATE_FIRE', { task_id: entry.task_id })
RETURN
ks = FETCH killswitch.status()
IF ks.active AND entry.task_class == 'trading':
log.info('CRON_RUNNER_KILL_SWITCH_ACTIVE', { task_id: entry.task_id })
RETURN
IF entry.disable_during_quiet_hours AND isQuietHour(now()):
log.info('CRON_RUNNER_QUIET_HOURS_SKIP', { task_id: entry.task_id })
EMIT OperationsReport(event_type='CRON_TASK_SUPPRESSED', ...)
RETURN
IF NOT entry.enabled_strategies:
log.warn('CRON_RUNNER_NO_TARGETS', { task_id: entry.task_id })
RETURN
trace_id = generateTraceId()
FOR target IN entry.enabled_strategies:
internal_bus.publish(topic='cron.trigger', payload={
task_id: entry.task_id,
target: target,
scheduled_at: scheduled_at_ms,
trace_id: trace_id
})
EMIT OperationsReport({
report_id: 'ops_cronrunner_' + scheduled_at_ms,
bot_id: 'gov.cron_runner',
event_type: 'CRON_TASK_DISPATCHED',
task_id: entry.task_id,
fired_at_ms: scheduled_at_ms,
targets: entry.enabled_strategies,
suppressed_count: 0,
trace_id: trace_id
})
// ---- MANIFEST RELOAD ----
FUNCTION reloadManifest():
// Called on config change event; validates and re-registers all tickers
FOR entry IN FETCH config_store.GET('/cron-manifest'):
validateCronExpression(entry.cron_expression)
registerTickers(manifest) // replaces existing registrations atomically
SDK calls used
config_store.GET('/cron-manifest')redis.SET(lock_key, NX=true, PX=5000)killswitch.status()internal_bus.publish(topic='cron.trigger', payload=...)alerting.emit('CRON_RUNNER_DUPLICATE_FIRE', metadata)
Complexity: O(T) per tick where T = tasks matching the current cron boundary
11. Wire Examples
Input — what arrives on the wire
{
"label": "Cron manifest entry",
"source": "config_store",
"payload": {
"task_id": "daily_pnl_reconcile",
"cron_expression": "0 0 * * *",
"enabled_strategies": [
"gov.pnl_reporter"
],
"disable_during_quiet_hours": false,
"task_class": "governance"
}
}
Output — what the bot emits
{
"label": "OperationsReport — CRON_TASK_DISPATCHED",
"payload": {
"report_id": "ops_cronrunner_1746792000000",
"bot_id": "gov.cron_runner",
"event_type": "CRON_TASK_DISPATCHED",
"task_id": "daily_pnl_reconcile",
"fired_at_ms": 1746792000000,
"targets": [
"gov.pnl_reporter"
],
"suppressed_count": 0,
"trace_id": "tr_abc123def456",
"report_kind": "OperationsReport"
}
}
12. Decision Logic
APPROVE
Not applicable — CronRunner dispatches triggers, it does not approve or reject trading decisions.
RESHAPE_REQUIRED
Not applicable.
REJECT
CronRunner will reject a task dispatch if KillSwitch is active for that task class, or if a duplicate lock is held.
WARNING_ONLY
Invalid cron expressions and empty target lists emit warnings but do not crash the scheduler.
13. Standard Decision Output
This bot returns a OperationsReport object. See OperationsReport schema.
{
"report_id": "ops_cronrunner_20260509T120000Z",
"bot_id": "gov.cron_runner",
"event_type": "CRON_TASK_DISPATCHED",
"task_id": "daily_pnl_reconcile",
"fired_at_ms": 1746792000000,
"targets": [
"gov.pnl_reporter"
],
"suppressed_count": 0,
"trace_id": "tr_abc123def456",
"report_kind": "OperationsReport"
}
14. Reason Codes
| Code | Severity | Meaning | Action | User-facing message |
|---|
CRON_RUNNER_TASK_DISPATCHED | INFO | A cron task fired on schedule and was dispatched to all configured targets. | No action — operational heartbeat. | |
CRON_RUNNER_INVALID_EXPRESSION | HARD_REJECT | A cron expression in the manifest failed to parse. | Reject the config entry at load time; alert on-call. | |
CRON_RUNNER_DUPLICATE_FIRE | WARN | A second CronRunner instance attempted to dispatch the same task at the same scheduled time; distributed lock was already held. | Skip dispatch; log warning. | |
CRON_RUNNER_QUIET_HOURS_SKIP | INFO | A task with disable_during_quiet_hours=true was suppressed because the current UTC time falls within a configured quiet window. | No action — expected behaviour. | A scheduled task was paused during a quiet period. |
CRON_RUNNER_NO_TARGETS | WARN | A task entry has an empty enabled_strategies list; there is nothing to dispatch. | Log warning; skip dispatch. | |
KILL_SWITCH_ACTIVE | WARN | KillSwitch is active; trading-class scheduled tasks are suppressed. | Suppress trading tasks; continue governance and health tasks. | Scheduled trading tasks are paused while the system kill switch is active. |
CRON_RUNNER_BUS_PUBLISH_FAILED | WARN | Internal bus publish for a task trigger failed after retries. | Log error; emit alert; retry on next tick. | |
CRON_RUNNER_MANIFEST_RELOAD_OK | INFO | Cron manifest reloaded successfully after a config change event. | No action. | |
15. Metrics & Logs
Metrics emitted
| Metric | Type | Unit | Labels | Meaning |
|---|
polytraders_gov_cronrunner_tasks_dispatched_total | counter | count | task_id, task_class | Total cron tasks dispatched successfully. |
polytraders_gov_cronrunner_tasks_suppressed_total | counter | count | reason, task_id | Tasks suppressed due to quiet hours, kill switch, or no targets. |
polytraders_gov_cronrunner_duplicate_fire_total | counter | count | task_id | Duplicate dispatch attempts blocked by distributed lock. |
polytraders_gov_cronrunner_dispatch_latency_ms | histogram | ms | task_id | Wall-clock latency from cron boundary to internal bus publish. |
polytraders_gov_cronrunner_manifest_entries | gauge | count | | Number of active entries in the loaded cron manifest. |
polytraders_gov_cronrunner_bus_failures_total | counter | count | task_id | Internal bus publish failures for cron triggers. |
Alerts
| Alert | Condition | Severity | Runbook |
|---|
CronRunnerNoDispatchIn15m | rate(polytraders_gov_cronrunner_tasks_dispatched_total[15m]) == 0 | page | #runbook-cronrunner-no-dispatch |
CronRunnerDuplicateFireRate | rate(polytraders_gov_cronrunner_duplicate_fire_total[5m]) > 0 | warn | #runbook-cronrunner-duplicate |
CronRunnerBusFailures | rate(polytraders_gov_cronrunner_bus_failures_total[5m]) > 0 | page | #runbook-cronrunner-bus-failure |
CronRunnerDispatchLatencyHigh | histogram_quantile(0.99, polytraders_gov_cronrunner_dispatch_latency_ms) > 500 | warn | #runbook-cronrunner-latency |
Dashboards
- Grafana — Governance / CronRunner task dispatch timeline
- Grafana — Governance / Scheduled task suppression rate
16. Developer Reporting
{
"bot_id": "gov.cron_runner",
"event_type": "CRON_TASK_DISPATCHED",
"task_id": "hourly_health_sweep",
"fired_at_ms": 1746788400000,
"targets": [
"gov.health_heartbeat"
],
"lock_acquired": true,
"trace_id": "tr_00deadbeef01"
}
17. Plain-English Reporting
| Situation | User-facing explanation |
|---|
| Scheduled maintenance task fired | The system ran a scheduled internal task on time. No action needed. |
| Task skipped during quiet hours | A scheduled task was paused during a quiet period to avoid interference with market activity. |
| Task suppressed by kill switch | Scheduled trading-related tasks are paused. Governance and health checks continue. |
18. Failure-Mode Block
| main_failure_mode | CronRunner process crashes or is partitioned from the internal bus, causing all scheduled tasks to miss their windows silently. |
|---|
| false_positive_risk | A task fires twice if two CronRunner instances start simultaneously and both acquire the dispatch lock before expiry. |
|---|
| false_negative_risk | A task is silently skipped if the system clock drifts more than the tick interval (1s), causing a cron boundary to be missed. |
|---|
| safe_fallback | On restart, CronRunner reloads the manifest and resumes from the current tick. Missed ticks in the recovery window are not retroactively fired — governance bots that depend on scheduled triggers include their own staleness checks. |
|---|
| required_dependencies | Config store (cron manifest), Internal message bus, Distributed lock (Redis or equivalent), KillSwitch status feed |
|---|
19. Failure-Injection Recipes
| Scenario | How to inject | Expected behaviour | Recovery |
|---|
REDIS_LOCK_UNAVAILABLE | Take Redis offline during a cron boundary | CronRunner falls back to in-process lock with WARN; CRON_RUNNER_DUPLICATE_FIRE suppression is weakened for the outage window | Once Redis reconnects, distributed locking resumes. Any duplicate dispatches during outage are handled by downstream idempotency. |
INVALID_MANIFEST_ENTRY | Push a cron manifest entry with cron_expression='99 * * * *' | CRON_RUNNER_INVALID_EXPRESSION raised; bad entry rejected; valid entries continue scheduling | Fix the manifest entry; reload triggers CRON_RUNNER_MANIFEST_RELOAD_OK. |
BUS_PUBLISH_FAILURE | Drop all packets to the internal bus for 60s | Dispatch attempts fail; CRON_RUNNER_BUS_PUBLISH_FAILED alert fired; tasks missed for that window | Bus reconnects; next tick dispatches normally. Downstream bots handle missed ticks via staleness checks. |
KILL_SWITCH_ACTIVE_DURING_TRADING_TASK | Activate KillSwitch; observe behaviour at next hourly tick for trading-class tasks | Trading tasks suppressed; governance and health tasks fire normally; KILL_SWITCH_ACTIVE logged | Deactivate KillSwitch; trading tasks resume at next tick. |
DUPLICATE_CRONRUNNER_INSTANCES | Start two CronRunner processes simultaneously | One instance acquires lock and dispatches; second emits CRON_RUNNER_DUPLICATE_FIRE and skips | Shut down duplicate instance; primary continues normally. |
20. State & Persistence
Cold-start recovery
On restart, cron manifest is re-fetched from config store and tickers are re-registered. Locks expire naturally. No missed ticks are retroactively fired.
21. Concurrency & Idempotency
| Aspect | Specification |
|---|
| Execution model | single-threaded event loop |
| Max in-flight | 100 |
| Idempotency key | task_id + scheduled_at_ms (distributed lock) |
| Per-call timeout (ms) | 2000 |
| Backpressure strategy | skip dispatch and emit CRON_RUNNER_BUS_PUBLISH_FAILED if bus is saturated |
| Locking / mutual exclusion | Redis SET NX PX per (task_id, scheduled_at_ms) |
22. Dependencies
Depends on (must run first)
| Bot | Why | Contract |
|---|
internal.config_store | Cron manifest is loaded from the config store on startup and on change events. | |
internal.killswitch | KillSwitch status is checked before dispatching trading-class tasks. | |
Emits to (downstream consumers)
External services
| Service | Endpoint | SLA assumed | On failure |
|---|
| Redis (distributed lock) | | 99.99% (internal SRE target) | |
23. Security Surfaces
Abuse vectors considered
- Injecting a malformed cron expression to crash the scheduler
- Racing the distributed lock to trigger double-dispatch of financial tasks
- Modifying the cron manifest to fire tasks outside of approved windows
Mitigations
- All cron expressions are validated at manifest load; invalid entries are rejected with HARD_REJECT
- Distributed lock (Redis SET NX) enforces single-dispatch per (task_id, scheduled_at_ms)
- Config store changes are audit-logged; manifest reload events are emitted as OperationsReport entries
- CronRunner has no CLOB or on-chain access — it only publishes to the internal bus
24. Polymarket V2 Compatibility
| Aspect | Value |
|---|
| CLOB version | v2 |
| Collateral asset | pUSD |
| EIP-712 Exchange domain version | 2 |
| Aware of builderCode field | no |
| Aware of negative-risk markets | no |
| Multi-chain ready | no |
| SDK used | internal-only |
| Settlement contract | none |
| Notes | CronRunner is internal-only and has no direct CLOB or on-chain interface. It dispatches triggers to other bots that operate on V2 surfaces. |
API surfaces declared
internal
Networks supported
polygon
25. Versioning & Migration
| Field | Value |
|---|
| spec | 2.0.0 |
| implementation | 2.1.0 |
| schema | 2 |
| released | 2026-04-28 |
Migration history
| Date | From | To | Reason | Action taken |
|---|
| 2026-04-28 | v1 | v2 | CLOB V2 cutover | No direct CLOB changes required — CronRunner has no CLOB surface. Updated OperationsReport schema to pUSD denomination in downstream task payloads. Removed legacy USDC.e references from task manifest documentation. |
26. Acceptance Tests
Unit Tests
| Test | Setup | Expected result |
|---|
| Valid cron expression parses without error | cron_expression='0 9 * * *' | Task scheduled for 09:00 UTC daily |
| Invalid cron expression raises ConfigError | cron_expression='99 * * * *' | ConfigError CRON_RUNNER_INVALID_EXPRESSION |
| Task suppressed during quiet hours | disable_during_quiet_hours=true, now=quiet window | CRON_RUNNER_QUIET_HOURS_SKIP logged; no dispatch |
| Duplicate lock prevents double-fire | Two instances attempt to dispatch same task_id at same scheduled_at_ms | Second instance emits CRON_RUNNER_DUPLICATE_FIRE and skips |
| KillSwitch suppresses trading tasks | KillSwitch active; task target=strat.some_strategy | Task suppressed; governance and health tasks fire normally |
Integration Tests
| Test | Expected result |
|---|
| Task dispatched end-to-end on cron boundary | OperationsReport emitted; target bot receives trigger on internal bus |
| Manifest reload after config change | New cron expressions take effect on next tick without restart |
Property Tests
| Property | Required behaviour |
|---|
| No task fires more than once per scheduled interval even under concurrent instances | Always true — distributed lock enforces single dispatch |
| Every dispatch produces an OperationsReport heartbeat | Always true |
27. Operational Runbook
CronRunner incidents fall into three categories: no dispatch (process down or bus failure), duplicate fires (lock race or multiple instances), or manifest errors (invalid expressions). All three block downstream governance tasks.
On-call actions
| Alert | First step | Diagnosis | Mitigation | Escalate to |
|---|
CronRunnerNoDispatchIn15m | Check CronRunner process status and internal bus connectivity. | | | Governance pod lead |
CronRunnerDuplicateFireRate | Check for multiple CronRunner instances; verify Redis lock is reachable. | | | SRE on-call |
CronRunnerBusFailures | Check internal message bus health; review bus failure logs for task_id. | | | SRE on-call |
CronRunnerDispatchLatencyHigh | Check Redis latency and internal bus throughput; reduce manifest size if overloaded. | | | Governance pod lead after 30 minutes |
Manual overrides
polytraders gov cron force-dispatch --task-id <id> — Manually trigger a missed cron task after investigating root cause.
Healthcheck
Endpoint: /internal/health/cron-runner | Green: Last dispatch < 65 minutes ago; manifest loaded; Redis lock reachable. | Red: No dispatch in 15 minutes; Redis unreachable; manifest parse errors.
29. Developer Checklist
Ready-to-ship score: 27/27 sections complete · 100%
| Requirement | Status |
|---|
| Purpose defined | ✓ done |
| Required inputs listed | ✓ done |
| Parameters defined | ✓ done |
| Defaults defined | ✓ done |
| Warning thresholds defined | ✓ done |
| Hard thresholds defined | ✓ done |
| Safe fallback defined | ✓ done |
| Structured output defined | ✓ done |
| Developer log defined | ✓ done |
| Plain-English explanation | ✓ done |
| Unit tests defined | ✓ done |
| Integration tests defined | ✓ done |
| Property tests defined | ✓ done |
| Failure-mode block complete | ✓ done |
| Reference implementation pseudocode | ✓ done |
| Wire examples (input + output) | ✓ done |
| Reason codes listed | ✓ done |
| Metrics & logs defined | ✓ done |
| State & persistence defined | ✓ done |
| Concurrency & idempotency defined | ✓ done |
| Dependencies declared | ✓ done |
| Security surfaces declared | ✓ done |
| Polymarket V2 compatibility declared | ✓ done |
| Version & migration history declared | ✓ done |
| Operational runbook defined | ✓ done |
| Promotion gates defined | ✓ done |
| Failure-injection recipes defined | ✓ done |