0.6 DuplicateMarketDetector

1. Bot Identity

2. Purpose

Detect semantically identical or dangerously overlapping Polymarket markets to prevent accidental correlated exposure and to surface cross-market arbitrage opportunities. Emits ObservationReports tagging each duplicate cluster with a similarity score.

3. Why This Bot Matters

• Duplicate markets not detected

  A strategy may take independent positions on two semantically identical markets, creating unintended double exposure that bypasses position-size limits.

• Near-duplicate neg-risk bundles missed

  Neg-risk outcome tokens across overlapping events can create correlated risk that is not visible from individual market inspection alone.

No worked examples on this bot yet. Worked examples are optional but strongly recommended — they turn an abstract failure mode into something a developer can verify in a fixture.

4. Required Polymarket Inputs

5. Required Internal Inputs

6. Parameter Guide

7. Detailed Parameter Instructions

{ "similarity_threshold": 0.85 }

if (score < params.hard) skip_pair();

{ "max_cluster_size": 10 }

if (cluster.size > params.hard) emit(LARGE_CLUSTER_WARN);

8. Default Configuration

{
  "bot_id": "disc.duplicate_market_detector",
  "version": "0.1.0",
  "mode": "shadow_only",
  "defaults": {
    "similarity_threshold": 0.85,
    "require_manual_review": false,
    "publish_to": [
      "disc.opportunityqueue"
    ],
    "max_cluster_size": 10
  }
}

9. Implementation Flow

1. On each detection cycle, fetch all active markets from Gamma API.
2. Check KillSwitch; if active, halt emissions.
3. Compute sentence embeddings for each market's (title + rules_text) using local embedding model.
4. Build a pairwise cosine similarity matrix across all candidate markets.
5. Cluster pairs with similarity >= similarity_threshold using Union-Find.
6. For clusters with size > max_cluster_size, emit LARGE_CLUSTER_WARN and flag for manual review if require_manual_review=true.
7. Emit one ObservationReport per duplicate cluster with market_ids, similarity_scores, and cluster_type (identical/overlap/negrisk_bundle).
8. Log cycle summary: total_pairs_evaluated, clusters_found, large_clusters.

10. Reference Implementation

Pseudocode is language-agnostic. FETCH = read input. EMIT = produce output. IF/THEN/ELSE = decision. Translate directly to TypeScript, Python, Go, or Rust.

FUNCTION detectionCycle():
  ks = FETCH internal.killswitch.status
  IF ks.active: RETURN

  candidates = FETCH disc.marketscanner.latest_candidates()
  markets = FETCH gamma.GET('/markets?ids=' + join(candidates.ids))
  IF markets IS NULL:
    LOG ERROR 'Gamma API unavailable — halting detection cycle'
    RETURN

  // Compute embeddings
  embeddings = {}
  FOR market IN markets:
    text = market.question + ' ' + (market.rules_text OR '')
    embeddings[market.condition_id] = embed_model.encode(text)

  // Pairwise similarity + clustering
  uf = UnionFind(markets.ids)
  FOR i, j IN pairs(markets):
    score = cosine(embeddings[i], embeddings[j])
    IF score >= params.similarity_threshold.hard:
      uf.union(i, j)

  FOR cluster IN uf.clusters(min_size=2):
    max_score = MAX(cosine(a,b) FOR a,b IN pairs(cluster))
    cluster_type = 'identical' IF max_score >= 0.85 ELSE 'overlap'
    warnings = []
    IF len(cluster) > params.max_cluster_size.default:
      warnings.append('LARGE_CLUSTER_WARN')
    IF max_score < params.similarity_threshold.default:
      warnings.append('POTENTIAL_OVERLAP')
    EMIT ObservationReport(cluster_id, cluster_type, cluster.ids, max_score, warnings)

  LOG detection cycle summary

SDK calls used

• gamma.GET('/markets?ids=<condition_id_list>')
• embed_model.encode(text)
• cosine(embedding_a, embedding_b)

Complexity: O(M²) where M = number of candidate markets; accelerated with FAISS ANN for large M

11. Wire Examples

Input — what arrives on the wire

Two near-duplicate markets from Gamma API — gamma_api

{
  "markets": [
    {
      "condition_id": "0x7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a",
      "question": "Will Candidate A win the Senate race?",
      "rules_text": "Resolves YES if Candidate A wins the Senate seat.",
      "resolution_date": "2026-11-04T00:00:00Z"
    },
    {
      "condition_id": "0x8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b",
      "question": "Will Candidate A win the Senate election?",
      "rules_text": "Resolves YES if Candidate A is elected to the Senate.",
      "resolution_date": "2026-11-04T00:00:00Z"
    }
  ]
}

Output — what the bot emits

ObservationReport — duplicate cluster detected

{
  "report_id": "0x1122334455667788990011223344556611223344556677889900112233445566",
  "bot_id": "disc.duplicate_market_detector",
  "cluster_id": "cluster-0001",
  "cluster_type": "identical",
  "market_ids": [
    "0x7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a",
    "0x8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b"
  ],
  "similarity_score": 0.93,
  "warnings": [],
  "detected_at_ms": 1746789000000
}

Reproduce locally

curl 'https://gamma-api.polymarket.com/markets?ids=0x7f8a...,0x8a9b...'

12. Decision Logic

13. Standard Decision Output

This bot returns a ObservationReport object. See ObservationReport schema.

{
  "report_id": "0x1122334455667788990011223344556611223344556677889900112233445566",
  "bot_id": "disc.duplicate_market_detector",
  "cluster_id": "cluster-0001",
  "cluster_type": "identical",
  "market_ids": [
    "0x7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a",
    "0x8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b"
  ],
  "similarity_score": 0.93,
  "warnings": [],
  "detected_at_ms": 1746789000000
}

14. Reason Codes

15. Metrics & Logs

Metrics emitted

Alerts

Dashboards

• Grafana — Discovery / DuplicateMarketDetector cluster overview

Log levels

16. Developer Reporting

{
  "bot_id": "disc.duplicate_market_detector",
  "cycle": 7,
  "markets_evaluated": 47,
  "pairs_compared": 1081,
  "clusters_found": 3,
  "large_clusters": 0,
  "killswitch_active": false,
  "detected_at": "2026-05-09T11:30:00Z"
}

17. Plain-English Reporting

18. Failure-Mode Block

19. Failure-Injection Recipes

20. State & Persistence

Cold-start recovery

On cold start, embeddings are recomputed from scratch on first cycle.

21. Concurrency & Idempotency

22. Dependencies

Depends on (must run first)

Emits to (downstream consumers)

Sibling bots (same OrderIntent)

External services

23. Security Surfaces

Abuse vectors considered

• Gamma API returning crafted market text designed to force a false duplicate cluster
• Embedding model poisoning via crafted input text

Mitigations

• Similarity threshold prevents low-confidence clusters from propagating
• Large-cluster warning and require_manual_review flag limit blast radius of false clusters

24. Polymarket V2 Compatibility

API surfaces declared

Networks supported

25. Versioning & Migration

Migration history

26. Acceptance Tests

Unit Tests

Integration Tests

Property Tests

27. Operational Runbook

DuplicateMarketDetector incidents are typically embedding model failures or large false clusters. Bot is read-only; incidents do not affect active positions.

On-call actions

Manual overrides

• —

Healthcheck

GET /internal/health/duplicatemarketdetector → green if Last detection cycle completed within 2× cycle interval; embedding model reachable.; red if No cycle in 2× interval or embedding model unreachable for >5 minutes.

28. Promotion Gates

A bot does not advance to the next readiness state until every gate below is green. Gates are observable from production data — no subjective sign-off.

Promote to Shadow

Promote to Limited live

Promote to General live

29. Developer Checklist

Ready-to-ship score: 27/27 sections complete · 100%