Protocol Specification

TopGun uses a custom WebSocket-based protocol for real-time synchronization, efficient delta updates, and distributed coordination. This document details the wire format and synchronization algorithms for developers building compatible clients.

Overview

Transport:WebSocket (Binary or JSON). Default is JSON for simplicity, MsgPack supported.

Model:Bidirectional, Event-driven. Client pushes ops, Server pushes updates.

Sync:Hybrid Logical Clocks (HLC) + Merkle Trees for convergence.

Message Format

All messages follow a standard envelope structure.

Message Envelope

{
  "type": "MESSAGE_TYPE",
  "payload": { ... },
  "timestamp": {
    "millis": 1678900000,
    "counter": 0,
    "nodeId": "client-1"
  },
  "requestId": "uuid..."
}

Connection Lifecycle

1. Handshake

Client connects to WebSocket. Server sends AUTH_REQUIRED.

2. Authentication

Client sends AUTH with JWT. Server responds with AUTH_ACK or AUTH_FAIL.

3. Synchronization

Client sends SYNC_INIT with last sync timestamp. Server responds with Merkle Root or Diff.

4. Live Updates

Bidirectional exchange of CLIENT_OP and SERVER_EVENT.

Message Types

Authentication

Client authentication flow with JWT tokens.

AUTH

{
  "type": "AUTH",
  "token": "eyJh..."
}

AUTH_ACK

{
  "type": "AUTH_ACK"
}

Data Operations

Operations are idempotent and commute based on HLC timestamps.

CLIENT_OP

{
  "type": "CLIENT_OP",
  "payload": {
    "mapName": "users",
    "key": "u1",
    "opType": "PUT", // or REMOVE, OR_ADD, OR_REMOVE
    "record": {
      "value": { "name": "Alice" },
      "timestamp": { ... }
    },
    "writeConcern": "PERSISTED", // Optional: FIRE_AND_FORGET | MEMORY | APPLIED | REPLICATED | PERSISTED
    "timeout": 5000              // Optional: timeout in ms
  }
}

OP_BATCH

{
  "type": "OP_BATCH",
  "payload": {
    "ops": [ ... ],              // Array of CLIENT_OP payloads
    "writeConcern": "APPLIED",   // Optional: batch-level Write Concern
    "timeout": 5000              // Optional: timeout in ms
  }
}

OP_ACK

Server acknowledgment with optional Write Concern level achieved.

{
  "type": "OP_ACK",
  "payload": {
    "lastId": "op-123",
    "achievedLevel": "PERSISTED",  // Optional: Write Concern level achieved
    "results": [                   // Optional: per-operation results
      {
        "opId": "op-123",
        "success": true,
        "achievedLevel": "PERSISTED",
        "latencyMs": 45
      }
    ]
  }
}

Write Concern Levels

Operations can specify a writeConcern to control acknowledgment:

Level	Description
FIRE_AND_FORGET	No acknowledgment, immediate return
MEMORY	Acknowledged when in server memory (default)
APPLIED	Acknowledged when CRDT merge complete
REPLICATED	Acknowledged when broadcast to peers
PERSISTED	Acknowledged when written to storage

See the Write Concern Guide for detailed usage.

Query Subscriptions

Subscribe to live query results.

QUERY_SUB

{
  "type": "QUERY_SUB",
  "payload": {
    "queryId": "q1",
    "mapName": "users",
    "query": {
      "where": { "role": "admin" },
      "limit": 10
    }
  }
}

QUERY_UNSUB

{
  "type": "QUERY_UNSUB",
  "payload": {
    "queryId": "q1"
  }
}

Synchronization (Merkle)

TopGun uses a prefix-trie Merkle Tree. The tree depth is fixed (default 3), and buckets are determined by the hex characters of the key’s hash.

SYNC_INIT

Client requests sync state.

{
  "type": "SYNC_INIT",
  "mapName": "users",
  "lastSyncTimestamp": 1678000000
}

SYNC_RESP_ROOT

Server returns root hash.

{
  "type": "SYNC_RESP_ROOT",
  "payload": {
    "mapName": "users",
    "rootHash": 12345678,
    "timestamp": { ... }
  }
}

MERKLE_REQ_BUCKET

Client requests bucket hashes if root differs.

{
  "type": "MERKLE_REQ_BUCKET",
  "payload": {
    "mapName": "users",
    "path": "a1" // Hex path in trie
  }
}

Distributed Locks

Distributed locking with fencing tokens.

LOCK_REQUEST

{
  "type": "LOCK_REQUEST",
  "payload": {
    "requestId": "uuid",
    "name": "resource-A",
    "ttl": 5000
  }
}

LOCK_GRANTED

{
  "type": "LOCK_GRANTED",
  "payload": {
    "requestId": "uuid",
    "fencingToken": 101
  }
}

LOCK_RELEASE

{
  "type": "LOCK_RELEASE",
  "payload": {
    "requestId": "uuid",
    "name": "resource-A",
    "fencingToken": 101
  }
}

Pub/Sub

Real-time messaging with topics.

TOPIC_SUB

{
  "type": "TOPIC_SUB",
  "payload": {
    "topic": "chat"
  }
}

TOPIC_UNSUB

{
  "type": "TOPIC_UNSUB",
  "payload": {
    "topic": "chat"
  }
}

TOPIC_PUB

{
  "type": "TOPIC_PUB",
  "payload": {
    "topic": "chat",
    "data": { "msg": "hello" }
  }
}

TOPIC_MESSAGE

{
  "type": "TOPIC_MESSAGE",
  "payload": {
    "topic": "chat",
    "data": { "msg": "hello" },
    "publisherId": "client-2",
    "timestamp": 1678900000
  }
}

React Hooks