Protocol Specification

TopGun uses a custom WebSocket-based protocol for real-time synchronization, efficient delta updates, and distributed coordination. This document details the wire format and synchronization algorithms for developers building compatible clients.

Overview

Transport:WebSocket (Binary or JSON). Default is JSON for simplicity, MsgPack supported.

Model:Bidirectional, Event-driven. Client pushes ops, Server pushes updates.

Sync:Hybrid Logical Clocks (HLC) + Merkle Trees for convergence.

Message Format

All messages follow a standard envelope structure.

Message Envelope

{
  "type": "MESSAGE_TYPE",
  "payload": { ... },
  "timestamp": {
    "millis": 1678900000,
    "counter": 0,
    "nodeId": "client-1"
  },
  "requestId": "uuid..."
}

Connection Lifecycle

1. Handshake

Client connects to WebSocket. Server sends AUTH_REQUIRED.

2. Authentication

Client sends AUTH with JWT. Server responds with AUTH_ACK or AUTH_FAIL.

3. Synchronization

Client sends SYNC_INIT with last sync timestamp. Server responds with Merkle Root or Diff.

4. Live Updates

Bidirectional exchange of CLIENT_OP and SERVER_EVENT.

Message Types

Authentication

Client authentication flow with JWT tokens.

AUTH

{
  "type": "AUTH",
  "token": "eyJh..."
}

AUTH_ACK

{
  "type": "AUTH_ACK"
}

Data Operations

Operations are idempotent and commute based on HLC timestamps.

CLIENT_OP

{
  "type": "CLIENT_OP",
  "payload": {
    "mapName": "users",
    "key": "u1",
    "opType": "PUT", // or REMOVE, OR_ADD, OR_REMOVE
    "record": {
      "value": { "name": "Alice" },
      "timestamp": { ... }
    }
  }
}

OP_BATCH

{
  "type": "OP_BATCH",
  "payload": {
    "ops": [ ... ] // Array of CLIENT_OP payloads
  }
}

Query Subscriptions

Subscribe to live query results.

QUERY_SUB

{
  "type": "QUERY_SUB",
  "payload": {
    "queryId": "q1",
    "mapName": "users",
    "query": {
      "where": { "role": "admin" },
      "limit": 10
    }
  }
}

QUERY_UNSUB

{
  "type": "QUERY_UNSUB",
  "payload": {
    "queryId": "q1"
  }
}

Synchronization (Merkle)

TopGun uses a prefix-trie Merkle Tree. The tree depth is fixed (default 3), and buckets are determined by the hex characters of the key’s hash.

SYNC_INIT

Client requests sync state.

{
  "type": "SYNC_INIT",
  "mapName": "users",
  "lastSyncTimestamp": 1678000000
}

SYNC_RESP_ROOT

Server returns root hash.

{
  "type": "SYNC_RESP_ROOT",
  "payload": {
    "mapName": "users",
    "rootHash": 12345678,
    "timestamp": { ... }
  }
}

MERKLE_REQ_BUCKET

Client requests bucket hashes if root differs.

{
  "type": "MERKLE_REQ_BUCKET",
  "payload": {
    "mapName": "users",
    "path": "a1" // Hex path in trie
  }
}

Distributed Locks

Distributed locking with fencing tokens.

LOCK_REQUEST

{
  "type": "LOCK_REQUEST",
  "payload": {
    "requestId": "uuid",
    "name": "resource-A",
    "ttl": 5000
  }
}

LOCK_GRANTED

{
  "type": "LOCK_GRANTED",
  "payload": {
    "requestId": "uuid",
    "fencingToken": 101
  }
}

LOCK_RELEASE

{
  "type": "LOCK_RELEASE",
  "payload": {
    "requestId": "uuid",
    "name": "resource-A",
    "fencingToken": 101
  }
}

Pub/Sub

Real-time messaging with topics.

TOPIC_SUB

{
  "type": "TOPIC_SUB",
  "payload": {
    "topic": "chat"
  }
}

TOPIC_UNSUB

{
  "type": "TOPIC_UNSUB",
  "payload": {
    "topic": "chat"
  }
}

TOPIC_PUB

{
  "type": "TOPIC_PUB",
  "payload": {
    "topic": "chat",
    "data": { "msg": "hello" }
  }
}

TOPIC_MESSAGE

{
  "type": "TOPIC_MESSAGE",
  "payload": {
    "topic": "chat",
    "data": { "msg": "hello" },
    "publisherId": "client-2",
    "timestamp": 1678900000
  }
}

React Hooks