Core
@topgunbuild/core ships the CRDT primitives, Hybrid Logical Clock, and Merkle tree that the client and server share. This page documents the public surface (the “what / how”). For background on why TopGun picks these primitives over alternatives, see Concepts → Data Structures.
The two primary data structures are LWWMap (last-write-wins) and ORMap (observed-remove). Both are exposed on TopGunClient via getMap() and getORMap() — most application code never instantiates them directly.
LWWMap
import { LWWMap, HLC } from '@topgunbuild/core';
const hlc = new HLC('node-1');
const map = new LWWMap<string, User>(hlc);
Last-write-wins semantics: each key holds exactly one value. Concurrent writes from different nodes are resolved by HLC timestamp (highest wins, ties broken by nodeId).
Constructor
new LWWMap<K, V>(hlc: HLC)
Methods
set(key, value, ttlMs?) — LWWRecord<V>
Write a value. Stamps the record with hlc.now(). Pass ttlMs to make the record expire on its own: see Expiring records.
const record = map.set('u1', { name: 'Alice', email: '[email protected]' });
// { value: {...}, timestamp: { millis, counter, nodeId } }
// With 1-hour TTL
map.set('session-123', sessionData, 3600000);
get(key) — V | undefined
Read a value. Returns undefined for deleted or expired keys.
getRecord(key) — LWWRecord<V> | undefined
Read the full record (value, timestamp, optional TTL).
remove(key) — LWWRecord<V>
Delete a key by writing a tombstone (value: null). Returns the tombstone record.
merge(key, remoteRecord) — boolean
Merge a remote record. Returns true if local state was updated (i.e., the remote record’s timestamp dominated).
const remote: LWWRecord<User> = {
value: { name: 'Bob' },
timestamp: { millis: Date.now(), counter: 0, nodeId: 'node-2' },
};
const updated = map.merge('u1', remote);
subscribe(callback) — () => void
Subscribe to map-change notifications. The callback receives entries: Array<[K, V]>. Returns an unsubscribe function.
const unsubscribe = map.subscribe((entries) => {
console.log('Map changed,', entries.length, 'live entries');
});
entries() — IterableIterator<[K, V]>
Iterate non-tombstoned entries (drops deleted and expired).
allKeys() — IterableIterator<K>
Iterate every key including tombstones.
size (getter) — number
Count of entries including tombstones.
prune(olderThan) — K[]
Drop tombstones older than the given Timestamp. Returns the removed keys.
pruneExpired(skip?) — K[]
Drop every live record whose TTL has run out (timestamp.millis + ttlMs is before now) from the map and its Merkle tree, and return the removed keys. Keys for which skip(key) returns true are kept (still hidden by get). Like prune, it is local housekeeping: it writes no tombstone and sends nothing. TopGunClient runs it for you (see Expiring records); call it yourself only when you use LWWMap without the client.
clear() — void
Reset to empty state.
getMerkleTree() — MerkleTree
Access the underlying Merkle tree for delta-sync diffing.
LWWRecord shape
interface LWWRecord<V> {
value: V | null; // null indicates tombstone
timestamp: Timestamp; // HLC stamp
ttlMs?: number; // optional TTL
}
ORMap
import { ORMap, HLC } from '@topgunbuild/core';
const hlc = new HLC('node-1');
const map = new ORMap<string, string>(hlc);
Observed-remove semantics: each key holds a set of values. Concurrent adds preserve every value (tagged with unique IDs). A remove operation tombstones only the tags it observed locally, so concurrent add-vs-remove resolves correctly (no spurious deletes).
Use this for tag clouds, comment threads, multi-select fields, presence rosters — anywhere two clients can legitimately contribute different values to the same key.
Constructor
new ORMap<K, V>(hlc: HLC)
Methods
add(key, value, ttlMs?) — ORMapRecord<V>
Add a value under a key. Generates a unique tag. Returns the record. Pass ttlMs to make this value expire on its own: see Expiring records.
const tags = new ORMap<string, string>(hlc);
tags.add('post:123', 'javascript');
tags.add('post:123', 'typescript');
// post:123 -> ['javascript', 'typescript']
remove(key, value) — string[]
Remove a specific value under a key. Returns the tombstone tags created.
get(key) — V[]
Read all live values for a key.
const postTags = tags.get('post:123');
// ['javascript', 'typescript']
getRecords(key) — ORMapRecord<V>[]
Read full records (value + tag + timestamp) for a key.
getTombstones() — string[]
Read all current tombstone tags.
apply(key, record) — boolean
Apply a remote ORMapRecord to local state. Returns true if state changed.
applyTombstone(tag, key?) — void
Apply a remote tombstone tag. The tag is suppressed under every key either way. Pass the key the removal belongs to whenever you know it: the tombstone is then also recorded against that key, which is what lets a removal that emptied the key still be compared with the server during sync. Without key the tombstone is only recorded map-wide.
import { ORMap, HLC } from '@topgunbuild/core';
const remote = new ORMap<string, string>(new HLC('node-2'));
const record = remote.add('post:123', 'draft');
// Replaying a removal that happened elsewhere:
remote.applyTombstone(record.tag, 'post:123');
remote.get('post:123'); // []
remote.getKeyTombstones('post:123'); // Set { record.tag }
getKeyTombstones(key) — Set<string>
Read the tombstone tags recorded against one key. Returns a copy; every tag in it is also in getTombstones(). remove(key, value) records its tombstones against key automatically.
setKeyTombstones(key, tags) — void
Replace the tombstone tags recorded against one key — used to mirror the set an authoritative peer reports for that key. Any record still carrying one of the tags is removed. A tag dropped from the key’s set stays in getTombstones(): the map-wide set only grows here. Passing no tags clears the key’s set.
merge(other) — void
Merge another ORMap of the same key/value types in bulk.
subscribe(callback) — () => void
Subscribe to change events. The callback receives entries: Array<[K, V[]]>.
allKeys() — K[]
All keys that currently hold at least one record. A key left with only tombstones is not listed.
size / totalRecords (getters) — number
size counts distinct keys; totalRecords counts every tagged record (including duplicates per key).
prune(olderThan) — string[]
Drop tombstones older than the timestamp. Returns dropped tags.
clear() — void
Reset to empty.
getMerkleTree() — ORMapMerkleTree
Underlying Merkle tree (a different shape than LWWMap’s tree because ORMap stores multi-value-per-key). A key contributes to it while it holds at least one record or has at least one tombstone recorded against it, so a key emptied by a removal still shows up in a sync comparison.
getSnapshot() — ORMapSnapshot<K, V>
The map’s internal state: items (key → tag → record), tombstones (every removed tag) and keyTombstones (key → the tombstone tags recorded against it). All three fields are required, so code that builds an ORMapSnapshot by hand must supply keyTombstones — an empty Map when there are none.
isTombstoned(tag) — boolean
Check whether a tag is in the tombstone set.
ORMapRecord shape
interface ORMapRecord<V> {
value: V;
tag: string; // unique add ID
timestamp: Timestamp; // HLC stamp
ttlMs?: number;
}
Expiring records
A record written with ttlMs disappears by itself: for sessions, invitations, one-time codes, presence and anything else that should not outlive a deadline. map.set(key, value, ttlMs) and orMap.add(key, value, ttlMs) both take it.
- When it expires. The TTL counts from the moment the server stores the write, not from when the client made it. Once that moment plus
ttlMshas passed, the server and every client hide the record: it is left out of query snapshots, HTTP/sync, search andget. A live query that already shows it drops it when the server deletes it (next point). - Then it is deleted. The server deletes the expired record the ordinary way, so live-query subscribers see it leave, the event journal records a delete with
reason: "expired", and every client drops it from its local map (anLWWMapby itself, anORMapwhen it receives the server’s delete). The delete can come up to the server’s sweep interval (TOPGUN_EXPIRY_SWEEP_INTERVAL_MS, 10 s by default) plus one sweep cycle after the record expired; until then it is already hidden. - Clocks. Each reader compares the deadline with its own clock, so readers whose clocks differ can disagree about an expiring record for as long as their clocks differ.
- Offline writes. A client hides its own entries that the server has not stored yet (unsynced, or written offline) by its local write time, which is earlier than the moment the server stores them. So such an entry can disappear on that client earlier than on the server and on other clients, by up to the time the client was offline. This affects only that client’s own view, and it ends when the client takes the server’s record (
LWWMap) or receives the server’s delete (ORMap). - Older records. Records written before servers kept the TTL do not expire.
TopGunClient drops expired LWWMap records from memory and from local storage once a second while at least one LWWMap is open, also while offline, and before it compares a map with the server, so its sync state matches the server’s after the server deleted them. A record that still has an unsynced local write is kept until the server has it.
HLC (Hybrid Logical Clock)
import { HLC } from '@topgunbuild/core';
const hlc = new HLC('node-1');
const ts = hlc.now();
// { millis: 1748025600000, counter: 0, nodeId: 'node-1' }
The HLC combines physical wall-clock time (millis) with a logical counter (counter) so total ordering is preserved across nodes even when clocks drift.
Constructor
new HLC(nodeId: string, options?: HLCOptions)
nodeId names this clock and becomes part of every timestamp it issues. It must not contain :, | or # — the constructor throws an Error if it does. Letters, digits, - and _ are always fine, and so is a UUID.
HLCOptions fields:
| Field | Default | Description |
|---|---|---|
strictMode | false | Throw on clock drift exceeding maxDriftMs. |
maxDriftMs | 60000 | Permitted drift threshold (ms). |
clockSource | wall clock | Custom ClockSource for deterministic tests. |
Methods
now() — Timestamp
Issue a fresh timestamp. Monotonic per node.
update(remote) — void
Update the local clock against a received remote timestamp. Bumps lastMillis/lastCounter as needed to maintain causal ordering.
getClockSource() — ClockSource
Returns the configured clock source.
getNodeId / getStrictMode / getMaxDriftMs (getters)
Inspect configured fields.
Static helpers
HLC.compare(a, b) — number
Total-order comparator. Returns negative if a < b, zero if equal, positive if a > b. Ordered by millis, then counter, then nodeId.
HLC.toString(ts) — string
Canonical string encoding for serialization or sort keys.
HLC.parse(str) — Timestamp
Inverse of toString.
HLC.assertValidNodeId(nodeId) — void
Check a node id without creating a clock. Throws the same Error as the constructor if nodeId contains :, | or #, and returns nothing otherwise. Useful for validating an id that comes from configuration or user input before you build anything with it.
Timestamp shape
interface Timestamp {
millis: number;
counter: number;
nodeId: string;
}
MerkleTree
import { MerkleTree } from '@topgunbuild/core';
const tree = new MerkleTree();
The Merkle tree powers delta sync: clients and servers compare hash trees and exchange only differing buckets instead of full state. LWWMap and ORMap maintain their own trees internally — use this directly only when implementing custom sync providers or diagnostics.
Constructor
new MerkleTree(records?: Map<string, LWWRecord<any>>, depth = 3)
depth controls fan-out (default 3 levels deep).
Methods
update(key, record) — void
Insert or replace the leaf for key and rehash the path to the root.
remove(key) — void
Drop the leaf and rehash.
getRootHash() — number
Top-level hash. Two trees with identical content produce the same root.
getNode(path) — MerkleNode | undefined
Return the node at a given path string (used during diff walks).
getBuckets(path) — Record<string, number>
Per-bucket hashes for the children of the node at path.
getKeysInBucket(path) — string[]
Leaf keys grouped under the given path.
Re-exports
@topgunbuild/core also re-exports types and helpers consumed across the stack. Highlights:
Predicates— composable filter builders (equal,gt,lt,contains,and,or,not,match,matchPhrase,matchPrefix). Used byclient.query,client.hybridQuery, and the React hooks.SearchOptions— shared FTS options shape used byclient.search/searchSubscribeanduseSearch.MergeRejection— event shape emitted when the built-in CRDT merge logic rejects a remote change. Observed viauseMergeRejections(React) orclient.getConflictResolvers().onRejection().WriteConcern/ConsistencyLevel— durability and consistency knobs for cluster writes.IndexedLWWMap/IndexedORMap— indexed variants used by the server’s query engine.- Full-Text Search —
Tokenizer,InvertedIndex,BM25Scorer,FullTextIndexfor embedding FTS directly. - Deterministic Simulation Testing —
VirtualClock,SeededRNG,VirtualNetwork,InvariantChecker,ScenarioRunnerfor property-based distributed tests.
The complete re-export list lives in packages/core/src/index.ts.