Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Gossip Protocol

Overview

Nodes communicate via GossipSub (libp2p) using CBOR-encoded messages. Two topics are used:

  • p2p-mes/commands -- outgoing commands and broadcasts
  • p2p-mes/responses -- query responses

Message deduplication is handled at GossipSub level using BLAKE3 hash of message data as message_id.

Audience: node implementers. This page documents the node-to-node gossip layer. Client applications never speak gossip directly -- they use the HTTP API (Building a Client, API Reference). Read on to understand how nodes propagate and reconcile data, or to build a node.

GossipMessage Enum

All gossip traffic is a single CBOR-encoded enum:

#![allow(unused)]
fn main() {
enum GossipMessage {
    PutMessage(PutMessage),             // Store a new message
    InboxFanout(InboxFanout),           // Disabled (full replication)
    BatchedInboxFanout(BatchedInboxFanout), // Disabled (full replication)
    Query(Query),                       // Request data
    QueryResponse(QueryResponse),       // Respond to query
    Ack(Ack),                          // Deprecated
    ReadProgress(ReadProgress),         // Mark-as-read
    ReadProgressAck(ReadProgressAck),   // Mark-as-read acknowledgment
    MembershipOp(MembershipOp),        // Membership change
    MembershipOpBatch(Vec<MembershipOp>), // Batch of ops
    PutIdentity(PutIdentity),          // User identity propagation (HLC LWW)
    MessageOp(MessageOpRecord),        // Edit / delete an existing message
}
}

Encoding/decoding:

#![allow(unused)]
fn main() {
let bytes = gossip_msg.encode();          // serde_cbor::to_vec
let msg = GossipMessage::decode(&bytes);  // serde_cbor::from_slice
}

CBOR Serialization Format

GossipMessage uses serde's internally-tagged representation. Each variant serializes as a CBOR map with one key (the variant name) whose value is the variant's payload:

PutMessage example:
  {"PutMessage": {"msg_id": <bytes32>, "chat_id": <bytes32>, ...}}

MembershipOp example:
  {"MembershipOp": {"chat_id": <bytes32>, "target": <bytes20>, "sig": <bytes65>,
                     "role": 0, "op_type": 0, "hlc": 111669149696005}}
  // `hlc` is a packed HlcTimestamp: 48 bits physical_ms + 16 bits logical.

MembershipOpBatch example:
  {"MembershipOpBatch": [<MembershipOp>, <MembershipOp>, ...]}

CBOR type mapping:

Rust TypeCBOR Type
[u8; N]array of N unsigned integers (major type 4) -- NOT a byte string
Vec<u8>array of unsigned integers (major type 4) -- NOT a byte string
Stringtext string (major type 3)
u8, u32, u64unsigned integer (major type 0)
boolsimple value (major type 7): false=0xF4, true=0xF5
Option<T>null (0xF6) if None, T if Some
Vec<T>array (major type 4)
ChatKindmap {"t": "0"|"1"|"2", "d": {...}} -- the tag is a CBOR text string, not an integer (see TYPES.md)
MembershipOpTypeunsigned integer: Add=0, Remove=1, Create=2, TransferOwnership=3, DeleteGroup=4

Note: serde_cbor is the serialization library. Fields with #[serde(default)] are omitted when at default value on some paths -- implementors should handle both presence and absence.

Byte arrays are CBOR arrays, not byte strings. No serde_bytes annotation is used, so [u8; N] and Vec<u8> fields encode as CBOR arrays of u8 integers (major type 4), never byte strings (major type 2). The <bytes32> notation in examples above is shorthand for such an array. See decoding msg_cbor in the Client Guide for a worked decoder.

Message Types

PutMessage

Initiated by HTTP API. Broadcasts a new chat message to the network.

#![allow(unused)]
fn main() {
struct PutMessage {
    msg_id: [u8; 32],           // BLAKE3(chat_id || sender || hlc_packed_be || text)
    chat_id: [u8; 32],          // Chat identifier
    kind: ChatKind,             // Dm { peer } | Group { title } | Channel { title }
    sender: [u8; 20],           // Sender's address
    members: Option<Vec<[u8; 20]>>, // Chat members (for inbox fanout)
    text: String,               // Message text (empty for control messages)
    hlc: HlcTimestamp,          // Server-stamped HLC (storage/CRDT/sync)
    origin_wall_ts: u64,        // Frozen originator wall-clock (UI display)
    origin: String,             // PeerId of originating node
    needs_ack: bool,            // Always false (fire-and-forget mode)
    msg_type: u8,               // client-defined; 0 = regular text
                                // (registry in CLIENT_GUIDE.md)
    control: Option<Vec<u8>>,   // protocol-structure payload (CBOR, opaque)
}
}

Node-enforced fields (node validates and uses these):

  • msg_id -- computed by node, used for dedup and Merkle tree
  • chat_id -- used for routing, storage, membership checks
  • sender -- verified against auth signature
  • members -- used for inbox fanout (DM only; groups use members CF)
  • hlc -- stamped server-side by the originating API node's HlcState; drives the messages CF key, retention cutoff comparisons, and inbox last_ts. Receiver-side gossip handler feeds the value through HlcState::receive and drops the message on drift violation
  • origin_wall_ts -- frozen sender wall-clock; UI display only, never participates in distributed logic
  • origin -- used for gossip routing
  • kind -- set once by the originating API node and stored verbatim. For DMs, Dm.peer is the original recipient (the {peer} path parameter of the send request) -- it is not rewritten per reader; only user_inbox entries get a viewer-relative peer (see TYPES.md)

Client-opaque fields (node stores and relays without interpretation):

  • msg_type: u8 -- client-defined, node does not switch on this value
  • control: Option<Vec<u8>> -- opaque payload for client-to-client protocols
  • text: String -- fully opaque to the node (size-capped at the API only)

All msg_type values are a client convention, not a protocol requirement -- any u8 value is valid. 0 = plain message (set by the plain message endpoints); the control-type registry (10/20/21 and friends) lives in CLIENT_GUIDE.md.

Flow:

  1. HTTP handler creates PutMessage with computed msg_id
  2. Publishes to p2p-mes/commands
  3. All nodes receive, each stores via DbOp::PutMessage
  4. Dedup via seen_msg CF prevents double storage
  5. On store success, process_db_op updates user inboxes locally

Query / QueryResponse

Request-reply pattern over gossip for data retrieval.

#![allow(unused)]
fn main() {
struct Query {
    query_id: [u8; 16],        // Correlation ID
    kind: QueryKind,
    requester: String,          // PeerId of requester
}

enum QueryKind {
    GetChatRange {
        user: [u8; 20],
        chat_id: [u8; 32],
        from_ts: u64,
        to_ts: Option<u64>,
        after_key: Option<Vec<u8>>,
        limit: usize,
        #[serde(default)]
        reverse: bool,          // false = oldest-first; true = newest-first
    },
    ListUserChats {
        user: [u8; 20],
        limit: usize,
        after_key: Option<Vec<u8>>,
    },
}
}

reverse is tagged #[serde(default)]: peers predating the field send no key and decode it as false, so the CBOR wire format stays backward-compatible. Under full replication this query path is dormant (every node answers GetChatRange from its local store), but the field is threaded through so the sharded path stays correct if it is ever re-enabled.

#![allow(unused)]
fn main() {
struct QueryResponse {
    query_id: [u8; 16],        // Matches Query.query_id
    kind: QueryResponseKind,
}

enum QueryResponseKind {
    GetChatRange(GetChatRangeResponseRaw),
    ListUserChats(ListUserChatsResponseRaw),
}
}

Flow:

  1. MPSC handler publishes Query to p2p-mes/commands
  2. Stores PendingQuery with oneshot channel for response
  3. Any node with data publishes QueryResponse to p2p-mes/responses
  4. First response wins, sent back to HTTP client
  5. Timeout: 30 seconds, cleaned up by ack_timeout_check timer

ReadProgress / ReadProgressAck

Broadcasts read progress across nodes.

#![allow(unused)]
fn main() {
struct ReadProgress {
    progress_id: [u8; 16],     // Correlation ID
    user: [u8; 20],
    chat_id: [u8; 32],
    seq: u32,                   // Mark all messages up to this seq as read
    origin: String,             // PeerId of origin node
}

struct ReadProgressAck {
    progress_id: [u8; 16],
    from: String,               // PeerId of acknowledging node
}
}

Each node stores read progress in user_read_progress CF. Update is monotonic: only advances if seq > current.

MembershipOp

Protocol-level membership change. Travels via gossip, updates members CF only (never messages CF).

#![allow(unused)]
fn main() {
struct MembershipOp {
    chat_id: [u8; 32],          // Target group chat
    target: [u8; 20],           // User being added/removed/created-as/promoted
    sig: Vec<u8>,               // ECDSA signature (65 bytes)
    role: u8,                   // 0=participant, 1=admin (2=owner is derived, never sent)
    op_type: MembershipOpType,  // see enum below
    hlc: HlcTimestamp,          // Originator's HLC stamp (packed u64)
    title: Option<String>,      // Create only: group title (chat_id preimage)
    nonce: Option<Vec<u8>>,     // Create only: 16-byte creation nonce (chat_id preimage)
}

enum MembershipOpType {
    Add = 0,
    Remove = 1,
    Create = 2,             // signer becomes the owner (role=2)
    TransferOwnership = 3,  // target = new owner; old owner demoted to admin
    DeleteGroup = 4,        // target unused; tombstones all members
}
}

Client signature deliberately excludes hlc -- clients have no access to node-level HLC state. The canonical signature message is keccak256(chat_id || target || op_type_byte) exactly as before. The node stamps hlc server-side via its HlcState immediately before publishing, mirroring how the legacy ts: u64 was set by now_millis(). See md/TIME_AND_CONSISTENCY.md for the rationale.

Flow:

  1. HTTP API POST /groups/{chat_id}/ops verifies ECDSA signature
  2. For Create ops: API verifies chat_id == blake3(domain || signer || nonce || title) (title validated: <= 128 bytes UTF-8, no control characters; nonce must be exactly 16 bytes)
  3. MPSC handler stamps each op with ctx.hlc.stamp() (one stamp per op)
  4. MPSC handler checks role-based authorization via DB. For Create it independently re-verifies the chat_id binding from step 2 (the command channel is reachable without the HTTP layer)
  5. Duplicate Create rejected if group already has members
  6. All valid ops published as one GossipMessage::MembershipOpBatch. Create ops carry title + nonce so receivers can re-verify
  7. Gossip handler feeds incoming hlc into ctx.hlc.receive(), dropping the op if it exceeds local now by more than the configured max-drift bound (default 5 minutes). Then re-verifies sig + auth (don't trust other nodes); for Create it also recomputes the chat_id from (signer, nonce, title) -- a relaying node cannot forge the title or fabricate a Create for a foreign chat_id
  8. Stored via DbOp::MembershipOp { hlc, ... }. The DB writer routes to add_member_synced / remove_member_synced, which perform the CRDT merge with removed_at semantics (see STORAGE.md). TransferOwnership and DeleteGroup route to their own atomic DbOps (DbOp::TransferOwnership, DbOp::DeleteGroup) so the multi-record updates commit in a single WriteBatch each. Verified Creates additionally queue DbOp::SetGroupCreateMeta, which records the immutable creation trio (title, creator, nonce) in chats_meta (write-once)

Anti-entropy sync ships the full MemberInfo state (role, added_at, optional removed_at) per record via SyncMemberRecord. Receivers apply via DbOp::ApplyMemberRecord, which calls apply_member_record_synced for a monotonic CRDT merge that preserves tombstones either side might have observed independently. See SYNC.md.

Role model and authorization rules:

Three roles: participant (0), admin (1), owner (2). Exactly one owner per group -- Create makes the signer the owner. Both MPSC and gossip handlers enforce the same rules independently (don't trust other nodes):

OpRule
Add (new participant)signer must be admin or owner
Add (assign admin / change existing role)signer must be the owner
Add (target is the owner, or role=2 requested)always rejected
Remove (other)signer must be admin or owner; removing the owner is always rejected
Remove (self) / LeaveGroupallowed for participant and admin; owner cannot leave -- transfer first
TransferOwnershipsigner must be the owner; target must be an active member; self-transfer rejected. Old owner becomes admin, target becomes owner
DeleteGroupsigner must be the owner

DeleteGroup tombstones every active member (removed_at = hlc in one WriteBatch) and clears each member's inbox entry, so the chat disappears from /conversations for everyone on every node. Stored messages are left untouched -- the retention GC removes them on its own schedule.

Group creation via compound endpoint: Group creation is done via POST /groups/{chat_id}/ops with a Create op. The client generates a random 16-byte nonce, picks an optional title, computes chat_id = blake3(domain || signer || nonce || title), and signs the Create op. The API verifies the derivation. This eliminates the security gap where the old POST /groups endpoint generated chat_id server-side and the client couldn't sign it.

Group creation metadata (title, creator, nonce): A verified Create materializes the immutable creation trio into chats_meta via DbOp::SetGroupCreateMeta (write-once -- later attempts are no-ops). Because the title and nonce are part of the chat_id preimage and the creator's signature covers the chat_id, the trio is self-verifying: any node can recompute the hash and reject a forged copy regardless of who delivered it. Consequences:

  • The title is immutable for the lifetime of the group (rename would change the chat_id). A future mutable-metadata layer can be added on top without breaking this anchor.
  • The trio travels over two channels: the gossip Create op (live path) and the members-sync piggyback on the owner record (anti-entropy path, see SYNC.md). No separate propagation round is needed.
  • Inbox entries (user_inbox) and /conversations responses expose the title through ChatKind::Group { title }; the DB hydrates it from chats_meta at inbox-write time, so messages themselves never carry the title.

LeaveGroup reuses MembershipOp(Remove): DELETE /groups/{chat_id}/membership publishes a standard MembershipOp with op_type=Remove and target=sender. No new gossip variant was needed.

Duplicate Create protection: Both MPSC handler and gossip handler reject Create ops when list_members(chat_id, limit=1) returns any results. This prevents group hijacking: once a group is created and has members, no subsequent Create op can overwrite ownership.

MembershipOpBatch: GossipMessage::MembershipOpBatch(Vec<MembershipOp>) carries all ops from a single compound call in one gossip message. The receiver processes ops in order within the batch, ensuring atomic delivery (e.g. Create+Add arrives together, preventing split-brain where Add arrives before Create).

Authorization inside a batch uses an in-memory virtual-state overlay (see handlers/membership_batch.rs): a Create populates the overlay with the new owner, and a subsequent Add in the same batch consults the overlay before falling back to the on-disk members CF. Without this overlay an [Create, Add, Add] batch would fail because the async DB writer is FIFO and has not committed the Create by the time the Add is authorized. TransferOwnership swaps the two roles in the overlay, enabling [Create, Transfer, Remove(self)] -- "create a group, hand it over, walk away" -- in one request.

The overlay also tracks chats deleted within the batch: after a DeleteGroup op, every subsequent op for the same chat_id in that batch is rejected (the deletion is terminal for the request).

Individual MembershipOp messages are still supported for backward compatibility and single-op flows (e.g. LeaveGroup).

Two delivery channels for compound membership:

One HTTP call to POST /groups/{chat_id}/ops produces:

  • 1 GossipMessage::MembershipOpBatch message (all membership ops)
  • M GossipMessage::PutMessage messages (client data: MLS Welcome/Commit)

Membership ops are processed before accompanying messages.

sequenceDiagram
    participant C as Client
    participant API as HTTP API
    participant MPSC as MPSC Handler
    participant G as GossipSub
    participant DB as DB Writer

    C->>API: POST /groups/{chat_id}/ops
    API->>API: Verify ECDSA sig + nonce/title binding (Create only)
    API->>MPSC: Command::MembershipOp

    MPSC->>MPSC: Role-based authorization + duplicate Create guard
    MPSC->>G: MembershipOpBatch (all ops)
    MPSC->>DB: DbOp::MembershipOp (per op)

    loop Each accompanying message
        MPSC->>G: PutMessage
        MPSC->>DB: DbOp::PutMessage
    end

    Note over DB: process_db_op updates inboxes locally
    MPSC-->>C: 200 OK

Membership Operations

Membership operations use a dedicated MembershipOp gossip variant (not embedded in PutMessage with msg_type routing).

Operation Types (op_type)

OpValueDescription
Add0Add member to group chat (or change role -- owner only)
Remove1Remove member from group chat / self-remove (leave)
Create2Create group; signer becomes the owner
TransferOwnership3Hand ownership to target; old owner becomes admin
DeleteGroup4Tombstone all members, clear inboxes; owner only

MembershipPayload (control field)

#![allow(unused)]
fn main() {
struct MembershipPayload {
    target: [u8; 20],   // User being added/removed
    sig: Vec<u8>,        // ECDSA signature (65 bytes)
    role: u8,            // 0=participant, 1=admin (default 0)
}
}

Admin Signature Format

Canonical 53-byte message: chat_id[32] || target[20] || op_type[1]

op_type values: Add=0, Remove=1, Create=2, TransferOwnership=3, DeleteGroup=4. For DeleteGroup target carries no protocol meaning; clients conventionally sign their own address.

Signature: sign(keccak256(canonical_message)) -- standard Ethereum ECDSA with recovery (r[32] || s[32] || v[1]).

verify_membership_sig in crates/crypto recovers the signer address for caller to verify against admin.

group_chat_id Computation

#![allow(unused)]
fn main() {
fn group_chat_id(creator: [u8; 20], nonce: &[u8; 16], title: &str) -> [u8; 32] {
    BLAKE3("p2p-mes:chat:group:v2:" || creator[20] || nonce[16] || title_utf8)
}
}
  • nonce is 16 bytes of crypto-random data generated client-side; the fixed length keeps the preimage unambiguous without length prefixes and guarantees unique chat_ids under concurrent creation by the same creator.
  • title is the group's creation name, hashed as raw UTF-8 bytes with no Unicode normalization -- the client must sign over a chat_id computed from the exact bytes it submits. An unnamed group contributes zero title bytes.
  • Because the creator signs the chat_id and the chat_id commits to the title, the title is authenticated end-to-end and immutable.

The trio (creator, nonce, title) is stored in chats_meta on every node (see STORAGE.md) so it can be re-served to peers that missed the Create -- each receiver re-verifies the hash before accepting it.

CRDT add-wins Semantics

Membership uses add-wins conflict resolution:

  • Each member entry has added_at timestamp for ordering
  • If both add and remove exist for the same user at the same timestamp, add wins
  • add_member_crdt only skips if existing added_at is strictly greater
  • Removed members (key deleted) can always be re-added

This ensures convergence across nodes: all nodes applying the same set of membership messages in any order will reach the same state.

PutIdentity

Propagates a user identity blob to all nodes. Uses last-write-wins semantics under HLC.

#![allow(unused)]
fn main() {
struct PutIdentity {
    user: [u8; 20],     // User address
    blob: Vec<u8>,      // Opaque identity blob (max 1024 bytes)
    hlc: HlcTimestamp,  // Server-side HLC stamp of the originating write
    origin: String,     // PeerId of the originating node
}
}

Flow:

  1. HTTP PUT /identity reaches the MPSC handler, which stamps hlc via the node's HlcState and sends DbOp::PutIdentity to the async DB writer
  2. DB writer applies HLC-LWW, updates seen_identity sync index, notifies Merkle tree
  3. MPSC handler publishes PutIdentity to p2p-mes/commands (real-time gossip)
  4. All nodes receive; gossip handler first calls HlcState::receive(msg.hlc) -- the incoming HLC is rejected if it exceeds local now by more than DEFAULT_MAX_DRIFT_MS, otherwise the local clock advances -- then routes through DbOp::PutIdentity (same HLC-LWW pipeline)
  5. If incoming.hlc > stored.hlc (or no stored value): overwrite with [hlc_packed:u64be:8][blob]
  6. If incoming.hlc <= stored.hlc: silently discard (stale delivery)

Last-write-wins: hlc is stamped server-side by the originating API node's HlcState; clients never specify time. The client signature scheme is unchanged.

Merkle tree sync: Identity is included in the anti-entropy protocol (domain Identity). Nodes that were offline during gossip will receive the data via Merkle-tree sync; the receiver routes each SyncIdentityRecord through the same HLC-LWW pipeline, so resurrecting an older blob is impossible.

MessageOp

Edits or deletes an already-stored message. The design never mutates the synced set: an operation is an append-only record, and the message row clients read is a materialized projection of (original message + all its ops). Because the record set only grows and the projection is computed by an order-independent rule, a node that is behind can never resurrect an old version.

#![allow(unused)]
fn main() {
struct MessageOpRecord {
    target: [u8; 32],      // msg_id of the message being changed
    chat_id: [u8; 32],     // chat the target belongs to
    editor: [u8; 20],      // must equal the target's original sender
    op_kind: u8,           // 0 = Edit, 1 = Delete
    new_text: String,      // replacement text (empty for Delete)
    hlc: HlcTimestamp,     // server-stamped by the originating node
    sig: Vec<u8>,          // 65-byte client ECDSA signature
}
}

The record is used verbatim as both the gossip payload and the anti-entropy wire record -- there is no separate encoding, so the two paths can never drift.

Conflict resolution (the fold). Applying operations to a message is an order-independent rule, so every node converges regardless of delivery order:

  • an operation applies only if editor == target.sender, op_kind is known, and msg.hlc < op.hlc <= msg.hlc + 48h (the edit window);
  • a Delete is terminal -- once a message is deleted, no later Edit revives it;
  • otherwise an Edit wins only if its hlc beats the edit currently in effect (ties broken by op_id bytes).

All comparisons use stamps carried inside the records, never a local clock, so the verdict is identical on every node.

Signature -- delete only. A delete carries a client ECDSA signature over keccak256(chat_id || target || op_kind || blake3("")); every node independently recovers the signer and requires it to equal editor. As with MembershipOp, the hlc is not signed (the node stamps it server-side).

An edit carries no signature (sig is empty). This is the same asymmetry as plain messages, and for the same reason: an edit writes new content, and content authenticity is the recipient's job, verified end-to-end on Layer 2 (the ciphertext is a sign-then-encrypt envelope -- see CLIENT_GUIDE.md). The node only relays it; a forged edit yields an envelope the recipient cannot verify and rejects. At the node, an edit is authorized exactly like a send: the originating node checks editor == target.sender against the authenticated request (X-Sig).

A delete is different in kind and is the one operation the node must authorize itself. It produces the absence of content: a recipient sees only a deleted: true stub and cannot tell "the author deleted this" from "a node erased it without permission" -- there is nothing left to authenticate end-to-end. So the delete is signed, the signature travels with the operation, and every node re-verifies it (a peer is never trusted to have checked). Blake3-hashing the empty text keeps the payload fixed-length and identical in shape to a would-be signed edit, so the scheme stays general even though only deletes are signed today.

Flow:

  1. HTTP PATCH/DELETE /messages/{msg_id} reaches the MPSC handler, which loads the target, checks authorship and the 48h window, stamps hlc, routes DbOp::MessageOp to the DB writer, then publishes MessageOp to p2p-mes/commands (live path)
  2. The DB writer records the op in CF message_ops, updates the seen_op sync index and the ops_by_chat feed, and -- if the target is stored locally and the op wins the fold -- rewrites the message row in place, all in one WriteBatch
  3. Gossip receivers feed hlc through HlcState::receive, re-verify the signature, then route the same DbOp::MessageOp. Dedup by op_id makes double delivery (gossip + sync) idempotent
  4. If the op arrives before the message it targets (normal during anti-entropy backfill), it simply waits in message_ops; the later-arriving message folds it in on write via fold_ops_for_message

Two delivery channels. Gossip is the live path (online nodes apply within milliseconds); the MessageOps anti-entropy domain is the backstop for nodes that were offline when the op was published. Neither alone suffices -- gossip is fast but lossy, sync is reliable but runs only every few sync ticks -- and double delivery is safe because both converge on one DbOp::MessageOp behind the seen_op dedup. See SYNC.md for the domain and RETENTION.md for how aged ops are reclaimed.

Materialized row. After an Edit the message's text is replaced and edited_at is set; after a Delete text is cleared and deleted is set, but the row survives as a stub (same key, same msg_id) so pagination stays stable and clients can evict cached copies. Readers never fold ops themselves -- they see a plain list of messages.

Deprecated and disabled variants

These variants remain in the GossipMessage enum for CBOR backward compatibility but are not part of the active protocol. New node implementations neither send nor process them, and clients never see them.

InboxFanout / BatchedInboxFanout (disabled)

Disabled under full replication. Because every node stores all messages, each node updates user inboxes locally inside process_db_op right after writing PutMessage, so explicit gossip fanout is unnecessary. If sharding is ever re-introduced (where the node storing a message may differ from the node owning a user's inbox), these would need to be re-enabled.

#![allow(unused)]
fn main() {
struct InboxFanout {
    users: Vec<[u8; 20]>,       // All chat members (targets for inbox update)
    chat_id: [u8; 32],
    kind: ChatKind,
    last_sender: [u8; 20],
    last_ts: u64,
    last_seq: u32,
    last_msg_id: [u8; 32],
}

struct BatchedInboxFanout {
    items: Vec<InboxFanout>,    // Up to 100 items per batch
}
}

Ack (deprecated)

Legacy acknowledgment for PutMessage. No longer used -- all messages are fire-and-forget (needs_ack = false).

#![allow(unused)]
fn main() {
struct Ack {
    msg_id: [u8; 32],
    from: String,
    response: PutMessageResponseRaw,
}
}

CBOR Compatibility Rules

When modifying gossip message types:

  • Add new fields with #[serde(default)] -- safe, backward compatible
  • Never remove fields -- old nodes will fail to deserialize
  • Never change enum tag names or numbering
  • Never change field types
  • New enum variants are safe -- unknown variants cause deserialization errors, but senders should be updated first

msg_id Computation

#![allow(unused)]
fn main() {
fn compute_msg_id(chat: &[u8], sender: &[u8], hlc: HlcTimestamp, text: &str) -> [u8; 32] {
    BLAKE3(chat || sender || hlc.to_packed().to_be_bytes() || text.as_bytes())
}
}

Same inputs always produce the same msg_id, enabling idempotent storage across nodes. The packed HLC is hashed as 8 big-endian bytes -- same shape as the legacy ts: u64, so the resulting digest stays 32 bytes.

msg_id is a stable identity, not a live content hash. It is computed once from the original text and never recomputed. Once a message is edited its stored text no longer hashes to its msg_id, and that is intentional: the id stays fixed so dedup, the messages Merkle tree, and every cross-node reference keep working. The node has never re-derived msg_id from stored content (messages carry no client signature; trust sits at the API edge), so nothing depends on the former msg_id == BLAKE3(current text) coincidence. Edits and deletes travel as MessageOp records in their own sync domain instead of perturbing the messages tree.

op_id Computation

#![allow(unused)]
fn main() {
fn compute_op_id(
    target: &[u8; 32], editor: &[u8; 20], op_kind: u8,
    hlc: HlcTimestamp, new_text: &str,
) -> [u8; 32] {
    BLAKE3("p2p-mes:msgop:v1:" || target || editor || [op_kind]
           || hlc.to_packed().to_be_bytes() || new_text.as_bytes())
}
}

The deterministic id of a MessageOp, used for dedup (seen_op) and as the MessageOps Merkle leaf. The signature is deliberately excluded: it is not guaranteed byte-identical across signers of the same payload, and hashing it in would let re-signing mint a second id for one logical operation.

DM chat_id Computation

#![allow(unused)]
fn main() {
fn dm_chat_id(a: [u8; 20], b: [u8; 20]) -> [u8; 32] {
    let (lo, hi) = if a <= b { (a, b) } else { (b, a) };
    BLAKE3("p2p-mes:chat:dm:v1:" || lo || hi)
}
}

Deterministic: both participants compute the same chat_id. No membership storage needed for DMs.

Identify-Based API Discovery

Nodes advertise their public HTTP API endpoint to peers through the libp2p identify protocol -- not through gossip. Identify metadata is exchanged automatically on every connection, so the announcement costs no extra network traffic and needs no dedicated message type.

Wire format. A node with public_api_url set in its config embeds the URL into the identify agent_version string as a whitespace-delimited token:

p2p-mes/1.0.0 api=https://node1.example.com:8080

Nodes without a public_api_url send a plain p2p-mes/1.0.0.

Receiving side. On each identify Received event the node parses the peer's agent_version (crates/node/src/handlers/kad/identify.rs):

  • A well-formed api=<url> token (http/https scheme, <= 256 bytes) is recorded in the in-memory API registry (PeerId -> URL)
  • A missing or malformed token removes any previous registry entry for that peer (the peer may have restarted without a public_api_url)
  • The entry is evicted when the last connection to the peer closes

Presence in the registry therefore means "connected right now" -- no TTL or liveness probing is needed. The registry is ephemeral state: it is never persisted, never synced, and rebuilds itself from identify exchanges within seconds of a restart.

Trust. The URL is self-reported by the peer over an authenticated libp2p connection (the transport proves the sender owns its PeerId). Consistent with the trust model (see PHILOSOPHY.md), a dishonest node could advertise an arbitrary URL; the network assumes honest replicas.

Client surface. The registry backs the public HTTP endpoint GET /network/nodes (see API.md), which clients use for bootstrap and failover instead of hardcoding node URLs.