.gitignore

* agent: use primary key index in setRcvServiceAssocs

Previous WHERE rcv_id = ? did not match the (host, port, rcv_id)
primary key prefix and fell back to a table scan via
idx_rcv_queues_client_notice_id. With ~390k rows per queue, each
update in a 1350-row batch scanned the whole table, yielding ~290s
per batch and a multi-hour rcv-services migration.

* agent: pass SMPServer explicitly to setRcvServiceAssocs

Avoid extracting host/port from the first queue inside setRcvServiceAssocs.
The caller already has SMPServer in scope (from tSess) and the call chain
is short, so threading it through is simpler than inspecting the list.
Removes the empty-list guard from setRcvServiceAssocs (it remains in
processRcvServiceAssocs).

.github/workflows/build.yml

@@ -9,10 +9,6 @@ on:
       - "v*"
   pull_request:
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: ${{ !startsWith(github.ref, 'refs/tags/v') }}
-
 jobs:
 
 # =============================
@@ -28,8 +24,6 @@ jobs:
       - name: Clone project
         if: startsWith(github.ref, 'refs/tags/v')
         uses: actions/checkout@v3
-        with:
-          submodules: recursive
 
       - name: Build changelog
         id: build_changelog
@@ -120,8 +114,6 @@ jobs:
       - name: Clone project
         if: matrix.should_run == true
         uses: actions/checkout@v3
-        with:
-          submodules: recursive
 
       - name: Set up Docker Buildx
         if: matrix.should_run == true
@@ -305,29 +297,3 @@ jobs:
             echo "All "$attempts" attempts failed."
             exit 1
           fi
-
-# =============================
-#      Resolver test job
-# =============================
-
-# The SNRC resolver is Python, so this job needs none of the Haskell toolchain
-# above and runs independently of it.
-
-  resolver-test:
-    name: "resolver (python)"
-    runs-on: ubuntu-latest
-    steps:
-      - name: Clone project
-        uses: actions/checkout@v3
-
-      - name: Set up uv
-        # uv takes Python from .python-version, the version of the Dockerfile's runtime stage.
-        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
-
-      - name: Lint
-        working-directory: scripts/resolver/service
-        run: uv run --frozen ruff check .
-
-      - name: Test
-        working-directory: scripts/resolver/service
-        run: uv run --frozen pytest -v

.github/workflows/docker-image.yml

@@ -20,8 +20,6 @@ jobs:
     steps:
       - name: Clone project
         uses: actions/checkout@v4
-        with:
-          submodules: recursive
 
       - name: Log in to Docker Hub
         uses: simplex-chat/docker-login-action@v3

.github/workflows/reproduce-schedule.yml

@@ -11,8 +11,6 @@ jobs:
     steps:
       - name: Checkout code
         uses: actions/checkout@v3
-        with:
-          submodules: recursive
 
       - name: Get latest release
         shell: bash

.gitignore

@@ -1,8 +1,4 @@
 *.lock
-# the resolver's Docker image and CI install exactly these
-!/scripts/resolver/service/uv.lock
-# local overrides of scripts/resolver/.env.example
-/scripts/resolver/.env
 *.db
 *.db.bak
 *.session.sql
@@ -16,4 +12,3 @@ cabal.project.local~
 *.tix
 .coverage
 
-__pycache__/

.gitmodules

@@ -1,6 +0,0 @@
-[submodule "cbits/libbbs"]
-	path = cbits/libbbs
-	url = https://github.com/simplex-chat/libbbs.git
-[submodule "cbits/blst"]
-	path = cbits/blst
-	url = https://github.com/supranational/blst.git

CODE.md

@@ -0,0 +1,210 @@
+# simplexmq — LLM Navigation Guide
+
+This file is the entry point for LLMs working on simplexmq. Read it before making any code changes.
+
+## Three-Layer Architecture
+
+simplexmq maintains three documentation layers alongside source code:
+
+| Layer | Directory | Answers | Audience |
+|-------|-----------|---------|----------|
+| **Product** | `product/` | What does this do? Who uses it? What must never break? | Anyone reasoning about behavior, privacy, security |
+| **Spec** | `spec/` | How does the code work? What does each function do? What are the security invariants? | LLMs and developers modifying code |
+| **Protocol** | `protocol/` | What is the wire protocol? What are the message formats and state machines? | Protocol implementors, formal verification |
+
+Additionally:
+- `rfcs/` — Protocol evolution: each RFC describes a delta to a protocol spec
+- `product/threat-model.md` — Comprehensive threat model across all protocols
+- `spec/security-invariants.md` — Every security invariant with enforcement and test coverage
+
+## Navigation Workflow
+
+When modifying code, follow this sequence:
+
+1. **Identify scope** — Find the relevant component in `product/concepts.md`
+2. **Load product context** — Read the component file in `product/components/` to understand what users depend on
+3. **Load spec context** — Read the relevant `spec/` file(s) for implementation details and call graphs
+4. **Check security** — Read `spec/security-invariants.md` for any invariants enforced by the code you're changing
+5. **Load source** — Read the actual source files referenced in spec/
+6. **Identify impact** — Trace the call graph to understand what your change affects
+7. **Implement** — Make the change
+8. **Update all layers** — Update spec/, product/, and protocol/ (if wire protocol changed) to stay coherent
+
+## Protocol Specifications
+
+Consolidated protocol specs live in `protocol/`. These describe the wire protocols as originally specified. Code has advanced beyond these versions — Phase 2 of this project will synchronize them.
+
+| File | Protocol | Spec version | Code version |
+|------|----------|-------------|--------------|
+| `simplex-messaging.md` | SMP (simplex messaging) | v9 | SMP relay v18, SMP client v4 |
+| `agent-protocol.md` | Agent (duplex connections) | v5 | Agent v7 |
+| `xftp.md` | XFTP (file transfer) | v2 | XFTP v3 |
+| `xrcp.md` | XRCP (remote control) | v1 | RCP v1 |
+| `push-notifications.md` | Push notifications | v2 | NTF v3 |
+| `pqdr.md` | PQDR (post-quantum double ratchet) | v1 | E2E v3 |
+| `overview-tjr.md` | Cross-protocol overview | — | — |
+
+Note: SMP has multiple version axes — `VersionSMP` (relay/transport, currently 18), `VersionSMPC` (client protocol, currently 4), and `VersionSMPA` (agent, currently 7). These are negotiated independently.
+
+Protocol specs are amended in place when implementation changes. RFCs in `rfcs/` track the evolution history.
+
+## Source Structure
+
+```
+src/Simplex/
+  Messaging/
+    Protocol.hs, Protocol/Types.hs      — SMP wire protocol types + encoding
+    Client.hs                            — SMP client (protocol operations, proxy relay)
+    Client/Agent.hs                      — Low-level async SMP agent
+    Server.hs                            — SMP server request handling
+    Server/Env/STM.hs                    — Server environment + STM state
+    Server/Main.hs, Server/Main/Init.hs  — Server CLI + initialization
+    Server/QueueStore/                   — Queue storage (STM, Postgres)
+    Server/MsgStore/                     — Message storage (STM, Journal, Postgres)
+    Server/MsgStore/Journal.hs           — Journal message store (1000 lines)
+    Server/StoreLog/                     — Store log (append-only write, read-compact-rewrite restore)
+    Server/NtfStore.hs                   — Message notification store
+    Server/Control.hs, Server/CLI.hs     — Control protocol + CLI utilities
+    Server/Stats.hs, Server/Prometheus.hs — Metrics
+    Server/Information.hs                — Server public information / metadata
+    Agent.hs                             — SMP agent: duplex connections, queue rotation
+    Agent/Client.hs                      — Agent's SMP/XFTP/NTF client management
+    Agent/Protocol.hs                    — Agent wire protocol types + encoding (2200 lines)
+    Agent/Store.hs                       — Agent storage types (queues, connections, messages)
+    Agent/Store/AgentStore.hs              — Agent storage implementation (3500 lines)
+    Agent/Store/                          — Agent storage backends (SQLite, Postgres)
+    Agent/Env/SQLite.hs                  — Agent environment + configuration
+    Agent/NtfSubSupervisor.hs            — Notification subscription management
+    Agent/TSessionSubs.hs                — Transport session subscriptions
+    Agent/Stats.hs                       — Agent statistics
+    Agent/RetryInterval.hs               — Retry interval logic
+    Agent/Lock.hs                        — Named locks
+    Agent/QueryString.hs                 — Query string parsing
+    Transport.hs                         — TLS transport abstraction + handshake
+    Transport/Client.hs, Transport/Server.hs — TLS client + server
+    Transport/HTTP2.hs                   — HTTP/2 transport setup
+    Transport/HTTP2/Client.hs            — HTTP/2 client
+    Transport/HTTP2/Server.hs            — HTTP/2 server
+    Transport/HTTP2/File.hs              — HTTP/2 file streaming
+    Transport/WebSockets.hs              — WebSocket adapter
+    Transport/Buffer.hs                  — Transport buffering
+    Transport/KeepAlive.hs               — TCP keepalive
+    Transport/Shared.hs                  — Certificate chain validation
+    Transport/Credentials.hs             — TLS credential generation
+    Crypto.hs                            — All cryptographic primitives
+    Crypto/File.hs                       — File encryption (NaCl secret box + lazy)
+    Crypto/Lazy.hs                       — Lazy hashing + encryption
+    Crypto/Ratchet.hs                    — Double ratchet + PQDR
+    Crypto/ShortLink.hs                  — Short link key derivation
+    Crypto/SNTRUP761.hs                  — Post-quantum KEM hybrid secret
+    Crypto/SNTRUP761/Bindings.hs         — sntrup761 C FFI bindings
+    Notifications/Protocol.hs            — NTF wire protocol types + encoding
+    Notifications/Types.hs               — NTF agent types (tokens, subscriptions)
+    Notifications/Transport.hs           — NTF transport handshake
+    Notifications/Client.hs              — NTF client operations
+    Notifications/Server.hs              — NTF server
+    Notifications/Server/Env.hs          — NTF server environment + config
+    Notifications/Server/Store.hs        — NTF server storage (STM)
+    Notifications/Server/Store/Postgres.hs — NTF server storage (Postgres)
+    Notifications/Server/Push/APNS.hs    — Apple push notification integration
+    Notifications/Server/Push/APNS/Internal.hs — APNS HTTP/2 client
+    Notifications/Server/Main.hs         — NTF server CLI
+    Notifications/Server/Stats.hs        — NTF server metrics
+    Notifications/Server/Prometheus.hs   — NTF Prometheus metrics
+    Notifications/Server/Control.hs      — NTF server control
+    Encoding.hs, Encoding/String.hs      — Binary + string encoding
+    Version.hs, Version/Internal.hs      — Version ranges + negotiation
+    Util.hs                              — Utilities (error handling, STM, grouping)
+    Parsers.hs                           — Attoparsec parser combinators
+    TMap.hs                              — Transactional map (STM)
+    Compression.hs                       — Zstd compression
+    ServiceScheme.hs                     — Service scheme + server location types
+    Session.hs                           — Session variables (TVar-based)
+    SystemTime.hs                        — Rounded system time types
+  FileTransfer/
+    Protocol.hs                          — XFTP wire protocol types + encoding
+    Client.hs                            — XFTP client operations
+    Client/Agent.hs                      — XFTP client agent (connection pooling)
+    Client/Main.hs                       — XFTP CLI client implementation
+    Client/Presets.hs                    — Default XFTP servers
+    Server.hs                            — XFTP server request handling
+    Server/Env.hs                        — XFTP server environment + config
+    Server/Store.hs                      — XFTP server storage
+    Server/StoreLog.hs                   — XFTP server store log
+    Server/Main.hs                       — XFTP server CLI
+    Server/Stats.hs                      — XFTP server metrics
+    Server/Prometheus.hs                 — XFTP Prometheus metrics
+    Server/Control.hs                    — XFTP server control
+    Agent.hs                             — XFTP agent operations
+    Description.hs                       — File description format
+    Transport.hs                         — XFTP transport
+    Crypto.hs                            — File encryption for transfer
+    Types.hs                             — File transfer types
+    Chunks.hs                            — Chunk sizing
+  RemoteControl/
+    Client.hs                            — XRCP client (ctrl device)
+    Invitation.hs                        — XRCP invitation handling
+    Discovery.hs                         — Local network discovery
+    Discovery/Multicast.hsc              — Multicast discovery (C FFI)
+    Types.hs                             — XRCP types + version
+
+apps/
+  smp-server/Main.hs                    — SMP server executable
+  smp-server/web/Static.hs              — SMP server web static files
+  xftp-server/Main.hs                   — XFTP server executable
+  xftp/Main.hs                          — XFTP CLI executable
+  ntf-server/Main.hs                    — Notification server executable
+  smp-agent/Main.hs                     — SMP agent (experimental, not in cabal)
+```
+
+## Linking Conventions
+
+### spec → src
+Fully qualified exported function names inline in prose: `Simplex.Messaging.Client.connectSMPProxiedRelay`. Use Grep/Glob to locate in source. For app targets: `xftp/Main.main`.
+
+### src → spec
+Comment above function:
+```haskell
+-- spec/crypto-tls.md#certificate-chain-validation
+-- Validates relay certificate chain to prevent proxy MITM (SI-XX)
+connectSMPProxiedRelay :: ...
+```
+
+### spec ↔ spec
+Named markdown heading anchors: `spec/crypto.md#ed25519-signing`
+
+### spec ↔ product
+Cross-references: `product/rules.md#pr-05`, `spec/security-invariants.md#si-01`
+
+### protocol/ references
+`protocol/simplex-messaging.md` with section name
+
+## Build Flags
+
+simplexmq builds with several flag combinations:
+
+| Flag | Effect |
+|------|--------|
+| (none) | Default: SQLite storage, all executables |
+| `-fserver_postgres` | Postgres backend for SMP server |
+| `-fclient_postgres` | Postgres backend for agent storage |
+| `-fclient_library` | Library-only build (no server executables) |
+| `-fswift` | Swift JSON format for mobile bindings |
+| `-fuse_crypton` | Use crypton in cryptostore |
+
+All flag combinations must compile with `--enable-tests`. Verify with:
+```
+cabal build all --ghc-options="-O0" [-flags] [--enable-tests]
+```
+
+## Change Protocol
+
+Every code change must maintain coherence across all three layers:
+
+1. **Code change** — Implement in src/
+2. **Spec update** — Update the relevant spec/ file(s): types, call graphs, security notes
+3. **Product update** — If user-visible behavior changed, update product/ files
+4. **Protocol update** — If wire protocol changed, amend protocol/ spec (requires user approval)
+5. **Security check** — If the change touches a trust boundary, update spec/security-invariants.md
+
+Protocol spec amendments require explicit user approval before committing.

README.md

@@ -1,86 +1,79 @@
-# SimpleXMQ
+# SimpleX Network
 
 [![GitHub build](https://github.com/simplex-chat/simplexmq/actions/workflows/build.yml/badge.svg)](https://github.com/simplex-chat/simplexmq/actions/workflows/build.yml)
 [![GitHub release](https://img.shields.io/github/v/release/simplex-chat/simplexmq)](https://github.com/simplex-chat/simplexmq/releases)
 
-📢 SimpleXMQ v1 is released - with many security, privacy and efficiency improvements, new functionality - see [release notes](https://github.com/simplex-chat/simplexmq/releases/tag/v1.0.0).
+The simplexmq package provides the software for [SimpleX Network](./protocol/overview-tjr.md) — a general-purpose packet routing network where endpoints exchange data through independently operated routers using resource-based addressing. Unlike IP networks, SimpleX addresses identify resources on routers (queues, data packets), not endpoint devices. Participants do not need globally unique identifiers to communicate.
 
-**Please note**: v1 is not backwards compatible, but it has the version negotiation built into all protocol layers for forwards compatibility of this version and backwards compatibility of the future versions, that will be backwards compatible for at least two versions back.
+The software is organized in three layers:
 
-If you have a server deployed please deploy a new server to a new host and retire the previous version once it is no longer used.
-
-## Message broker for unidirectional (simplex) queues
-
-SimpleXMQ is a message broker for managing message queues and sending messages over public network. It consists of SMP server, SMP client library and SMP agent that implement [SMP protocol](./protocol/simplex-messaging.md) for client-server communication and [SMP agent protocol](./protocol/agent-protocol.md) to manage duplex connections via simplex queues on multiple SMP servers.
-
-SMP protocol is inspired by [Redis serialization protocol](https://redis.io/topics/protocol), but it is much simpler - it currently has only 10 client commands and 8 server responses.
-
-SimpleXMQ is implemented in Haskell - it benefits from robust software transactional memory (STM) and concurrency primitives that Haskell provides.
-
-## SimpleXMQ roadmap
-
-- SimpleX service protocol and application template - to enable users building services and chat bots that work over SimpleX protocol stack. The first such service will be a notification service for a mobile app.
-- SMP queue redundancy and rotation in SMP agent connections.
-- SMP agents synchronization to share connections and messages between multiple agents (it would allow using multiple devices for [simplex-chat](https://github.com/simplex-chat/simplex-chat)).
-
-## Components
-
-### SMP server
-
-[SMP server](./apps/smp-server/Main.hs) can be run on any Linux distribution, including low power/low memory devices. OpenSSL library is required for initialization.
-
-To initialize the server use `smp-server init -n <fqdn>` (or `smp-server init --ip <ip>` for IP based address) command - it will generate keys and certificates for TLS transport. The fingerprint of offline certificate is used as part of the server address to protect client/server connection against man-in-the-middle attacks: `smp://<fingerprint>@<hostname>[:5223]`.
-
-SMP server uses in-memory persistence with an optional append-only log of created queues that allows to re-start the server without losing the connections. This log is compacted on every server restart, permanently removing suspended and removed queues.
-
-To enable store log, initialize server using `smp-server -l` command, or modify `smp-server.ini` created during initialization (uncomment `enable = on` option in the store log section). Use `smp-server --help` for other usage tips.
-
-Starting from version 2.3.0, when store log is enabled, the server would also enable saving undelivered messages on exit and restoring them on start. This can be disabled via a separate setting `restore_messages` in `smp-server.ini` file. Saving messages would only work if the server is stopped with SIGINT signal (keyboard interrupt), if it is stopped with SIGTERM signal the messages would not be saved.
-
-> **Please note:** On initialization SMP server creates a chain of two certificates: a self-signed CA certificate ("offline") and a server certificate used for TLS handshake ("online"). **You should store CA certificate private key securely and delete it from the server. If server TLS credential is compromised this key can be used to sign a new one, keeping the same server identity and established connections.** CA private key location by default is `/etc/opt/simplex/ca.key`.
-
-SMP server implements [SMP protocol](./protocol/simplex-messaging.md).
-
-#### Running SMP server on MacOS
-
-SMP server requires OpenSSL library for initialization. On MacOS OpenSSL library may be replaced with LibreSSL, which doesn't support required algorithms. Before initializing SMP server verify you have OpenSSL installed:
-
-```sh
-openssl version
+```
+  Application (e.g. SimpleX Chat)
++----------------------------------+
+|         SimpleX Agent            |  Layer 3 — duplex connections, e2e encryption
++----------------------------------+
+|      SimpleX Client Libraries    |  Layer 2 — protocol clients for SMP, XFTP
++----------------------------------+
+|        SimpleX Routers           |  Layer 1 — network infrastructure (SMP, XFTP, NTF)
++----------------------------------+
 ```
 
-If it says "LibreSSL", please install original OpenSSL:
+[SimpleX Chat](https://github.com/simplex-chat/simplex-chat) is one application built on Layer 3. IoT devices, AI services, monitoring systems, and automated services are other applications that can use Layers 2 or 3 directly.
 
-```sh
-brew update
-brew install openssl
-echo 'PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zprofile # or follow whatever instructions brew suggests
-. ~/.zprofile # or restart your terminal to start a new session
-```
+The simplexmq package is implemented in Haskell, benefiting from robust software transactional memory (STM) and concurrency primitives.
 
-Now `openssl version` should be saying "OpenSSL". You can now run `smp-server init` to initialize your SMP server.
+See the [SimpleX Network overview](./protocol/overview-tjr.md) for the full protocol architecture, trust model, and [security analysis](./protocol/security.md).
 
-### SMP client library
+## Architecture
 
-[SMP client](./src/Simplex/Messaging/Client.hs) is a Haskell library to connect to SMP servers that allows to:
+### SimpleX Routers
 
-- execute commands with a functional API.
-- receive messages and other notifications via STM queue.
-- automatically send keep-alive commands.
+Routers are the network infrastructure — they accept, buffer, and deliver packets. Three router types serve different purposes:
 
-### SMP agent
+- **SMP routers** provide messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes). Protocol: [SMP](./protocol/simplex-messaging.md). Module spec: [`Simplex.Messaging.Server`](./spec/modules/Simplex/Messaging/Server.md).
+- **XFTP routers** accept and deliver data packets over HTTP/2 — individually addressed blocks in fixed sizes (64KB–4MB) for larger payloads. Protocol: [XFTP](./protocol/xftp.md). Module spec: [`Simplex.FileTransfer.Server`](./spec/modules/Simplex/FileTransfer/Server.md).
+- **NTF routers** bridge to platform push services (APNS) for mobile notification delivery. Protocol: [Push Notifications](./protocol/push-notifications.md). Module spec: [`Simplex.Messaging.Notifications.Server`](./spec/modules/Simplex/Messaging/Notifications/Server.md).
 
-[SMP agent library](./src/Simplex/Messaging/Agent.hs) can be used to run SMP agent as part of another application and to communicate with the agent via STM queues, without serializing and parsing commands and responses.
 
-Haskell type [ACommand](./src/Simplex/Messaging/Agent/Protocol.hs) represents SMP agent protocol to communicate via STM queues.
+#### Running an SMP router
 
-See [simplex-chat](https://github.com/simplex-chat/simplex-chat) terminal UI for the example of integrating SMP agent into another application.
+[SMP server](./apps/smp-server/Main.hs) runs on any Linux distribution. OpenSSL is required for initialization.
 
-[SMP agent executable](./apps/smp-agent/Main.hs) can be used to run a standalone SMP agent process that implements plaintext [SMP agent protocol](./protocol/agent-protocol.md) via TCP port 5224, so it can be used via telnet. It can be deployed in private networks to share access to the connections between multiple applications and services.
+Initialize: `smp-server init -n <fqdn>` (or `--ip <ip>`). This generates TLS certificates. The CA certificate fingerprint becomes part of the server address: `smp://<fingerprint>@<hostname>[:5223]`.
 
-## Using SMP server and SMP agent
+By default, the server uses in-memory persistence with an optional append-only store log for queue persistence across restarts. Enable with `smp-server init -l` or in `smp-server.ini`. The log is compacted on every restart.
 
-You can either run your own SMP server locally or deploy using [Linode StackScript](https://cloud.linode.com/stackscripts/748014), or try local SMP agent with the deployed servers:
+When store log is enabled, undelivered messages are saved on exit (SIGINT only, not SIGTERM) and restored on start. Control this independently with the `restore_messages` setting.
+
+> **Please note:** On initialization, SMP server creates a certificate chain: a self-signed CA certificate ("offline") and a server certificate for TLS ("online"). **Store the CA private key securely and delete it from the server.** If the server TLS credential is compromised, this key can sign a new one while keeping the same server identity. Default location: `/etc/opt/simplex/ca.key`.
+
+See [docs/ROUTERS.md](./docs/ROUTERS.md) for XFTP/NTF router setup, advanced configuration, MacOS notes, and all deployment options (Docker, installation script, building from source, Linode, DigitalOcean).
+
+### SimpleX Client Libraries
+
+[Client libraries](./docs/CLIENT.md) provide low-level protocol access to SimpleX routers. They implement the wire protocols (SMP, XFTP, NTF) and handle connection lifecycle, command authentication, and keep-alive.
+
+The [SMP client](./src/Simplex/Messaging/Client.hs) ([module spec](./spec/modules/Simplex/Messaging/Client.md)) offers a functional Haskell API with STM queues for asynchronous event delivery. The [XFTP client](./src/Simplex/FileTransfer/Client.hs) ([module spec](./spec/modules/Simplex/FileTransfer/Client.md)) sends and receives data packets over HTTP/2 with per-request forward secrecy. The [NTF client](./src/Simplex/Messaging/Notifications/Client.hs) ([module spec](./spec/modules/Simplex/Messaging/Notifications/Client.md)) manages push notification tokens and subscriptions.
+
+Applications that manage their own encryption and connection logic — IoT devices, sensors, simple data pipelines — can use this layer directly. See [docs/CLIENT.md](./docs/CLIENT.md).
+
+### SimpleX Agent
+
+The [Agent](./docs/AGENT.md) builds duplex encrypted connections on top of the client libraries. It manages:
+
+- Duplex connections from simplex queue pairs
+- End-to-end encryption with double ratchet and post-quantum extensions
+- File transfer with chunking, encryption, and multi-router distribution
+- Queue rotation for metadata privacy
+- Push notification subscriptions
+
+The [Agent library](./src/Simplex/Messaging/Agent.hs) ([module spec](./spec/modules/Simplex/Messaging/Agent.md)) communicates via STM queues using the [ACommand](./src/Simplex/Messaging/Agent/Protocol.hs) type — no serialization needed. The Agent implements the [Agent protocol](./protocol/agent-protocol.md) for duplex connections and uses the [PQDR protocol](./protocol/pqdr.md) for end-to-end encryption. Cross-device remote control uses the [XRCP protocol](./protocol/xrcp.md).
+
+See [docs/AGENT.md](./docs/AGENT.md).
+
+## Quick start
+
+Public SMP routers for testing:
 
 `smp://[email protected]`
 
@@ -88,233 +81,16 @@ You can either run your own SMP server locally or deploy using [Linode StackScri
 
 `smp://[email protected]`
 
-It's the easiest to try SMP agent via a prototype [simplex-chat](https://github.com/simplex-chat/simplex-chat) terminal UI.
+## Deploy routers
 
-## Deploy SMP/XFTP servers on Linux
-
-You can run your SMP/XFTP server as a Linux process, optionally using a service manager for booting and restarts.
-
-Notice that `smp-server` and `xftp-server` requires `openssl` as run-time dependency (it is used to generate server certificates during initialization). Install it with your packet manager:
+You can run SMP/XFTP routers on any Linux distribution. OpenSSL is required:
 
 ```sh
-# For Ubuntu
+# Ubuntu
 apt update && apt install openssl
 ```
 
-### Install binaries
-
-#### Using Docker
-
-On Linux, you can deploy smp and xftp server using Docker. This will download image from [Docker Hub](https://hub.docker.com/r/simplexchat).
-
-1. Create directories for persistent Docker configuration:
-
-   ```sh
-   mkdir -p $HOME/simplex/{xftp,smp}/{config,logs} && mkdir -p $HOME/simplex/xftp/files
-   ```
-
-2. Run your Docker container.
-
-   - `smp-server`
-
-     You must change **your_ip_or_domain**. `-e "pass=password"` is optional variable to password-protect your `smp` server:
-     ```sh
-     docker run -d \
-         -e "ADDR=your_ip_or_domain" \
-         -e "PASS=password" \
-         -p 5223:5223 \
-         -v $HOME/simplex/smp/config:/etc/opt/simplex:z \
-         -v $HOME/simplex/smp/logs:/var/opt/simplex:z \
-         simplexchat/smp-server:latest
-     ```
-
-   - `xftp-server`
-
-     You must change **your_ip_or_domain** and **maximum_storage**.
-     ```sh
-     docker run -d \
-         -e "ADDR=your_ip_or_domain" \
-         -e "QUOTA=maximum_storage" \
-         -p 443:443 \
-         -v $HOME/simplex/xftp/config:/etc/opt/simplex-xftp:z \
-         -v $HOME/simplex/xftp/logs:/var/opt/simplex-xftp:z \
-         -v $HOME/simplex/xftp/files:/srv/xftp:z \
-         simplexchat/xftp-server:latest
-     ```
-
-#### Using installation script
-
-**Please note** that currently, only Ubuntu distribution is supported.
-
-You can install and setup servers automatically using our script:
-
-```sh
-curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/simplex-chat/simplexmq/stable/install.sh -o simplex-server-install.sh &&\
-if echo '53fcdb4ceab324316e2c4cda7e84dbbb344f32550a65975a7895425e5a1be757 simplex-server-install.sh' | sha256sum -c; then
-  chmod +x ./simplex-server-install.sh
-  ./simplex-server-install.sh
-  rm ./simplex-server-install.sh
-else
-  echo "SHA-256 checksum is incorrect!"
-  rm ./simplex-server-install.sh
-fi
-```
-
-### Build from source
-
-#### Using Docker
-
-> **Please note:** to build the app use source code from [stable branch](https://github.com/simplex-chat/simplexmq/tree/stable).
-
-On Linux, you can build smp server using Docker.
-
-1. Build your images:
-
-   ```sh
-   git clone https://github.com/simplex-chat/simplexmq
-   cd simplexmq
-   git checkout stable
-   DOCKER_BUILDKIT=1 docker build -t local/smp-server --build-arg APP="smp-server" --build-arg APP_PORT="5223" . # For xmp-server
-   DOCKER_BUILDKIT=1 docker build -t local/xftp-server --build-arg APP="xftp-server" --build-arg APP_PORT="443" . # For xftp-server
-   ```
-
-2. Create directories for persistent Docker configuration:
-
-   ```sh
-   mkdir -p $HOME/simplex/{xftp,smp}/{config,logs} && mkdir -p $HOME/simplex/xftp/files
-   ```
-
-3. Run your Docker container.
-
-   - `smp-server`
-
-     You must change **your_ip_or_domain**. `-e "pass=password"` is optional variable to password-protect your `smp` server:
-     ```sh
-     docker run -d \
-         -e "ADDR=your_ip_or_domain" \
-         -e "PASS=password" \
-         -p 5223:5223 \
-         -v $HOME/simplex/smp/config:/etc/opt/simplex:z \
-         -v $HOME/simplex/smp/logs:/var/opt/simplex:z \
-         simplexchat/smp-server:latest
-     ```
-
-   - `xftp-server`
-
-     You must change **your_ip_or_domain** and **maximum_storage**.
-     ```sh
-     docker run -d \
-         -e "ADDR=your_ip_or_domain" \
-         -e "QUOTA=maximum_storage" \
-         -p 443:443 \
-         -v $HOME/simplex/xftp/config:/etc/opt/simplex-xftp:z \
-         -v $HOME/simplex/xftp/logs:/var/opt/simplex-xftp:z \
-         -v $HOME/simplex/xftp/files:/srv/xftp:z \
-         simplexchat/xftp-server:latest
-     ```
-
-#### Using your distribution
-
-1. Install dependencies and build tools (`GHC`, `cabal` and dev libs):
-
-   ```sh
-   # On Ubuntu. Depending on your distribution, use your package manager to determine package names.
-   sudo apt-get update && apt-get install -y build-essential curl libffi-dev libffi7 libgmp3-dev libgmp10 libncurses-dev libncurses5 libtinfo5 pkg-config zlib1g-dev libnuma-dev libssl-dev
-   export BOOTSTRAP_HASKELL_GHC_VERSION=9.6.3
-   export BOOTSTRAP_HASKELL_CABAL_VERSION=3.10.3.0
-   curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | BOOTSTRAP_HASKELL_NONINTERACTIVE=1 sh
-   ghcup set ghc "${BOOTSTRAP_HASKELL_GHC_VERSION}"
-   ghcup set cabal "${BOOTSTRAP_HASKELL_CABAL_VERSION}"
-   source ~/.ghcup/env
-   ```
-
-2. Build the project:
-
-   ```sh
-   git clone https://github.com/simplex-chat/simplexmq
-   cd simplexmq
-   git checkout stable
-   cabal update
-   cabal build exe:smp-server exe:xftp-server
-   ```
-
-3. List compiled binaries:
-
-   `smp-server`
-   ```sh
-   cabal list-bin exe:smp-server
-   ```
-
-   `xftp-server`
-   ```sh
-   cabal list-bin exe:xftp-server
-   ```
-
-- Initialize SMP server with `smp-server init [-l] -n <fqdn>` or `smp-server init [-l] --ip <ip>` - depending on how you initialize it, either FQDN or IP will be used for server's address.
-
-- Run `smp-server start` to start SMP server, or you can configure a service manager to run it as a service.
-
-- Optionally, `smp-server` can be setup for having an onion address in `tor` network. See: [`scripts/tor`](./scripts/tor/). In this case, the server address can have both public and onion hostname pointing to the same server, to allow two people connect when only one of them is using Tor. The server address would be: `smp://<fingerprint>@<public_hostname>,<onion_hostname>`
-
-See [this section](#smp-server) for more information. Run `smp-server -h` and `smp-server init -h` for explanation of commands and options.
-
-[<img alt="Linode" src="./img/linode.svg" align="right" width="200">](https://cloud.linode.com/stackscripts/748014)
-
-## Deploy SMP server on Linode
-
-\* You can use free credit Linode offers when [creating a new account](https://www.linode.com/) to deploy an SMP server.
-
-Deployment on Linode is performed via StackScripts, which serve as recipes for Linode instances, also called Linodes. To deploy SMP server on Linode:
-
-- Create a Linode account or login with an already existing one.
-- Open [SMP server StackScript](https://cloud.linode.com/stackscripts/748014) and click "Deploy New Linode".
-- You can optionally configure the following parameters:
-  - SMP Server store log flag for queue persistence on server restart, recommended.
-  - [Linode API token](https://www.linode.com/docs/guides/getting-started-with-the-linode-api#get-an-access-token) to attach server address etc. as tags to Linode and to add A record to your 2nd level domain (e.g. `example.com` [domain should be created](https://cloud.linode.com/domains/create) in your account prior to deployment). The API token access scopes:
-    - read/write for "linodes"
-    - read/write for "domains"
-  - Domain name to use instead of Linode IP address, e.g. `smp1.example.com`.
-- Choose the region and plan, Shared CPU Nanode with 1Gb is sufficient.
-- Provide ssh key to be able to connect to your Linode via ssh. If you haven't provided a Linode API token this step is required to login to your Linode and get the server's fingerprint either from the welcome message or from the file `/etc/opt/simplex/fingerprint` after server starts. See [Linode's guide on ssh](https://www.linode.com/docs/guides/use-public-key-authentication-with-ssh/) .
-- Deploy your Linode. After it starts wait for SMP server to start and for tags to appear (if a Linode API token was provided). It may take up to 5 minutes depending on the connection speed on the Linode. Connecting Linode IP address to provided domain name may take some additional time.
-- Get `address` and `fingerprint` either from Linode tags (click on a tag and copy it's value from the browser search panel) or via ssh.
-- Great, your own SMP server is ready! If you provided FQDN use `smp://<fingerprint>@<fqdn>` as SMP server address in the client, otherwise use `smp://<fingerprint>@<ip_address>`.
-
-Please submit an [issue](https://github.com/simplex-chat/simplexmq/issues) if any problems occur.
-
-[<img alt="DigitalOcean" src="/img/digitalocean.png" align="right" width="300">](https://marketplace.digitalocean.com/apps/simplex-server)
-
-## Deploy SMP server on DigitalOcean
-
-> 🚧 DigitalOcean snapshot is currently not up to date, it will soon be updated 🏗️
-
-\* When creating a DigitalOcean account you can use [this link](https://try.digitalocean.com/freetrialoffer/) to get free credit. (You would still be required either to provide your credit card details or make a confirmation pre-payment with PayPal)
-
-To deploy SMP server use [SimpleX Server 1-click app](https://marketplace.digitalocean.com/apps/simplex-server) from DigitalOcean marketplace:
-
-- Create a DigitalOcean account or login with an already existing one.
-- Click 'Create SimpleX server Droplet' button.
-- Choose the region and plan according to your requirements (Basic plan should be sufficient).
-- Finalize Droplet creation.
-- Open "Console" on your Droplet management page to get SMP server fingerprint - either from the welcome message or from `/etc/opt/simplex/fingerprint`. Alternatively you can manually SSH to created Droplet, see [DigitalOcean instruction](https://docs.digitalocean.com/products/droplets/how-to/connect-with-ssh/).
-- Great, your own SMP server is ready! Use `smp://<fingerprint>@<ip_address>` as SMP server address in the client.
-
-Please submit an [issue](https://github.com/simplex-chat/simplexmq/issues) if any problems occur.
-
-> **Please note:** SMP server uses server address as a Common Name for server certificate generated during initialization. If you would like your server address to be FQDN instead of IP address, you can log in to your Droplet and run the commands below to re-initialize the server. Alternatively you can use [Linode StackScript](https://cloud.linode.com/stackscripts/748014) which allows this parameterization.
-
-```sh
-smp-server delete
-smp-server init [-l] -n <fqdn>
-```
-
-## SMP server design
-
-![SMP server design](./design/server.svg)
-
-## SMP agent design
-
-![SMP agent design](./design/agent2.svg)
+See [docs/ROUTERS.md](./docs/ROUTERS.md) for Docker, binary installation, building from source, and cloud deployment (Linode, DigitalOcean).
 
 ## License
 

apps/xftp-server/XFTPWeb.hs

@@ -58,7 +58,7 @@ xftpSubsts XFTPServerConfig {fileExpiration, logStatsInterval, allowNewFiles, ne
   [("smpConfig", Nothing), ("xftpConfig", Just "y")] <> substConfig <> serverInfoSubsts simplexmqSource information <> [("onionHost", strEncode <$> onionHost), ("iniFileName", Just "file-server.ini")]
   where
     substConfig =
-      [ ("fileExpiration", Just . fromString . timedTTLText . ttl $ fileExpiration),
+      [ ("fileExpiration", Just $ maybe "Never" (fromString . timedTTLText . ttl) fileExpiration),
         ("statsEnabled", Just . yesNo $ isJust logStatsInterval),
         ("newUploadsAllowed", Just . yesNo $ allowNewFiles),
         ("basicAuthEnabled", Just . yesNo $ isJust newFileBasicAuth)

apps/xftp-server/static/xftp-web-bundle/crypto.worker.js

@@ -619,6 +619,7 @@ async function handleEncrypt(id, data, fileName) {
   const digest = sha512Streaming([encData], (done) => {
     self.postMessage({ id, type: "progress", done: source.length + done, total });
   }, encDataLen);
+  console.log(`[WORKER-DBG] encrypt: encData.len=${encData.length} digest=${_whex(digest, 64)} chunkSizes=[${chunkSizes.join(",")}]`);
   const dir = await getSessionDir();
   const fileHandle = await dir.getFileHandle("upload.bin", { create: true });
   const writeHandle = await fileHandle.createSyncAccessHandle();
@@ -642,7 +643,9 @@ function handleReadChunk(id, offset, size) {
 }
 async function handleDecryptAndStore(id, dhSecret, nonce, body, chunkDigest, chunkNo) {
   const bodyArr = new Uint8Array(body);
+  console.log(`[WORKER-DBG] store chunk=${chunkNo} body.len=${bodyArr.length} nonce=${_whex(nonce, 24)} dhSecret=${_whex(dhSecret)} digest=${_whex(chunkDigest, 32)} body[0..8]=${_whex(bodyArr)} body[-8..]=${_whex(bodyArr.slice(-8))}`);
   const decrypted = decryptReceivedChunk(dhSecret, nonce, bodyArr, chunkDigest);
+  console.log(`[WORKER-DBG] decrypted chunk=${chunkNo} len=${decrypted.length} [0..8]=${_whex(decrypted)} [-8..]=${_whex(decrypted.slice(-8))}`);
   if (useMemory) {
     memoryChunks.set(chunkNo, decrypted);
     self.postMessage({ id, type: "stored" });
@@ -657,6 +660,7 @@ async function handleDecryptAndStore(id, dhSecret, nonce, body, chunkDigest, chu
   currentDownloadOffset += decrypted.length;
   chunkMeta.set(chunkNo, { offset, size: decrypted.length });
   const written = downloadWriteHandle.write(decrypted, { at: offset });
+  console.log(`[WORKER-DBG] OPFS write chunk=${chunkNo} offset=${offset} size=${decrypted.length} written=${written}`);
   if (written !== decrypted.length) {
     console.warn(`[WORKER] OPFS write failed chunk=${chunkNo}: ${written}/${decrypted.length}, falling back to in-memory storage`);
     for (const [cn, meta] of chunkMeta.entries()) {
@@ -680,16 +684,23 @@ async function handleDecryptAndStore(id, dhSecret, nonce, body, chunkDigest, chu
     return;
   }
   downloadWriteHandle.flush();
+  const verifyBuf = new Uint8Array(Math.min(8, decrypted.length));
+  downloadWriteHandle.read(verifyBuf, { at: offset });
+  const verifyEnd = new Uint8Array(Math.min(8, decrypted.length));
+  downloadWriteHandle.read(verifyEnd, { at: offset + decrypted.length - verifyEnd.length });
+  console.log(`[WORKER-DBG] OPFS verify chunk=${chunkNo} readBack[0..8]=${_whex(verifyBuf)} readBack[-8..]=${_whex(verifyEnd)} expected[0..8]=${_whex(decrypted)} expected[-8..]=${_whex(decrypted.slice(-8))}`);
   self.postMessage({ id, type: "stored" });
 }
 async function handleVerifyAndDecrypt(id, size, digest, key, nonce) {
+  console.log(`[WORKER-DBG] verify: expectedSize=${size} expectedDigest=${_whex(digest, 64)} useMemory=${useMemory} chunkMeta.size=${chunkMeta.size} memoryChunks.size=${memoryChunks.size}`);
   const chunks = [];
   let totalSize = 0;
   const total = size * 3;
   let done = 0;
   if (useMemory) {
     const sorted = [...memoryChunks.entries()].sort((a, b) => a[0] - b[0]);
-    for (const [, data] of sorted) {
+    for (const [chunkNo, data] of sorted) {
+      console.log(`[WORKER-DBG] verify memory chunk=${chunkNo} size=${data.length}`);
       chunks.push(data);
       totalSize += data.length;
       done += data.length;
@@ -704,10 +715,12 @@ async function handleVerifyAndDecrypt(id, size, digest, key, nonce) {
     const dir = await getSessionDir();
     const fileHandle = await dir.getFileHandle("download.bin");
     const readHandle = await fileHandle.createSyncAccessHandle();
+    console.log(`[WORKER-DBG] verify: OPFS file size=${readHandle.getSize()}`);
     const sortedEntries = [...chunkMeta.entries()].sort((a, b) => a[0] - b[0]);
-    for (const [, meta] of sortedEntries) {
+    for (const [chunkNo, meta] of sortedEntries) {
       const buf = new Uint8Array(meta.size);
-      readHandle.read(buf, { at: meta.offset });
+      const bytesRead = readHandle.read(buf, { at: meta.offset });
+      console.log(`[WORKER-DBG] verify read chunk=${chunkNo} offset=${meta.offset} size=${meta.size} bytesRead=${bytesRead} [0..8]=${_whex(buf)} [-8..]=${_whex(buf.slice(-8))}`);
       chunks.push(buf);
       totalSize += meta.size;
       done += meta.size;
@@ -732,9 +745,20 @@ async function handleVerifyAndDecrypt(id, size, digest, key, nonce) {
   }
   const actualDigest = r.crypto_hash_sha512_final(state);
   if (!digestEqual(actualDigest, digest)) {
+    console.error(`[WORKER-DBG] DIGEST MISMATCH: expected=${_whex(digest, 64)} actual=${_whex(actualDigest, 64)} chunks=${chunks.length} totalSize=${totalSize}`);
+    const state2 = r.crypto_hash_sha512_init();
+    for (let i = 0; i < chunks.length; i++) {
+      const chunk = chunks[i];
+      for (let off = 0; off < chunk.length; off += hashSEG) {
+        r.crypto_hash_sha512_update(state2, chunk.subarray(off, Math.min(off + hashSEG, chunk.length)));
+      }
+      const chunkDigest = sha512Streaming([chunk]);
+      console.error(`[WORKER-DBG] chunk[${i}] size=${chunk.length} sha512=${_whex(chunkDigest, 32)}… [0..8]=${_whex(chunk)} [-8..]=${_whex(chunk.slice(-8))}`);
+    }
     self.postMessage({ id, type: "error", message: "File digest mismatch" });
     return;
   }
+  console.log(`[WORKER-DBG] verify: digest OK`);
   const result = decryptChunks(BigInt(size), chunks, key, nonce, (d) => {
     self.postMessage({ id, type: "progress", done: size * 2 + d, total });
   });

apps/xftp-server/static/xftp-web-bundle/index.js

@@ -334,6 +334,11 @@ class WorkerBackend {
     const nonceCopy = new Uint8Array(nonce);
     const digestCopy = new Uint8Array(digest);
     const buf = this.toTransferable(body);
+    const hex = (b, n = 8) => {
+      const u = b instanceof ArrayBuffer ? new Uint8Array(b) : b;
+      return Array.from(u.slice(0, n)).map((x) => x.toString(16).padStart(2, "0")).join("");
+    };
+    console.log(`[BACKEND-DBG] chunk=${chunkNo} body.len=${body.length} body.byteOff=${body.byteOffset} buf.byteLen=${buf.byteLength} nonce=${hex(nonceCopy, 24)} dhSecret=${hex(dhSecretCopy)} digest=${hex(digestCopy, 32)} buf[0..8]=${hex(buf)} body[-8..]=${hex(body.slice(-8))}`);
     await this.send(
       { type: "decryptAndStoreChunk", dhSecret: dhSecretCopy, nonce: nonceCopy, body: buf, chunkDigest: digestCopy, chunkNo },
       [buf]
@@ -10908,12 +10913,14 @@ async function sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunk
   const block = encodeAuthTransmission(client.sessionId, corrId, entityId, cmdBytes, privateKey);
   const reqBody = chunkData ? concatBytes$1(block, chunkData) : block;
   const fullResp = await client.transport.post(reqBody);
+  console.log(`[XFTP-DBG] sendOnce: fullResp.length=${fullResp.length} entityId=${_hex(entityId)} cmdTag=${cmdBytes[0]}`);
   if (fullResp.length < XFTP_BLOCK_SIZE) {
     console.error("[XFTP] Response too short: %d bytes (expected >= %d)", fullResp.length, XFTP_BLOCK_SIZE);
     throw new Error("Server response too short");
   }
   const respBlock = fullResp.subarray(0, XFTP_BLOCK_SIZE);
   const body = fullResp.subarray(XFTP_BLOCK_SIZE);
+  console.log(`[XFTP-DBG] sendOnce: body.length=${body.length} body.byteOffset=${body.byteOffset} body.buffer.byteLength=${body.buffer.byteLength}`);
   const raw = blockUnpad(respBlock);
   if (raw.length < 20) {
     const text = new TextDecoder().decode(raw);
@@ -10932,13 +10939,18 @@ async function sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunk
   }
   return { response, body };
 }
+function _hex(b, n = 8) {
+  return Array.from(b.slice(0, n)).map((x) => x.toString(16).padStart(2, "0")).join("");
+}
 async function sendXFTPCommand(agent, server, privateKey, entityId, cmdBytes, chunkData, maxRetries = 3) {
   let clientP = getXFTPServerClient(agent, server);
   let client = await clientP;
   for (let attempt = 1; attempt <= maxRetries; attempt++) {
     try {
+      if (attempt > 1) console.log(`[XFTP-DBG] sendCmd: retry attempt=${attempt}/${maxRetries}`);
       return await sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunkData);
     } catch (e) {
+      console.log(`[XFTP-DBG] sendCmd: attempt=${attempt} failed: ${e instanceof Error ? e.message : String(e)} retriable=${isRetriable(e)}`);
       if (!isRetriable(e)) {
         throw categorizeError(e);
       }
@@ -10967,6 +10979,7 @@ async function downloadXFTPChunkRaw(agent, server, rpKey, fId) {
   const { response, body } = await sendXFTPCommand(agent, server, rpKey, fId, cmd);
   if (response.type !== "FRFile") throw new Error("unexpected response: " + response.type);
   const dhSecret = dh(response.rcvDhKey, privateKey);
+  console.log(`[XFTP-DBG] dlChunkRaw: body.length=${body.length} nonce=${_hex(response.nonce, 24)} dhSecret=${_hex(dhSecret)} body[0..8]=${_hex(body)} body[-8..]=${_hex(body.slice(-8))}`);
   return { dhSecret, nonce: response.nonce, body };
 }
 async function downloadXFTPChunk(agent, server, rpKey, fId, digest) {
@@ -10999,6 +11012,7 @@ function encryptFileForUpload(source, fileName) {
   const encSize = BigInt(chunkSizes.reduce((a, b) => a + b, 0));
   const encData = encryptFile(source, fileHdr, key, nonce, fileSize, encSize);
   const digest = sha512Streaming([encData]);
+  console.log(`[AGENT-DBG] encrypt: encData.len=${encData.length} digest=${_dbgHex(digest, 64)} chunkSizes=[${chunkSizes.join(",")}]`);
   return { encData, digest, key, nonce, chunkSizes };
 }
 const DEFAULT_REDIRECT_THRESHOLD = 400;
@@ -11155,7 +11169,9 @@ async function downloadFileRaw(agent, fd, onRawChunk, options) {
   if (err) throw new Error("downloadFileRaw: " + err);
   const { onProgress} = options ?? {};
   if (fd.redirect !== null) {
+    console.log(`[AGENT-DBG] resolving redirect: outer size=${fd.size} chunks=${fd.chunks.length}`);
     fd = await resolveRedirect(agent, fd);
+    console.log(`[AGENT-DBG] resolved: size=${fd.size} chunks=${fd.chunks.length} digest=${Array.from(fd.digest.slice(0, 16)).map((x) => x.toString(16).padStart(2, "0")).join("")}…`);
   }
   const resolvedFd = fd;
   let downloaded = 0;
@@ -11173,6 +11189,7 @@ async function downloadFileRaw(agent, fd, onRawChunk, options) {
       const seed = decodePrivKeyEd25519(replica.replicaKey);
       const kp = ed25519KeyPairFromSeed(seed);
       const raw = await downloadXFTPChunkRaw(agent, server, kp.privateKey, replica.replicaId);
+      console.log(`[AGENT-DBG] chunk=${chunk.chunkNo} body.len=${raw.body.length} expectedChunkSize=${chunk.chunkSize} digest=${_dbgHex(chunk.digest, 32)} body.byteOffset=${raw.body.byteOffset} body.buffer.byteLength=${raw.body.buffer.byteLength}`);
       await onRawChunk({
         chunkNo: chunk.chunkNo,
         dhSecret: raw.dhSecret,

cbits/getentropy_win.c

@@ -1,24 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// getentropy() shim for Windows, where it is absent from the CRT.
-// Follows the POSIX contract: fills `buffer` with `length` random bytes
-// (length must not exceed 256), returns 0 on success or -1 with errno set.
-#ifdef _WIN32
-#include <errno.h>
-#include <stddef.h>
-#include <windows.h>
-#include <bcrypt.h>
-
-int getentropy(void *buffer, size_t length) {
-    if (length > 256) {
-        errno = EIO;
-        return -1;
-    }
-    NTSTATUS status = BCryptGenRandom(NULL, (PUCHAR)buffer, (ULONG)length,
-                                      BCRYPT_USE_SYSTEM_PREFERRED_RNG);
-    if (!BCRYPT_SUCCESS(status)) {
-        errno = EIO;
-        return -1;
-    }
-    return 0;
-}
-#endif

cbits/sntrup761.c

@@ -719,32 +719,23 @@ Small_random (small * out, void *random_ctx, sntrup761_random_func * random)
 
 /* ----- Streamlined NTRU Prime Core */
 
-/* x^p-x-1 has a degree-19 factor mod 3, so a random g is not invertible in R3
-   with probability about 3^-19; KeyGen_attempts failures in a row mean a broken RNG */
-#define KeyGen_attempts 10
-
 /* h,(f,ginv) = KeyGen() */
-/* returns 0 if KeyGen succeeded; else -1 */
-static int
+static void
 KeyGen (Fq * h, small * f, small * ginv, void *random_ctx,
         sntrup761_random_func * random)
 {
   small g[p];
   Fq finv[p];
-  int i;
 
-  for (i = 0; i < KeyGen_attempts; ++i)
+  for (;;)
     {
       Small_random (g, random_ctx, random);
       if (R3_recip (ginv, g) == 0)
         break;
     }
-  if (i == KeyGen_attempts)
-    return -1;
   Short_random (f, random_ctx, random);
   Rq_recip3 (finv, f);          /* always works */
   Rq_mult_small (h, finv, g);
-  return 0;
 }
 
 /* c = Encrypt(r,h) */
@@ -893,21 +884,18 @@ typedef small Inputs[p];        /* passed by reference */
 #define PublicKeys_bytes Rq_bytes
 
 /* pk,sk = ZKeyGen() */
-/* returns 0 if KeyGen succeeded; else -1 */
-static int
+static void
 ZKeyGen (unsigned char *pk, unsigned char *sk, void *random_ctx,
          sntrup761_random_func * random)
 {
   Fq h[p];
   small f[p], v[p];
 
-  if (KeyGen (h, f, v, random_ctx, random) != 0)
-    return -1;
+  KeyGen (h, f, v, random_ctx, random);
   Rq_encode (pk, h);
   Small_encode (sk, f);
   sk += Small_bytes;
   Small_encode (sk, v);
-  return 0;
 }
 
 /* C = ZEncrypt(r,pk) */
@@ -972,21 +960,19 @@ HashSession (unsigned char *k, int b, const unsigned char *y,
 /* ----- Streamlined NTRU Prime */
 
 /* pk,sk = KEM_KeyGen() */
-int
+void
 sntrup761_keypair (unsigned char *pk, unsigned char *sk, void *random_ctx,
                    sntrup761_random_func * random)
 {
   int i;
 
-  if (ZKeyGen (pk, sk, random_ctx, random) != 0)
-    return -1;
+  ZKeyGen (pk, sk, random_ctx, random);
   sk += SecretKeys_bytes;
   for (i = 0; i < PublicKeys_bytes; ++i)
     *sk++ = pk[i];
   random (random_ctx, Inputs_bytes, sk);
   sk += Inputs_bytes;
   Hash_prefix (sk, 4, pk, PublicKeys_bytes);
-  return 0;
 }
 
 /* c,r_enc = Hide(r,pk,cache); cache is Hash4(pk) */

cbits/sntrup761.h

@@ -19,8 +19,7 @@
 
 typedef void sntrup761_random_func (void *ctx, size_t length, uint8_t *dst);
 
-/* returns 0 on success, -1 if the RNG never produced an invertible polynomial */
-int
+void
 sntrup761_keypair (uint8_t *pk, uint8_t *sk,
                    void *random_ctx, sntrup761_random_func *random);
 

contributing/CODE.md

@@ -7,7 +7,6 @@ This file provides guidance on coding style and approaches and on building the c
 When designing code and planning implementations:
 - Apply adversarial thinking, and consider what may happen if one of the communicating parties is malicious.
 - Formulate an explicit threat model for each change - who can do which undesirable things and under which circumstances.
-- Use the default cryptographic primitive for each purpose from the [primitive registry](../protocol/crypto-registry.md); any exception requires review, and the registry must be updated with every new primitive use or domain-separation string.
 
 ## Code Quality Standards
 
@@ -93,6 +92,7 @@ cabal list-bin exe:smp-server
 ### Cabal Flags
 
 - `swift`: Enable Swift JSON format
+- `use_crypton`: Use crypton in cryptostore (default: enabled)
 - `client_library`: Build without server code
 - `client_postgres`: Use PostgreSQL instead of SQLite for agent persistence
 - `server_postgres`: PostgreSQL support for server queue/notification store

contributing/PROJECT.md

@@ -13,7 +13,7 @@ Key components:
 - **SMP Client**: Functional API with STM-based message delivery ([code](../src/Simplex/Messaging/Client.hs)).
 - **SMP Agent**: High-level duplex connections via multiple simplex queues with E2E encryption ([code](../src/Simplex/Messaging/Agent.hs)). Implements Agent-to-agent protocol ([code](../src/Simplex/Messaging/Agent/Protocol.hs), [spec](../protocol/agent-protocol.md)) via intermediary agent client ([code](../src/Simplex/Messaging/Agent/Client.hs)).
 - **XFTP**: SimpleX File Transfer Protocol, server and CLI client ([code](../src/Simplex/FileTransfer/), [spec](../protocol/xftp.md)).
-- **XRCP**: SimpleX Remote Control Protocol ([code](`../src/Simplex/RemoteControl/`), [spec](../protocol/xrcp.md)).
+- **XRCP**: SimpleX Remote Control Protocol ([code](../src/Simplex/RemoteControl/), [spec](../protocol/xrcp.md)).
 - **Notifications**: Push notifications server requires PostgreSQL ([code](../src/Simplex/Messaging/Notifications), [executable](../apps/ntf-server/)). Client protocol is used for clients to communicate with the server ([code](../src/Simplex/Messaging/Notifications/Protocol.hs), [spec](../protocol/push-notifications.md)). For subscribing to SMP notifications the server uses [lightweight SMP client](../src/Simplex/Messaging/Client/Agent.hs).
 
 ## Architecture

docs/AGENT.md

@@ -0,0 +1,92 @@
+# SimpleX Agent
+
+The SimpleX Agent builds duplex encrypted connections on top of [SimpleX client libraries](CLIENT.md). It manages the full lifecycle of secure communication: connection establishment, end-to-end encryption, queue rotation, file transfer, and push notifications.
+
+This is **Layer 3** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers; Layer 2 is the [client libraries](CLIENT.md) that speak the wire protocols. The Agent adds the connection semantics that applications need. For internal architecture diagrams (thread topology, message processing flows), see [`spec/agent.md`](../spec/agent.md).
+
+**Source**: [`Simplex.Messaging.Agent`](../src/Simplex/Messaging/Agent.hs). **Module spec**: [`spec/modules/Simplex/Messaging/Agent.md`](../spec/modules/Simplex/Messaging/Agent.md)
+
+## Connections
+
+The Agent turns simplex (unidirectional) SMP queues into duplex connections, implementing the [Agent protocol](../protocol/agent-protocol.md):
+
+- **Duplex connections**: each connection uses a pair of SMP queues - one for each direction. The queues can be on different routers chosen independently by each party. See the [duplex connection procedure](../protocol/agent-protocol.md) for the full handshake.
+- **Connection establishment**: one party creates a connection and generates an invitation (containing router address, queue ID, and public keys). The invitation is passed out-of-band (QR code, link, etc.). The other party joins by creating a reverse queue and completing the handshake.
+- **Connection links**: the Agent supports connection links (long and short) for sharing connection invitations via URLs. Short links use a separate SMP queue to store the full invitation, allowing compact QR codes.
+- **Queue rotation**: the Agent has API to rotate the SMP queues, to reduce metadata correlation. The connection identity remain the same when the underlying queue changes.
+
+## Encryption
+
+The Agent provides end-to-end encryption with forward secrecy and break-in recovery, specified in the [Post-Quantum Double Ratchet protocol](../protocol/pqdr.md):
+
+- **Double ratchet**: messages are encrypted using a double ratchet protocol. Each message uses a unique key; compromising one key does not reveal past or future messages. See the [PQDR specification](../protocol/pqdr.md) for the full ratchet state machine.
+- **Post-quantum extensions**: the ratchet supports hybrid key exchange using SNTRUP761 (a lattice-based KEM) combined with X25519 DH. This provides protection against future quantum computers that could break classical DH. See the [SNTRUP761 module spec](../spec/modules/Simplex/Messaging/Crypto/SNTRUP761.md) and [Ratchet module spec](../spec/modules/Simplex/Messaging/Crypto/Ratchet.md) for implementation details.
+- **Ratchet synchronization**: if the ratchet state becomes desynchronized (e.g., due to message loss or device restore), the Agent detects this and can negotiate resynchronization with the peer.
+- **Per-queue encryption**: in addition to end-to-end encryption, the [SMP protocol](../protocol/simplex-messaging.md) provides a separate encryption layer on each queue between sender and router, preventing traffic correlation even if TLS is compromised.
+
+## File Transfer
+
+The Agent handles file transfer over [XFTP](../protocol/xftp.md) routers. File transfer orchestration is implemented in the [XFTP Agent module](../spec/modules/Simplex/FileTransfer/Agent.md):
+
+- **Chunking**: files are split into chunks, each sent as a data packet to an XFTP router. Chunk sizes are fixed powers of 2 (64KB to 4MB), hiding the actual file size. See the [file description module spec](../spec/modules/Simplex/FileTransfer/Description.md) for chunk size selection and file descriptor format.
+- **Client-side encryption**: files are encrypted and padded before being sent to XFTP routers. The recipient decrypts after receiving all chunks. The encryption key and file metadata are sent through the SMP connection, not through XFTP. See [file crypto module spec](../spec/modules/Simplex/FileTransfer/Crypto.md).
+- **Multi-router distribution**: chunks can be sent to different XFTP routers, and each chunk can have multiple replicas on different routers for redundancy.
+- **Redirect chains**: for metadata privacy, file descriptors can be sent as XFTP data packets themselves, creating an indirection layer between the SMP message and the actual file location.
+
+## Notifications
+
+The Agent manages push notification subscriptions for mobile devices, using the [Push Notifications protocol](../protocol/push-notifications.md). Notification supervision is handled by the [NtfSubSupervisor](../spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md):
+
+- **Token registration**: registers device push tokens with NTF (notification) routers, which bridge to platform push services (APNS). See the [NTF client module spec](../spec/modules/Simplex/Messaging/Notifications/Client.md).
+- **Notification subscriptions**: creates NTF subscriptions for SMP queues so that incoming messages trigger push notifications without requiring persistent connections.
+- **Privacy preservation**: push notifications contain only a notification ID, not message content. The device wakes, connects to the SMP router, and retrieves the actual message. See the [Push Notifications protocol](../protocol/push-notifications.md) for the full flow.
+
+## Integration
+
+The Agent is designed to be embedded as a Haskell library:
+
+- **STM queues**: the application communicates with the Agent via STM queues. Commands go in (`ACommand`), events come out (`AEvent`). No serialization or parsing - direct Haskell values. The command/event types are defined in the [Agent Protocol module](../spec/modules/Simplex/Messaging/Agent/Protocol.md).
+- **Async operation**: all network operations are asynchronous. The Agent manages internal worker threads for each router connection, message processing, and background tasks (cleanup, statistics, notification supervision). See the [Agent Client module spec](../spec/modules/Simplex/Messaging/Agent/Client.md) for worker architecture.
+- **Background mode**: on mobile platforms, the Agent can run in a reduced mode with only the message receiver active, minimizing resource usage when the app is backgrounded.
+- **Dual database backends**: the Agent supports both SQLite (for mobile/desktop) and PostgreSQL (for server deployments) as persistence backends, selected at compile time. See [Agent Store Interface](../spec/modules/Simplex/Messaging/Agent/Store/Interface.md) and [Agent Store Postgres](../spec/modules/Simplex/Messaging/Agent/Store/Postgres.md).
+
+## Use cases
+
+- **Chat applications**: [SimpleX Chat](https://github.com/simplex-chat/simplex-chat) is the reference application, using the full Agent API for messaging, file sharing, groups, and calls.
+- **Bots and automated services**: services that need duplex encrypted communication with SimpleX Chat users or other Agent-based applications.
+- **Any application needing secure duplex communication** over the SimpleX Network without implementing the connection management, encryption, and queue rotation logic directly.
+
+## What this layer adds over client libraries
+
+| Capability | Client (Layer 2) | Agent (Layer 3) |
+|---|---|---|
+| Queue operations | Direct | Managed transparently |
+| Connection model | Simplex (unidirectional) queues | Duplex connections |
+| Encryption | Application's responsibility | Double ratchet with PQ extensions |
+| File transfer | Raw data packet send/receive | Chunking, encryption, reassembly |
+| Identity | Per-queue keys | Per-connection, rotatable |
+| Notifications | Direct NTF protocol operations | Automated subscription supervision |
+
+## Protocol references
+
+- [Agent Protocol](../protocol/agent-protocol.md) - duplex connection procedure, message format
+- [SimpleX Network overview](../protocol/overview-tjr.md) - architecture, trust model
+- [PQDR](../protocol/pqdr.md) - post-quantum double ratchet specification
+- [SimpleX Messaging Protocol](../protocol/simplex-messaging.md) - SMP queue operations used by the Agent
+- [XFTP Protocol](../protocol/xftp.md) - data packet operations for file transfer
+- [Push Notifications Protocol](../protocol/push-notifications.md) - NTF token and subscription management
+## Peer library: Remote Control
+
+The Agent exposes the [XRCP protocol](../protocol/xrcp.md) API for cross-device remote control (e.g., controlling a mobile app from a desktop). The actual logic is in the standalone [`Simplex.RemoteControl.Client`](../src/Simplex/RemoteControl/Client.hs) library - the Agent provides thin wrappers that pass through its random and multicast state. XRCP is not a managed Agent capability (no workers, persistence, or background supervision). See the [RemoteControl module specs](../spec/modules/Simplex/RemoteControl/Types.md).
+
+## Module specs
+
+- [Agent](../spec/modules/Simplex/Messaging/Agent.md) - main Agent module, connection lifecycle, message processing
+- [Agent Client](../spec/modules/Simplex/Messaging/Agent/Client.md) - worker threads, router connections, subscription management
+- [Agent Protocol](../spec/modules/Simplex/Messaging/Agent/Protocol.md) - ACommand/AEvent types, connection invitations
+- [Agent Store Interface](../spec/modules/Simplex/Messaging/Agent/Store/Interface.md) - database abstraction for SQLite/Postgres
+- [Agent Store (AgentStore)](../spec/modules/Simplex/Messaging/Agent/Store/AgentStore.md) - connection, queue, and message persistence
+- [NtfSubSupervisor](../spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) - notification subscription management
+- [XFTP Agent](../spec/modules/Simplex/FileTransfer/Agent.md) - file transfer orchestration
+- [Ratchet](../spec/modules/Simplex/Messaging/Crypto/Ratchet.md) - double ratchet implementation
+- [SNTRUP761](../spec/modules/Simplex/Messaging/Crypto/SNTRUP761.md) - post-quantum KEM

docs/CLIENT.md

@@ -0,0 +1,96 @@
+# SimpleX Client Libraries
+
+SimpleX client libraries provide low-level protocol access to SimpleX routers. They implement the wire protocols ([SMP](../protocol/simplex-messaging.md), [XFTP](../protocol/xftp.md), [NTF](../protocol/push-notifications.md)) and handle connection lifecycle, but leave encryption, identity management, and connection orchestration to the application.
+
+This is **Layer 2** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers themselves; Layer 3 is the [Agent](AGENT.md), which builds duplex encrypted connections on top of these libraries. For internal architecture diagrams (thread topology, command processing flows), see [`spec/clients.md`](../spec/clients.md).
+
+## SMP Client
+
+**Source**: [`Simplex.Messaging.Client`](../src/Simplex/Messaging/Client.hs). For architecture and module specs, see [SMP Client](../spec/clients.md#smp-client-protocolclient).
+
+The SMP client connects to SMP routers and manages simplex messaging queues, the fundamental addressing primitive of the SimpleX Network. Each simplex queue is a unidirectional, ordered sequence of fixed-size packets (16,384 bytes) with separate cryptographic credentials for sending and receiving. The queue model and command set are defined in the [SMP protocol](../protocol/simplex-messaging.md).
+
+### Capabilities
+
+- **Queue management**: create, secure, subscribe to, and delete queues on any SMP router. Queue operations use the [SMP command set](../protocol/simplex-messaging.md) (NEW, KEY, SUB, DEL, etc.).
+- **Message sending and receiving**: send messages to a queue's sender address; receive messages from a queue's recipient address
+- **Command authentication**: each queue operation is authenticated with per-queue cryptographic keys (Ed25519, Ed448, or X25519). See the [SMP protocol security model](../protocol/simplex-messaging.md) for key roles.
+- **Keep-alive**: automatic ping loop detects and recovers from half-open connections
+- **Proxy forwarding**: send messages through a proxy router via 2-hop onion routing (PRXY/PFWD/RFWD commands), protecting the sender's IP address from the destination router. See [proxy forwarding details](../spec/modules/Simplex/Messaging/Client.md) in the module spec.
+- **Batched commands**: multiple commands can be sent in a single transmission for efficiency
+
+### API model
+
+The client uses a functional Haskell API with STM queues for asynchronous event delivery:
+
+- **Commands** are sent via `sendProtocolCommand` (single) or `sendBatch` (multiple). Each returns a result synchronously or via timeout.
+- **Router events** (incoming messages, subscription notifications) arrive on `msgQ`, an STM `TBQueue` that the application reads from its own thread.
+- **Connection lifecycle** is managed automatically: the client maintains send, receive, process, and monitor threads internally. When any thread fails, all are torn down and the `disconnected` callback fires.
+
+### Router identity
+
+Routers are identified by the SHA-256 hash of their CA certificate fingerprint, not by hostname. The client validates the full X.509 certificate chain on every TLS connection and compares the CA fingerprint against the expected hash from the queue address. This means a DNS or IP-level attacker who cannot produce the correct certificate is detected at connection time.
+
+## SMPClientAgent
+
+**Source**: [`Simplex.Messaging.Client.Agent`](../src/Simplex/Messaging/Client/Agent.hs). For architecture and module specs, see [SMPClientAgent](../spec/clients.md#smpclientagent).
+
+Connection manager that multiplexes multiple SMP client connections. Maintains one ProtocolClient per SMP router, tracks queue and service subscriptions, and handles reconnection with exponential backoff. Used by the SMP router (for proxy forwarding) and the NTF router (for message subscriptions).
+
+### Capabilities
+
+- **Connection pooling**: maintains a pool of ProtocolClient connections keyed by SMP router, creating connections on demand and reusing existing ones
+- **Subscription tracking**: tracks active and pending subscriptions (both queue-based and service-based) with automatic state transitions on connect/disconnect
+- **Automatic reconnection**: on connection loss, moves subscriptions from active to pending, then spawns a background worker that retries with backoff and resubscribes
+- **Session-scoped disconnect handling**: uses session IDs to ensure only subscriptions belonging to the disconnected session are affected, preventing races with newly established connections
+
+## XFTP Client
+
+**Source**: [`Simplex.FileTransfer.Client`](../src/Simplex/FileTransfer/Client.hs). For architecture and module specs, see [XFTP Client](../spec/clients.md#xftp-client).
+
+The XFTP client connects to XFTP routers and manages data packets, individually addressed blocks used for larger payload delivery. Data packets come in fixed sizes (64KB, 256KB, 1MB, 4MB), hiding the actual payload size. The XFTP protocol runs over HTTP/2, simplifying browser integration. The data packet lifecycle and command set are defined in the [XFTP protocol](../protocol/xftp.md).
+
+### Capabilities
+
+- **Data packet creation**: create data packets on routers with sender, recipient, and optional additional recipient credentials. See the [XFTP protocol](../protocol/xftp.md) for credential roles and packet lifecycle.
+- **Send** (FPUT): send encrypted data to the router in a single HTTP/2 streaming request (command + body)
+- **Receive** (FGET): receive data packets with per-request ephemeral Diffie-Hellman key exchange, providing forward secrecy: compromising one DH key does not reveal other received data packets
+- **Acknowledgment and deletion**: recipients acknowledge receipt; senders delete data packets after delivery
+
+## NTF Client
+
+**Source**: [`Simplex.Messaging.Notifications.Client`](../src/Simplex/Messaging/Notifications/Client.hs). For architecture and module specs, see [NTF Client](../spec/clients.md#ntf-client).
+
+The NTF client connects to NTF (notification) routers and manages push notification tokens and subscriptions. It implements the [Push Notifications protocol](../protocol/push-notifications.md).
+
+### Capabilities
+
+- **Token management**: register, verify, replace, and delete push notification tokens on NTF routers
+- **Subscription management**: create, check, and delete notification subscriptions that link SMP queues to push tokens
+- **Batch operations**: create or check multiple subscriptions in a single request, with per-item error handling for partial success
+
+## Use cases
+
+These libraries are appropriate when the application manages its own encryption and connection logic:
+
+- **IoT sensor data collection**: a sensor creates an SMP queue and sends readings; a collector subscribes and receives them. The queue address (router + queue ID + keys) is provisioned once, out-of-band.
+- **Device control**: a controller sends commands to an actuator's queue. Separate queues for commands and telemetry provide unidirectional isolation.
+- **Bulk data delivery**: an application encrypts and chunks a file, sends data packets to XFTP routers, and shares the packet addresses with the recipient out-of-band.
+- **Custom protocols**: any application that needs unidirectional, router-mediated packet delivery without the overhead of the Agent's connection management.
+
+## What this layer does NOT provide
+
+The following capabilities require the [Agent](AGENT.md) (Layer 3):
+
+- **Duplex connections** - the Agent pairs two simplex queues into a duplex connection
+- **End-to-end encryption** - the Agent manages double ratchet with post-quantum extensions
+- **File transfer** - the Agent handles chunking, encryption, padding, multi-router distribution, and reassembly
+- **Queue rotation** - the Agent transparently rotates queues to limit metadata correlation
+- **Connection discovery** - connection links, short links, and contact addresses are Agent-level abstractions
+- **Push notifications** - notification token management and subscription is Agent-level
+
+## Protocol references
+
+- [SimpleX Messaging Protocol](../protocol/simplex-messaging.md) - SMP wire format, commands, and security properties
+- [XFTP Protocol](../protocol/xftp.md) - XFTP wire format, data packet lifecycle
+- [SimpleX Network overview](../protocol/overview-tjr.md) - architecture, trust model, and design rationale

docs/ROUTERS.md

@@ -0,0 +1,181 @@
+# SimpleX Routers: Deployment and Configuration
+
+SimpleX routers are the network infrastructure of the [SimpleX Network](../protocol/overview-tjr.md). They accept, buffer, and deliver data packets between endpoints. Each router operates independently and can be run by any party on standard computing hardware.
+
+This document covers deployment and advanced configuration. For an overview of the router architecture and trust model, see the [SimpleX Network overview](../protocol/overview-tjr.md). For internal architecture diagrams (thread topology, command processing flows), see [`spec/routers.md`](../spec/routers.md).
+
+## SMP Router
+
+The SMP router provides messaging queues - unidirectional, ordered sequences of fixed-size packets (16,384 bytes each). It implements the [SimpleX Messaging Protocol](../protocol/simplex-messaging.md). For architecture and module specs, see [SMP Router](../spec/routers.md#smp-router).
+
+### Advanced configuration
+
+`smp-server.ini` is created during initialization and controls all runtime behavior.
+
+**Message persistence**: when store log is enabled (`enable = on`), the server saves undelivered messages on exit and restores them on start. This only works with SIGINT (keyboard interrupt); SIGTERM does not trigger message saving. The `restore_messages` setting can be used to override this behavior independently of the store log setting.
+
+**Tor onion addresses**: the server can have both a public hostname and an onion hostname, allowing two users to connect when only one is using Tor. Configure as: `smp://<fingerprint>@<public_hostname>,<onion_hostname>`. See [`scripts/tor/`](../scripts/tor/) for setup instructions.
+
+### Running on MacOS
+
+SMP server requires OpenSSL for initialization. MacOS may ship LibreSSL instead, which doesn't support the required algorithms.
+
+```sh
+openssl version
+```
+
+If it says "LibreSSL", install OpenSSL:
+
+```sh
+brew update
+brew install openssl
+echo 'PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zprofile
+. ~/.zprofile
+```
+
+## XFTP Router
+
+The XFTP router accepts and delivers data packets over HTTP/2 - individually addressed blocks in fixed sizes (64KB, 256KB, 1MB, 4MB). It implements the [XFTP protocol](../protocol/xftp.md). Data packets are used for larger payload delivery (files, media) where SMP queue packet sizes would be inefficient. The use of HTTP/2 simplifies browser integration. For architecture and module specs, see [XFTP Router](../spec/routers.md#xftp-router).
+
+Initialize with `xftp-server init` and configure storage quota in `xftp-server.ini`.
+
+## NTF Router
+
+The NTF router bridges SimpleX Network to platform push notification services (APNS). It implements the [Push Notifications protocol](../protocol/push-notifications.md). Mobile clients register push tokens with the NTF router, which subscribes to their SMP queues and sends push notifications when messages arrive. The push notification contains only a notification ID, not message content. For architecture and module specs, see [NTF Router](../spec/routers.md#ntf-router).
+
+Initialize with `ntf-server init` and configure APNS credentials in `ntf-server.ini`.
+
+## Deployment methods
+
+All routers require `openssl` as a runtime dependency for certificate generation during initialization:
+
+```sh
+# Ubuntu
+apt update && apt install openssl
+```
+
+### Docker (prebuilt images)
+
+Prebuilt images are available from [Docker Hub](https://hub.docker.com/r/simplexchat).
+
+1. Create directories for persistent configuration:
+
+   ```sh
+   mkdir -p $HOME/simplex/{xftp,smp}/{config,logs} && mkdir -p $HOME/simplex/xftp/files
+   ```
+
+2. Run:
+
+   **SMP router** - change `your_ip_or_domain`; `-e "PASS=password"` is optional:
+   ```sh
+   docker run -d \
+       -e "ADDR=your_ip_or_domain" \
+       -e "PASS=password" \
+       -p 5223:5223 \
+       -v $HOME/simplex/smp/config:/etc/opt/simplex:z \
+       -v $HOME/simplex/smp/logs:/var/opt/simplex:z \
+       simplexchat/smp-server:latest
+   ```
+
+   **XFTP router** - change `your_ip_or_domain` and `maximum_storage`:
+   ```sh
+   docker run -d \
+       -e "ADDR=your_ip_or_domain" \
+       -e "QUOTA=maximum_storage" \
+       -p 443:443 \
+       -v $HOME/simplex/xftp/config:/etc/opt/simplex-xftp:z \
+       -v $HOME/simplex/xftp/logs:/var/opt/simplex-xftp:z \
+       -v $HOME/simplex/xftp/files:/srv/xftp:z \
+       simplexchat/xftp-server:latest
+   ```
+
+### Installation script (Ubuntu)
+
+```sh
+curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/simplex-chat/simplexmq/stable/install.sh -o simplex-server-install.sh &&\
+if echo '53fcdb4ceab324316e2c4cda7e84dbbb344f32550a65975a7895425e5a1be757 simplex-server-install.sh' | sha256sum -c; then
+  chmod +x ./simplex-server-install.sh
+  ./simplex-server-install.sh
+  rm ./simplex-server-install.sh
+else
+  echo "SHA-256 checksum is incorrect!"
+  rm ./simplex-server-install.sh
+fi
+```
+
+### Build from source
+
+#### Using Docker
+
+Build from the [stable branch](https://github.com/simplex-chat/simplexmq/tree/stable):
+
+```sh
+git clone https://github.com/simplex-chat/simplexmq
+cd simplexmq
+git checkout stable
+DOCKER_BUILDKIT=1 docker build -t local/smp-server --build-arg APP="smp-server" --build-arg APP_PORT="5223" .
+DOCKER_BUILDKIT=1 docker build -t local/xftp-server --build-arg APP="xftp-server" --build-arg APP_PORT="443" .
+```
+
+Then run with the same Docker commands as above, replacing `simplexchat/smp-server:latest` with `local/smp-server` (and similarly for XFTP).
+
+#### Native build
+
+1. Install dependencies:
+
+   ```sh
+   # Ubuntu
+   sudo apt-get update && apt-get install -y build-essential curl libffi-dev libffi7 libgmp3-dev libgmp10 libncurses-dev libncurses5 libtinfo5 pkg-config zlib1g-dev libnuma-dev libssl-dev
+   export BOOTSTRAP_HASKELL_GHC_VERSION=9.6.3
+   export BOOTSTRAP_HASKELL_CABAL_VERSION=3.10.3.0
+   curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | BOOTSTRAP_HASKELL_NONINTERACTIVE=1 sh
+   ghcup set ghc "${BOOTSTRAP_HASKELL_GHC_VERSION}"
+   ghcup set cabal "${BOOTSTRAP_HASKELL_CABAL_VERSION}"
+   source ~/.ghcup/env
+   ```
+
+2. Build:
+
+   ```sh
+   git clone https://github.com/simplex-chat/simplexmq
+   cd simplexmq
+   git checkout stable
+   cabal update
+   cabal build exe:smp-server exe:xftp-server
+   ```
+
+3. Find binaries:
+
+   ```sh
+   cabal list-bin exe:smp-server
+   cabal list-bin exe:xftp-server
+   ```
+
+4. Initialize and run:
+
+   ```sh
+   smp-server init [-l] -n <fqdn>   # or --ip <ip>
+   smp-server start
+   ```
+
+### Linode StackScript
+
+[Deploy via Linode StackScript](https://cloud.linode.com/stackscripts/748014). Shared CPU Nanode with 1GB is sufficient.
+
+Configuration options:
+- SMP Server store log flag for queue persistence (recommended)
+- [Linode API token](https://www.linode.com/docs/guides/getting-started-with-the-linode-api#get-an-access-token) for automatic DNS and tagging (scopes: read/write for "linodes" and "domains")
+- Domain name (e.g., `smp1.example.com`) - the [domain must exist](https://cloud.linode.com/domains/create) in your Linode account
+
+After deployment (up to 5 minutes), get the server address from Linode tags or SSH: `smp://<fingerprint>@<fqdn>`.
+
+## Monitoring
+
+SMP and XFTP routers expose Prometheus metrics via a control port. The control port also supports commands for runtime inspection (queue counts, client counts, statistics). See module specs linked from each router section in [`spec/routers.md`](../spec/routers.md) (Control, Prometheus, Stats).
+
+## Protocol references
+
+- [SimpleX Messaging Protocol](../protocol/simplex-messaging.md) - SMP wire format and security properties
+- [XFTP Protocol](../protocol/xftp.md) - data packet protocol
+- [Push Notifications Protocol](../protocol/push-notifications.md) - NTF protocol
+- [SimpleX Network overview](../protocol/overview-tjr.md) - architecture and trust model

docs/smp-server-db-load.md

@@ -1,198 +0,0 @@
-# SMP server database load: root causes and fixes
-
-## Summary
-
-The recurring `pMsgFwdsOwn_pErrorsOther` spikes are a symptom of chronic disk saturation of the
-`smp_server` PostgreSQL database, not a proxy or forwarding bug. The baseline read load is the
-Prometheus scrape running whole-table `COUNT` scans of `msg_queues` (48.5M rows) every 30 seconds,
-compounded by the message-expiration sweep. The write load comes from indexing `updated_at`:
-`updateQueueTime` stamps it on the first send to each active queue per UTC day, and these non-HOT
-updates bloat the indexes to 104 GB and amplify WAL to about 425 GB per day (37% full-page images,
-~31% from `updateQueueTime`). Because the stamp is day-granular, this write burst clusters at 00:00 UTC,
-tipping the saturated disk into the observed spike and loading the pgBackRest backup repository.
-
-Code fixes replace the largest metric scan with an estimate and index the sweep and notifier counts.
-Reclaiming the index bloat, which also clears the remaining metric scan, and keeping it down are
-operator actions, because PostgreSQL never shrinks indexes automatically.
-
-## Environment
-
-The affected deployment runs the pure PostgreSQL backend (`store_queues: database`,
-`store_messages: database`), schema `smp_server`, `db_pool_size 20`, `prometheus_interval 30`,
-`[NAMES]` enabled, own-server proxying enabled, and forwards routed over a SOCKS proxy.
-
-`msg_queues`: 48.5M live rows, 12.3% dead tuples, 14 GB heap, 104 GB indexes, 118 GB total.
-
-## Evidence
-
-`pg_stat_statements` over a 5 minute window, ordered by disk reads (`track_io_timing` was off, so
-`shared_blks_read` is the disk proxy; the two `EXPLAIN ANALYZE` rows are manual and excluded):
-
-| query | calls | mean ms | reads | share of all reads |
-| --- | --- | --- | --- | --- |
-| `getEntityCounts` (the six-`COUNT` metric query) | 30 | 111,456 | 990 GB | 70.7% |
-| queue-record load (`SELECT recipient_id, …`) | 47,027 | 121 | 89 GB | 6.3% |
-| expiration sweep batch (`array_agg` in `expire_old_messages`) | 13 | 233,143 | 62 GB | 4.5% |
-| message peek (`DISTINCT ON (recipient_id)`) | 46,932 | 76 | 43 GB | 3.0% |
-| `updateQueueTime` (`UPDATE msg_queues SET updated_at`) | 172,885 | 4 | 7.7 GB | 0.5% |
-| `write_message` | 110,911 | 5 | 7.0 GB | 0.5% |
-
-`EXPLAIN (ANALYZE, BUFFERS)` of each `msg_queues` count inside `getEntityCounts`:
-
-| count | plan | time |
-| --- | --- | --- |
-| `queue_count` (`deleted_at IS NULL`) | Parallel Seq Scan | 30 s |
-| `notifier_count` (`deleted_at IS NULL AND notifier_id IS NOT NULL`) | Parallel Seq Scan | 32.6 s |
-| `ntf_service_queues_count` (`ntf_service_id IS NOT NULL AND deleted_at IS NULL`) | Parallel Seq Scan | 30.2 s |
-| `rcv_service_queues_count` (`rcv_service_id IS NOT NULL AND deleted_at IS NULL`) | Index Only Scan | 1.4 ms |
-
-The expiration sweep batch: 229 s for one 10,000-row batch, walking `msg_queues_pkey` and discarding
-809,729 rows by filter to find 10,000 expirable ones.
-
-`pg_stat_wal` over the measurement window: 27 TB of WAL accumulated, 37.2% of records full-page
-images, averaging about 425 GB per day; recent days reach ~470 GB (from the `archive-push` log) as
-the queue count grows.
-
-Per-statement WAL from `pg_stat_statements` (`wal_bytes`), leaf statements only (this instance runs
-`pg_stat_statements.track = all`, so the `write_message` wrapper double-counts the `INSERT` and
-`UPDATE` it runs, and is excluded):
-
-| category | share of WAL | main statements |
-| --- | --- | --- |
-| `msg_queues` updates | ~42% | `updateQueueTime` ~31%, `msg_can_write` flags ~9%, insert/delete ~2% |
-| `messages` insert and delete | ~34% | message `INSERT`, `DELETE` by `message_id` and `recipient_id` |
-| read path (hint bits, FPIs on SELECT) | ~24% | message peek ~15%, `msg_queue_size`, `getEntityCounts` |
-
-## Root causes
-
-1. Prometheus scrape. `getEntityCounts` (`QueueStore/Postgres.hs:154`) runs six `COUNT(1)` subqueries
-   every `prometheus_interval` (30 s), called from the metrics path (`Server.hs:698`). Four count
-   `msg_queues`; three of those seq-scan the 48.5M-row heap. At 70.7% of all reads it is the dominant
-   disk consumer, and it runs continuously (mean 111 s per scrape exceeds the 30 s interval).
-
-2. Message-expiration sweep. `expireMessagesThread` (`Server.hs:477`) calls the `expire_old_messages`
-   procedure (`server_schema.sql`), whose inner batch query selects expirable queues ordered by
-   `recipient_id`. With `oldQueue = 0` the `updated_at` filter matches everything, so the planner
-   walks the primary key and filters `msg_queue_expire`, discarding ~99% of rows per batch.
-
-3. Index bloat. 104 GB of indexes on a 14 GB table. `updated_at` is part of
-   `idx_msg_queues_updated_at_recipient_id` (`server_schema.sql:526`) and `updateQueueTime`
-   (`QueueStore/Postgres.hs:443`) updates it on the day's first send per queue (172,885 times in the
-   window above). Updating an indexed column prevents HOT updates, so every update writes a new row
-   version plus new entries in every index on the table, and the old index entries accumulate as bloat.
-
-4. WAL amplification. Only 15.6% of `msg_queues` updates are HOT, because its composite index covers
-   `updated_at` and `msg_queue_expire`, both changed on the hot path (cause 3); the rest rewrite every
-   index on the table. With the 104 GB of bloated indexes and frequent checkpoints (`max_wal_size 4GB`,
-   `checkpoint_timeout 5min`, `wal_compression off`), 37% of WAL records are full-page images, and the
-   server generates about 425 GB of WAL per day (27 TB accumulated; recent days near 470 GB).
-   `updateQueueTime`, stamping `updated_at` on the ~5.3M daily-active queues, is the single largest
-   statement at ~31% of WAL (see Evidence). pgBackRest `archive-push` ships all of it, loading the
-   primary disk and the backup repository. The load peaks
-   at 00:00 UTC because `getSystemDate` rounds `updated_at` to the UTC day, so the first send to each
-   active queue after midnight runs `updateQueueTime`, clustering these writes and the `archive-push`
-   volume in the 00h hour.
-
-## Index bloat and autovacuum
-
-Autovacuum is running (60 runs on this table) and is not misconfigured, but two limits apply.
-
-Autovacuum reclaims heap dead tuples and refreshes planner statistics; it does not shrink indexes.
-Only `REINDEX` or `pg_repack` rebuilds a bloated B-tree. Index bloat therefore has no automatic
-remedy.
-
-Default autovacuum pacing assumes moderate churn. `autovacuum_vacuum_scale_factor = 0.2` waits until
-20% of the table is dead (about 9.7M rows here) before vacuuming, and cost-based throttling caps its
-I/O, so a hot table stays around 12% dead. Per-table tuning makes it run sooner and faster.
-
-The amplifier is indexing `updated_at`, a column updated on every send, which the fixes and operator
-actions below address.
-
-## Implemented fixes (batch 1)
-
-Branch `sh/leaks-batch-1`, two commits.
-
-| commit | change | effect |
-| --- | --- | --- |
-| `88673eee` | migration `20260916_prometheus_indexes` adds partial indexes `idx_msg_queues_expire (recipient_id) WHERE deleted_at IS NULL AND msg_queue_expire` and `idx_msg_queues_notifier_active (notifier_id) WHERE deleted_at IS NULL AND notifier_id IS NOT NULL` | the sweep batch and `notifier_count` become index scans |
-| `2912be1e` | `getEntityCounts.queue_count` uses `pg_class.reltuples` (`QueueStore/Postgres.hs:154`) | removes the 30 s, 990 GB whole-table `COUNT` on every scrape |
-
-Verified: both indexes are created and used as index-only scans, the schema-dump test passes, and
-`smp-server` builds. `queueCount` is consumed only by metrics, logs, and display
-(`Server.hs:528,698,815,2369`), so an estimate is safe.
-
-Not fixed by new indexes:
-
-- `ntf_service_queues_count` seq-scans only because of index bloat; `idx_msg_queues_ntf_service_id`
-  already covers it. `REINDEX` restores the index-only scan, as proven by `rcv_service_queues_count`.
-- `rcv_service_queues_count` and the two `services` counts are already cheap.
-
-## Implemented fixes (batch 2)
-
-Migration `20260917_msg_queues_hot` and `MsgStore/Postgres.hs`.
-
-| change | effect |
-| --- | --- |
-| drop `idx_msg_queues_updated_at_recipient_id`, set `msg_queues` `fillfactor = 80` | `updateQueueTime` updates become HOT, so they rewrite no index entries; removes its index bloat and cuts its WAL |
-| remove the dead `updated_at > p_old_queue` filter and its parameter from `expire_old_messages`, matching the `CALL` (`MsgStore/Postgres.hs:113`) | the sweep query runs as an index-only scan on `idx_msg_queues_expire`; `updated_at` is no longer read on the hot path |
-| autovacuum reloptions on `msg_queues`, `messages` (and its TOAST), and `services` | keeps dead tuples and bloat low per table (see the table below) |
-
-Verified on PostgreSQL 16 with the daily-active pattern (~11% of queues updated per day): `updateQueueTime`
-goes from 0% HOT (indexed `updated_at`) to 100% HOT at `fillfactor = 80` (96.7% at 90), and per-update
-WAL drops about 4x. The HOT ratio depends on how many rows per page change before autovacuum reclaims
-space, so it is workload-dependent; `fillfactor = 80` reached 100% for this fraction, and updating a much
-larger share of queues at once would need a lower value. `fillfactor` applies to pages rewritten after
-the change, so the ratio ramps up as the heap turns over, or immediately after `pg_repack`.
-
-The Postgres message store does not read `updated_at`, so no query depends on the dropped index.
-
-Autovacuum reloptions applied by the migration:
-
-| table | reloptions | reason |
-| --- | --- | --- |
-| `msg_queues` | `fillfactor 80`, `autovacuum_vacuum_scale_factor 0.02`, `autovacuum_analyze_scale_factor 0.01`, `autovacuum_vacuum_cost_limit 1000` | HOT headroom; vacuum/analyze at ~2%/1% churn instead of 20%/10%; faster vacuum |
-| `messages` (and TOAST) | `autovacuum_vacuum_scale_factor 0.02`, `autovacuum_analyze_scale_factor 0.01`, `toast.autovacuum_vacuum_scale_factor 0.02` | high insert and delete churn; 152 GB, mostly TOASTed message bodies |
-| `services` | `fillfactor 70`, `autovacuum_vacuum_threshold 1000`, `autovacuum_vacuum_scale_factor 0` | tiny, very hot table was autovacuumed tens of thousands of times; make updates HOT and vacuum it far less |
-
-## Operator actions
-
-1. Reclaim index bloat (online, no lock; needs free disk near the index size; run largest first):
-   ```
-   REINDEX INDEX CONCURRENTLY smp_server.idx_msg_queues_updated_at_recipient_id;
-   REINDEX INDEX CONCURRENTLY smp_server.idx_msg_queues_notifier_id;
-   REINDEX INDEX CONCURRENTLY smp_server.idx_msg_queues_ntf_service_id;
-   REINDEX INDEX CONCURRENTLY smp_server.idx_msg_queues_rcv_service_id;
-   REINDEX INDEX CONCURRENTLY smp_server.idx_msg_queues_sender_id;
-   REINDEX INDEX CONCURRENTLY smp_server.idx_msg_queues_link_id;
-   ANALYZE smp_server.msg_queues;
-   ```
-   Expect `pg_indexes_size('smp_server.msg_queues')` to drop from 104 GB to roughly 12 GB (measured
-   after reindexing). `ANALYZE` restores the index-only scan for `ntf_service_queues_count`. With the
-   batch-2 `fillfactor` the index bloat no longer rebuilds, so this is a one-time reclaim.
-
-2. Autovacuum reloptions are applied by the batch-2 migration (`msg_queues`, `messages` and its TOAST,
-   `services`), so no manual `ALTER TABLE` is needed. Reclaim the current `messages` bloat once with
-   `pg_repack smp_server.messages` (152 GB heap and TOAST, roughly 30-50 GB reclaimable); running
-   `pg_repack` on `msg_queues` also makes `fillfactor = 80` effective immediately rather than as pages
-   turn over. With `fillfactor = 80` the `msg_queues` index bloat no longer rebuilds, so scheduled
-   reindexing is rarely needed. To automate it anyway, run `reindexdb --concurrently` or `pg_repack`
-   from a scheduler; `pg_cron` runs each job inside a transaction and cannot execute
-   `REINDEX ... CONCURRENTLY`, so use `pg_cron` only for `VACUUM`/`ANALYZE` or a non-concurrent
-   maintenance-window `REINDEX`.
-
-3. Cut WAL and checkpoint pressure (all reloadable, no restart):
-   ```
-   ALTER SYSTEM SET wal_compression = 'lz4';
-   ALTER SYSTEM SET max_wal_size = '32GB';
-   ALTER SYSTEM SET checkpoint_timeout = '30min';
-   SELECT pg_reload_conf();
-   ```
-   Fewer checkpoints produce fewer full-page images; `wal_compression` shrinks those that remain.
-   Together these reduce the WAL written on the primary and shipped by `archive-push`. The message
-   insert and delete volume itself is fixed by traffic, but most of its WAL cost is amplification
-   (full-page images and hint-bit writes on bloated pages), which these settings and `REINDEX` reduce.
-   This is separate from pgBackRest repo-side compression, which does not reduce WAL on the primary.
-   Verify with `pg_stat_reset_shared('wal')`, wait, then re-check `wal_fpi` and `wal_bytes` in
-   `pg_stat_wal`.
-
-4. Enable `track_io_timing = on` so disk wait time is measurable in `pg_stat_statements`.

plans/2026-06-01-bbs-bindings.md

@@ -1,126 +0,0 @@
-# BBS+ Bindings for simplexmq
-
-Haskell FFI bindings to libbbs for BBS+ signatures. General-purpose - the module knows nothing about specific applications.
-
-## How BBS+ works
-
-BBS+ signs a fixed list of N messages. Each message is an arbitrary byte array. The signer signs all N messages at once with one signature.
-
-The holder of the signature can then generate a proof that selectively discloses some messages and hides others. The verifier learns the disclosed messages and confirms they were signed by the signer, but learns nothing about the hidden messages. Different proofs from the same signature are unlinkable.
-
-Key constraint: the total number of messages N is fixed at signing time. The verifier must know N. A proof generated from a 3-message signature cannot be verified as a 2-message proof.
-
-## Types
-
-```haskell
-newtype BBSSecretKey = BBSSecretKey ByteString  -- 32 bytes
-newtype BBSPublicKey = BBSPublicKey ByteString  -- 96 bytes (BLS12-381 G2 point)
-newtype BBSSignature = BBSSignature ByteString  -- 80 bytes
-newtype BBSProof = BBSProof ByteString          -- 272 + 32 * numUndisclosed bytes
-newtype BBSHeader = BBSHeader ByteString        -- always-disclosed context (e.g. protocol identifier)
-newtype BBSPresHeader = BBSPresHeader ByteString -- random nonce for proof unlinkability
-```
-
-All newtypes get StrEncoding (base64url), ToJSON/FromJSON (via strToJSON/strParseJSON), Eq, Show.
-
-## Functions
-
-```haskell
-bbsKeyGen :: IO (Either String BBSKeyPair)  -- BBSKeyPair = (BBSPublicKey, BBSSecretKey)
-
--- pk is derived from sk internally, so it is not a parameter
-bbsSign
-  :: BBSSecretKey
-  -> BBSHeader            -- always-disclosed context
-  -> [ByteString]         -- all N messages
-  -> IO (Either String BBSSignature)
-
--- C order: pk, signature, header, presentation_header, disclosed_indexes, messages
-bbsProofGen
-  :: BBSPublicKey
-  -> BBSSignature
-  -> BBSHeader            -- must match what was signed
-  -> BBSPresHeader        -- random nonce bound into the proof
-  -> [Int]                -- disclosed indexes (0-based)
-  -> [ByteString]         -- all N messages (needed internally, hidden ones not revealed in proof)
-  -> IO (Either String BBSProof)
-
--- C order: pk, proof, header, presentation_header, disclosed_indexes, n, messages
-bbsProofVerify
-  :: BBSPublicKey
-  -> BBSProof
-  -> BBSHeader            -- must match what was signed
-  -> BBSPresHeader        -- must match what was used in bbsProofGen
-  -> [Int]                -- disclosed indexes
-  -> Int                  -- total message count N
-  -> [ByteString]         -- disclosed messages only
-  -> IO Bool
-```
-
-## How applications use it
-
-An application defines:
-- A message layout: which index means what
-- Which indexes are disclosed vs hidden
-- How to encode application values as ByteString messages
-
-### Badge example (in simplex-chat, not in this module)
-
-Message layout (always 3 messages):
-- Index 0: master secret (32 random bytes) - HIDDEN
-- Index 1: expiry (UTF-8 encoded timestamp string) - DISCLOSED
-- Index 2: badge type (UTF-8 encoded, e.g. "supporter") - DISCLOSED
-
-Signing (v2, on the server):
-```
-bbsSign sk header [ms, encodeUtf8 "2026-07-31", encodeUtf8 "supporter"]
-```
-
-Proof generation (v2, on the client):
-```
-bbsProofGen pk sig header presHeader [1, 2] [ms, encodeUtf8 "2026-07-31", encodeUtf8 "supporter"]
-```
-
-Proof verification (v1, on the recipient):
-```
-bbsProofVerify pk proof header presHeader 3 [1, 2] [encodeUtf8 "2026-07-31", encodeUtf8 "supporter"]
-```
-
-The recipient only sees the proof, presentationHeader, expiry string, and badge type string. They verify these were signed by the server (pk is hardcoded). They never see the master secret.
-
-Expiry is always present as a string. Monthly badges use a date like `"2026-07-31"`, lifetime badges use `"lifetime"`. BBS+ doesn't interpret the bytes - expiry semantics are the application's responsibility. This keeps the message count fixed at 3 for all badge types.
-
-## libbbs C API mapping
-
-```c
-int bbs_keygen_full(ciphersuite, sk, pk)
-int bbs_sign(ciphersuite, sk, pk, signature, header, header_len, n, messages, message_lens)
-int bbs_proof_gen(ciphersuite, pk, signature, proof, header, header_len, presentation_header, presentation_header_len, disclosed_indexes, disclosed_indexes_len, n, messages, message_lens)
-int bbs_proof_verify(ciphersuite, pk, proof, proof_len, header, header_len, presentation_header, presentation_header_len, disclosed_indexes, disclosed_indexes_len, n, messages, message_lens)
-```
-
-We use `bbs_sha256_ciphersuite`. The header parameter is exposed in all Haskell functions - the application decides what to put there. Tests use `"SimpleX"` as header.
-
-The `presentation_header` parameter is what we call `presentationHeader`.
-
-In `bbs_proof_verify`, the `n` parameter is the total number of messages (not the number of disclosed messages). The `messages` array contains only the disclosed messages, and `disclosed_indexes` maps each to its position in the original message list.
-
-## Build
-
-Submodules in cbits/:
-- `cbits/libbbs` - https://github.com/Fraunhofer-AISEC/libbbs
-- `cbits/blst` - https://github.com/supranational/blst (libbbs dependency)
-
-C sources in cabal: `cbits/blst/src/server.c`, `cbits/blst/build/assembly.S`, libbbs source files.
-Include dirs: `cbits/blst/bindings/`, `cbits/blst/src/`, `cbits/libbbs/include/`, `cbits/libbbs/src/`.
-C flags: `-D__BLST_PORTABLE__` for cross-CPU-generation compatibility.
-
-## Tests
-
-- Keygen produces keys of correct size
-- Sign + proofGen + proofVerify roundtrip succeeds
-- Tampered proof fails verification
-- Tampered disclosed message fails verification
-- Wrong public key fails verification
-- Two proofs from same credential with different nonces both verify
-- Proof size matches expected (272 + 32 * numUndisclosed)

plans/2026-06-27-service-subscriptions-memory-leak.md

@@ -1,57 +0,0 @@
-## Root cause: orphaned `Sub` entries in the service client's `subscriptions` map
-
-**The leak is service-specific and was introduced by PR #1667 "messaging services" (`f0b7a4be`).** A long-lived messaging-service connection accumulates per-queue `Sub` records in its `Client.subscriptions` map that are **never removed** when the associated queues are deleted or unassociated — only the counter is decremented. Over normal queue churn the map grows monotonically for the entire lifetime of the service connection.
-
-### The proof — an asymmetry between two handlers in `serverThread`
-
-Both individual queue subscriptions and service subscriptions store a `Sub` per queue in `Client.subscriptions` (= `clientSubs` for the SMP subscriber thread, wired at `Server.hs:189`). When a queue ends/is deleted, the two paths diverge:
-
-**Individual subscriber — entry IS removed** (`Server.hs:332`, `346`):
-```haskell
-CSAEndSub qId -> atomically (endSub c qId) >>= a unsub_   -- :332
-  ...
-endSub c qId = TM.lookupDelete qId (clientSubs c) >>= (removeWhenNoSubs c $>)  -- :346
-```
-
-**Service subscriber — entry is NOT removed** (`Server.hs:336-340`):
-```haskell
-CSAEndServiceSub qId -> atomically $ do
-  modifyTVar' (clientServiceSubs c) decrease   -- decrements serviceSubsCount
-  modifyTVar' totalServiceSubs decrease        -- decrements global count
-  where decrease = subtractServiceSubs (1, queueIdHash qId)
-  -- never touches (clientSubs c) — the Sub for qId stays forever
-```
-
-### Where the orphaned entries are added (both new in this PR)
-- `Server.hs:1860-1862` — on service subscribe (`SSUB`), one `Sub` inserted per queue that has a pending message.
-- `Server.hs:2039-2043` (`newServiceDeliverySub`) — on **every** `SEND` to a service-associated queue with no existing sub, a `Sub` is inserted into the service client's `subscriptions`. After delivery the thread state resets to `NoSub` (`:2069`) but the map entry remains as a "already delivering" marker (`:1856-1859`).
-
-### Why they leak
-The only places the service client's `subscriptions` map is cleared are:
-- `clientDisconnected` — `swapTVar subscriptions M.empty` (`Server.hs:1097`) — only on disconnect.
-- `CSADecreaseSubs` — `swapTVar (clientSubs c) M.empty` (`Server.hs:343`) — only on full service takeover by another connection.
-- `delQueueAndMsgs` — `TM.lookupDelete entId $ subscriptions clnt` (`Server.hs:2164`) — but `clnt` here is **the recipient deleting its own queue, not the service client**. The service's entry for that queue is reached only via the `CSDeleted → endServiceSub → CSAEndServiceSub` path (`Server.hs:306, 313, 336`), which decrements the counter but leaves the map entry.
-
-**Concrete scenario (fully traced):** Service `S` subscribes (`SSUB`) and stays connected for days. Recipient `R` owns service-associated queue `Q`. A `SEND` to `Q` inserts a `Sub` into `S.subscriptions[Q]` (`:2042`). `R` later deletes `Q` → `delQueueAndMsgs` runs on `R`'s connection, removes `Q` from `R.subscriptions`, decrements counters, enqueues `CSDeleted Q (Just S)` (`:2167`) → `serverThread` runs `CSAEndServiceSub Q` for `S` (`:336`), decrementing `S.serviceSubsCount` but **leaving `S.subscriptions[Q]` in place**. Net: one orphaned `Sub` (record + 2 TVars) per service-associated queue ever deleted/unassociated, never reclaimed until `S` disconnects. The logical counter `serviceSubsCount` correctly drops, so the map size diverges from the counter — making the leak invisible to the existing service-sub metric.
-
-### Verdict
-This is a deterministic, static-provable memory leak — no production logging needed to confirm the existence; the asymmetry between `CSAEndSub` (removes) and `CSAEndServiceSub` (doesn't) is the smoking gun. It is specific to messaging-service certificate clients, which is exactly the population added by the services/certificate PR.
-
-### Secondary findings (lower impact, same PR area, not the primary cause)
-- **`forkClient` register-after-fork race** (`Server.hs:1356-1359`): if the forked action's `finally` delete (`:1358`) runs before the parent's `IM.insert` (`:1359`), a `Weak ThreadId` of a dead thread is left in `endThreads` until disconnect. Pre-existing, tiny per-entry, but exercised far more by the PR's higher END/DELD volume.
-- **Wrong-client counter decrement** (`Server.hs:2166`): `delQueueAndMsgs` decrements `serviceSubsCount` of the *deleting* client, not the service; harmless for non-service deleters (floored at 0) but corrupts accounting if a service deletes its own queue.
-
----
-
-### Recommended fix (mirror `endSub` in the service path)
-Make `CSAEndServiceSub` also delete the per-queue `Sub` and cancel its delivery thread, exactly as `CSAEndSub`/`endSub` do for individual subscribers. Roughly:
-
-```haskell
-CSAEndServiceSub qId -> do
-  s_ <- atomically $ do
-    modifyTVar' (clientServiceSubs c) decrease
-    modifyTVar' totalServiceSubs decrease
-    TM.lookupDelete qId (clientSubs c) <* removeWhenNoSubs c
-  forM_ unsub_ $ \unsub -> mapM_ unsub s_
-  where decrease = subtractServiceSubs (1, queueIdHash qId)
-```

plans/2026-07-11-service-rpc-implementation.md

@@ -1,135 +0,0 @@
-# Service RPC implementation plan
-
-RFC: [../rfcs/2026-07-11-service-rpc.md](../rfcs/2026-07-11-service-rpc.md)
-
-Depends on: [2026-07-12-address-dr-implementation.md](2026-07-12-address-dr-implementation.md). RPC establishes the double ratchet from the address exactly as address-DR does; this is the RPC layer on top of it.
-
-**Status: implemented and tested in this repo.** Service-side idempotency (single execution by request hash) is deferred. The `simplex-chat` integration is a separate repo.
-
-**Scope.** One request, one response — no continuation, no streaming.
-
-One DR-advertising contact address serves both flows: the owner branches per incoming request on the decrypted inner message — `AgentConnInfoReply` opens a connection (`REQ`), `AgentServiceRequest` answers an RPC (`SREQ`). A request gets exactly one reply, a response or a rejection; both are the single confirming message on the requester's reply queue Q_A, after which the ephemeral reply connection is torn down. Response and rejection are the same operation parameterized by the inner message (`AgentServiceResponse` payload vs `AgentRejection` reason) and outcome (the call returns the payload vs throws an agent error).
-
-## RPC messages
-
-No new outer envelope: the request reuses `AgentContactRequest` (tag `'A'`); the reply reuses `AgentConfirmation` (the only message on Q_A).
-
-Inner `AgentMessage` (ratchet-encrypted, parsed by `parseMessage`), siblings of `AgentConnInfoReply`:
-
-```haskell
-  | AgentServiceRequest (NonEmpty SMPQueueInfo) MsgBody  -- 'A': reply queue Q_A + opaque payload
-  | AgentServiceResponse MsgBody                         -- 'P': response payload (single, terminal)
-  | AgentRejection ByteString                            -- 'J': refusal reason (single, terminal)
-```
-
-`AgentServiceRequest` carries Q_A (as `AgentConnInfoReply` does); its constructor is the only thing that distinguishes `REQ` from `SREQ`. Delivery `msgType`: `AM_SRV_RESP` routes to `sendConfirmation` (the reply is the confirming first message on Q_A). `AM_SRV_REQ` is never stored — the request is sent synchronously inside `joinConnSrv'` via `sendInvitation`, so its arms in the delivery worker are unreachable and assert (`logError`).
-
-## Ratchet establishment — reuse of the address-DR flow
-
-`joinConnSrv'` takes `mkInner :: SMPQueueInfo -> AgentMessage`; `joinConnSrv` is the one-line wrapper passing `AgentConnInfoReply`. `sendServiceRequest'` passes `AgentServiceRequest`.
-
-**Request (client).** `serviceRequest_` fails fast with `A_SERVICE ASENotDRAddress` if the address carries no ratchet keys, then creates the client connection via `newConnToJoin` with `serviceRequestExpiresAt = Just (now + reqTimeout)` (the persisted per-request deadline), registers a one-shot `TMVar` in `serviceRequests`, sends the request, and blocks on the `TMVar` up to `reqTimeout`. The connection is `RcvConnection` (Q_A) with the send ratchet.
-
-**Request (service).** `smpContactRequest` decrypts `encConnInfo` and branches on the inner message; both branches call the same `storeInvitation` → `conn_invitations`, differing only in the kind column and event:
-
-- `AgentConnInfoReply` → `REQ` (`service_request = 0`).
-- `AgentServiceRequest _ payload` → `SREQ invId payload` (`service_request = 1`).
-
-Before storing, it **deduplicates** by the sender's ratchet-key hash (`checkRatchetKeyHashExists`/`addProcessedRatchetKeyHash`, the mechanism `newRatchetKey` uses): a redelivered/retried request reuses the same Q_A and the same `e2eSndParams`, so the hash matches and the duplicate is dropped — one invitation and one `REQ`/`SREQ` per request. Receive-time establishment on unauthenticated input — the address-DR abuse bound applies.
-
-**The reply (service).** `prepareReply` fetches the invitation, enforces the kind (`CMD PROHIBITED` on the wrong one), and rejects a stale request (`A_SERVICE ASETimeout` + delete) older than `serviceResponseTimeout`; then `newConnToAccept` + `startJoinInvitationDR` build the one-directional `SndQueue` to Q_A (no reply queue back), and `storeConfirmation` queues the inner message. `sendReplySync` secures Q_A, submits the message, and deletes the connection with wait-for-delivery — **deleting the connection on failure too** (`catchAllErrors`), so a failed secure/submit does not orphan it. `sendServiceReplyAsync` defers secure+deliver+delete to the `ICReplyDel` command (retried, survives a down server). `sendServiceReply`/`Async` and `replyRequest_` return the reply `ConnId` so the caller can correlate the `SENT` event on that throwaway connection.
-
-**The response (client).** The single `AgentConfirmation` on Q_A reaches `processConnInfo` (the `RcvConnection … New` branch). Dispatch is gated on `serviceRequestExpiresAt` and the kinds are mutually exclusive:
-
-- `AgentConnInfoReply` **only when `isNothing serviceRequestExpiresAt`** (a contact connection) → `processConf`.
-- `AgentServiceResponse` only when `isJust` → the request `TMVar` gets `Right payload`.
-- `AgentRejection` when `isJust` → `Left (A_SERVICE (ASERejected reason))`; when `isNothing` → contact `RJCT`.
-- anything else → `prohibited`.
-
-The `isNothing` guard on `AgentConnInfoReply` is a security boundary: without it a malicious service could send `AgentConnInfoReply` on an RPC reply queue and drive it into the contact-`CONF` path. `dispatchServiceReply` puts the result into the `serviceRequests` `TMVar`; a reply with no pending request (e.g. post-restart) is `ERR (A_SERVICE ASENoPendingRequest)`.
-
-## Rejection
-
-A rejection is `AgentRejection reason` — the same single confirming message on Q_A as a response.
-
-- **Kind guard.** `rejectContact` only on a contact invitation, `rejectServiceRequest`/`sendServiceReply` only on a request; wrong kind is `CMD PROHIBITED`. `rejectRequest_` enforces the kind **even on a `Nothing` (silent-drop) reject** — it fetches the invitation and checks before deleting, so `rejectContact … Nothing` cannot delete a service request (or vice versa).
-- `reject*` take `Maybe ByteString`: `Nothing` → delete the invitation, send nothing (the requester times out); `Just reason` → the reply path with `AgentRejection`.
-- Requester side: `AgentRejection` on a contact reply queue → `RJCT`; on an RPC reply queue → a thrown `A_SERVICE (ASERejected reason)`.
-
-## Reply connections and cleanup
-
-No reply-queue table and no new connection type.
-
-- **Requester reply queue** (`RcvConnection` on Q_A): `connections.service_request_expires_at` is non-null only here; it is the persisted request deadline, used both to gate CONF dispatch and to reap the connection. In-memory routing is `serviceRequests :: TMap ConnId (TMVar (Either AgentErrorType MsgBody))`.
-- **Timeout race.** The async `JOIN` worker holds `withConnLock c connId` around `joinConnSrv'`, and `serviceRequest_`'s cleanup holds the same lock around `TM.delete` + `deleteConnectionAsync'`. This serializes the send with the timeout teardown, so a timing-out call cannot delete the connection mid-send; after cleanup the worker's re-check of `serviceRequests` finds nothing and skips.
-- **Service reply connection** (`SndConnection` to Q_A): ephemeral — created, sends the one reply, deleted with wait-for-delivery in the same operation.
-- **Cleanup** (`cleanupManager`, `deleteExpiredServiceReqs`): `deleteExpiredServiceRequests` reaps unanswered `conn_invitations` (service side) older than `serviceResponseTimeout`; `getExpiredServiceConns` (`service_request_expires_at < now`) → `deleteConnectionsAsync'` reaps orphaned requester reply queues.
-
-## Database schema
-
-`M20260712_address_dr_rpc` (SQLite + PostgreSQL) creates `address_ratchet_keys` (address-DR) and adds:
-
-```sql
-ALTER TABLE conn_invitations ADD COLUMN service_request INTEGER NOT NULL DEFAULT 0; -- service side: 1 = RPC request
-ALTER TABLE connections ADD COLUMN service_request_expires_at TEXT;                 -- client side: request deadline; gating + cleanup (nullable)
-```
-
-The down migration drops the columns then the table/index. Schema dump tests pass (up, down, STRICT).
-
-## Agent API — `Simplex.Messaging.Agent`
-
-```haskell
--- service: send the one response, return the reply ConnId, then tear the reply connection down.
-sendServiceReply      :: AgentClient -> NetworkRequestMode -> UserId -> InvitationId -> MsgBody -> AE ConnId
-sendServiceReplyAsync :: AgentClient -> ACorrId -> UserId -> InvitationId -> MsgBody -> AE ConnId
-
--- refuse a request (Just reason = AgentRejection; Nothing = silent drop). PROHIBITED on wrong kind.
-rejectServiceRequest      :: AgentClient -> NetworkRequestMode -> UserId -> InvitationId -> Maybe ByteString -> AE ()
-rejectServiceRequestAsync :: AgentClient -> ACorrId -> UserId -> InvitationId -> Maybe ByteString -> AE ()
-rejectContact      :: AgentClient -> NetworkRequestMode -> UserId -> ConfirmationId -> Maybe ByteString -> AE ()
-rejectContactAsync :: AgentClient -> ACorrId -> UserId -> ConfirmationId -> Maybe ByteString -> AE ()
-
--- client: establish the ratchet from the address, send the request, block on the reply TMVar up to the timeout
--- (Nothing = serviceRequestTimeout; Just t overrides per request), returning the payload. Sync fails fast if the
--- server is down; async enqueues a retried JOIN command that survives an outage.
-sendServiceRequest      :: AgentClient -> NetworkRequestMode -> UserId -> ConnectionRequestUri 'CMContact -> Maybe NominalDiffTime -> MsgBody -> AE MsgBody
-sendServiceRequestAsync :: AgentClient -> UserId -> ConnectionRequestUri 'CMContact -> Maybe NominalDiffTime -> MsgBody -> AE MsgBody
-```
-
-Both client calls share `serviceRequest_`; the async `JOIN` worker branches on the `JRServiceReq` command to send `AgentServiceRequest`. The call blocks on the `TMVar` and returns synchronously — no events, no correlation for the app.
-
-Events (`AEvent`, entity is the address connection):
-
-```haskell
-SREQ :: InvitationId -> MsgBody -> AEvent AEConn  -- payload = the request; mirrors REQ.
-RJCT :: ConnInfo -> AEvent AEConn                 -- contact-request rejection reason.
-```
-
-Errors (`SMPAgentError`):
-
-```haskell
-| A_SERVICE {serviceError :: AgentServiceError}
-
-data AgentServiceError
-  = ASERejected {rejectReason :: Text}  -- service refused (Text: JSON-serializable, UTF-8-decoded from the reason bytes)
-  | ASETimeout                          -- no reply within the timeout
-  | ASENoPendingRequest                 -- a reply arrived with no pending request (e.g. post-restart)
-  | ASENotDRAddress                     -- the target address advertises no ratchet keys (fail fast, no send)
-```
-
-Config (`AgentConfig`): `serviceRequestTimeout` (30 s, client wait, overridable per request) and `serviceResponseTimeout` (180 s, service reply window and cleanup TTL; must exceed `serviceRequestTimeout`).
-
-## Idempotency (deferred)
-
-Not built. When built, the service will key a request by hash and cache the one response for a retention period, answering a repeat from storage without reaching the bot — single execution over at-least-once delivery, with its own tables.
-
-## Tests
-
-In `FunctionalAPITests` (plus the encoding roundtrip in `ConnectionRequestTests`), passing with `-O0`:
-
-- Request → one response, sync (`sendServiceReply`) and async (`sendServiceReplyAsync`).
-- Request → rejection (`rejectServiceRequest (Just reason)` → thrown `A_SERVICE (ASERejected …)`).
-- Resilience: `server down → send → up → receive → down → reply → up → receive response`.
-- No regression: the contact rejection and DR-join suites still pass.
-
-The `simplex-chat` end-to-end tests (happy path, drop-when-off, non-DR fail-fast) live in that repo.

plans/2026-07-12-address-dr-implementation.md

@@ -1,279 +0,0 @@
-# Establishing the double ratchet from address data - implementation plan
-
-RFC: [../rfcs/2026-07-12-address-pqdr-keys.md](../rfcs/2026-07-12-address-pqdr-keys.md)
-
-All references are to the current tree. Names of new constructors, fields, tables and functions are provisional.
-
-Goal: a contact address advertises the owner's X3DH parameters in link data; a requester establishes the double ratchet in its first message, so that message and the profile in it are under the ratchet with post-quantum protection. The change reuses the invitation/confirmation machinery, with the requester in the joiner role and the owner in the initiator role - opposite to today's contact flow, but every message and code path below is reused.
-
-Version: `addressDRVersion = VersionSMPA 8`, a plain agent-layer bump; `currentSMPAgentVersion` goes 7 → 8 (Agent/Protocol.hs:317-324). It gates the `AgentConfirmation.ratchetKeyId` field and the DR-from-address behavior. The receive-at-address path relies on ratchet-on-confirmation, already present since `ratchetOnConfSMPAgentVersion = 7` (Agent/Protocol.hs:317), so there is no cross-layer version dependency; the SMP and e2e-encryption versions are unchanged.
-
-Scope of this change: the **synchronous** DR handshake in join, gated on the address advertising `ratchetKeys`. `joinConnection`/`joinConn`/`joinConnSrv` gain an optional `Maybe AddressRatchetKeys` (the advertised `RcvE2ERatchetParamsUri` + `ratchetKeyId`), passed in from the link data the caller fetched at plan time (`LGET`); present → DR path (R2'/R3'), absent → the classic `AgentInvitation`. Chat wires that argument later (a chat change); the agent supports it now and tests pass it directly. Making the send **async** (worker retry, a "connecting" UX, the `CreatedConnLink` LGET-gate) is **deferred** - kept below under "Deferred" as future work, not part of this change.
-
-### Implementation status (as built; `lib:simplexmq` compiles)
-
-**Done** (compiles): version bump; `RatchetKeyId`/`AddressRatchetKeys` types + `Encoding`, `UserContactData.ratchetKeys` (appended, backward-compatible); `AgentConfirmation.ratchetKeyId` (version-gated encode/decode); `ContactRequest`/`DRRequest` sum with tagged `Encoding` + `cr_invitation` `ToField`/`FromField` (legacy-URI fallback); `address_ratchet_keys` table + `createAddressRatchetKeys`/`getAddressRatchetKeys` (SQLite + Postgres migrations `M20260712_address_dr`); join threading (`Maybe AddressRatchetKeys`); requester R2'/R3' (`joinAddressDR` + `sendConfirmationToAddress`); owner O1' dispatch, O2' `smpAddressConfirmation`, O3' (`acceptContact'` continue-ratchet branch), all three `connReq` readers (`acceptContact'`, `acceptContactAsync'` → `CMD PROHIBITED` for DR, `newConnToAccept` → shell from `drAgentVersion`/`drPQSupport`); requester R5' (`smpConfirmation` `RcvConnection … Nothing` branch, guarded on a ratchet existing); address-creation bundle generation (`mkAddressRatchetKeys`) wired into `createConnectionForLink'` (`IKUsePQ`-for-`SCMContact` prohibition lifted there).
-
-**Deltas from the plan discovered while building:**
-- **R5' emits `CONF` and reuses the allow step** (not auto-complete). The DR requester is a `RcvConnection` receiving the owner's reply - the same position as the classic contact requester, which goes `CONF` → `allowConnection'` → `connectReplyQueues` (msg 3). R5' mirrors that (differing only in that the ratchet already exists, so it `getRatchet` + `rcDecrypt` instead of building it), so the app supplies `ownConnInfo` for msg 3 at allow, exactly as today. No new storage.
-- **`DRRequest` carries `drAgentVersion` + `drPQSupport`** (Part 3): the sync accept creates the connection shell via `newConnToAccept`→`newConnToJoin` before O3', and there is no URI to derive the version/PQ from.
-- `cr_invitation` serialization is downgrade-safe: `CRInvitation` keeps the legacy URI (`strEncode`, byte-identical to before), so an older agent still reads classic invitations; `CRConfirmation` is JSON (`DRRequest` has manual `ToJSON`/`FromJSON`), told apart on read by the leading `{` (a URI never starts with it). JSON keeps `DRRequest` extensible. `SMPQueueInfo` gained a base64 `StrEncoding` + JSON (it only had `Encoding`) so it can sit in the JSON.
-- **DR is opt-in per address**: `createConnectionForLink'`/`createConnectionForLink` gain a `Maybe InitialKeys` DR parameter (separate from the existing connection-PQ `InitialKeys`) - `Nothing` = no DR (old behavior, existing callers), `Just ik` = advertise the bundle with `ik`. The `IKUsePQ`-for-`SCMContact` prohibition stays on the connection-PQ parameter and is lifted only for the DR bundle.
-
-**Test-matrix consequence of the version bump:** `currentSMPAgentVersion` 7 → 8 moves the version-matrix "prev" (`current − 1`) from v6 to v7. v7 ≥ `ratchetOnConfSMPAgentVersion (7)`, so a joiner/acceptor at "prev" now secures the send queue on confirmation - the `sqSecured` expectation for the prev variants in `testMatrix2`/`testMatrix2Stress`/`testBasicMatrix2` flips `False → True`. (Standard version-bump maintenance; the pre-`ratchetOnConf` unsecured path is now two versions back and no longer exercised by these matrices.)
-
-**Not yet done:** rotation (`rotateRatchetKeys`, Part 4), cleanup step (Part 4), the app-driven `LSET` upgrade API (Part 5), wiring the DR parameter into the non-prepared-link `newRcvConnSrv` path, DR-specific tests (Part 6), regenerating `agent_schema.sql` if a schema-consistency test requires it, and chat wiring (deferred by design).
-
-## Part 1 - the current contact-address handshake, step by step
-
-Requester Alice connects to owner Bob's contact address. Q_A is Alice's receive queue (Bob to Alice), Q_B is Bob's receive queue (Alice to Bob).
-
-Requester side, in `joinConnSrv … CRContactUri` (Agent.hs:1398-1428):
-
-- R1. `compatibleContactUri` (Agent.hs:1370) - version check, yields the address queue `SMPQueueInfo`.
-- R2. `mkJoinInvitation` (Agent.hs:1411): creates or reuses the receive queue Q_A; `getRatchetX3dhKeys` or `generateRcvE2EParams` produces Alice's Rcv X3DH parameters, stored by `createRatchetX3dhKeys` (Agent.hs:1424); builds `cReq = CRInvitationUri crData aliceRcvParams` (Agent.hs:1426).
-- R3. `sendInvitation` (Agent.hs:1408; Agent/Client.hs:1924-1934): sends `AgentInvitation {connReq = cReq, connInfo = aliceProfile}` to the address queue, per-queue encrypted with a fresh ephemeral key by `agentCbEncryptOnce` (Agent/Client.hs:1929-1934), unauthenticated. **`connInfo` (Alice's profile) is under the per-queue X25519 layer only - the gap this plan closes.**
-
-Owner side, receiving on the contact address:
-
-- O1. `processClientMsg` dispatch (Agent.hs:3185): state `(Nothing, Just e2ePubKey)`, `(PHEmpty, AgentInvitation {connReq, connInfo})` -> `smpInvitation` (Agent.hs:3186).
-- O2. `smpInvitation` (Agent.hs:3610): stores an `Invitation`, emits `REQ` with Alice's `connInfo`.
-- O3. `acceptContact'` (Agent.hs:1477): `getInvitation`, then `joinConn` with Alice's `connReq` (Agent.hs:1480).
-- O4. `joinConnSrv … CRInvitationUri` (Agent.hs:1383) -> `startJoinInvitation` (Agent.hs:1395).
-- O5. `startJoinInvitation` (Agent.hs:1310-1350): creates Bob's send queue to Q_A (`newSndQueue`, Agent.hs:1335); `createRatchet_` (Agent.hs:1343-1350) runs `generateSndE2EParams`, `pqX3dhSnd` against Alice's Rcv parameters, `initSndRatchet`, `createSndRatchet`.
-- O6. `secureConfirmQueue` (Agent.hs:1396, 3747-3765): `agentSecureSndQueue` secures Q_A with `SKEY` (Agent.hs:3749); `mkAgentConfirmation` (Agent.hs:3780-3785) calls `createReplyQueue` to create Bob's receive queue Q_B and returns `AgentConnInfoReply (Q_B :| []) bobInfo`; `mkConfirmation` ratchet-encrypts it and wraps `AgentConfirmation {e2eEncryption_ = Just bobSndParams, encConnInfo}`; `sendConfirmation` sends it to Q_A. This is confirmation #1.
-
-Requester side, receiving confirmation #1 on Q_A:
-
-- R4. dispatch (Agent.hs:3181-3183): state `(Nothing, Just e2ePubKey)`, `AgentConfirmation` -> `smpConfirmation`.
-- R5. `smpConfirmation`, initiating-party branch `RcvConnection … Just e2eEncryption` (Agent.hs:3405-3444): `getRatchetX3dhKeys`, `pqX3dhRcv` (Agent.hs:3408), `initRcvRatchet` (Agent.hs:3411), `createRatchet` (Agent.hs:3436), `setRcvQueueConfirmedE2E` (Agent.hs:3440); decrypts `AgentConnInfoReply` (Agent.hs:3420); `processConf` emits `CONF` (Agent.hs:3444).
-- R6. `allowConnection'` (Agent.hs:1467-1474): `acceptConfirmation`, then `ICAllowSecure` secures Q_A with Bob's sender key.
-- R7. `connectReplyQueues` (Agent.hs:3724-3737): `upgradeConn` creates Alice's send queue to Q_B; `agentSecureSndQueue` secures Q_B; `enqueueConfirmation … Nothing` (Agent.hs:3733) stores `AgentConnInfo aliceInfo` and sends `AgentConfirmation {e2eEncryption_ = Nothing, encConnInfo}` to Q_B. This is confirmation #2.
-
-Owner side, receiving confirmation #2 on Q_B:
-
-- O7. dispatch (Agent.hs:3182): `AgentConfirmation` -> `smpConfirmation`.
-- O8. `smpConfirmation`, accepting-party branch `DuplexConnection … Nothing` (Agent.hs:3447-3462): `agentRatchetDecrypt` with the established ratchet; `AgentConnInfo` -> `INFO` (Agent.hs:3452); `ICDuplexSecure` or `CON`.
-
-Completion is direct `CON` on `senderCanSecure` (SKEY) messaging-mode queues (the sender on `AgentConnInfo`, Agent.hs:2252; the receiver with no `senderKey`, Agent.hs:3459-3461); the separate `HELLO` via `helloMsg` (Agent.hs:3466) is the older non-`senderCanSecure` (duplexHandshake v2, in-band-securing) path.
-
-## Part 2 - the DR-from-address handshake, mapped to Part 1
-
-The address advertises Bob's Rcv X3DH parameters in link data (Part 3). Alice, when the address advertises them and versions are compatible, takes the joiner role; Bob takes the initiator role.
-
-Requester side - a new branch in `joinConnSrv … CRContactUri`, taken when the passed `Maybe AddressRatchetKeys` is present (the caller's plan-time `LGET`):
-
-- R2'. Replaces R2/R3. Read the passed bundle - `ratchetKeyId` and `e2eParams :: RcvE2ERatchetParamsUri 'C.X448` - and negotiate the concrete version with `compatibleVersion` against the client e2e range, as `compatibleInvitationUri` does (Agent.hs:1362-1368). Create the receive queue Q_A subscribed (`newRcvQueue` with `subMode`), messaging mode so Bob can secure it. Choose the requester's KEM with `replyKEM_ v ownerKem_ pqSup` (Ratchet.hs:839): if the bundle advertises a KEM (owner `IKUsePQ`) the requester `AcceptKEM` - a **double KEM**: it both encapsulates to the address KEM (ciphertext) and includes its own new KEM public key (`generateSndE2EParams` → `sntrup761Enc` + a fresh keypair, Ratchet.hs:433-435), so PQ is bidirectional from message 1; if the bundle has no KEM and the requester wants PQ, it `ProposeKEM` (its own key only, PQ from message 2 if the owner supports it). Run `generateSndE2EParams g v (replyKEM_ …)`, `pqX3dhSnd` against the negotiated parameters, `initSndRatchet`, `createSndRatchet` - the body of `createRatchet_` (Agent.hs:1343-1350), with parameters from the passed bundle rather than a received invitation.
-- R3'. Build `AgentConfirmation {e2eEncryption_ = Just aliceSndParams, ratchetKeyId = Just ratchetKeyId, encConnInfo = ratchetEncrypt(AgentConnInfoReply (Q_A :| []) aliceProfile)}` - the `mkAgentConfirmation`/`mkConfirmation` bodies (Agent.hs:3780-3765) with the reply queue being Alice's own Q_A. Send it to the address queue unauthenticated with `agentCbEncryptOnce`, one-shot (as `sendInvitation` sends, Agent/Client.hs:1929-1934) - **synchronous**, with the same send-failure UX as today's classic contact join. Nothing is stored: a retry (chat re-invokes the join → `mkJoinInvitation` reuses Q_A + keys, 1418) re-builds the confirmation, advancing the send ratchet, and the owner absorbs the advance - a **failed send** is skipped when the owner establishes the ratchet (`maxSkip = 512`, Ratchet.hs:988), and a **lost reply** carries the current content and updates the owner's request by `XContactId` (ContactRequest.hs:99-101, 269); both testable. The requester does **not** SKEY the address (`QMContact`, not `senderCanSecure`); rotation is handled because the passed params are the current advertised keys. **Alice's profile is now inside `encConnInfo`, under the ratchet.** Alice's connection is `RcvConnection` (Q_A) with a send ratchet, until she receives Q_B. This "New `RcvConnection` + `ratchets` row" is a new state (today a New `RcvConnection` holds x3dh keys but no ratchet - the classic initiator builds the ratchet only at R5, `createRatchet` Agent.hs:3436), and it composes: connection type is derived from queue rows alone while the `ratchets` table is keyed independently by `conn_id`, so subscription (Agent.hs:1551), `connectionStats` (2658), and `allowConnectionAsync'` (888) never read the ratchet for a `RcvConnection`; the only handshake reader on it is `smpConfirmation` (R5').
-
-### Deferred (future work): async delivery + connect UX
-
-The synchronous send above fails in the user's face on a lost reply (the same wart as today's classic contact join), even though the request may have been delivered. Making it async is a separate, later change, not part of this DR work:
-
-- Delivery cannot use the message-delivery worker: a `SndQueue` is unique per `(host, port, snd_id)` and belongs to one connection (schema PK), while a contact address is one queue that many connections send to, so no per-connection SndQueue to it can exist. It would go through the **async command worker**, keyed by `(connId, server)` (`getAsyncCmdWorker`, Agent.hs:1856-1858), which already retries the `JOIN` command (`tryMoveableCommand` → `retrySndOp`, 2016-2024); each retry re-runs `joinConnSrv` (re-build + ratchet advance, which the owner absorbs - above), so nothing is stored. (`joinConnSrvAsync` for `CRContactUri` is `CMD PROHIBITED` today, Agent.hs:1452, and the `JOIN` handler falls back to sync `joinConnSrv`, 1899-1902; the `TBC` at Agent.hs:1897 is about async *receive*-queue creation - Q_A - and is orthogonal.)
-- The async join returns "connecting" early and completes via the events chat already handles (`joinContact` sets `ConnJoined`; the DR requester emits `CONF` in R5' and the chat allows it, exactly as the classic contact requester, driving msg 3 → `CON`; a permanent send failure still surfaces as `ERR → ConnFailed`).
-- This needs a chat change: the join API takes a `CreatedConnLink` (full + short link), not the bare `ConnectionRequestUri` it takes today, so the agent can LGET-gate on the owner's server (a real reachability check) and verify the fetched `linkConnReq` equals the passed full link before reporting success. Used only for DR addresses (link data advertises `ratchetKeys`); old / non-DR addresses stay on the current sync path.
-
-Owner side - a new dispatch branch and a new receive handler:
-
-- O1'. In `processClientMsg` (Agent.hs:3176-3187), add a branch in state `(Nothing, Just e2ePubKey)`: an `AgentConfirmation` with `ratchetKeyId = Just _` **and** `e2eEncryption_ = Just _` on a `ContactConnection` -> `smpAddressConfirmation` (new). A `ratchetKeyId` without `e2eEncryption_` is ignored (it does not match this branch and falls through as a non-DR confirmation). It must be placed **before** the existing `(PHEmpty, AgentConfirmation) | senderCanSecure queueMode` case (Agent.hs:3182-3184), because a contact-address queue is `QMContact` (not `senderCanSecure`) and would otherwise fall into `prohibited "handshake: missing sender key"` (Agent.hs:3184). The address queue's `e2eDhSecret` stays `Nothing` (it is never set for a contact address - `smpInvitation` does not set it, Agent.hs:3609-3622), so every request is decrypted with its own ephemeral key via this `(Nothing, Just e2ePubKey)` path.
-- O2'. `smpAddressConfirmation` (new, modeled on `smpConfirmation` initiating branch, Agent.hs:3405-3444): select the private triple `(pk1, pk2, pKem)` by `ratchetKeyId` from `address_ratchet_keys`; `pqX3dhRcv pk1 pk2 pKem aliceSndParams`; `initRcvRatchet` with the address connection's stored `PQSupport` (`connPQEncryption` of the address `InitialKeys` - `On` for `IKUsePQ` and `IKPQOn`, `Off` for `IKPQOff`; this is what lets `IKPQOn` accept the requester's proposed KEM), combined with version compatibility as `smpConfirmation` derives `pqSupport'` (Agent.hs:3410); `rcDecrypt` of `encConnInfo` performs the first ratchet step, giving the ratchet its send side too (as it does for the initiator today), so the owner can later reply. Parse `AgentConnInfoReply (Q_A :| []) aliceProfile`. Store the request with `createInvitation` on the address connection (`contact_conn_id`), exactly as a classic invitation - except the request value is the `CRConfirmation` variant (Part 3) carrying the post-decrypt ratchet state and Q_A, and `recipient_conn_info` is `aliceProfile` - so **no connection or `ratchets` row is created at receive**, as with a classic invitation. Emit `REQ` with the `invitation_id`. A resend is not deduplicated: like a resent classic invitation it produces another `REQ` (the connect-UX fix for that is separate chat work). An unknown or expired `ratchetKeyId`, or a decryption failure: discard and acknowledge, as an undecryptable message is dropped today. This establishes ratchet state on unauthenticated input before the user accepts - see "Receive-time establishment, state, and abuse".
-- O3'. `acceptContact'` for a DR request - a new branch that continues the ratchet instead of `joinConn`. `getInvitation` returns the request; its `CRConfirmation` variant gives the stored ratchet state and Q_A. Create the connection now (as `joinConn` does for a classic invitation) and `createRatchet` (AgentStore.hs:1419) from the stored ratchet state. Reuse `mkAgentConfirmation` (Agent.hs:3780-3785) to create Bob's receive queue Q_B and return `AgentConnInfoReply (Q_B :| []) bobInfo`; create Bob's send queue to Q_A (`newSndQueue`, generating Bob's own sender key) and secure Q_A with `SKEY` using that key (`agentSecureSndQueue`, valid because Q_A is messaging mode) - the securing key is Bob's own, not taken from Alice's message; send the response to Q_A as `AgentConfirmation {e2eEncryption_ = Nothing, ratchetKeyId = Nothing, encConnInfo = ratchetEncrypt(AgentConnInfoReply (Q_B :| []) bobInfo)}` via `sendConfirmation` (`agentCbEncrypt` over Bob's send queue to Q_A, `PHEmpty` because Q_A is `senderCanSecure`) - exactly the current contact msg 2 path (Client.hs:1916), not `agentCbEncryptOnce`. The reply content is `AgentConnInfoReply`, not `AgentConnInfo`: it takes the `mkAgentConfirmation` path with `e2eEncryption_ = Nothing`, not the `enqueueConfirmation` path (which produces `AgentConnInfo`, Agent.hs:3789). `rejectContact'` deletes the `conn_invitations` row (the current behaviour), discarding the inline ratchet; no connection was created, so there is nothing else to clean up.
-
-Requester side, receiving the response on Q_A:
-
-- R5'. `smpConfirmation` needs a new branch `RcvConnection … Nothing` (today only `RcvConnection … Just` and `DuplexConnection … Nothing` exist, Agent.hs:3403-3447). It looks up the ratchet first (`getRatchet`) and, if there is none, falls through to `prohibited "conf: incorrect state"` - so a classic initiator (a New `RcvConnection` with x3dh keys but no ratchet) that receives a stray `Nothing`-confirmation keeps today's exact outcome; only a DR requester, which holds a send ratchet, takes the new path. Alice already holds the send ratchet, so `rcDecrypt` advances it and creates the receive side; parse `AgentConnInfoReply (Q_B :| []) bobInfo`. **This mirrors the classic contact requester exactly**: `setRcvQueueConfirmedE2E` on Q_A, `createRatchet` the advanced ratchet, store the reply as a `NewConfirmation`, and emit **`CONF`** - the app then calls `allowConnection'` (supplying `ownConnInfo` for msg 3), which drives `connectReplyQueues` (create Alice's send queue to Q_B, `SKEY`, upgrade to `DuplexConnection`, `enqueueConfirmation` the `AgentConnInfo` msg 3). Because Q_B is sender-securable, sending `AgentConnInfo` completes Alice with `CON` (Agent.hs:2252) - no `HELLO`. The only difference from the classic requester is that the ratchet is pre-built (from R2') rather than built from Bob's Snd params here, so there is no `CONF`-less auto-completion and no separate storage of Alice's own info.
-- R6'/completion. Unchanged from the current contact handshake, and modern (no `HELLO`). The exchange is three agent↔agent wire messages - Alice → address queue (msg 1), Bob → Q_A (msg 2, an `AgentConfirmation` carrying `AgentConnInfoReply` with Q_B), Alice → Q_B (msg 3, an `AgentConfirmation` carrying `AgentConnInfo`) - the same shape as the current contact flow, where msg 1 was `AgentInvitation`; here it is the ratchet-establishing `AgentConfirmation`. (`CON` is not a wire message - it is the agent→app event; `HELLO` and `AgentConnInfo` are the wire messages.) `HELLO` belongs to the older non-`senderCanSecure` path (duplexHandshake v2, before SKEY): there the confirmation secures the queue in-band (`PHConfirmation` carries the sender key, Client.hs:1918) and the receiver replies with `HELLO` (`ICDuplexSecure` → `enqueueDuplexHello`, Agent.hs:3457-3458). Both Q_A and Q_B here are messaging-mode - Q_A by R2', Q_B via `createReplyQueue` → `SCMInvitation` → `QMMessaging` (Agent.hs:1233,1458,3783) - so the sender secures with SKEY and sends `PHEmpty` (Client.hs:1918), the dispatch takes the `senderCanSecure` branch (Agent.hs:3182-3184), and each agent raises the `CON` app event locally off msg 3 - Bob on receiving it (`senderKey = Nothing`, Agent.hs:3459-3461), Alice on sending it (Agent.hs:2252) - with no separate `HELLO` wire message. (msg 2's `AgentConnInfoReply` only sets Q_A `Confirmed`, Agent.hs:2254.) Invitations are two messages because the initiator's queue is already in the link; a contact address needs three because Bob's receive queue Q_B is only delivered in msg 2. The third message no longer has a ratchet role: Bob's X3DH params are pre-published, so the agreement is complete once Bob receives msg 1 (in the current flow Bob's Snd params instead arrive in msg 2). msg 2 and msg 3 are queue setup - msg 2 delivers Q_B, msg 3 secures Q_B so Alice can send to Bob and signals Bob's `CON`; neither negotiates the ratchet. A one-directional exchange (the RPC) needs no Q_B and is two messages.
-
-Net code touch points: `joinConnSrv` (new requester branch), `processClientMsg` (new owner dispatch), `smpConfirmation` (new `RcvConnection … Nothing` branch and `AgentConnInfoReply` acceptance), `acceptContact'` (new continue-ratchet branch), a new `smpAddressConfirmation` reusing `createInvitation`/`getInvitation` with the sum request value, and the link data and storage of Part 3-4. `rejectContact'` is unchanged (it deletes the `conn_invitations` row either way).
-
-### Receive-time establishment, state, and abuse
-
-This is the substantive departure from the current flow. Today `smpInvitation` creates only a lightweight `NewInvitation` and emits `REQ` (Agent.hs:3618-3621); no connection or ratchet exists until the user accepts. For DR the request is under the ratchet, so to show the requester's profile in `REQ` the owner must decrypt it, which means establishing the ratchet at **receive**, before accept.
-
-Design decision (Q1): decrypt at receive. Both use cases need the request content at `REQ` - a person decides to accept from the profile, and a service bot needs the request payload to act. Deferring decryption to accept would make `REQ` contentless and does not fit the service case, so it is not done.
-
-Consequences:
-
-- No connection is created at receive, exactly as for a classic invitation. O2' stores the request with `createInvitation` on the address connection; the post-decrypt ratchet state and Q_A live inline in the `CRConfirmation` request value (`cr_invitation`). O3' (accept) creates the connection, `createRatchet` from the stored state, and adds Bob's queues, becoming a `DuplexConnection`; `rejectContact'` deletes the `conn_invitations` row.
-- Per incoming `AgentConfirmation` the owner does one `pqX3dhRcv` (three DH plus, with PQ, one `sntrup761` decapsulation) and one `rcDecrypt`, on unauthenticated input, and writes one `conn_invitations` row - more CPU than the current `NewInvitation`, the same order of state (no connection, no `ratchets` row until accept).
-
-Abuse (Q2): a contact address already accepts and processes unauthenticated invitations today, so this is a degree-worse version of an existing surface, not a new class. It is bounded by the address queue quota (an attacker fills it, the owner drains and acknowledges) and, optionally, by basic auth on the address (already supported for contact addresses, `optBasicAuth`). The per-request state is a single `conn_invitations` row - the same class as a classic contact request - so it is subject to the same limits and lifecycle, with no DR-specific dedup or TTL. Proof-of-work or a stricter gate can be added later; it is out of scope here and noted as a follow-up.
-
-`acceptContact'`/`rejectContact'` keep taking the `invitation_id` from `REQ` unchanged; the only difference is that `getInvitation` returns a request that is either a `CRInvitation` URI (current `joinConn` path, O3-O6) or a `CRConfirmation` (continue-ratchet path, O3'). Nothing in the `REQ`/accept/reject flow or the chat client changes - the change is contained in the agent.
-
-### The four communication layers, per message (verified against code)
-
-Layers, outermost (server-visible) first:
-
-- **L1 `ClientMsgEnvelope`** (Protocol.hs:1089), `PubHeader {phVersion, phE2ePubDhKey :: Maybe PublicKeyX25519}` (1096) - **this is where per-queue encryption is agreed** (not L2). `phE2ePubDhKey` is the sender's e2e DH public key; the recipient combines it with the queue's e2e private key: `(e2eDhSecret, e2ePubKey_) -> (Nothing, Just e2ePubKey) -> e2eDh = dh' e2ePubKey e2ePrivKey` (Agent.hs:3172-3178). `agentCbEncryptOnce` (Client.hs:2214) puts a **fresh ephemeral** pubkey (generated 2217, set 2223) - used when the sender has no send queue (the address queue), whose `e2eDhSecret` stays `Nothing`, so it decrypts every message with the per-message ephemeral. `agentCbEncrypt` (Client.hs:2203) puts the **send queue's persistent** e2e pubkey (`Just` on a confirmation, 2210); the recipient stores the secret via `setRcvQueueConfirmedE2E`, and *later* messages send `phE2ePubDhKey = Nothing` (`sendAgentMessage`, 2080).
-- **L2 `ClientMessage PrivHeader`** (Protocol.hs:1113), `PrivHeader = PHConfirmation APublicAuthKey | PHEmpty` (1115) - **queue securing / authorization, not encryption**. `PHConfirmation` carries the sender's AUTH key for in-band securing (v2, non-`senderCanSecure`); `PHEmpty` when the sender secured the queue with SKEY out-of-band. `PHEmpty` on every message here is about securing, and says nothing about encryption (that is L1). Set in `sendConfirmation` (Client.hs:1918), `sendInvitation` (1934), `sendAgentMessage` (2079).
-- **L3 `AgentMsgEnvelope`** (Agent/Protocol.hs:829, encoding 851) - outside the ratchet. `AgentConfirmation` ('C') carries `e2eEncryption_` (Snd X3DH params, agrees DR) + `encConnInfo`; `AgentInvitation` ('I') carries `connReq` (Rcv X3DH params) + plaintext `connInfo` (no DR); `AgentMsgEnvelope` ('M') carries `encAgentMessage`.
-- **L4 `AgentMessage`** (Agent/Protocol.hs:883, encoding 893) - inside the ratchet. `AgentConnInfo` ('I'), `AgentConnInfoReply` ('D', reply queues + info), `AgentMessage APrivHeader AMessage` ('M'; `AMessage` includes `HELLO`, Agent/Protocol.hs:1018-1020). **Absent when L3 is `AgentInvitation`** (that profile is per-queue-only - the gap this plan closes).
-
-Send routing: msg 1 (to address) → `sendInvitation` today / a new `agentCbEncryptOnce` confirmation send for DR; msg 2 → `secureConfirmQueue` → `sendConfirmation` (Agent.hs:3747); msg 3 → `connectReplyQueues` → `enqueueConfirmation` → delivery worker `AM_CONN_INFO` → `sendConfirmation` (Agent.hs:3733,3789,2183). `AM_CONN_INFO`/`AM_CONN_INFO_REPLY` both go through `sendConfirmation` (2183-2184); other `AMessage`s go through `sendAgentMessage` wrapping `AgentMsgEnvelope` 'M' (2192-2193).
-
-Current contact handshake (address does **not** advertise DR):
-
-| msg | L1 `PubHeader.phE2ePubDhKey` (per-queue enc) | L2 `PrivHeader` (securing) | L3 `AgentMsgEnvelope` | L4 `AgentMessage` |
-|---|---|---|---|---|
-| 1 Alice→addr | `Just` fresh ephemeral, `agentCbEncryptOnce` (Client.hs:1933,2223) | `PHEmpty` (1934) | `AgentInvitation` {connReq = Alice Rcv params, connInfo = profile} (Client.hs:1932) | — none (profile per-queue only) |
-| 2 Bob→Q_A | `Just` Bob's send-queue e2e pubkey, `agentCbEncrypt` (1920,2210) | `PHEmpty` [`senderCanSecure`] (1918) | `AgentConfirmation` {e2eEncryption_ = **Just Bob Snd params**, encConnInfo} (Agent.hs:3765) | `AgentConnInfoReply` (Q_B) bobInfo, DR-enc (Agent.hs:3785) |
-| 3 Alice→Q_B | `Just` Alice's send-queue e2e pubkey, `agentCbEncrypt` (1920,2210) | `PHEmpty` [`senderCanSecure`] (1918) | `AgentConfirmation` {e2eEncryption_ = **Nothing**, encConnInfo} (Agent.hs:3802) | `AgentConnInfo` aliceInfo, DR-enc (Agent.hs:3789) |
-
-New DR handshake (address advertises DR):
-
-| msg | L1 `PubHeader.phE2ePubDhKey` (per-queue enc) | L2 `PrivHeader` (securing) | L3 `AgentMsgEnvelope` | L4 `AgentMessage` |
-|---|---|---|---|---|
-| 1 Alice→addr | `Just` fresh ephemeral, `agentCbEncryptOnce` [same] | `PHEmpty` [same] | **`AgentConfirmation`** {e2eEncryption_ = **Just Alice Snd params**, **ratchetKeyId = Just**, encConnInfo} [was `AgentInvitation`] | **`AgentConnInfoReply`** (Q_A) aliceProfile, **DR-enc** [was plaintext connInfo] |
-| 2 Bob→Q_A | `Just` Bob's send-queue e2e pubkey, `agentCbEncrypt` [same] | `PHEmpty` [same] | `AgentConfirmation` {e2eEncryption_ = **Nothing**, ratchetKeyId = Nothing, encConnInfo} [was Just Bob Snd params] | `AgentConnInfoReply` (Q_B) bobInfo, DR-enc [same] |
-| 3 Alice→Q_B | `Just` Alice's send-queue e2e pubkey, `agentCbEncrypt` [same] | `PHEmpty` [same] | `AgentConfirmation` {e2eEncryption_ = Nothing, encConnInfo} [same] | `AgentConnInfo` aliceInfo, DR-enc [same] |
-
-Net difference: **only msg 1 and msg 2's L3/L4 change.** msg 1's L3 becomes `AgentConfirmation` (was `AgentInvitation`) carrying Alice's Snd params + `ratchetKeyId`, and the profile moves from plaintext L3 to DR-encrypted L4 (`AgentConnInfoReply`) - the whole point of the change. msg 2 drops `e2eEncryption_` (Bob no longer sends Snd params - the ratchet is agreed from msg 1). msg 3 is unchanged. L1 (per-queue encryption - each queue agrees its own secret via the sender's e2e pubkey in the `PubHeader` on the first message to it) and L2 (securing, `PHEmpty` because SKEY is used) are unchanged throughout; the DR change is entirely at L3/L4. The only new send code is msg 1 (an `AgentConfirmation` fired to the address with `agentCbEncryptOnce`, like `sendInvitation` but with a confirmation envelope).
-
-## Part 3 - types and link data
-
-### Fixed data - unchanged
-
-`FixedLinkData` (Protocol.hs:1824) is not touched. The double-ratchet keys go entirely in mutable data, so an existing address advertises them without a new link (the fixed data is hash-committed and cannot change). Fixed data keeps only `agentVRange`, `rootKey`, `linkConnReq`, `linkEntityId`.
-
-### Mutable data - ratchet keys bundle
-
-Appended to `UserContactData` (Protocol.hs:1840); the encoding stops at a trailing tail (Protocol.hs:1981), so earlier versions ignore it:
-
-```haskell
-newtype RatchetKeyId = RatchetKeyId ByteString -- opaque short id; one Encoding instance, shared below
-
-data AddressRatchetKeys = AddressRatchetKeys
-  { ratchetKeyId :: RatchetKeyId, -- identifies this bundle; changes on rotation, echoed in the request
-    e2eParams :: CR.RcvE2ERatchetParamsUri 'C.X448 -- version range + both X3DH keys + optional KEM
-  }
-instance Encoding AddressRatchetKeys where ... -- the key-bundle instance; both fields required
-
-data UserContactData = UserContactData
-  { direct :: Bool, owners :: [OwnerAuth], relays :: [ConnShortLink 'CMContact],
-    userData :: UserLinkData,
-    ratchetKeys :: Maybe AddressRatchetKeys -- whole bundle optional, one Encoding instance
-  }
-```
-
-`e2eParams` is the existing `RcvE2ERatchetParamsUri 'C.X448` (`E2ERatchetParamsUri VersionRangeE2E k1 k2 (Maybe (RKEMParams s))`, Ratchet.hs:282-286) - the same type a `CRInvitationUri` advertises - with `StrEncoding`/`Encoding` already defined (Ratchet.hs:302-374). There is no bespoke key type and no reconstruction: the requester negotiates the concrete version with `compatibleVersion` against its own e2e range, exactly as `compatibleInvitationUri` does for an invitation (Agent.hs:1362-1368), giving `RcvE2ERatchetParams` for `pqX3dhSnd`. The KEM is optional: `Nothing` gives an X448-only ratchet (as when `PQSupport` is off), `Just` a hybrid one, matching `generateRcvE2EParams`'s `PQSupport` gate (Ratchet.hs:439-445).
-
-The address-creation parameter is `InitialKeys` (Ratchet.hs:864) - the same 3-way choice as invitations, not a bare `PQSupport`. Currently `IKUsePQ` is prohibited for `SCMContact` (Agent.hs:990,1198) because a contact address carries no owner keys; this change lifts that prohibition. The bundle plays the published-contact-request role, so its KEM follows `initialPQEncryption False pqInitKeys` (Ratchet.hs:882) - exactly as the requester's contact request does today (Agent.hs:1422):
-
-- `IKUsePQ` - the bundle advertises the KEM; the requester encapsulates to it, so PQ from message 1.
-- `IKPQOn` (`IKLinkPQ PQSupportOn`) - the bundle is X448-only (no KEM advertised), but the owner's ratchet supports PQ (`connPQEncryption` = On, Ratchet.hs:888); the requester proposes its own KEM (R2'), so PQ from message 2.
-- `IKPQOff` (`IKLinkPQ PQSupportOff`) - X448-only, and the owner's ratchet does not support PQ even if the requester proposes it.
-
-Advertising the KEM adds ~1158 B to the rotated, widely-fetched link data, which is why `IKPQOn` exists (PQ one round later, without the size cost). The owner generates the bundle with `generateRcvE2EParams g v (initialPQEncryption False pqInitKeys)` (Ratchet.hs:439), stores the private triple `(pk1, pk2, pKem)` (Part 4), and advertises `e2eParams` by wrapping the public `E2ERatchetParams` in the address's e2e version range (`toVersionRangeT`; or `mkRcvE2ERatchetParams` from the stored privates, Ratchet.hs:412) - the same private-key shape `createRatchetX3dhKeys`/`getRatchetX3dhKeys` already store (AgentStore.hs:1362-1367). `ratchetKeys` is set by the agent when it signs mutable link data (`Crypto.ShortLink.encodeSignUserData`), not by the application.
-
-### Authentication of the advertised keys
-
-No signature is added on the keys: the mutable link data already signs them. `decryptLinkData` (Crypto/ShortLink.hs:106-114) verifies `sig2` over the mutable `UserContactData` by `rootKey`, so `ratchetKeys` is root-signed. This is the X3DH anti-substitution property: an SMP server cannot substitute the keys without forging the root signature. The signer is the root Ed25519 key (the address's signing identity); the X3DH keys are separate DH keys (X448, which cannot sign). A single owner signs address data ("we don't use multiple owners"), so the root signature alone is sufficient - no per-key signature. A malicious server can still serve an older but validly-signed `UserContactData` (rollback to a retired bundle); this is bounded by the retention window and by the ratchet advancing after the first message, and a signature does not prevent it. Inline ratchet params in a `CRInvitationUri` contact request are not in signed link data and remain unsigned - a separate change, out of scope here.
-
-### Request envelope
-
-`AgentConfirmation` (Protocol.hs:830-834) gains an optional `ratchetKeyId` - the `ratchetKeyId` of the `AddressRatchetKeys` bundle the requester used, so the owner selects the matching private keys:
-
-```haskell
-AgentConfirmation
-  { agentVersion :: VersionSMPA,
-    e2eEncryption_ :: Maybe (SndE2ERatchetParams 'C.X448), -- reused: Alice's Snd params in DR msg 1
-    ratchetKeyId :: Maybe RatchetKeyId,                    -- selects the owner's key generation
-    encConnInfo :: ByteString
-  }
-```
-
-`ratchetKeyId` is a separate optional selector (the shared `RatchetKeyId` newtype), not the bundle - the owner already holds the published public bundle and looks up its private keys by this id. It reuses the existing `e2eEncryption_` for Alice's Snd params rather than a new combined bundle; the minor cost is two correlated `Maybe`s (`ratchetKeyId = Just` is only meaningful with `e2eEncryption_ = Just`). **A `ratchetKeyId` with `e2eEncryption_ = Nothing` is ignored** - O2' requires both (there are no Snd params to run `pqX3dhRcv`), so such a message falls through to the current dispatch as if it had no `ratchetKeyId`.
-
-Encoding (extends Protocol.hs:853-866): from `addressDRVersion`, `smpEncode (agentVersion, 'C', e2eEncryption_, ratchetKeyId, Tail encConnInfo)`; `ratchetKeyId` is `Just` for an address-DR confirmation and `Nothing` for the current joiner-to-initiator and initiator-to-joiner confirmations; earlier versions omit the field entirely and use `smpEncode (agentVersion, 'C', e2eEncryption_, Tail encConnInfo)`. Parsing gates the field on `agentVersion`. `CRInvitationUri` is unchanged - a connection request URI holds Rcv parameters and must not hold Snd parameters.
-
-### Stored request - the invitation record
-
-The `conn_invitations` record stays; only the type of the stored request widens. From chat's point of view a DR request is still an invitation - it "contains a confirmation" instead of an invitation URI - so `REQ`, `acceptContact'`/`rejectContact'`, and the chat side are unchanged; the change is contained in the agent. The `NewInvitation`/`Invitation` request field (`cr_invitation`, stays `NOT NULL`) becomes a sum:
-
-```haskell
-data ContactRequest
-  = CRInvitation (ConnectionRequestUri 'CMInvitation) -- classic: joinConn on accept (O3-O6)
-  | CRConfirmation DRRequest                          -- DR: continue the ratchet on accept (O3')
-
-data DRRequest = DRRequest
-  { drRatchet :: RatchetX448,        -- post-decrypt receiving ratchet (with send side), stored inline
-    drReplyQueue :: SMPQueueInfo,    -- Q_A, where the owner replies
-    drAgentVersion :: VersionSMPA,   -- negotiated at receive; needed to build the connection shell at accept
-    drPQSupport :: PQSupport         -- the address's PQ setting for this connection
-  }
-```
-
-`recipient_conn_info` holds the profile in both cases. `getInvitation`/`createInvitation` carry `ContactRequest`; `acceptContact'` branches on the constructor. There is no dedup column: a resent request produces another `REQ`, exactly as a resent classic invitation does.
-
-`drAgentVersion`/`drPQSupport` are stored because the accept flow creates the connection **shell** through `newConnToAccept` → `newConnToJoin` (via `prepareConnectionToAccept`, called by chat's sync accept before `acceptContact'`, Internal.hs:914,925) and `newConnToJoin` today derives `connAgentVersion`/`pqSupport` from the `ConnectionRequestUri` (Agent.hs:1277-1293); a `CRConfirmation` has no URI, so the values negotiated at receive (O2') are stored and used to build the shell.
-
-Three readers of the widened `connReq` field (all via `getInvitation`) branch on the constructor:
-- `acceptContact'` (Agent.hs:1479, sync): `CRInvitation cr` → `joinConn … cr` (classic, unchanged); `CRConfirmation dr` → the O3' continue-ratchet path.
-- `newConnToAccept` (Agent.hs:1296, via `prepareConnectionToAccept`): `CRInvitation cr` → `newConnToJoin … cr` (unchanged); `CRConfirmation dr` → create the `NewConnection` shell from `drAgentVersion`/`drPQSupport` (`createNewConn`, generating the connId).
-- `acceptContactAsync'` (Agent.hs:900): `CRInvitation cr` → `joinConnAsync … cr` (unchanged); `CRConfirmation _` → `throwE $ CMD PROHIBITED` (async DR accept is deferred; DR requests accept synchronously). Chat's REQ/accept is unaffected either way - it only ever passes `invId`, never the `ContactRequest`, which stays internal to the agent.
-
-Storage: `cr_invitation`'s `ToField`/`FromField` encode `CRInvitation` as the legacy `strEncode` URI (unchanged from before, so the format is downgrade-safe) and `CRConfirmation` as JSON (`J.encode` of `DRRequest`). `FromField` peeks the first byte: `{` → JSON `CRConfirmation`, else `strDecode` → `CRInvitation` (a URI never starts with `{`). `DRRequest` uses manual `ToJSON`/`FromJSON` (extensible), and `SMPQueueInfo` gets a base64 `StrEncoding` + JSON to sit inside it. `smpInvitation` (Agent.hs:3618) wraps its `connReq` in `CRInvitation`; `smpAddressConfirmation` (O2') writes `CRConfirmation`.
-
-## Part 4 - key rotation (client-driven)
-
-Rotation is independent of the handshake above and is driven by the client app, not the agent. The agent never rotates on its own - it lacks the app's intent and the mutable link data (profile/badge and other short-link data). The app rotates by calling `setConnShortLink` with the rotate flag; the whole ratchet-keys bundle - both X448 keys and the KEM - is generated fresh each time.
-
-### Schema
-
-```sql
--- one row per ratchet-keys generation for an address; the current generation plus the most recent retained ones.
--- private side of the advertised RcvE2ERatchetParamsUri - same shape as the ratchets x3dh
--- columns and createRatchetX3dhKeys (AgentStore.hs).
-CREATE TABLE address_ratchet_keys(
-  address_ratchet_key_id INTEGER PRIMARY KEY AUTOINCREMENT,
-  conn_id BLOB NOT NULL REFERENCES connections ON DELETE CASCADE,
-  ratchet_key_id BLOB NOT NULL,     -- the published id echoed by requests
-  x3dh_priv_key_1 BLOB NOT NULL,    -- X448
-  x3dh_priv_key_2 BLOB NOT NULL,    -- X448
-  pq_priv_kem BLOB,                 -- RcvPrivRKEMParams (sntrup761 keypair); NULL when PQ is off for this address
-  created_at TEXT NOT NULL DEFAULT(datetime('now'))
-);
-CREATE UNIQUE INDEX idx_address_ratchet_keys ON address_ratchet_keys(conn_id, ratchet_key_id);
-
--- a DR request stays in conn_invitations with NO schema change: cr_invitation now holds a ContactRequest
--- sum (an invitation URI or a confirmation carrying the post-decrypt ratchet + reply queue), so it stays
--- NOT NULL - no nullable change, no new column on conn_invitations, no new table for the request.
-```
-
-`cr_invitation` stays `NOT NULL` - only its decoded value gains a variant (Part 3), so the invitations flow, `REQ`, and chat are unchanged; the only new storage is the `address_ratchet_keys` table. The link signing key is already on the address queue (`rcv_queues.link_priv_sig_key`, M20250322), so nothing is added there - rotation and retrofit re-sign mutable data with it. PostgreSQL mirrors this. Migration `M20260712_address_dr`.
-
-### Rotation logic
-
-The app rotates by calling `setConnShortLink` with the rotate flag; there is no automatic, agent-driven rotation. On rotation:
-
-1. `generateRcvE2EParams` for a fresh generation - two X448 keys, and an sntrup761 keypair only if PQ is on for this address - with a fresh `ratchetKeyId`.
-2. Recompute mutable link data with the new `AddressRatchetKeys` (the public `e2eParams`), re-sign with the root key (`encodeSignUserData`, key from `rcv_queues.link_priv_sig_key`), and `LSET` it to the address queue (`setConnShortLink` path).
-3. Insert the new `address_ratchet_keys` row (`x3dh_priv_key_1`, `x3dh_priv_key_2`, `pq_priv_kem`).
-
-### Retention
-
-Retention is count-based, not time-based: on each rotation `deleteOldAddressRatchetKeys` keeps the newest `keepAddressKeys` generations per address (default 3, ordered by `address_ratchet_key_id`) and deletes older ones. There is no `retired_at` column, no time window, and no `cleanupManager` step. A request that used a recently-retired bundle still decrypts while that generation is retained; how long a recorded first message stays decryptable after a compromise of the current private keys is therefore bounded by the retained-generation count and the app's rotation cadence (both app-controlled). Unaccepted DR request rows in `conn_invitations` are handled exactly like unaccepted classic invitation requests - no DR-specific cleanup (a DR request is one `conn_invitations` row, the same class of state as a classic contact request).
-
-## Part 5 - backward compatibility
-
-- A requester older than `addressDRVersion`, or an address without `ratchetKeys`, uses R2/R3 (`AgentInvitation`); the owner uses O1-O8. Unchanged.
-- The owner dispatches on the envelope: `AgentInvitation` -> `smpInvitation` (current); `AgentConfirmation` with `ratchetKeyId` on a `ContactConnection` -> `smpAddressConfirmation` (new). Both coexist.
-- `AgentConfirmation` without `ratchetKeyId` remains the current confirmation on established connections.
-- An existing address gains `ratchetKeys` via a new agent API (e.g. `updateContactAddressLink`) that the app calls with the mutable link data (profile/badge and any other short-link data): the agent generates the DR bundle if absent (the first `address_ratchet_keys` row and its stored private keys), adds `ratchetKeys` to `UserContactData`, re-signs with `rcv_queues.link_priv_sig_key`, and `LSET`s it. **Only mutable data changes - the address (link) is unchanged**, because the keys are in mutable, not fixed, data. Requesters that fetch the updated data use DR; older ones still use `AgentInvitation`. The agent does not do this on its own (it lacks the profile and the user's intent); the app drives it, combined with the full→short address migration.
-
-## Part 6 - tests
-
-- Encoding roundtrips: `UserContactData` with and without `ratchetKeys`, and with the KEM present and absent; `AgentConfirmation` with and without `ratchetKeyId`, across versions.
-- Address creation advertises `ratchetKeys` (the `RcvE2ERatchetParamsUri`); `decryptLinkData` (Crypto/ShortLink.hs:100) verifies signatures and the requester negotiates the advertised params to a concrete version, with and without the KEM.
-- Both PQ modes: an address whose bundle carries a KEM gives a hybrid ratchet (`pqEncryption` on); one without gives an X448-only ratchet.
-- End to end: a DR-advertising address; a new requester establishes the ratchet, sends its profile under it, owner emits `REQ`, accepts, both reach `CON`; assert the profile never travels under per-queue-only encryption; assert `pqEncryption` on.
-- Rotation and retrofit: request against the current bundle; against a just-retired bundle within the window still decrypts; against a bundle past the window is discarded and the requester times out; an address that adds `ratchetKeys` via `LSET` is then reached by DR while an old requester still uses `AgentInvitation`.
-- Backward compatibility: old requester against a DR address connects via `AgentInvitation`; new requester against a non-DR address falls back to `AgentInvitation`.
-
-## Part 7 - phases
-
-1. Link data: `AddressRatchetKeys` in `UserContactData` (reusing `RcvE2ERatchetParamsUri`), encoding, `encodeSignUserData`; `AgentConfirmation.ratchetKeyId`; address creation taking `InitialKeys` (lifting the `IKUsePQ`-for-`SCMContact` prohibition), generating (`generateRcvE2EParams`, KEM per `initialPQEncryption False`) and storing the first `address_ratchet_keys` row.
-2. Handshake: thread the optional `Maybe AddressRatchetKeys` through `joinConnection`/`joinConn`/`joinConnSrv` (present → DR branch, absent → classic); requester R2'/R3' (synchronous one-shot send); owner O1'/O2'/O3' storing the DR request as a `conn_invitations` row whose request value is the `CRConfirmation` variant; `smpConfirmation` `RcvConnection … Nothing` branch and `AgentConnInfoReply` acceptance; end-to-end connection with the profile under the ratchet. Tests pass `AddressRatchetKeys` directly (chat wiring is later work).
-3. Rotation and retrofit: schema migration, `rotateRatchetKeys`, retention window, cleanup step, app-driven `LSET` retrofit (with the full→short address migration), rotation/retrofit tests.

plans/2026-07-24-signed-service-requests.md

@@ -1,59 +0,0 @@
-# Signed service requests
-
-Optional Ed25519 signature on service RPC requests, constructed and verified in the agent (not the bot), bound to the request's double ratchet. Requests only — responses stay authenticated by the address/ratchet. Signing is optional; a bot decides whether to require it. The agent is stateless: the meaning of a signer key (identity, resource) is the bot's concern.
-
-## Wire — `Simplex.Messaging.Agent.Protocol`
-
-Extend the existing `'A'` inner message; `Maybe` absent = unsigned, so the unsigned path is unchanged:
-
-```haskell
-AgentServiceRequest (NonEmpty SMPQueueInfo) (Maybe RequestSignature) MsgBody
-
-data RequestSignature = RequestSignature C.PublicKeyEd25519 (C.Signature 'C.Ed25519)
-```
-
-## Binding
-
-```
-binding = sha3-256("SimpleXService" <> rcAD)
-sig     = Ed25519.sign(sk, binding <> payload)
-```
-The service recomputes `binding` from its own `rcAD` and verifies.
-
-- `rcAD` = the ratchet associated data (`Ratchet.rcAD`) — the shared connection security code: identical on both ratchets by construction (`pubKey(requester ephemeral) <> pubKey(service key)`), stable, and unique per request (fresh requester ephemeral). Already on the ratchet; nothing derived or stored.
-- sha3-256 here is not for uniformity or secrecy (both moot: the value is signed, not keyed, and only a ratchet holder can craft a valid request). It gives a canonical fixed-length, domain-tagged binding; the 32-byte fixed prefix also makes `binding <> payload` unambiguous.
-- Domain string `"SimpleXService"`: separates this signature from other uses of the signing key.
-- Not covered: reply queues (the AEAD protects them in transit; addresses may use redundant queues).
-- Anti-relay: a signature bound to one session's rcAD does not verify under another's (both parties' keys differ). Replay of the encrypted blob is handled separately by transport dedup.
-
-## Sign (requester) — `Simplex.Messaging.Agent`
-
-- `sendServiceRequest` / `sendServiceRequestAsync` gain a `Maybe` Ed25519 signing key.
-- `joinConnSrv'` DR path takes the ratchet straight from the `createRatchet_`/`getSndRatchet` line (both now yield `(RatchetX448, params)`) and computes `serviceReqBinding` from its `rcAD`; the `mkInner :: SMPQueueInfo -> ByteString -> AgentMessage` closure calls `signServiceReq signKey_ binding payload` — `RequestSignature pub (sign' pk (binding <> payload))` when a key is given, `Nothing` otherwise.
-- Async carries the key in `JRServiceReq {requestKey :: Maybe C.PrivateKeyEd25519}` (enabled `StrEncoding (PrivateKey Ed25519)`); the JOIN worker deserializes it and signs after building the ratchet.
-
-**Status:** implemented and tested in simplexmq-3 (sync + async); invalid signature → `A_SERVICE ASEBadSignature` (logs + `ERR` event), no invitation.
-
-## Verify (service) — `smpContactRequest`
-
-After `initRcvRatchet_` + decrypt, on `AgentServiceRequest (replyQueue :| _) sig_ payload`, one helper does the check:
-
-`verifyServiceReq rc payload sig_ :: Either String (Maybe C.PublicKeyEd25519)`
-- `Nothing` → `Right Nothing` (unsigned).
-- `Just (RequestSignature key sig)` → recompute `serviceReqBinding rc` and `C.verify' key sig (binding <> payload)`; `Right (Just key)` if valid, else `Left err`.
-
-Then:
-- `Right key_` → `storeInvitation … True` + `notify $ SREQ invId key_ payload`.
-- `Left err` → `logError` + `notify (ERR (AGENT (A_SERVICE ASEBadSignature)))`, no invitation.
-
-Dedup unchanged.
-
-## Event / API
-
-- `SREQ :: InvitationId -> Maybe C.PublicKeyEd25519 -> MsgBody -> AEvent AEConn`.
-- `StrEncoding (PrivateKey Ed25519)` enabled so a caller (e.g. via `JRServiceReq`) can carry the signing key.
-
-## Tests
-
-- `testSignedServiceRequest` (sync) + `testSignedServiceRequestAsync` — signed round-trip delivers the exact signer key on `SREQ` (`sigKey_ == Just signPub`).
-- Unsigned path unchanged (existing service tests carry `Nothing` for the new field).

plans/2026-08-05-proxy-relay-error-attribution.md

@@ -1,81 +0,0 @@
-## Root cause: PRXY errors are attributed to the forwarding server instead of the destination relay
-
-When private routing is enabled and the destination relay is unreachable, the client reports
-**"Error connecting to forwarding server smp5.simplex.im"** — naming a preset server that the client
-connected to successfully. Retrying rotates to the next proxy (`getNextServer`, `Agent/Client.hs:689`)
-and produces the same message with a different preset server, so the destination server is never named
-and the failure looks like an outage of our own infrastructure.
-
-### Reproduction
-
-Connecting to a contact address on an unresolvable host (`simplex.server.home`, no DNS record):
-
-```
--- private routing off (correct)
-BROKER {brokerAddress = "smp://VvXX…@simplex.server.home:5223",
-        brokerErr = NETWORK {networkError = NEConnectError {connectError = "…does not exist (Name or service not known)"}}}
-
--- private routing on (misattributed)
-SMP {serverAddress = "smp://…@smp5.simplex.im,…onion",
-     smpErr = PROXY {proxyErr = BROKER {brokerErr = NETWORK {networkError = NEFailedError}}}}
-```
-
-### The asymmetry between the two proxied paths
-
-A server returns `PROXY (BROKER …)` only from `smpProxyError` (`Client.hs:804-815`), which is called
-exclusively where the proxy failed to reach the relay — `PRXY` (`Server.hs:1444`) and `PFWD`
-(`Server.hs:1466`). The error therefore *always* describes the proxy→relay hop. The two paths then
-diverge in how the agent wraps it:
-
-**PFWD — keeps both addresses** (`Agent/Client.hs:1183-1189`): the proxy's error arrives as
-`Left ProxyClientError` and is thrown as `PROXY {proxyServer, relayServer, proxyErr}`.
-
-**PRXY — drops the relay** (`Agent/Client.hs:713`): `connectSMPProxiedRelay` has no `Either` layer, so
-the error arrives as `PCEProtocolError` and `liftClient SMP` maps it to `SMP <proxyAddr> (PROXY …)`
-(`Agent/Client.hs:1244`). The destination address is discarded.
-
-Both clients read the second shape as a client→proxy failure and word it accordingly
-(`SimpleXAPI.kt:2692`, `ErrorAlert.swift:117`), which is never what it means.
-
-### Fix
-
-In `newProxiedRelay`, map proxy-reported `PROXY (BROKER …)` errors to the same shape `PFWD` already
-produces:
-
-```haskell
-proxyRelayError :: HostName -> ErrorType -> AgentErrorType
-proxyRelayError proxyHost = \case
-  e@(SMP.PROXY (SMP.BROKER _)) -> PROXY {proxyServer = protocolClientServer smp, relayServer = …destSrv, proxyErr = ProxyProtocolError e}
-  e -> SMP proxyHost e
-```
-
-`liftClient` applies this only to `PCEProtocolError`, so genuine client↔proxy failures (response
-timeout, network error, proxy transport version) still map to `BROKER <proxy> …` and remain attributed
-to the proxy. Both apps already render the resulting shape correctly, with no client change:
-*"Forwarding server smp5.simplex.im failed to connect to destination server simplex.server.home."*
-
-The guard is `BROKER` rather than every `ProxyError`, so the remap covers exactly the misattributed
-class and nothing else. `BASIC_AUTH` is deliberately excluded — the proxy returns it when proxying is
-disabled or the basic auth does not match (`Server.hs:1416-1420`), which is a client↔proxy fact and is
-correctly attributed today. `NO_SESSION` is returned only for `PFWD`. `PROTOCOL` describes the relay
-but is not rendered as a proxy-connection error by either client, so leaving it unchanged keeps the
-diff to the errors that actually produce a wrong message.
-
-### Blast radius
-
-- `temporaryAgentError` (`Agent/Client.hs:1572-1580`) and `serverHostError` (`:1594-1596`) already match
-  both shapes with the same helpers — retry and proxy-fallback behaviour is unchanged.
-- `clientServiceError` (`:1268-1273`) has no `PROXY`-shape twin for `BROKER NO_SERVICE`, but both ends
-  document that case as unreachable (`Client.hs:812`); left as is.
-- simplex-chat `Subscriber.hs:1819-1820` handles both shapes; send failures move from `SndErrProxy` to
-  `SndErrProxyRelay`, i.e. "Destination server error" rather than "Error" — also more accurate.
-- `SMP _ (PROXY _)` becomes unreachable, making `smpProxyErrorAlert` in both clients dead code. Removing
-  it is a follow-up in simplex-chat, not required by this change.
-
-### Verification
-
-- Reproduced before/after with a CLI built against this branch: the error now carries
-  `relayServer = "smp://VvXX…@simplex.server.home:5223"`, and the direct (non-proxied) path is
-  byte-identical to before.
-- `SMPProxyTests`: 45 examples, 0 failures — including `fails when fallback is prohibited` and both
-  retry tests, which exercise `newProxiedRelay` and the error classification.

plans/2026-08-09-fast-queue-rotation-plan.md

@@ -1,187 +0,0 @@
-# Fast queue rotation — implementation plan
-
-Branch: ep/drop-agent-versions. RFC: ../rfcs/2026-08-09-fast-queue-rotation.md.
-
-Model: redundant delivery, no flip. `QADD` adds the new receive queue R'. From `QADD` until R' is
-secured the sender writes every message to both old and R' (double delivery, not a move); once R' is
-secured the sender writes new messages to R' only, while old delivers its already-scheduled tail and
-the `QEND` appended to it. `QEND` removes a named queue. The recipient drops duplicates (double
-ratchet), so the order and which queue delivers do not matter, as long as every message arrives on at
-least one queue. Rotation away from a dead server works because every message up to securing is
-scheduled on R' too.
-
-Roles: A initiates (its receive queue rotates; A receives on R'). B sends to A and secures R'.
-
-## Why redundant delivery removes the hard parts
-
-- No boundary, no drain, no last-message id. A never decides how much of old to read.
-- A dead old server loses nothing B still holds: every undelivered message is scheduled on R' as well.
-- A dead new server does not suspend delivery: old keeps delivering until R' is secured.
-- The double ratchet already drops duplicates (`AGENT A_DUPLICATE`) and tolerates bounded reordering,
-  and the delivery schema already writes one message to several send queues (`enqueueMessageB` +
-  `enqueueSavedMessageB`).
-
-## The one ordering constraint
-
-A must hold R''s secret before it reads any R' data message. A data message reaching R' before the
-confirmation is dropped as "no keys" (`processClientMsg`, `(Nothing, Nothing)` arm, line 3611), which
-loses it when old is dead. So the confirmation is the first message B sends on R'. R''s delivery
-worker does not start while R' is securing, so its accumulated rows cannot outrun the confirmation.
-`ICQSndSecure` sends the confirmation and only then starts the worker. This holds on restart too (see
-Worker gate).
-
-## New definitions
-
-Agent/Protocol.hs
-- Condition fast rotation on the existing `rpcAddressSMPAgentVersion` (v8, `Protocol.hs:322`).
-- New `AMessage` constructor `QEND SndQAddr` (tag `QE`), the address of the queue to remove. v8-only,
-  and only sent during fast rotation, so peers below v8 never parse it.
-- `SndSwitchStatus` constructors `SSSecuringQueue` (old, while R' secures) and `SSSendingQEND` (old,
-  after R' is secured — it drains its tail and `QEND` but takes no new messages).
-- `InternalCommand` constructor `ICQSndSecure SMP.SenderId`.
-
-No receive-side switch status, no boundary, no drain state.
-
-## Schema
-
-None. `SSSecuringQueue` uses `snd_queues.switch_status`; R' is secured into `rcv_queues.e2e_dh_secret`.
-No new columns, no migration.
-
-## Sender B
-
-### Dual scheduling from QADD
-
-`enqueueMessageB` writes a delivery row for the head send queue and for each `filter isActiveSndQ`
-tail queue (`Agent.hs:2345`). Adjust the selection two ways: additionally include a securing
-replacement queue on a v8 connection (`connAgentVersion cData >= rpcAddressSMPAgentVersion && status == New && isJust dbReplaceQueueId`),
-and exclude a terminating queue (`sndSwchStatus == Just SSSendingQEND`). The version guard keeps the
-slow path unchanged — there R' is also `New` with a replace reference during `QKEY`/`QUSE`, but it must
-not be dual-scheduled. `SSSendingQEND` is a fast-path-only status, so the exclusion never affects the
-slow path. The gate below is inert for the slow path anyway, since it never starts R''s worker while
-`New`.
-
-- `QADD` until R' secured: old is the head (active) and R' is the securing replacement, so every `SEND`
-  writes both rows. old delivers at once; R''s rows accumulate behind its gate.
-- R' secured: R' is the head (primary) and old is `SSSendingQEND` (excluded), so a `SEND` writes R'
-  only. old keeps its worker and delivers whatever was already scheduled on it, plus `QEND`.
-
-### Worker gate
-
-`submitPendingMsg` (`Agent.hs:2437`) and `resumeMsgDelivery` (`2421`) — the two `getDeliveryWorker`
-callers that start delivery — skip a queue with `status == New && isJust dbReplaceQueueId`, so neither
-a `SEND` nor startup starts R''s worker while it secures. Startup resumes delivery through
-`resumeMsgDelivery` (`resumeDelivery` line 1848, and `getAllSndQueuesForDelivery` line 1943), so R' is
-skipped there; `resumeAllCommands` (1883) resumes R''s `ICQSndSecure`, which secures R' and only then
-starts its worker.
-
-### Steps
-
-`qAddMsg` (fast branch, under the connection lock, `Agent.hs:3855`):
-- Add R' as the slow path does (line 3870): `addConnSndQueue (sq_) {primary = True, dbReplaceQueueId = Just old}`, `New`.
-- Duplicate **every** undelivered message on old to R': for each pending row on old
-  (`SELECT internal_id FROM snd_message_deliveries WHERE conn_id = ? AND snd_queue_id = old AND failed = 0`),
-  `createSndMsgDelivery db R' internalId`. This is the loss-prevention step: old's not-yet-sent
-  messages are duplicated onto R', so if old later fails they are already on R'. If old is already
-  down, nothing was sent and the whole backlog is duplicated.
-- `enqueueCommand (Just newSrv) (ICQSndSecure sndId)`; `setSndSwitchStatus SSSecuringQueue` on old
-  (where the slow path sets `SSSendingQKEY`, line 3874); notify `SWITCH QDSnd SPStarted`. old keeps
-  delivering; R''s worker is gated.
-
-`ICQSndSecure sId` (retryable, under `tryWithLock`):
-1. If old is already gone, a prior attempt finished; return. Otherwise find R' by `sId`.
-2. `secureSndQueue` (SKEY) R' — idempotent, since `sndPrivateKey` was persisted by `qAddMsg`
-   (`QueueStore/STM.hs:213`: same key → `Right ()`, different key → `AUTH`).
-3. Send the confirmation on R' (`sendConfirmation`; empty body, both peers already know each other;
-   `e2eEncryption_ = Nothing`, no ratchet step). It is the first message on R'.
-4. On success, in one transaction: `setSndQueueStatus R' Active` (the gate lifts), `setSndQueuePrimary R'`
-   (R' becomes the head; its replace reference is cleared), and `setSndSwitchStatus old (Just SSSendingQEND)`
-   (old takes no new messages but keeps its worker). Then `submitPendingMsg c R'` (the worker starts and
-   flushes the accumulated rows after the confirmation), `enqueueMessages [old, R'] (QEND oldAddr)`
-   (appended after old's tail, delivered on both), and notify `SWITCH QDSnd SPSecured`. From here a
-   `SEND` goes to R' only; old delivers its tail then `QEND` and is removed when `QEND` is sent.
-
-A temporary error retries from step 1; nothing is torn down. A permanent `AUTH` (should not occur, R'
-was secured with B's own key) leaves both queues and surfaces `A_QUEUE`.
-
-`QEND` sent — new `AM_QEND_` arm in `runSmpQueueMsgDelivery`, modelled on `AM_QTEST_` (`Agent.hs:2567`):
-on a successful send of `QEND addr`, remove the named send queue (`TM.delete` its worker,
-`deleteConnSndQueue addr`), make the remaining queue the sole primary (`setSndQueuePrimary`, which
-clears its `replace_snd_queue_id`), and notify `SWITCH QDSnd SPCompleted` (as `AM_QTEST_` does, line
-2591). This re-primary is a no-op on the send side — step 4 already made R' primary before `QEND` was
-enqueued. The handler is idempotent — a second `QEND` send finds the named queue already gone and does
-nothing. `QEND` is sent on both queues; removing old's send queue also drops any `QEND` still pending
-on old. The R' copy reliably removes old
-and reaches A even when old is dead; the old copy is best effort. Once old's send queue is gone, `SEND`
-schedules to R' only.
-
-## Recipient A
-
-A is subscribed to old (primary, `RSSendingQADD`) and R' (created at rotation start,
-`dbReplaceQueueId = old`).
-
-- **Confirmation on R'.** In `processClientMsg`, `(Nothing, Just e2ePubKey)` case, add an arm before the
-  `senderCanSecure` arm (`Agent.hs:3476`), guarded by `isJust (dbReplaceQueueId rq)`. In one
-  transaction: `setRcvQueueConfirmedE2E rq (C.dh' e2ePubKey e2ePrivKey) (min v phVer)` (secures R') and
-  `setRcvQueuePrimary R'` (clears R''s replace reference). Then `ack`, and notify `SWITCH QDRcv SPConfirmed`.
-  No conn-info processing, no ratchet step, no deferral. Redelivery is idempotent: R' now has `e2e_dh_secret`, so a re-sent
-  confirmation reaches the `(Just e2eDh, Just _)` arm (line 3608) and is acked — correct here, since
-  there is no backlog to hold.
-- **Data on R'.** With R''s replace reference cleared, a data message on R' takes the ordinary path
-  (`(_, dbReplaceQueueId=Nothing)`, line 3503) — no old-deletion, no `RSSendingQUSE` check. A copy
-  already read on old is dropped as `A_DUPLICATE`; a copy read first on R' advances the ratchet and
-  old's copy is then the duplicate.
-- **`QEND oldAddr` on either queue.** New `AMessage` handler (`qEndMsg`, a `qDuplex` handler like
-  `qAddMsg`): `findRQ oldAddr` the receive queue to remove. Mark it deleted (`setRcvQueueDeleted`, so
-  `getRcvQueuesByConnId_`'s `deleted = 0` filter excludes it at once and a restart does not resurrect
-  it) and `enqueueCommand (Just oldServer) (ICDeleteRcvQueue oldRcvId)` for the server `DEL` and record
-  removal — the async, crash-safe path `abortConnectionSwitch` uses, which resumes on restart and does
-  not block `QEND`, **not** the synchronous `deleteQueue` of `finalizeSwitch`, which would stall if old
-  is unreachable. `ICDeleteRcvQueue` (`Agent.hs:2224`) currently retries a temporary error forever;
-  bound it with the same persisted `rcv_queues.delete_errors`/`deleteErrorCount` mechanism `deleteQueueRec`
-  uses (2884): on a temporary error `incRcvDeleteErrors`, and at the limit `deleteConnRcvQueue` and
-  stop. The count is in the database, so the bound survives restarts, and its only other caller
-  (`abortConnectionSwitch'`, 2739) deletes an alive queue that succeeds well before the limit. `qEndMsg`
-  does **not** re-primary R' — the confirmation arm (above) owns R''s primary flag and replace
-  reference. `QEND` on old and the confirmation on R' travel on different queues with no order between
-  them, so `QEND` on old can be processed first (it sits only behind old's tail); re-primarying then
-  would clear R''s `dbReplaceQueueId` and the later confirmation would miss the rotation arm and never
-  secure R'. So `qEndMsg` only removes the named queue. Re-create the notification subscription
-  (`when enableNtfs $ sendNtfSubCommand ns (NSCCreate, [connId])`); notify
-  `SWITCH QDRcv SPCompleted`; `ackDel` the `QEND`. Received on both queues, the second finds it already
-  marked deleted and is a no-op.
-
-No drain, no boundary, no finalize command. Old is removed when `QEND` arrives, not by counting.
-
-## Abort / version
-
-- Fast rotation runs only when `connAgentVersion >= v8`; otherwise the `QKEY`/`QUSE` slow path runs
-  unchanged.
-- `canAbortRcvSwitch` (`Agent/Store.hs:210`) returns false for `RSSendingQADD` when `connAgentVersion >= v8`
-  (at v8 B always chooses fast, so A treats a sent `QADD` as committed). Its signature gains
-  `connAgentVersion`; both callers pass it from `cData` — `abortConnectionSwitch'` (2730) and
-  `rcvQueueInfo` in `connectionStats` (2976).
-
-## Losses and duplication
-
-- No boundary loss: the `QADD` step **duplicates** old's entire undelivered backlog onto R', and every
-  later message up to securing is scheduled on both queues, so if old fails it loses nothing B still
-  holds. The only messages old can strand are those its server already accepted but had not handed to
-  A — the ordinary store-and-forward risk, present whenever a server fails with unread messages, and
-  empty if old was already down (a down server accepted nothing).
-- Duplicates: the double ratchet drops them (`A_DUPLICATE`); `checkMsgIntegrity`'s `MsgDuplicate` is
-  only a flag, not the mechanism.
-- The 512 skip bound (`Crypto/Ratchet.hs:953`) does not bite on the rotation: each queue delivers in
-  order and every message B still holds is on R', so A reads a contiguous stream with only small
-  cross-queue reordering. A store-and-forward residual (above) is an ordinary loss, not introduced here.
-
-## Tests
-
-- new/new, old stopped right after `QADD`: rotation completes; all messages delivered on R'; old
-  removed by `QEND` on R'.
-- new/new, both alive: messages delivered on both, deduped; old removed by `QEND`.
-- new/old and old/new: fall back to the `QKEY`/`QUSE` path.
-- crash during securing: restart does not start R''s worker; `ICQSndSecure` resumes, secures R',
-  starts the worker, sends `QEND`.
-- `QEND` received on both queues: old removed once, the second receipt is a no-op.
-- `QEND` on old processed before the confirmation on R': R' still secures, because `qEndMsg` does not
-  clear R''s replace reference; rotation completes.

plans/2026-08-22-xftp-file-storage-time.md

@@ -1,168 +0,0 @@
-# Implementation plan: XFTP variable file storage time
-
-Proposal: `../rfcs/2026-08-22-xftp-file-storage-time.md`.
-
-## simplexmq: entitlement crypto
-
-New module `Simplex.Messaging.Crypto.Entitlement`, over `Simplex.Messaging.Crypto.BBS`:
-
-Types:
-
-```
-newtype MasterKey = MasterKey ByteString
-
-data Entitlement = Entitlement
-  { expiresAt :: UTCTime,
-    entitlementName :: Text,
-    extraInfo :: Text
-  }
-
-data EntitlementCredential = EntitlementCredential
-  { issuerKeyIdx :: Word16,
-    masterKey :: MasterKey,
-    entitlement :: Entitlement,
-    issuerSignature :: BBSSignature
-  }
-
-data EntitlementProof = EntitlementProof
-  { issuerKeyIdx :: Word16,
-    entitlement :: Entitlement,
-    entProof :: BBSProof
-  }
-```
-
-Functions and constants:
-
-- the disclosed-message encoding: the master key is message 0 and stays undisclosed; `expiresAt`, `entitlementName`, and `extraInfo` are messages 1 to 3 and are disclosed. The protocol encoding of `Entitlement` and its field order follow the same order
-- the BBS header string `"SimpleX badges v1"` (shared with chat's badges, which sign under it), the message count, and the disclosed indexes
-- `generateEntitlementProof :: Map Word16 BBSPublicKey -> EntitlementCredential -> BBSPresHeader -> IO (Either String EntitlementProof)` (the issuer key is looked up by the credential's index; an absent index is `Left`)
-- `verifyEntitlement :: Map Word16 BBSPublicKey -> BBSPresHeader -> EntitlementProof -> IO EntitlementVerification`, where `data EntitlementVerification = EVValid | EVInvalid | EVUnknownIssuer` (the caller supplies the presentation header; the server reconstructs it, the proof never includes it)
-- the issuer public keys constant `Map Word16 BBSPublicKey`
-
-## simplexmq: protocol, new XFTP version
-
-In `Simplex.FileTransfer.Transport`:
-
-- add the next `VersionXFTP` and set `currentXFTPVersion` to 4
-- add `entitlementProof :: Maybe EntitlementProof` to `XFTPClientHandshake`, encoded before the `Tail` and only from this version
-- the presentation header is the session id alone
-
-In `Simplex.FileTransfer.Protocol`:
-
-- add `GrantedStorageTime` and its encoding; retain the one-character sum prefix for future variants:
-
-```
-data GrantedStorageTime = GSTExpires {epochSeconds :: Int64}
-```
-
-- add the storage time (`Maybe Word32`: `Nothing` and `Just 0` request the server maximum, a value above zero requests that number of hours) to `FNEW`
-- add the granted storage to `FRSndIds` as `Maybe GrantedStorageTime` (`Nothing` when decoding a response from a server below this version)
-
-## simplexmq: server configuration
-
-In `Simplex.FileTransfer.Server.Env` and `Simplex.FileTransfer.Server.Main`:
-
-- make `fileExpiration` non-optional (`ExpirationConfig`, no longer `Maybe`); the server always expires files, so the server maximum is always a concrete number of seconds
-- read a maximum storage time (a number of hours) for each entitlement name from the `[STORE_LOG]` INI section, from the keys `expire_files_hours_for_supporter` and `expire_files_hours_for_legend`, into `fileStorageEntitlements :: Map Text EntitlementConfig`, where `newtype EntitlementConfig = EntitlementConfig {storageTime :: Int64}` holds seconds; an absent key is skipped (that name gets the default), a present but malformed value fails startup
-- exit at startup if any name's maximum is below the default file expiration
-- add `entitlementKeys :: Map Word16 BBSPublicKey` to the server config (default = the shared constant, set from `Main`); the handshake verifies the proof against it, so the trusted keys never come from the sender
-
-## simplexmq: server session
-
-In `Simplex.FileTransfer.Server`:
-
-- `processClientHandshake` verifies the proof from the handshake, once per session, and resolves the maximum storage time for the entitlement name. `HandshakeSent` carries `EntitlementChecked`, so the first handshake verifies and every later one on that connection reuses the result, including after `processHello` returns the session to `HandshakeSent`
-- verify only when the answer can change: the name is configured (startup rejects a maximum below the default), and the entitlement expired less than 24 hours ago. A proof that fails these checks gets no verification; a proof that fails to verify is logged. In both cases the session gets the default maximum
-- a verified proof becomes `peerEntitlement :: Maybe SessionEntitlement` in `THAuthServer`, where `data SessionEntitlement = SessionEntitlement {expiresAt :: SystemSeconds, entConfig :: EntitlementConfig}`; `processXFTPRequest` takes it from there, so no proof is verified while a command is processed
-- `createFile` caps the requested storage time by the session maximum, which is the entitlement's storage time when the entitlement is still valid, and the default otherwise
-
-## simplexmq: server store and expiration
-
-The `files` table gets a nullable `expires_at`. Every new file stores a concrete `expires_at`. It is NULL only for pre-feature rows, which the migration must not re-date (it has no access to the operator's configured TTL); those are expired at query time as `created_at + ttl`.
-
-Common to both stores, in `Simplex.FileTransfer.Server.Store`:
-
-- add `expiresAt :: Maybe RoundedFileTime` to `FileRec`
-- in `createFile`, cap the requested hours at the session maximum, round the expiry up to the hour, store it, and return that same value as the granted storage
-- `expiredFiles` receives `now` and `old` (= `now - ttl`). A stored expiry is deleted when `expires_at < now` (no grace — it is already rounded up); a legacy row (no `expires_at`) is deleted when `created_at + fileTimePrecision < old` (the grace covers `created_at` being floored to the hour)
-- retain `created_at` for statistics, export, and the legacy fallback
-
-STM store:
-
-- in `expiredFiles`, expire a new file when `roundedSeconds expiresAt < now`, and a legacy file (no `expiresAt`) when `created_at + fileTimePrecision < old`
-
-PostgreSQL store, in `Simplex.FileTransfer.Server.Store.Postgres` and its migrations:
-
-- add the nullable column `expires_at BIGINT` (no backfill)
-- add one composite index `idx_files_expiry ON files (expires_at, created_at)`
-- `expiredFiles` query: `(SELECT ... WHERE expires_at < ? LIMIT ?) UNION ALL (SELECT ... WHERE expires_at IS NULL AND created_at < ? LIMIT ?)` with `(now, limit, old - fileTimePrecision, limit)`. The first arm deletes stored (already rounded-up) expiries; the second drains legacy rows, with the grace folded into `old - fileTimePrecision` so the columns stay bare and sargable. Each arm is one range over the composite index and stops at its own limit; a single `OR` predicate builds a bitmap of every match before the limit applies. A `COALESCE(expires_at, created_at + ttl)` predicate is avoided (not sargable, would force a sequential scan). No `ORDER BY` — the batch loop deletes all expired rows regardless of order. New files always store an expiry, so the second arm drains permanently once the legacy rows expire, and is then removed.
-
-Store log, in `Simplex.FileTransfer.Server.StoreLog`:
-
-- add the optional expiration to the `AddFile` record; a record without it parses to `Nothing` (the configured default), never a hardcoded value
-
-## simplexmq: agent
-
-The credential belongs to the user, so the agent holds it the way it holds the user's servers: in memory, supplied when the agent is created and replaced through an API. It is not stored by the agent.
-
-Per-user state in `Simplex.Messaging.Agent.Env.SQLite` and `Simplex.Messaging.Agent.Client`:
-
-- add `entitlements :: Map UserId EntitlementCredential` to `InitialAgentServers`, beside the servers
-- add `userEntitlements :: TMap UserId EntitlementCredential` to `AgentClient`, filled from it by `newAgentClient`
-- add `entitlementKeys :: Map Word16 BBSPublicKey` to `AgentConfig` (default = the shared constant), for the issuer key that proof generation needs
-
-Public API in `Simplex.Messaging.Agent`:
-
-- add storage time (`Maybe Word32` hours) to `xftpSendFile`
-- add `setUserEntitlement :: AgentClient -> UserId -> Maybe EntitlementCredential -> IO ()`, in the shape of `setProtocolServers`: it replaces the entry, and when the credential changed it closes that user's XFTP clients, so the next upload presents the new credential
-
-Store, in both the SQLite and PostgreSQL agent stores:
-
-- add a nullable storage time column (integer hours; NULL means the server maximum) to `snd_files`
-- add the migration to both stores
-- in `createSndFile`, store the storage time
-
-Upload, in `Simplex.Messaging.Agent.Client` and `Simplex.FileTransfer.Client`:
-
-- `getXFTPClient` takes a proof for the session as a parameter, `SessionId -> IO (Maybe EntitlementProof)`, beside the callback it already takes for a closed client. The client config holds no credential and no keys
-- `getXFTPServerClient` passes a function that generates the proof over the session id from the configured issuer keys, and only for a server of this user, matched by the key hash that TLS pins, so a file description of the sender cannot direct the entitlement to another server. A server that is not the user's, a missing credential, or a failure to generate gives `Nothing`, with the failure logged
-- `xftpClientHandshakeV1` calls it with the session id from the connection, and sends the result in the handshake
-- `agentXFTPNewChunk` reads the storage time from the send record and sends FNEW with it
-- `createXFTPChunk` returns the granted expiry (epoch seconds); `agentXFTPNewChunk` stores it on `NewSndChunkReplica`
-
-Completion:
-
-- `createXFTPChunk` returns the granted expiry as `Maybe GrantedStorageTime`; `SndFileChunkReplica` and `NewSndChunkReplica` carry `expiresAt :: Maybe GrantedStorageTime`
-- persist it in a nullable `replica_expires_at` column on `snd_file_chunk_replicas` (added to the entitlement migration): `createSndFileReplica` stores `epochSeconds`, `getSndFile` and `getNextSndChunkToUpload` read it back into `GSTExpires`
-- on `SFDONE`, report the file expiry: a chunk expires when its last replica expires (`max` over replicas, absent replicas ignored, `Nothing` only if none report); the file expires when its first chunk expires (`min` over chunks, `Nothing` if any chunk is unknown). `GrantedStorageTime` derives `Ord`
-- `SFDONE` gains a trailing `Maybe GrantedStorageTime` (not str-encoded); chat consumes it (wired later)
-
-Testing:
-
-- e2e test in `tests/XFTPAgent.hs`: generate a BBS keypair, sign a supporter credential (issuer key index 1), run the server with `entitlementKeys = {1: testPk}` and a supporter maximum above the default, run the sender agent with the same `entitlementKeys` and the credential for the user, send a file requesting a number of hours above the default and below that maximum, and assert `SFDONE`'s granted expiry rounds up `now + requested` (proof of the entitlement raising the max above the default)
-- the same upload without the credential is capped at the default maximum
-- the expiry is written before the status, because `BlockingInfo`'s notice parser is terminal (`A.takeByteString`), so any field after the blocking info makes the record unparseable and the file is dropped on restart. Every field added to `AddFile` must go before the status
-- store log round trip in `tests/CoreTests/StoreLogTests.hs`, in the shape of the SMP store log test: a file record survives a write, a read into the store, and compaction, with and without the expiry, including a file blocked with a notice in both cases
-
-## simplex-chat
-
-- remove lifetime badges: make `badgeExpiry` a `UTCTime`, drop the `"lifetime"` encoding, and remove the lifetime option from the UI and the CLI
-- map `BadgeInfo` to `Entitlement` (`entitlementName = textEncode badgeType`, `expiresAt = badgeExpiry`, `extraInfo = badgeExtra`) when calling the agent
-- pass the user's credential to the agent when it is created, and through `setUserEntitlement` when the badge changes, in the same places that pass and update the user's servers
-- pass the storage time to `xftpSendFile`
-- retain the `maxXFTPFileSize` size limit
-- reuse `verifyEntitlement` for peer-badge verification
-- import the issuer public keys from the shared simplexmq constant
-
-## State
-
-Steps 2 to 5 are implemented in simplexmq. Step 1 and step 6 belong to the chat branch that carries badges.
-
-## Order
-
-1. Add the entitlement crypto module; move chat's badge verification onto it and remove lifetime badges.
-2. Add the new XFTP version, the FNEW storage time, and the response.
-3. Change the server configuration, store, expiration, and store log.
-4. Move the proof to the handshake: the handshake field, the session state on the server, and the proof for the session on the client.
-5. Hold the credential per user in the agent, and add the API to replace it.
-6. Wire chat to pass the credential and the storage time.

plans/20260522_01_smp_public_namespaces.md

@@ -1,455 +0,0 @@
-# Server: SMP support for public namespaces
-
-> **⚠ Implementation diverged from this plan.** Six audit rounds reshaped the
-> original design. **The shipped code differs in several load-bearing ways:**
->
-> - **Wire format**: `NameRecord` is now JSON (aeson), not the custom binary
->   ABNF this plan documents. See `protocol/simplex-messaging.md` §Resolver
->   commands and `src/Simplex/Messaging/Protocol.hs` ToJSON/FromJSON instances.
-> - **No cache**: the TTL + FIFO + byte-cap cache, in-flight coalescing,
->   `psqueues` dep, and `cache_*` INI keys are all gone. Every RSLV becomes
->   one `eth_call` bounded by `rpcMaxConcurrency` + `rpcTimeoutMs`. See
->   `src/Simplex/Messaging/Server/Names.hs`.
-> - **No `allow_dangerous_colocation` flag**: the proxy co-location guard
->   was demoted to a startup `logWarn` (the flag was always-on because
->   `[PROXY]` has no enable toggle).
-> - **Module shape**: `Names/Resolver.hs` was merged into `Names.hs`; only
->   `Names/Eth/RPC.hs` and `Names/Eth/SNRC.hs` remain as separate modules.
-> - **Test list**: of the 15 specs listed below, ~7 shipped; the rest were
->   either superseded by the cache removal (CacheSpec) or deferred
->   (ForwardedRslvSpec, MockRpcSpec, StartupGuardSpec, UrlValidationSpec,
->   EipChecksumSpec).
->
-> Sources of truth: `CHANGELOG.md` (release notes),
-> `protocol/simplex-messaging.md` §Resolver commands (wire format),
-> `src/Simplex/Messaging/Server/Names*.hs` (implementation). This file is
-> retained as historical context; do not treat it as a specification.
-
-Implementation plan for Part 2 of [RFC 2026-05-21-public-namespaces](https://github.com/simplex-chat/simplex-chat/blob/ep/namespace/docs/rfcs/2026-05-21-public-namespaces.md). Adds a forwarded-only `RSLV <lookup_key>` SMP command that returns `NAME <NameRecord>` read from the SNRC contract via a Reth+Nimbus JSON-RPC endpoint. Smp-server becomes name-capable by `[NAMES] enable: on`.
-
-Out of scope: `Simplex.Messaging.Client` API, agent-side resolution flow, `ServerRoles.names` in the agent, default-router list, reverse resolution, multicoin/text records, state proofs.
-
-## Architecture
-
-```mermaid
-sequenceDiagram
-  participant C as Client
-  participant P as Proxy (storage role)
-  participant N as Name server (names role)
-  participant E as Ethereum endpoint<br/>(Reth+Nimbus)
-
-  C ->> P: PFWD(enc(RSLV key))
-  P ->> N: RFWD(enc(RSLV key))
-  note over N: verifyTransmission True →<br/>vc SResolver (RSLV _) → VRVerified
-  N ->> N: cache lookup
-  alt cache miss
-    N ->> E: eth_call(SNRC, namehash(key))
-    E -->> N: ABI bytes
-    note over N: ABI decode + zero-owner check + cache insert
-  end
-  N -->> P: RFWD(enc(NAME rec | ERR AUTH))
-  P -->> C: PRES(enc(NAME rec | ERR AUTH))
-```
-
-RSLV is **forwarded-only** — direct RSLV is rejected `CMD PROHIBITED`. This preserves the RFC's two-server resolution: the name server sees the lookup key but never the client's IP, session, or identity.
-
-## Protocol
-
-Shared library: `src/Simplex/Messaging/Protocol.hs` and `src/Simplex/Messaging/Transport.hs`.
-
-**Version.** `Transport.hs:226`: `namesSMPVersion = VersionSMP 20`. Bump `currentClientSMPRelayVersion`, `currentServerSMPRelayVersion`, `proxiedSMPRelayVersion` to 20. Pre-v20 binaries lack the `RSLV_` tag; v20 binaries with sessions negotiated at v < 20 reject `RSLV_` at the parameter parser. The proxied-version bump 18 → 20 is safe (v19's `RecipientService`/`NotifierService` aren't in the forwarded whitelist; v18's `BLOCKED info` is already version-branched at `Protocol.hs:1943`).
-
-**Party kind.** Append `Resolver` to `Party` (line 335); add `SResolver` (line 349), `TestEquality` clause (line 361), `PartyI Resolver` (line 394). `queueParty SResolver = Nothing` (falls through line 412). `partyClientRole SResolver = Nothing`.
-
-**`RSLV` command.**
-
-```haskell
-RSLV :: LookupKey -> Command Resolver
-newtype LookupKey = LookupKey ByteString
-
-instance Encoding LookupKey where
-  smpEncode (LookupKey s) = smpEncode s
-  smpP = do
-    n <- lenP
-    when (n > 64) $ fail "LookupKey too large"
-    LookupKey <$> A.take n
-```
-
-Name-syntax validation is client-side per RFC; the server treats the key as opaque bytes. Tag `"RSLV"`, version guard inside `protocolP v (CT SResolver RSLV_)`: `| v >= namesSMPVersion -> Cmd SResolver . RSLV <$> _smpP`.
-
-**Testnet/mainnet selector**: how the `#testnet:name` namespace appears in `LookupKey` bytes is determined by the SNRC contract (Part 1) — confirm with Part 1 before merging.
-
-**`NAME` response.**
-
-```haskell
-NAME :: NameRecord -> BrokerMsg
-```
-
-Tag `"NAME"`. Symmetric version guards on encode (in `encodeProtocol v`) and decode (in `protocolP v NAME_`): `| v >= namesSMPVersion -> ...`. `NameRecord` has **no `Encoding` typeclass instance** — the typeclass cannot version-branch. Use top-level helpers `nameRecBytes :: VersionSMP -> NameRecord -> ByteString` and `parseNameRec :: VersionSMP -> Parser NameRecord`, mirroring the `IDS QIK` precedent at `Protocol.hs:1912–1979`.
-
-**`NameRecord` schema and wire layout.**
-
-```haskell
-data NameRecord = NameRecord
-  { nrDisplayName  :: Text                    -- ≤255 bytes UTF-8
-  , nrOwner        :: NameOwner               -- 20 raw bytes
-  , nrChannelLinks :: [NameLink]
-  , nrContactLinks :: [NameLink]
-  , nrAdminAddress :: Maybe Text
-  , nrAdminEmail   :: Maybe Text
-  , nrExpiry       :: Int64                   -- Unix seconds, ≥ 0
-  , nrIsTest       :: Bool
-  }
-
-newtype NameOwner = NameOwner ByteString   -- bare ctor NOT exported; smart ctor enforces length 20
-newtype NameLink  = NameLink  Text         -- bare ctor NOT exported; smart ctor enforces ≤1024 bytes
-
-unNameOwner :: NameOwner -> ByteString
-unNameOwner (NameOwner bs) = bs
-
-unNameLink :: NameLink -> Text
-unNameLink (NameLink t) = t
-```
-
-Field additions are gated by future SMP version bumps (matching the `IDS QIK` precedent at `Protocol.hs:1912–1979`) — no separate record-version field.
-
-| Field | Encoding | Max bytes |
-|---|---|---|
-| `nrDisplayName` | 1-byte length prefix + UTF-8 | 1 + 255 |
-| `nrOwner` | 20 raw bytes, no prefix | 20 |
-| `nrChannelLinks`, `nrContactLinks` | 1-byte count + per-element (Word16 BE len + UTF-8); combined cap **8 entries** across both lists | 1 + Σ(2 + ≤1024) |
-| `nrAdminAddress`, `nrAdminEmail` | `'0'` or `'1'` + (1-byte length + UTF-8 if `'1'`) | 1 + 1 + 255 |
-| `nrExpiry` | two big-endian `Word32` | 8 |
-| `nrIsTest` | `'T'` or `'F'` | 1 |
-
-`Encoding NameLink` reads the Word16 length **before** `A.take` allocates — going through the existing `Large` wrapper allows up to 65 535 bytes per element. There is no `Encoding [a]` instance — use `smpEncodeList` / `smpListP` / a bounded variant:
-
-```haskell
-smpListPUpTo :: Encoding a => Int -> Parser [a]
-smpListPUpTo cap = do
-  n <- lenP
-  when (n > cap) $ fail "list too long"
-  A.count n smpP
-
-parseNameRec _v = do
-  nrDisplayName  <- smpP
-  nrOwner        <- smpP
-  nrChannelLinks <- smpListPUpTo 8
-  nrContactLinks <- smpListPUpTo (8 - length nrChannelLinks)
-  nrAdminAddress <- smpP
-  nrAdminEmail   <- smpP
-  nrExpiry       <- smpP
-  when (nrExpiry < 0) $ fail "expiry must be non-negative"
-  nrIsTest       <- smpP
-  pure NameRecord{..}
-```
-
-Both list parsers fail at the count step before allocating; the second inherits the residual budget. Canonical encoding by construction: every primitive has exactly one valid byte form — two name servers reading the same SNRC state produce byte-identical responses.
-
-**Wire-size budget.** `paddedProxiedTLength = 16226` is the plaintext input to `cbEncrypt` (`Server.hs:2117`); `pad` reserves 2 bytes → framed transmission ≤ 16 224 bytes. Combined-link cap 8 yields max payload ≈ 9 050 bytes — generous margin.
-
-**Error semantics.** A single wire code: `ERR AUTH`. Per RFC, this collapses every failure (name not found, malformed key, names disabled, RPC unreachable, decode error, timeout). Resolver internally distinguishes the cause for stats only.
-
-**Forwarded-only access.** Direct RSLV is rejected with `CMD PROHIBITED`. The shape of `THAuthServer` alone cannot discriminate direct from forwarded (`Transport.hs:852` sets `sessSecret' = Just _` for every v6+ direct client too). An explicit `forwarded :: Bool` flag is threaded through `verifyTransmission` (see below).
-
-## Server changes
-
-All edits in `src/Simplex/Messaging/Server.hs`.
-
-**`forwarded :: Bool` plumbing.** Three signatures change:
-
-- `verifyTransmission :: Bool -> ...` (line 1233) — direct path passes `False` (lines 1152–1153), forwarded path passes `True` (line 2129).
-- `verifyLoadedQueue :: Bool -> ...` (line 1238) — receives the flag from `verifyTransmission` (lines 1235, 1240).
-- `verifyQueueTransmission :: Bool -> ...` (line 1244) — receives and uses the flag.
-
-New `vc` clauses inside `verifyQueueTransmission`:
-
-```haskell
-vc SResolver (RSLV _) | forwarded = VRVerified Nothing
-                      | otherwise = VRFailed (CMD PROHIBITED)
-vc SResolver _ = VRFailed (CMD PROHIBITED)   -- defensive catch-all
-```
-
-**Forwarded whitelist** (`Server.hs:2132`):
-
-```haskell
-Cmd SResolver (RSLV _) -> True
-```
-
-**`processCommand` branch** (alongside line 1481):
-
-```haskell
-Cmd SResolver (RSLV (LookupKey key)) -> do
-  st <- asks (rslvStats . serverStats)
-  incStat (rslvReqs st)
-  asks namesEnv >>= \case
-    Nothing -> incStat (rslvDisabled st)  $> response (corrId, NoEntity, ERR AUTH)
-    Just nenv -> liftIO (resolveName nenv key) >>= \case
-      Right rec     -> incStat (rslvSucc st)     $> response (corrId, NoEntity, NAME rec)
-      Left NotFound -> incStat (rslvNotFound st) $> response (corrId, NoEntity, ERR AUTH)
-      Left _        -> incStat (rslvEthErrs st)  $> response (corrId, NoEntity, ERR AUTH)
-```
-
-**Shutdown.** Add `closeNamesEnv :: NamesEnv -> IO ()` calling `closeManager`. Wire into `closeServer` (`Server.hs:247`):
-
-```haskell
-closeServer = do
-  asks (smpAgent . proxyAgent) >>= liftIO . closeSMPClientAgent
-  asks namesEnv >>= liftIO . mapM_ closeNamesEnv
-```
-
-In-flight `resolveName` calls during shutdown receive `ConnectionClosed` → `EthHttpErr` → masked-leader cleanup runs → waiters unblock with `ERR AUTH`.
-
-**`incStat` relocation.** Defined at `Server.hs:2220`, currently unexported. Move to `Server/Stats.hs` (one-line transplant + export) so `Resolver.hs` can use it.
-
-**Co-located proxy warning.** `newEnv` logs a startup warning whenever `allowSMPProxy = True` and `namesConfig = Just _`. RSLV is the first slow forwarded command; on a proxy host it can serialise other forwarded commands on the same proxy-relay session up to `rpcTimeoutMs` per cache miss. The warning is not a hard refusal because `[PROXY]` has no `enable: on/off` toggle — proxy is always on for every smp-server. `forkForwardedCmd` async dispatch is the longer-term fix, tracked as a follow-up; once the proxy role is gateable per-server, the warning can be tightened back to a refusal.
-
-## Resolver subtree
-
-New module tree at `src/Simplex/Messaging/Server/Names/`:
-
-| Module | Contents |
-|---|---|
-| `Names.hs` | Façade — re-exports `NamesConfig`, `NamesEnv`, `ResolveError`, `resolveName`, `newNamesEnv`, `closeNamesEnv`. |
-| `Names/Resolver.hs` | All types + cache + in-flight + `resolveName`. Helpers exported directly (no `.Internal` per codebase convention). **Test seam**: `NamesEnv` holds `ethCall` as a function value, so tests construct stubs via `newNamesEnvWith`. |
-| `Names/Eth/RPC.hs` | `EthRpcEnv`; `ethCallReal` via `http-client` + `withResponse` + `brReadSome rpcMaxResponseBytes`. JSON-RPC error / HTTP error split. `rpcMaxConcurrency` semaphore. `Authorization` header from `rpcAuth`. |
-| `Names/Eth/SNRC.hs` | `EthAddress`, Keccak-256 namehash via `crypton`'s `Crypto.Hash.Algorithms.Keccak_256` (mirroring `Crypto.hs:1023–1025` for SHA3), hand-rolled bounded Solidity ABI codec, `getRecord` with zero-owner detection. **Ethereum's Keccak ≠ NIST SHA3-256.** |
-
-**ABI codec invariants**, enforced before any allocation: `offset + 32 ≤ buf.length`; `offset + 32 + length ≤ buf.length`; `offset ≥ headEnd` (no backward jumps); every length ≤ per-field cap; `string[]` outer length × 32 ≤ buf.length; recursion depth ≤ 2; `uint256 → Int64` rejects if any high 24 bytes non-zero; UTF-8 via `decodeUtf8'` returns `EthDecodeErr`.
-
-**Zero-owner → `NotFound`**: ENS-style resolvers return zeroed records for non-existent names. After ABI decode, if `nrOwner == NameOwner (B.replicate 20 0)` return `Left NotFound`.
-
-**Errors.**
-
-```haskell
-data ResolveError = NotFound | EthHttpErr | EthRpcErr { rpcCode :: Int, rpcMessage :: Text }
-                  | EthDecodeErr | TimedOut
-```
-
-All collapse to `ERR AUTH`. `EthRpcErr` carries JSON-RPC `error` object — method-not-found (SNRC not deployed at `snrc_address`) is logged immediately on the first error after a recent success: `logError "NAMES: JSON-RPC error from endpoint — check snrc_address: <code> <message>"`. No automatic retry.
-
-**Cache.** TTL + FIFO eviction. `TVar (OrdPSQ LookupKey Word64 NameRecord, Int)` — priority = monotonic-ns at insert; the `Int` is running byte count. `cacheLookup` is one STM transaction (read, expiry-check, expired-delete-with-byte-decrement). `cacheInsert` is one STM transaction: while `size > cacheMaxEntries` OR `bytes + sizeOf(rec) > cacheMaxBytes`, `minView` to drop oldest, then `insert`. Byte counter prevents `100 000 × 9 KB ≈ 900 MB` worst-case blow-up.
-
-**Request coalescing** (async-exception safe via `E.mask`):
-
-```haskell
-resolveName env bs = do
-  let k = LookupKey bs
-  now <- getMonotonicTimeNSec
-  atomically (cacheLookup env k now) >>= \case
-    Just rec -> incStat (rslvCacheHits ...) $> Right rec
-    Nothing  -> do
-      incStat (rslvCacheMiss ...)
-      ticket <- atomically $ TM.lookup k (inflight env) >>= \case
-        Just mv -> pure (Waiter mv)
-        Nothing -> newEmptyTMVar >>= \mv -> TM.insert k mv (inflight env) $> Leader mv
-      case ticket of
-        Waiter mv -> atomically (readTMVar mv)
-        Leader mv -> E.mask $ \restore -> do
-          r <- restore (fetchOnceTimed env bs)
-                 `E.catch` \(e :: E.SomeException) -> pure (Left (mapEthErr e))
-          atomically $ putTMVar mv r >> TM.delete k (inflight env)
-          case r of Right rec -> atomically (cacheInsert env k now rec); Left _ -> pure ()
-          pure r
-
-fetchOnceTimed env bs =
-  System.Timeout.timeout (rpcTimeoutMs (config env) * 1000) (fetchOnce env bs) >>= \case
-    Just r  -> pure r
-    Nothing -> pure (Left TimedOut)
-```
-
-`E.mask` ensures `putTMVar + TM.delete` runs even on async exception; `fetchOnceTimed` runs under `restore` so it remains interruptible. Waiters always see a value; the in-flight TMap entry is always removed.
-
-`fetchOnce`, `mapEthErr`, `scrubUrl`, `cacheLookup`, `cacheInsert` are internal to `Resolver.hs`. `getMonotonicTimeNSec` from `GHC.Clock` — first monotonic-clock use in the codebase; clock-jump safe.
-
-**STM contention.** Cache hits are read-only `readTVar` — STM scales. Cache writes under sustained miss traffic can retry; `CacheSpec` asserts < 5% retry at 4 readers + 1 writer @ 1k RPS. If observed higher, swap `TVar` for `IORef` + `atomicModifyIORef'`.
-
-**Multicoin and text records** are not in `NameRecord`. If Part 1 contract returns them from `getRecord`, extend `NameRecord` and the wire-size budget. **Confirm with Part 1 author before implementing `Eth/SNRC.hs`.**
-
-## Configuration
-
-`ServerConfig` (`Env/STM.hs:142`) gains one field `namesConfig :: Maybe NamesConfig`. `Env` (`Env/STM.hs:261`) gains `namesEnv :: Maybe NamesEnv`. `newEnv` constructs it after `proxyAgent` (line 605) with the co-location guard.
-
-```haskell
-data NamesConfig = NamesConfig
-  { ethereumEndpoint     :: Text          -- http(s), no userinfo, explicit port required
-  , snrcAddress          :: NameOwner     -- 20 bytes
-  , rpcAuth              :: Maybe RpcAuth -- required when https & non-loopback host
-  , cacheSeconds         :: Int           -- 300
-  , cacheMaxEntries      :: Int           -- 100000
-  , cacheMaxBytes        :: Int           -- 67108864 (64 MB)
-  , rpcTimeoutMs         :: Int           -- 3000
-  , rpcMaxResponseBytes  :: Int           -- 262144 (256 KB)
-  , rpcMaxConcurrency    :: Int           -- 8
-  }
-
-data RpcAuth = AuthBearer Text | AuthBasic Text Text
-```
-
-INI parsing in `Server/Main.hs`:
-
-- `validateUrl` (using new `network-uri` dep): accepts only http(s), non-empty host, **explicit port** (rejects `http://localhost` defaulting to 80 while Reth is on 8545), no userinfo, no query/fragment. Rejects `https://...` without `rpc_auth` when host is non-loopback. On rejection: `logError` + `exitFailure`.
-- `parseEthAddr`: accepts `0x[0-9a-fA-F]{40}` and the same without `0x`. Mixed-case → verify EIP-55 checksum and reject mismatch (catches typos).
-- `parseRpcAuth`: reads optional `rpc_auth` key; format `bearer <token>` or `basic <user>:<pass>`.
-- `scrubUrl`: strips userinfo from all log lines mentioning the endpoint, including inside `mapEthErr`.
-- Transition-aware error logging: log immediately on first error after a recent success, then at most hourly while persisting + summary at every stats reset.
-
-Default INI template (`Server/Main/Init.hs`, after `[PROXY]`):
-
-```
-[NAMES]
-# Public-namespace resolution (SNRC on Ethereum).
-# Requires an Ethereum JSON-RPC endpoint (Reth+Nimbus). See deployment guide.
-# Cannot be combined with [PROXY] enable: on by default — see allow_dangerous_colocation.
-# Restart required to change settings.
-enable: off
-# Same-host:
-# ethereum_endpoint: http://127.0.0.1:8545
-# Central Reth via Caddy:
-# ethereum_endpoint: https://eth.simplex.chat:443
-# rpc_auth: basic <username>:<password>
-# snrc_address: 0x0000000000000000000000000000000000000000
-# cache_seconds: 300
-# cache_max_entries: 100000
-# cache_max_bytes: 67108864
-# rpc_timeout_ms: 3000
-# rpc_max_response_bytes: 262144
-# rpc_max_concurrency: 8
-# allow_dangerous_colocation: off
-```
-
-Upgrade from a pre-v6.6 INI: missing `[NAMES]` section → disabled. No operator action required.
-
-## Operator deployment
-
-Two supported topologies. smp-server is agnostic — only `ethereum_endpoint` changes.
-
-**Topology A (same-host)**: smp-server, Caddy (optional), Reth, Nimbus all on one box. `ethereum_endpoint: http://127.0.0.1:8545`.
-
-**Topology B (central Reth, N smp-server hosts — recommended for fleets)**: one operator runs one eth host with Reth+Nimbus behind Caddy on public HTTPS. Each smp-server has its own credential.
-
-```mermaid
-flowchart LR
-  subgraph eth-host
-    Caddy["Caddy<br/>(public :443, basic auth)"]
-    Reth["Reth<br/>(127.0.0.1:8545)"]
-    Nimbus["Nimbus"]
-    Caddy --> Reth
-    Nimbus -- Engine API (jwt.hex) --> Reth
-  end
-  subgraph smp-host-1
-    S1["smp-server #1"]
-  end
-  subgraph smp-host-N
-    SN["smp-server #N"]
-  end
-  S1 -- HTTPS + Authorization --> Caddy
-  SN -- HTTPS + Authorization --> Caddy
-  Reth <-- Ethereum p2p --> internet
-  Nimbus <-- beacon sync --> internet
-```
-
-Sharing one Reth across **multiple operators** is **not** supported — collapses the RFC's two-server resolution privacy.
-
-**Reth + Nimbus**: Reth (execution layer) holds Ethereum state on ~260 GB pruned NVMe; Nimbus (consensus light client) follows beacon-chain headers. Paired via Engine API on `127.0.0.1:8551` with a shared `jwt.hex`. Recommended Reth flags:
-
-```bash
-reth node \
-  --http.addr 127.0.0.1 \
-  --http.api eth \                     # only eth namespace
-  --rpc.gascap 50000000 \              # cap gas per eth_call
-  --rpc.max-response-size 5242880 \    # 5 MB
-  --http.corsdomain none \
-  --authrpc.jwtsecret /opt/eth/jwt.hex \
-  --authrpc.addr 127.0.0.1 --authrpc.port 8551
-```
-
-**Caddy + Let's Encrypt + Basic auth** (Topology B):
-
-```caddy
-eth.simplex.chat {
-  basicauth {
-    smp-server-1 $2a$14$<bcrypt-hash-1>
-    smp-server-2 $2a$14$<bcrypt-hash-2>
-  }
-  log { format filter { wrap json; fields { request>headers>Authorization delete } } }
-  reverse_proxy 127.0.0.1:8545
-}
-```
-
-Caddy auto-fetches Let's Encrypt cert. Each smp-server has its own credential; revoking one = delete the line. `Authorization` stripped from access logs. Port 80 needed for the ACME HTTP-01 challenge (use TLS-ALPN-01 or DNS-01 to drop it). The threat being defended against is DoS (SNRC state is public); mTLS would be overkill. WireGuard/Tailscale are alternative network-layer approaches — both compatible with the plan.
-
-**Capacity.** One Reth+Nimbus box handles a realistic operator fleet by 10–1000× margin. Per-smp-server peak RSLV ≈ 1700 RPS (pessimistic); cache hit rate ≥ 95% → ~85 RPS cache miss per smp-server; 10 smp-servers → ~850 RPS aggregate cache miss reaching Reth; Reth `eth_call` throughput on warm NVMe ≈ 1k–10k RPS. Sizing: 8 vCPU, 32 GB RAM, 1 TB NVMe is comfortable. Scale-out path: more Reth+Nimbus pairs, smp-servers round-robin or shard.
-
-## Implementation
-
-**Order**:
-
-1. Protocol: party/SParty/PartyI, RSLV+tag, NAME+tag, NameRecord + helpers, version constants in `Transport.hs`.
-2. `verifyTransmission`/`verifyLoadedQueue`/`verifyQueueTransmission` `forwarded :: Bool` flag + `vc SResolver` clauses.
-3. Forwarded whitelist + `processCommand` branch + `incStat` move to `Stats.hs`.
-4. Env plumbing: `Server/Env/STM.hs`, `Server/Main.hs` INI parse, `Server/Main/Init.hs` template.
-5. Resolver subtree: `Eth/SNRC.hs` → `Eth/RPC.hs` → `Resolver.hs`.
-6. `NameResolverStats` sub-record + CSV log + Prometheus `names =` block.
-7. Replace stub in (3) with real `resolveName`.
-8. Tests.
-9. `protocol/simplex-messaging.md`: header version line 1 (`19 → 20`), sentence at line 86, version-history list (lines 93–105) v20 entry, TOC (lines 25–68) "Resolver commands" subsection, new section with ABNF + byte layout + error semantics, "Router security requirements" paragraph about names-role outbound HTTP, cross-ref `Transport.hs:226`.
-10. `CHANGELOG.md`: v6.6 entry.
-
-**Cabal** (`simplexmq.cabal`): bump `version: 6.6.0.0`. Add to `if !flag(client_library)` block: `http-client >=0.7 && <0.8`, `http-client-tls >=0.3 && <0.4`, `network-uri >=2.6 && <2.7`, `psqueues >=0.2.7 && <0.3`. Expose 4 new `Server.Names.*` modules in the same block. `crypton` already provides `Keccak_256`.
-
-**Files changed**:
-
-| File | Change |
-|---|---|
-| `Protocol.hs` | Resolver party + RSLV/NAME tags + version guards; `NameRecord` + newtypes + smart ctors; `nameRecBytes`/`parseNameRec`/`smpListPUpTo` helpers (no Encoding NameRecord instance); `LookupKey` parser-side cap |
-| `Transport.hs` | `namesSMPVersion = 20`; bump current/proxied SMP versions |
-| `Server.hs` | Thread `forwarded :: Bool`; `vc SResolver` clauses; whitelist (2132); Resolver branch in `processCommand` (1481); `closeServer` calls `closeNamesEnv`; CSV log (579–618); **remove** local `incStat` |
-| `Server/Env/STM.hs` | `namesConfig` field; `namesEnv` field; `newEnv` constructs `NamesEnv` with co-location guard |
-| `Server/Main.hs` | `[NAMES]` parse: `validateUrl`/`parseEthAddr`/`parseRpcAuth`; `scrubUrl` in logs |
-| `Server/Main/Init.hs` | `[NAMES]` block in default INI |
-| `Server/Stats.hs` | `incStat` moved here + exported; `NameResolverStats` sub-record + helpers; `rslvStats` field |
-| `Server/Prometheus.hs` | `names =` metric block |
-| `Server/Names.hs` (new) | Façade re-exports |
-| `Server/Names/Resolver.hs` (new) | All resolver types + cache + coalescing + `fetchOnceTimed` + `newNamesEnv[With]` + `closeNamesEnv` |
-| `Server/Names/Eth/RPC.hs` (new) | `EthRpcEnv`, `ethCallReal` with bounded body + concurrency semaphore + `Authorization` header |
-| `Server/Names/Eth/SNRC.hs` (new) | `EthAddress`, Keccak namehash, bounded ABI (8 invariants), `getRecord` with zero-owner detection |
-| `simplexmq.cabal` | Bump `6.6.0.0`; 4 new deps + 4 new modules in `if !flag(client_library)` block |
-| `protocol/simplex-messaging.md` | Header version, version-history v20 entry, new "Resolver commands" section |
-| `CHANGELOG.md` | v6.6 entry |
-
-## Testing
-
-`tests/SMPNamesTests/` registered in `tests/Test.hs:112–151`. Build only when `client_library = False`.
-
-1. **ProtocolEncodingSpec** — `nameRecBytes` ↔ `parseNameRec` round-trip; oversized fields rejected at parse; combined-list cap 8 enforced; negative `nrExpiry` rejected; canonical encoding byte-stable.
-2. **MaxSizeSpec** — max `NameRecord` encodes ≤ ~9 KB; `encodeTransmission v ≤ paddedProxiedTLength - 2`; `cbEncrypt` succeeds.
-3. **CommandTagSpec** — `"RSLV"`/`"NAME"` parse; v < 20 sessions reject `RSLV_` at parameter parser.
-4. **ForwardedGateSpec** — direct RSLV → `CMD PROHIBITED`; forwarded RSLV reaches handler.
-5. **ForwardedRslvSpec** — RSLV wrapped in PFWD reaches the handler end-to-end. **Test infra cost**: first protocol-level PFWD test; budget for `runProxiedSmpCommand` helper performing `PRXY`/`PKEY`/`PFWD` manually.
-6. **CacheSpec** — hit avoids RPC; TTL expiry forces re-fetch; bytes cap evicts before entries cap on large records; concurrent same-key callers issue one RPC; leader exception → all waiters get `Left _`, TMap entry removed; leader async-cancel → cleanup STM still runs.
-7. **AbiSpec** — encode/decode against pinned fixtures (`tests/fixtures/snrc/`); QuickCheck fuzz on random buffers ≤ `rpcMaxResponseBytes` must never crash.
-8. **NamehashSpec** — Keccak-256 reference vectors; assert Keccak ≠ SHA3-256.
-9. **MockRpcSpec** — fake HTTP server; missing → `EthHttpErr`; slow → `TimedOut`; multi-GB body truncated → `EthDecodeErr`. `rpcAuth = AuthBasic` sends correct header.
-10. **Uint256OverflowSpec** — `expiry > Int64.maxBound` → `EthDecodeErr`.
-11. **ZeroOwnerSpec** — `owner = 0x000...000` → `NotFound`.
-12. **StartupGuardSpec** — `allowSMPProxy + names.enable` aborts; `allow_dangerous_colocation = on` starts with warning.
-13. **UrlValidationSpec** — userinfo/scheme/host/port edge cases; rejects `https://` without `rpc_auth` for non-loopback.
-14. **EipChecksumSpec** — `parseEthAddr` accepts lower/upper; verifies mixed-case checksum; rejects typos.
-15. **AbiBoundsSpec** — each of 8 ABI invariants triggers `EthDecodeErr` without crash/allocation blow-up.
-
-Integration against real Reth+Nimbus mainnet deferred to ops.
-
-## Threat model, scope, coordination
-
-| Actor | Can | Cannot |
-|---|---|---|
-| Name server | See lookup-key bytes; see query timing; see Eth endpoint URL (operator-self) | See client IP/session; correlate clients across queries |
-| Compromised Eth endpoint | Poison this server's cache for one TTL window; see every lookup key the server queries | Bypass two-server agreement (client-side, out of scope) |
-| Adversarial client (high-rate unique keys) | Cache-thrash DoS; fill `Manager` connection pool up to `managerConnCount = 8` | Bypass `rpcMaxResponseBytes` or `fetchOnceTimed` |
-| Adversarial proxy (slow inner RSLVs) | Block other forwarded commands on that proxy connection up to `rpcTimeoutMs` per miss | Affect other proxy connections |
-| Operator with footgun config (https no auth, public Eth RPC) | (rejected at startup, or operator-acknowledged data leak) | — |
-
-Mitigations: caching + coalescing + `rpcTimeoutMs` + `rpcMaxResponseBytes` + `rpcMaxConcurrency`; co-location refused at startup; URL validation; Caddy + auth in front of Reth; Reth's own gas/size caps. Timing side-channels (cache-hit vs miss latency) not mitigated — flagged for post-MVP. State proofs deferred to post-MVP per RFC.
-
-**Cross-repo coordination.** The `simplex-chat` `ep/namespace` branch currently contains only the RFC commit — no agent-side wire-format code yet. This plan's wire format is validated only by simplexmq's own tests until a matching agent PR lands (structurally weak — encoder/decoder bugs are mutually consistent with themselves). Coordinate with the agent-side implementer **before merging** on: exact `NameRecord` field order and types; `LookupKey` namespace-prefix convention; error-code semantics; Part 1 SNRC contract `getRecord` ABI surface.

protocol/agent-protocol.md

@@ -85,7 +85,7 @@ SMP agent protocol has 2 main parts:
 
 ![Duplex connection procedure](./diagrams/duplex-messaging/duplex-creating.svg)
 
-The procedure of establishing a duplex connection is explained on the example of Alice (the initiating party) and Bob (the joining party) creating a bi-directional connection consisting of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP routers (which could be the same router). It is shown on the diagram above and has these steps:
+The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection consisting of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP routers (which could be the same router). It is shown on the diagram above and has these steps:
 
 1. Alice requests the new connection from the SMP agent A using agent `createConnection` api function.
 2. Agent A creates an SMP queue on the router (using [SMP protocol](./simplex-messaging.md) `NEW` command) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
@@ -208,7 +208,7 @@ This syntax of decrypted SMP client message body is defined by `decryptedAgentMe
 Decrypted SMP message client body can be one of 4 types:
 - `agentConnInfo` - used by the initiating party when confirming reply queue - sent in `agentConfirmation` envelope.
 - `agentConnInfoReply` - used by accepting party, includes reply queue(s) in the initial confirmation - sent in `agentConfirmation` envelope.
-- `agentRatchetInfo` - used to pass additional information when renegotiating double ratchet encryption - sent in `agentRatchetKey` envelope. A key sent in reply to another key includes the hash of that key; agents do not reply to such keys.
+- `agentRatchetInfo` - used to pass additional information when renegotiating double ratchet encryption - sent in `agentRatchetKey` envelope.
 - `agentMessage` - all other agent messages.
 
 `agentMessage` contains these parts:
@@ -233,8 +233,7 @@ connInfo = *OCTET
 agentConnInfoReply = %s"D" smpQueues connInfo
 smpQueues = length 1*newQueueInfo ; NonEmpty list of reply queues
 agentRatchetInfo = %s"R" ratchetInfo
-ratchetInfo = [answeredKeyHash *OCTET] ; bytes after answeredKeyHash are ignored
-answeredKeyHash = %s"0" / (%s"1" shortString) ; "0" in a key that starts renegotiation, otherwise SHA-256 of the two raw public keys of the answered key
+ratchetInfo = *OCTET
 
 agentMessage = %s"M" agentMsgHeader aMessage
 agentMsgHeader = agentMsgId prevMsgHash

protocol/crypto-registry.md

@@ -1,189 +0,0 @@
-Revision 1, 2026-10-01
-
-# SimpleX Network: cryptographic primitive registry
-
-The cryptographic primitives, constructions and domain-separation strings used by this repository, and which of them new code may use. For the threat model see [Security](./security.md).
-
-Module aliases follow the codebase: `C` = [Simplex.Messaging.Crypto](../src/Simplex/Messaging/Crypto.hs), `LC` = [Crypto.Lazy](../src/Simplex/Messaging/Crypto/Lazy.hs), `CR` = [Crypto.Ratchet](../src/Simplex/Messaging/Crypto/Ratchet.hs), `SL` = [Crypto.ShortLink](../src/Simplex/Messaging/Crypto/ShortLink.hs), `KEM` = [Crypto.SNTRUP761](../src/Simplex/Messaging/Crypto/SNTRUP761.hs) and [its bindings](../src/Simplex/Messaging/Crypto/SNTRUP761/Bindings.hs), `BBS` = [Crypto.BBS](../src/Simplex/Messaging/Crypto/BBS.hs). Lengths are in bytes.
-
-## Table of contents
-
-- [Policy](#policy)
-- [Defaults for new code](#defaults-for-new-code)
-- [Primitives](#primitives)
-- [SimpleX box constructions](#simplex-box-constructions)
-- [Domain separation and KDF inputs](#domain-separation-and-kdf-inputs)
-- [Nonces and IVs](#nonces-and-ivs)
-- [Implementation sources](#implementation-sources)
-- [xftp-web (TypeScript)](#xftp-web-typescript)
-- [Tests](#tests)
-
-## Policy
-
-- New code uses the default primitive for its purpose from [Defaults for new code](#defaults-for-new-code).
-- Using any other primitive, or a primitive marked *restricted* outside its listed purpose, requires a written justification in the pull request and review by a maintainer responsible for cryptography.
-- *Legacy-only* primitives stay only for the existing wire formats and stored data listed below. Do not add call sites.
-- Every new HKDF info string or hash prefix must be unique, start with `SimpleX`, and be added to [Domain separation and KDF inputs](#domain-separation-and-kdf-inputs).
-- Changing a primitive, key length, KDF input, info string or nonce construction changes the wire format. It requires a protocol version and an update of this registry and of the affected `protocol/` specification.
-
-Status values used below:
-
-| Status | Meaning |
-|---|---|
-| Default | Approved and preferred for new code for this purpose |
-| Approved | Approved for new code for this purpose when the default does not fit |
-| Restricted | Approved only for the listed purpose (external interoperability or a single protocol) |
-| Legacy-only | Kept for existing wire formats or stored data; no new call sites |
-
-## Defaults for new code
-
-| Purpose | Default | Functions |
-|---|---|---|
-| Signature | Ed25519 | `C.sign'`, `C.verify'` with `C.SEd25519` |
-| Key agreement | X25519 | `C.generateKeyPair`, `C.dh'` |
-| Post-quantum KEM | sntrup761 | `KEM.sntrup761Keypair`, `sntrup761Enc`, `sntrup761Dec` |
-| Hybrid DH + KEM secret | HKDF-SHA512 over `dh \|\| kemSecret` with a new info string, as in `CR.rootKdf` | `C.hkdf` |
-| Authenticated encryption | XSalsa20-Poly1305 [SimpleX secretbox](#simplex-box-constructions) | `C.sbEncrypt`/`C.sbDecrypt`, `LC.sbEncryptTailTag` for streams |
-| Authenticated encryption with a DH secret | XSalsa20-Poly1305 [SimpleX crypto_box](#simplex-box-constructions) | `C.cbEncrypt`/`C.cbDecrypt` |
-| Forward-secret symmetric session | HKDF-SHA512 key chain | `C.sbcInit`, `C.sbcHkdf` |
-| KDF | HKDF-SHA512 | `C.hkdf` |
-| Hash commitment, derived identifier | SHA3-256 | `C.sha3_256` |
-| Certificate fingerprint | SHA-256 X.509 fingerprint | `C.signedFingerprint`, `C.certificateFingerprint` |
-| Randomness | ChaChaDRG seeded from system entropy | `C.newRandom`, `C.randomBytes` and the `random*` helpers |
-
-## Primitives
-
-| Primitive | Status | Purpose and protocol | Implementation | Lengths |
-|---|---|---|---|---|
-| Ed25519 | Default | SMP/NTF/XFTP queue and file keys (agent default `rcvAuthAlg`, `sndAuthAlg` in [Agent/Env/SQLite.hs](../src/Simplex/Messaging/Agent/Env/SQLite.hs)), short-link signatures and owner auth (`SL.encodeSign`, `SL.newOwnerAuth`), XRCP identity and session keys ([RemoteControl/Invitation.hs](../src/Simplex/RemoteControl/Invitation.hs)), agent service request signatures, client/service TLS certificates ([Transport/Credentials.hs](../src/Simplex/Messaging/Transport/Credentials.hs)) | crypton `Crypto.PubKey.Ed25519` via `C.sign'`/`C.verify'` | pub 32, priv 32, sig 64; X.509 SPKI 44 |
-| Ed448 | Approved | Server CA and online certificates (default `signAlgorithm = ED448` in [Server/CLI.hs](../src/Simplex/Messaging/Server/CLI.hs)); accepted for SMP command signatures and TLS | crypton `Crypto.PubKey.Ed448`; server key generation by the `openssl genpkey` CLI | pub 57, priv 57, sig 114; SPKI 69 |
-| X25519 | Default | Per-queue e2e and server-to-recipient `crypto_box` keys, SMP session DH, command authenticator (SMP v7+), proxy (PRXY/PFWD), notification tokens, XFTP chunk transport, XRCP hello and announcements | crypton `Crypto.PubKey.Curve25519` via `C.dh'` | pub 32, priv 32, secret 32; SPKI 44 |
-| X448 | Restricted | E2E double ratchet only: X3DH and DH ratchet (`CR.RatchetX448`, E2E v3) | crypton `Crypto.PubKey.Curve448` | pub 56, priv 56, secret 56 |
-| sntrup761 | Default | PQ KEM in double ratchet (E2E v3, agent v5+) and XRCP v1 hello | Vendored C [cbits/sntrup761.c](../cbits/sntrup761.c) via FFI; randomness from ChaChaDRG through `haskell_rng_func` ([Bindings/RNG.hs](../src/Simplex/Messaging/Crypto/SNTRUP761/Bindings/RNG.hs)) | pk 1158, sk 1763, ct 1039, shared 32 |
-| XSalsa20-Poly1305, SimpleX crypto_box | Default | SMP message bodies (server to recipient), per-queue e2e envelope, confirmations, proxy forwarding, notifications, XRCP hello, XFTP chunk transport | crypton `Crypto.Cipher.XSalsa`, `Crypto.MAC.Poly1305`; `C.cryptoBox`, `LC.cbInit` | key 32 (DH secret), nonce 24, tag 16 |
-| XSalsa20-Poly1305, SimpleX secretbox | Default | SMP transport block encryption (SMP v11+), short-link data (SMP v15+), XFTP file encryption, local file encryption ([Crypto/File.hs](../src/Simplex/Messaging/Crypto/File.hs)), XRCP session | same; `C.sbEncrypt`, `LC.sbEncryptTailTag`, `LC.sbInit` | key 32, nonce 24, tag 16 |
-| crypto_box authenticator | Approved | Deniable sender command authorization with X25519 queue keys (SMP v7+); agent currently creates Ed25519 keys | `C.cbAuthenticate`, `C.cbVerify`: `crypto_box(sha512(msg))` | 80 = 64 + 16 |
-| AES-256-GCM, 16-byte IV | Legacy-only | Double ratchet header and body (E2E v3, [pqdr.md](./pqdr.md)) | crypton `Crypto.Cipher.AES`, `Crypto.Cipher.Types`; `C.encryptAEAD`, `C.decryptAEAD`, `C.initAEAD`; J0 = GHASH(IV) as NIST SP 800-38D defines for non-96-bit IVs | key 32, IV 16, tag 16; padded header 88 (PQ off) or 2310 (PQ on), `CR.paddedHeaderLen` |
-| AES-256-GCM, 12-byte IV | Restricted | WebRTC frame encryption in simplex-chat (`Simplex.Chat.Mobile.WebRTC`), for WebCrypto interoperability | `C.encryptAESNoPad`, `C.decryptAESNoPad`, `C.initAEADGCM`, `C.GCMIV` | key 32, IV 12, tag 16 |
-| HKDF-SHA512 | Default | All KDFs listed in [Domain separation](#domain-separation-and-kdf-inputs) | crypton `Crypto.KDF.HKDF` with `SHA512`; `C.hkdf` (extract + expand) | output per use, at most 255 * 64 |
-| SHA-256 | Default for X.509 fingerprints, otherwise Restricted | X.509 fingerprints (`C.KeyHash`, server identity, XRCP CA, service certificate hash, SMP v16+), XFTP chunk digests, agent message hashes, `requestCode`, ratchet key dedup hash, security code `codeAD = sha256(rcAD)` | crypton; `C.sha256Hash`, `LC.sha256Hash`, `getFingerprint ... HashSHA256` | 32 |
-| SHA-512 | Restricted | XFTP file digests and file description hash, input of the crypto_box authenticator | crypton; `C.sha512Hash`, `LC.sha512Hash` | 64 |
-| SHA-512 (inside sntrup761) | Restricted | sntrup761 internal hash | [cbits/sha512.c](../cbits/sha512.c) calls OpenSSL `SHA512()` from the system `libcrypto` (`extra-libraries: crypto`) | 64 |
-| SHA3-256 | Default | Short-link key `sha3_256(fixedData)` (SMP v15+), XRCP hybrid secret, service request binding (agent v8) | crypton; `C.sha3_256`, `KEM.kemHybridSecret` | 32 |
-| SHA3-384 | Restricted | Client-supplied sender ID: first 24 bytes of `sha3_384(corrId)`, computed by the agent (`prepareConnectionLink'`, `newRcvConnSrv`) and the server (`createQueue`) | crypton; `C.sha3_384` | 48, truncated to 24 |
-| Keccak-256 | Restricted | Label hash for SMP name queries (`NameQuery NQHash`), must match the on-chain registry | crypton `Keccak_256`; `labelHash` in [SimplexName.hs](../src/Simplex/Messaging/SimplexName.hs) | 32 |
-| MD5 | Legacy-only, non-security | XOR-aggregated queue ID hash `IdsHash` for service subscription reconciliation (SUBS, NSUBS, SOKS, ENDS) | crypton (`C.md5Hash`, `Protocol.queueIdHash`); SQLite UDF `simplex_xor_md5_combine` ([Agent/Store/SQLite.hs](../src/Simplex/Messaging/Agent/Store/SQLite.hs)); PostgreSQL pgcrypto `digest(..., 'md5')` in server and NTF schemas | 16 |
-| ChaChaDRG | Default | Keys, nonces, correlation IDs, random IDs, sntrup761 randomness | crypton `Crypto.Random`; `C.newRandom` seeds with `drgNew` from system entropy | n/a |
-| BBS+ over BLS12-381, SHA-256 suite | Restricted | Badge entitlement credentials and proofs ([Crypto/Entitlement.hs](../src/Simplex/Messaging/Crypto/Entitlement.hs)), XFTP storage-time extension | Vendored submodules libbbs and blst (C and assembly) via FFI; randomness from libbbs `getentropy`, `SecRandomCopyBytes` with the `commoncrypto` flag, `BCryptGenRandom` on Windows ([cbits/getentropy_win.c](../cbits/getentropy_win.c)) | sk 32, pk 96, sig 80, proof 272 + 32 per undisclosed message |
-| ECDSA with SHA-256 (ES256) | Restricted | APNS provider JWT ([Push/APNS.hs](../src/Simplex/Messaging/Notifications/Server/Push/APNS.hs)) | crypton `Crypto.PubKey.ECC.ECDSA.sign`; key read by cryptostore `Crypto.Store.PKCS8`; curve taken from the key file | signature DER `SEQUENCE {r, s}`, base64url |
-| TLS 1.3 / 1.2 | Restricted | SMP, XFTP, NTF, XRCP transport: `TLS_CHACHA20_POLY1305_SHA256` (1.3), `ECDHE_ECDSA_CHACHA20POLY1305_SHA256` (1.2), groups X448 and X25519, Ed448 and Ed25519 signatures (`defaultSupportedParams`) | `tls` 1.9 | n/a |
-| TLS for browsers | Restricted | XFTP server with HTTPS credentials (`defaultSupportedParamsHTTPS`: `ciphersuite_strong`, FFDHE, P-521, ECDSA and RSA signatures); SMP server HTTPS requires RSA-4096 (`checkHTTPSCredentials`) | `tls`, `warp-tls` | n/a |
-| X.509 | Restricted | Certificate chains, fingerprints, signed session keys (`C.signX509`, `C.verifyX509`) | `crypton-x509`, `crypton-x509-validation`, `cryptostore` | n/a |
-| SQLCipher | Restricted | Agent SQLite database at rest (`PRAGMA key`); cipher parameters are SQLCipher defaults, not set in this repository | `direct-sqlcipher` (git dependency, [cabal.project](../cabal.project)) | n/a |
-
-## SimpleX box constructions
-
-`C.cryptoBox` and `LC.sbInit_` compute, for a 32-byte key `k` and 24-byte nonce `n`:
-
-```
-k1 = HSalsa20(k, 0^16)
-subkey = HSalsa20(k1, n[0..16])
-stream = Salsa20(subkey, n[16..24])
-polyKey = stream[0..32]
-ct = msg XOR stream[32..]
-tag = Poly1305(polyKey, ct)
-```
-
-| Construction | Key `k` | Equivalent libsodium call | Layout |
-|---|---|---|---|
-| SimpleX crypto_box (`C.cbEncrypt`, `LC.cbInit`) | X25519 shared secret | `crypto_box_easy` (`crypto_box_beforenm` is `HSalsa20(dh, 0^16)`) | strict: `tag \|\| ct`; lazy tail-tag: `ct \|\| tag` |
-| SimpleX secretbox (`C.sbEncrypt`, `LC.sbInit`, `KEM.kcb*`) | 32-byte symmetric key | `crypto_secretbox_easy` with key `crypto_core_hsalsa20(0^16, k)`, not with `k` | same |
-
-Padding before encryption: `C.pad` prefixes a 2-byte big-endian length and fills with `#` (message at most 65533 bytes); `LC.pad` prefixes an 8-byte length. `*NoPad` variants skip padding.
-
-## Domain separation and KDF inputs
-
-HKDF is `C.hkdf salt ikm info len` (HKDF-SHA512).
-
-| Info string | Salt | IKM | Output (split) | Function | Use |
-|---|---|---|---|---|---|
-| `"SimpleXX3DH"` | 64 zero bytes | `dh1 \|\| dh2 \|\| dh3 [\|\| kemShared]` | 96: `hk`, `nhk`, root key (32 each) | `CR.pqX3dh` | Ratchet initialization, added in E2E v2; KEM secret from E2E v3 (current minimum) |
-| `"SimpleXVerifyCode"` | 64 zero bytes | same as `SimpleXX3DH` | 32: `rcVCPQ` | `CR.pqX3dh` | Verification code covering all handshake keys, new ratchets |
-| `"SimpleXRootRatchet"` | root key | `dh [\|\| kemShared]` | 96: root key, chain key, next header key | `CR.rootKdf` | DH/PQ ratchet step |
-| `"SimpleXChainRatchet"` | empty | chain key | 96: chain key, message key (32 each), message IV (16), header IV (16) | `CR.chainKdf` | Per-message keys |
-| `"SimpleXSbChainInit"` | SMP session ID | X25519 session secret | 64: two 32-byte chain keys | `C.sbcInit` from `Transport.blockEncryption` | SMP transport block encryption, SMP v11+ |
-| `"SimpleXSbChainInit"` | empty | XRCP hybrid secret (below) | 64: two 32-byte chain keys | `C.sbcInit` in [RemoteControl/Client.hs](../src/Simplex/RemoteControl/Client.hs) | XRCP v1 session |
-| `"SimpleXSbChain"` | empty | chain key | 88: chain key (32), secretbox key (32), nonce (24) | `C.sbcHkdf` | Each SMP block and XRCP message |
-| `"SimpleXContactLink"` | empty | link key (32) | 56: link ID (24), secretbox key (32) | `SL.contactShortLinkKdf` | Contact short links, SMP v15+ |
-| `"SimpleXInvLink"` | empty | link key (32) | 32: secretbox key | `SL.invShortLinkKdf` | Invitation short links, SMP v15+ |
-
-Other derivations:
-
-| Value | Construction | Function | Use |
-|---|---|---|---|
-| Service request binding | `sha3_256("SimpleXService" \|\| rcAD)` | `serviceReqBinding` in [Agent.hs](../src/Simplex/Messaging/Agent.hs) | Agent RPC, agent v8 |
-| BBS header | `"SimpleX badges v1"` | `entitlementBBSHeader` | Badge credentials |
-| XRCP hybrid secret | `sha3_256(x25519Dh \|\| kemShared)` | `KEM.kemHybridSecret` | XRCP v1 |
-| Short-link key | `sha3_256(smpEncode FixedLinkData)` | `SL.encodeSignFixedData`, checked in `SL.decryptLinkData` | SMP v15+ |
-| Sender ID | `take 24 (sha3_384 corrId)` | `prepareConnectionLink'`, `newRcvConnSrv` in Agent.hs; `createQueue` in [Server.hs](../src/Simplex/Messaging/Server.hs) | Client-supplied sender ID with link data, SMP v15+; the server rejects a mismatch to prevent an ID oracle |
-| Ratchet associated data | `pubKeyBytes sk1 \|\| pubKeyBytes rk1` (raw X448, 112) | `CR.pqX3dh` | AD of every ratchet AEAD |
-| Security code | `sha256(rcAD)` | `ratchetVerifyCodes` in [AgentStore.hs](../src/Simplex/Messaging/Agent/Store/AgentStore.hs) | Connection verification |
-| Contact request code | `sha256(smpEncode (k1, k2, kem, sndId))` | `requestCode` in Agent.hs | Contact request binding |
-| Command authenticator | `crypto_box(sha512(authorized))` | `C.cbAuthenticate` | SMP v7+ |
-| Queue IDs hash | `xor` of `md5(queueId)` | `Protocol.queueIdsHash` | Service subscriptions |
-
-`"SimpleXSbChainInit"` and `"SimpleXSbChain"` are shared by SMP block encryption and XRCP. Their outputs are separated only by the IKM (and the salt for `sbcInit`).
-
-## Nonces and IVs
-
-| Use | Construction |
-|---|---|
-| SMP command authenticator, proxied command | `C.cbNonce corrId`, where `corrId` is a random 24-byte nonce (`C.randomCbNonce`) |
-| SMP proxied responses | `C.reverseNonce` of the request nonce |
-| Server-to-recipient message body | `C.cbNonce msgId`; `msgIdBytes = 24` in [Server/Main.hs](../src/Simplex/Messaging/Server/Main.hs) |
-| Per-queue e2e envelope, short-link data, notifications, XRCP hello, XFTP chunk download | Random 24-byte nonce, sent with the ciphertext |
-| XFTP file, local encrypted file | Random key and nonce per file (`C.randomSbKey`, `C.randomCbNonce`) |
-| SMP block, XRCP session messages | Key and nonce from `C.sbcHkdf`, one per block |
-| Ratchet header, body | 16-byte IVs from `CR.chainKdf`; header IV is sent, body IV is not |
-| WebRTC frames | 12-byte `C.GCMIV` supplied by the caller in simplex-chat |
-
-## Implementation sources
-
-| Source | Version or pin | Provides |
-|---|---|---|
-| crypton | 0.34 | Ed25519, Ed448, X25519, X448, XSalsa20, Poly1305, AES-GCM, HKDF, SHA-2, SHA-3, Keccak, MD5, ChaChaDRG, ECDSA |
-| tls, crypton-x509, crypton-x509-validation, cryptostore | 1.9.0, 1.7.6, 1.6.12, 0.3.0.1 | TLS, X.509, PKCS#8 |
-| [cbits/sntrup761.c](../cbits/sntrup761.c) | Copy of draft-josefsson-ntruprime-streamlined-00 ([cbits/README.md](../cbits/README.md)) | sntrup761 |
-| [cbits/sha512.c](../cbits/sha512.c) and system OpenSSL `libcrypto` | system | SHA-512 for sntrup761 |
-| `cbits/libbbs` submodule ([simplex-chat/libbbs](https://github.com/simplex-chat/libbbs)) | `59a0f4bf` | BBS+ (`bbs_sha256_ciphersuite`), SHA-256, SHAKE256 |
-| `cbits/blst` submodule ([supranational/blst](https://github.com/supranational/blst)) | `db3defd0` | BLS12-381 (C and `build/assembly.S`, `-D__BLST_PORTABLE__`) |
-| direct-sqlcipher | git `f814ee68` | SQLCipher |
-| `openssl` CLI | system | Server CA and certificate key generation (`createServerX509_`) |
-
-## xftp-web (TypeScript)
-
-[xftp-web](../xftp-web/) reimplements a subset for the browser XFTP client. It has no AES-GCM, HKDF, SHA-3, MD5, X448 or sntrup761 code.
-
-| Primitive | Implementation | Haskell counterpart |
-|---|---|---|
-| Ed25519 sign, verify, keys | libsodium-wrappers-sumo (`crypto_sign_*`), [src/crypto/keys.ts](../xftp-web/src/crypto/keys.ts) | `C.sign'`, `C.verify'` |
-| Ed448 verify | `@noble/curves/ed448`, keys.ts `verifyEd448` | `C.verify'` |
-| X25519 | libsodium `crypto_scalarmult`, `crypto_box_keypair` | `C.dh'` |
-| SimpleX crypto_box, secretbox, tail-tag streaming | Salsa20 block function written in TypeScript, libsodium `crypto_core_hsalsa20` and `crypto_onetimeauth_*`, [src/crypto/secretbox.ts](../xftp-web/src/crypto/secretbox.ts) | `C.cryptoBox`, `LC.sbInit_` |
-| crypto_box authenticator | [src/protocol/client.ts](../xftp-web/src/protocol/client.ts) `cbAuthenticate`, `cbVerify` | `C.cbAuthenticate`, `C.cbVerify` |
-| SHA-256, SHA-512 | libsodium `crypto_hash_sha256`, `crypto_hash_sha512*`, [src/crypto/digest.ts](../xftp-web/src/crypto/digest.ts) | `C.sha256Hash`, `C.sha512Hash` |
-| Random | WebCrypto `crypto.getRandomValues`; libsodium for key generation | `C.randomBytes` |
-
-## Tests
-
-None of the tests below compares against published reference vectors (NIST, RFC 8032, RFC 7748, RFC 5869, NaCl, the sntrup761 draft or the BBS draft). Interoperability is tested between this repository's own implementations.
-
-| Area | Test module (hspec group) | Kind |
-|---|---|---|
-| Ed25519, Ed448, X25519 crypto_box, secretbox, lazy and tail-tag secretbox, AES-GCM 12-byte IV, X.509 key encoding, X.509 chains, sntrup761, BBS+, entitlements, padding | [tests/CoreTests/CryptoTests.hs](../tests/CoreTests/CryptoTests.hs) | Round-trip; fixed lengths for BBS+; fixed bytes for padding |
-| Double ratchet (X25519 and X448), PQ KEM agreement, AES-GCM 16-byte IV | [tests/AgentTests/DoubleRatchetTests.hs](../tests/AgentTests/DoubleRatchetTests.hs) | Round-trip; decoding of a stored v2 ratchet JSON |
-| Short links (HKDF info strings, SHA3-256 link key) | [tests/AgentTests/ShortLinkTests.hs](../tests/AgentTests/ShortLinkTests.hs) | Round-trip, tamper rejection |
-| File encryption | [tests/CoreTests/CryptoFileTests.hs](../tests/CoreTests/CryptoFileTests.hs) | Round-trip |
-| XRCP session (SHA3-256 hybrid, sb chain) | [tests/RemoteControl.hs](../tests/RemoteControl.hs) | End-to-end |
-| SMP authenticators, block encryption, `IdsHash` (MD5) | [tests/ServerTests.hs](../tests/ServerTests.hs) | End-to-end; with the PostgreSQL queue store this checks Haskell MD5 against pgcrypto |
-| Haskell and xftp-web: SHA-256/512, Ed25519, Ed448 identity proof, X25519, DER keys, crypto_box, secretbox, tail-tag, authenticator, file encryption | [tests/XFTPWebTests.hs](../tests/XFTPWebTests.hs) (`XFTP Web Client`) | Byte-identical cross-language output |

protocol/diagrams/duplex-messaging/queue-rotation-fast.mmd

@@ -5,18 +5,13 @@ sequenceDiagram
   participant S as Server<br>that has A's send queue<br>(B's receive queue)
   participant B as Bob
 
-  A ->> R': NEW: create new queue (SKEY allowed)
-  A ->> S: SEND: QADD (R')
+  A ->> R': NEW: create new queue<br>(allow SKEY)
+  A ->> S: SEND: QADD (R'): send address<br>of the new queue(s)
   S ->> B: MSG: QADD (R')
-  B ->> R: SEND: messages (also scheduled on R')
-  R ->> A: MSG: messages
-  B ->> R': SKEY: authorize B as sender
-  B ->> R': SEND: confirmation (establishes R' secret)
-  R' ->> A: MSG: confirmation (A secures R')
-  B ->> R': SEND: held copies (deduped), then new messages
-  R' ->> A: MSG: messages
-  B ->> R: SEND: remaining tail, then QEND
-  R ->> A: MSG: QEND
-  B ->> R': SEND: QEND
-  R' ->> A: MSG: QEND
-  A ->> R: DEL: delete the current queue
+  B ->> R': SKEY: secure new queue
+  B ->> R': SEND: QTEST
+  R' ->> A: MSG: QTEST
+  A ->> R: DEL: delete the old queue
+  B ->> R': SEND: send messages to the new queue
+  R' ->> A: MSG: receive messages from the new queue
+  

protocol/diagrams/xrcp/session.mmd

@@ -17,11 +17,11 @@ note over CC, HC: TLS handshake
 par
     note over CC: validate client X509 credentials
     CC->>CI: session code from tlsUnique
-    CI-->>CC: user confirmation (optional if known)
+    CI-->>CC: user confirmation
 and
     note over HC: validate server X509 credentials
     HC->>HI: session code from tlsUnique
-    HI-->>HC: user confirmation (optional if known)
+    HI-->>HC: user confirmation
 end
 
 note over CI, HI: 3. Session verification and protocol negotiation

protocol/pqdr.md

@@ -62,16 +62,14 @@ It is possible to reduce size overhead by using only one KEM agreement and makin
 
 ## Double ratchet with encrypted headers augmented with double PQ KEM
 
-Algorithm below assumes that in addition to shared secret from the initial key agreement, there will be an encapsulation key available from the initiating party, that sent its keys in the connection invitation to the joining party (see [agent protocol](./agent-protocol.md)). Following the double ratchet specification, the pseudo-code below names the joining party Alice and the initiating party Bob.
-
-Unlike X3DH prekey bundles, these keys are not reusable keys published for any party to use. The initiating party generates two X448 key pairs and, optionally, a sntrup761 KEM key pair for each connection, sends the public keys in the invitation, and deletes the stored private keys once the ratchet is initialized; only the second X448 private key and the agreed KEM key pair remain in the ratchet state as its initial ratchet keys. The joining party generates its keys when joining and keeps only its KEM key pair, in the ratchet state. The exception is a contact address that publishes ratchet keys in its link data: all requesters use these keys until the address owner rotates them, and the owner keeps the private keys of a few recent generations.
+Algorithm below assumes that in addition to shared secret from the initial key agreement, there will be an encapsulation key available from the party that published its keys (Bob).
 
 ### Initialization
 
 The double ratchet initialization is defined in pseudo-code. This pseudo-code is identical to Signal algorithm specification except for that parts that add post-quantum key agreement.
 
 ```
-// Alice (joining party) received Bob's keys in the invitation and initializes ratchet first
+// Alice obtained Bob's keys and initializes ratchet first
 def RatchetInitAlicePQ2HE(state, SK, bob_dh_public_key, shared_hka, shared_nhkb, bob_pq_kem_encapsulation_key):
     state.DHRs = GENERATE_DH()
     state.DHRr = bob_dh_public_key
@@ -91,7 +89,7 @@ def RatchetInitAlicePQ2HE(state, SK, bob_dh_public_key, shared_hka, shared_nhkb,
     state.HKr = None
     state.NHKr = shared_nhkb
 
-// Bob (initiating party) initializes ratchet second, having received Alice's first message
+// Bob initializes ratchet second, having received Alice's connection request
 def RatchetInitBobPQ2HE(state, SK, bob_dh_key_pair, shared_hka, shared_nhkb, bob_pq_kem_key_pair):
     state.DHRs = bob_dh_key_pair
     state.DHRr = None
@@ -212,8 +210,6 @@ The outer envelope contains the encrypted header (used as associated data for bo
 
 The message body is encrypted with AES-256-GCM using the message key derived from the sending chain key (`KDF_CK`). The associated data for body encryption is the concatenation of the ratchet associated data and the encoded encrypted header.
 
-`KDF_CK(CK)` is HKDF-SHA512 with empty salt, `CK` as input key material and info `"SimpleXChainRatchet"`, producing 96 bytes split into the next chain key (32 bytes), the message key (32 bytes), the message body IV (16 bytes, not transmitted) and `headerIV` (16 bytes). Both IVs are used as 16-byte AES-256-GCM IVs, not the 12-byte IVs recommended by NIST SP 800-38D, so the initial counter block is J0 = GHASH(IV || 0^64 || [128]_64) as defined there for non-96-bit IVs. WebCrypto and other conforming implementations compute it when given the full 16-byte IV; truncating the IV to 12 bytes produces different ciphertext.
-
 ```abnf
 encRatchetMessage = versionedLength encMessageHeader msgAuthTag encMsgBody
 ; encMessageHeader is used as associated data for body decryption: AD = rcAD || encMessageHeader
@@ -278,7 +274,7 @@ As SimpleX Messaging Protocol pads messages to a fixed size, using 16kb transpor
 
 Sharing the initial keys in case of SimpleX Chat it is equivalent to sharing the invitation link. As encapsulation key is large, it may be inconvenient to share it in the link in some contexts, e.g. when QR codes are used.
 
-It is possible to postpone sharing the encapsulation key until the first message from the joining party (confirmation message in SMP protocol). The upside here is that the invitation link size would not increase. The downside is that the user profile shared in this confirmation will not be encrypted with PQ-resistant algorithm.
+It is possible to postpone sharing the encapsulation key until the first message from Alice (confirmation message in SMP protocol), the party sending connection request. The upside here is that the invitation link size would not increase. The downside is that the user profile shared in this confirmation will not be encrypted with PQ-resistant algorithm.
 
 Another consideration is pairwise ratchets in groups. Key generation in sntrup761 is quite slow - on slow devices it can be as slow as 10-20 keys per second, so using this primitive in groups larger than 10-20 members would result in slow performance.
 

protocol/security.md

@@ -15,7 +15,7 @@ This document describes the cryptographic primitives and threat model for the Si
   - [SimpleX Messaging Protocol router that proxies the messages to another SMP router](#simplex-messaging-protocol-router-that-proxies-the-messages-to-another-smp-router)
   - [An attacker who obtained Alice's (decrypted) chat database](#an-attacker-who-obtained-alices-decrypted-chat-database)
   - [A user's contact](#a-users-contact)
-  - [An attacker who observes the initiating party showing an introduction message to the joining party](#an-attacker-who-observes-the-initiating-party-showing-an-introduction-message-to-the-joining-party)
+  - [An attacker who observes Alice showing an introduction message to Bob](#an-attacker-who-observes-alice-showing-an-introduction-message-to-bob)
   - [An attacker with Internet access](#an-attacker-with-internet-access)
 
 
@@ -37,8 +37,6 @@ This document describes the cryptographic primitives and threat model for the Si
   - AES-GCM AEAD cipher,
   - SHA512-based HKDF for key derivation.
 
-All primitives in use, their lengths, domain-separation strings and the policy for new code are listed in the [cryptographic primitive registry](./crypto-registry.md).
-
 
 ## Threat Model
 
@@ -170,7 +168,7 @@ All primitives in use, their lengths, domain-separation strings and the policy f
 
 *cannot:*
 
-- impersonate a sender and send messages to the user whose database was stolen, unless the database contains a new rotation queue that is not secured yet. Doing so requires also compromising the router (to place the message in the queue, that is possible until the Double-Ratchet advances forward) or the user's device at a subsequent time (to place the message in the database).
+- impersonate a sender and send messages to the user whose database was stolen. Doing so requires also compromising the router (to place the message in the queue, that is possible until the Double-Ratchet advances forward) or the user's device at a subsequent time (to place the message in the database).
 
 - undetectably communicate at the same time as Alice with her contacts. Doing so would result in the contact getting different messages with repeated IDs.
 
@@ -192,15 +190,15 @@ All primitives in use, their lengths, domain-separation strings and the policy f
 
 - cannot collaborate with another of the user's contacts to confirm they are communicating with the same user.
 
-### An attacker who observes the initiating party showing an introduction message to the joining party
+### An attacker who observes Alice showing an introduction message to Bob
 
 *can:*
 
-- Impersonate the joining party to the initiating party.
+- Impersonate Bob to Alice.
 
 *cannot:*
 
-- Impersonate the initiating party to the joining party.
+- Impersonate Alice to Bob.
 
 ### An attacker with Internet access
 

protocol/simplex-messaging.md

@@ -1,4 +1,4 @@
-Version 21, 2026-07-05
+Version 19, 2025-01-24
 
 # Simplex Messaging Protocol (SMP)
 
@@ -67,9 +67,6 @@ Version 21, 2026-07-05
     - [Queue deleted notification](#queue-deleted-notification)
     - [Error responses](#error-responses)
     - [OK response](#ok-response)
-  - [Resolver commands](#resolver-commands)
-    - [Resolve name command](#resolve-name-command)
-    - [Name record response](#name-record-response)
 - [Transport connection with the SMP router](#transport-connection-with-the-SMP-router)
   - [General transport protocol considerations](#general-transport-protocol-considerations)
   - [TLS transport encryption](#tls-transport-encryption)
@@ -86,7 +83,7 @@ It's designed with the focus on communication security and integrity, under the
 
 It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system.
 
-This document describes SMP protocol version 23. Versions 1-5 are discontinued. The version history:
+This document describes SMP protocol version 19. Versions 1-5 are discontinued. The version history:
 
 - v1: binary protocol encoding
 - v2: message flags (used to control notifications)
@@ -106,10 +103,6 @@ This document describes SMP protocol version 23. Versions 1-5 are discontinued.
 - v17: create notification credentials with NEW command
 - v18: support client notices in BLOCKED error
 - v19: service subscriptions to messages (SUBS, NSUBS, SOKS, ENDS, ALLS commands)
-- v20: public namespaces resolver (RSLV command, RNAME response) — direct or forwarded via PFWD
-- v21: server public information in handshake
-- v22: `RNAME` says whether a name can be registered, not only what it resolves to
-- v23: version in nonces of forwarded commands, random nonces for forwarded responses
 
 ## Introduction
 
@@ -431,8 +424,6 @@ Simplex messaging router implementations MUST NOT create, store or send to any o
 
 - Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using simplex messaging routers (the routers cannot compromise forward secrecy of any application layer protocol, such as double ratchet).
 
-Routers with the names role make outbound HTTP calls to a backing resolver service (the reference implementation is `scripts/resolver/snrc-resolve.py`, which in turn makes JSON-RPC calls to an Ethereum endpoint) to read `NameRecord` data; the lookup key reaches that resolver and its upstream RPC endpoint. Operators MUST run both the resolver process and its upstream RPC endpoint themselves (loopback Reth + Nimbus, or a self-hosted central deployment) — sharing them across multiple operators collapses the two-server privacy property because the resolver / RPC operator would see every lookup key across all of them. The names role and the SMP-proxy role MUST NOT be enabled on the same router by default: a client forwarding `RSLV` through a proxy that is also the names router would expose both its connection and the lookup key to one operator, collapsing the two-server privacy property. (Resolution itself runs on a forked thread, so a slow `RSLV` does not serialise other forwarded commands on the session.)
-
 ## Message delivery notifications
 
 Supporting message delivery while the client mobile app is not running requires sending push notifications with the device token. All alternative mechanisms for background message delivery are unreliable, particularly on iOS platform.
@@ -1121,15 +1112,13 @@ When the client receives PKEY response it MUST validate that:
 - the fingerprint of the received certificate matches fingerprint in the router address - it mitigates MITM attack by proxy.
 - the router session key is correctly signed with the received certificate.
 
-`smpVersionRange` in `PKEY` is not signed.
-
 The proxy router may respond with error response in case the destination router is not available or in case it has an earlier version that does not support proxied commands.
 
 #### Send command via proxy
 
 Sender can send `SKEY` and `SEND` commands via proxy after obtaining the session ID with `PRXY` command (see [Request proxied session](#request-proxied-session)).
 
-Transmission sent to proxy router should use session ID as entity ID and use a random correlation ID of 24 bytes as a nonce for crypto_box encryption of transmission to the destination router. When `smpVersion` in `PFWD` is 23 or higher, the first 2 bytes of this nonce are XOR-ed with `smpVersion`. The random ephemeral X25519 key to encrypt transmission should be unique per command, and it should be combined with the key sent by the router in the handshake header to proxy and to the client in `PKEY` command.
+Transmission sent to proxy router should use session ID as entity ID and use a random correlation ID of 24 bytes as a nonce for crypto_box encryption of transmission to the destination router. The random ephemeral X25519 key to encrypt transmission should be unique per command, and it should be combined with the key sent by the router in the handshake header to proxy and to the client in `PKEY` command.
 
 Encrypted transmission should use the received session ID from the connection between proxy router and destination router in the authorized body.
 
@@ -1141,11 +1130,10 @@ commandKey = length x509encoded
 
 The proxy router will forward the encrypted transmission in `RFWD` command (see below).
 
-Having received the `RRES` response from the destination router, proxy router will forward `PRES` response to the client. `PRES` response should use the same correlation ID as `PFWD` command. The destination router will use this correlation ID increased by 1 as a nonce for encryption of the response. When `smpVersion` in `PFWD` is 23 or higher, the destination router uses a random nonce instead, and sends it before the encrypted response.
+Having received the `RRES` response from the destination router, proxy router will forward `PRES` response to the client. `PRES` response should use the same correlation ID as `PFWD` command. The destination router will use this correlation ID increased by 1 as a nonce for encryption of the response.
 
 ```abnf
-proxyResponse = %s"PRES" SP [responseNonce] <encrypted padded(forwardedResponse, 16226)>
-responseNonce = %s"0" / (%s"1" 24*24 OCTET) ; from v23
+proxyResponse = %s"PRES" SP <encrypted padded(forwardedResponse, 16226)>
 forwardedResponse = *OCTET ; client-encrypted SMP response, decrypted by client using per-command DH secret
 ```
 
@@ -1172,7 +1160,7 @@ The shared secret for encrypting transmission bodies between proxy router and de
 
 
 ```abnf
-relayResponse = %s"RRES" SP [responseNonce] <encrypted(responseTransmission)>
+relayResponse = %s"RRES" SP <encrypted(responseTransmission)>
 responseTransmission = fwdCorrId forwardedResponse
   ; fwdCorrId and forwardedResponse defined above in RFWD section
 ```
@@ -1434,199 +1422,6 @@ When the command is successfully executed by the router, it should respond with
 ok = %s"OK"
 ```
 
-### Resolver commands
-
-Resolver commands implement public-namespace name resolution on the names-role
-router. A names router translates an opaque lookup key (such as `alice` or
-`alice.simplex.eth`) into a `NameRecord` carrying the channel and contact links
-the named party publishes.
-
-**Direct or forwarded.** RSLV is an unauthenticated command accepted both
-directly from a transport client and inside a `PFWD` block via the SMP proxy;
-the client chooses. Forwarded delivery preserves the two-server privacy property
-of the resolver design: the names router sees the lookup key but never the
-client IP, session, or identity, while the proxy router sees the client
-connection but cannot read the encrypted lookup key inside the forwarded
-transmission. Direct delivery is simpler but exposes the client's connection to
-the names router, so clients SHOULD prefer the forwarded path when proxying is
-available.
-
-**Backing store.** This protocol does not prescribe where the names router
-reads `NameRecord` from. The reference implementation forwards each RSLV to a
-companion REST resolver process (`scripts/resolver/snrc-resolve.py`) that
-queries the SNRC contract on Ethereum; alternative backings (different chains,
-DHT, etc.) are valid as long as they expose the documented HTTP shape (`GET
-/v2/resolve/<query>` returning a `NameRegistration` on 200 for every
-registration shape, 400 for unknown TLDs, 502 for upstream failures) or
-substitute a different transport returning the same JSON. The resolver API is
-versioned separately from this protocol: `/v1/resolve/<name>` returns a bare
-`NameRecord` and is what relays before v22 call as `/resolve/<name>`.
-
-#### Resolve name command
-
-The `RSLV` command carries the query as text, not JSON. A client sends the
-hashed form only from v22, and the name itself below it:
-
-```abnf
-rslv   = %s"RSLV" SP query
-query  = domain / hashed        ; hashed only from v22
-domain = 1*253 OCTET            ; the name as text
-hashed = "[" 64HEXDIG "]" tld   ; keccak-256 of the second-level label
-tld    = %s".simplex" / %s".testing"
-```
-
-`domain` is the UTF-8 canonical fully-qualified name with the TLD always
-explicit (e.g. `privacy.simplex`, `test.testing`, `example.com`), bounded to
-253 bytes.
-
-**Hashed labels.** The query is a name, or the keccak-256 of a second-level
-label in ENS's bracketed form. A label can only be letters, digits and hyphens,
-so `[` tells the two apart and no tag is needed.
-
-Only a second-level name may be hashed. A name with subnames is sent as text: it
-is resolved rather than priced, and its record names it anyway. A web TLD has no
-registry to key a hash on.
-
-From v22 a client MUST send the hash. Older routers can only read the name, so a
-client on an older session sends the name. A router answering a hashed query
-does not know the label's length, so it cannot check a minimum-length policy
-either: the client does that, from the pricing it is sent.
-
-The same form reaches the backing resolver, which is what its HTTP API takes, so
-the query is one string end to end.
-
-A hashed query still answers with the name. The registrar records the plaintext
-label when a name is registered, keyed by the hash of that label, so a router can
-look up what the hash stands for without ever being told. The router is not
-trusted for it: a client MUST check that the record names the name it asked
-about, and reject the answer otherwise. A registry that does not record the
-label cannot answer a hashed query at all, and the router answers `ERR NAME
-RESOLVER` rather than a record it knows the client will reject. An unregistered
-name has no recorded label, so it cannot be looked up either.
-
-**Server-side validation.** The names router parses `domain` as a
-fully-qualified name (TLD required — bare labels are rejected) and forwards it
-to the configured backing resolver, which is the source of truth for which
-on-chain registry maps to each TLD.
-
-The names router responds with either an `RNAME` response saying what it knows
-about the name, or an `ERR NAME` error whose subcode a client iterating across
-several configured servers can act on distinctly:
-
-| Response | Condition | Client action |
-|---|---|---|
-| `RNAME` | the router read the registry | use it |
-| `ERR NAME NOT_FOUND` | below v22 only: every name that does not resolve. From v22 a router never sends it | stop, and do not read it as registrable |
-| `ERR NAME NO_RESOLVER` | this router has no resolver (names role not enabled) | skip this server, try the next |
-| `ERR NAME RESOLVER <detail>` | the router cannot state an answer completely: no registrar or price oracle for the TLD, an unreachable chain, a transport failure, a timeout, a registration it could not date or resolve | surface `<detail>`; retry only if it reads as transient |
-
-A client SHOULD NOT broadcast a `name` to further servers after a name-capable
-router has answered (`NOT_FOUND` or `RESOLVER`), since that router has already
-seen the lookup key; `NO_RESOLVER` discloses nothing about the name beyond the
-fact that this router cannot resolve, so iterating past it is safe.
-
-#### Name response
-
-`RNAME` answers both what a name resolves to and whether it can be registered.
-
-```abnf
-rname = %s"RNAME" SP nameResponse
-```
-
-`nameResponse` is a UTF-8 JSON object consuming the remainder of the
-transmission: the `registration` below, and `lastBlockTs`, the block timestamp the
-registry was read at. Money is US cents, times are seconds since the Unix epoch, and lengths are
-characters.
-
-| Field | JSON type | Constraints |
-|---|---|---|
-| `lastBlockTs` | number | block timestamp the answer was read at. Absent only from a v20/v21 router, which sent the record alone |
-| `registration` | object | below |
-
-A router reads the registry through a node of its own, which can lag. A name
-registered or changed after `lastBlockTs` still reads as it did before, so a client
-MUST NOT treat an answer as current without checking it.
-
-`registration`'s `type` selects which of the three answers it is.
-
-| `type` | Meaning |
-|---|---|
-| `registered` | held by someone until `expires`, renewable by its owner alone until `graceUntil`. It always carries `nameRecord`: where the owner set none, every field is unset and the resolver address is zero |
-| `available` | held by nobody and registrable now, at `pricing` |
-| `reserved` | held back by the registry and not registered. It carries no price, and a router MUST NOT quote one |
-
-| Field | On | JSON type | Constraints |
-|---|---|---|---|
-| `expires` | `registered` | number | absent only from a v20/v21 router, which sent the record alone |
-| `graceUntil` | `registered` | number | after `expires`; until here only the owner may renew. Absent on the same condition as `expires` |
-| `reservedReason_` | `registered` | string | a reason word, present only when the name is held back as well |
-| `nameRecord` | `registered` | object | the record, schema below |
-| `pricing` | `available` | object | `registrationPrices`, `basePrice` and `minLabelLength`, below |
-| `reservedReason` | `reserved` | string | a reason word |
-
-| `pricing` field | JSON type | Constraints |
-|---|---|---|
-| `registrationPrices` | object | label length, as a decimal string, to US cents per year, for the lengths the registry prices specially |
-| `basePrice` | number | US cents per year for every other length |
-| `minLabelLength` | number | characters; the registry refuses shorter labels |
-
-A reason word is `internal`, `trademark`, `community`, or a word a later version
-reserves under, at most 32 printable ASCII characters. A router truncates an
-unknown word to that and otherwise passes it through unchanged. A client MUST
-read a word it does not know as unknown and still treat the name as reserved.
-
-**Computing the price.** In US cents, for a duration in seconds:
-
-```
-price len duration = tier len * duration / 31536000
-tier len = the entry for len in registrationPrices, or basePrice when there is none
-```
-
-The registry's minimum registration is 730 days, a contract constant, so it is
-specified here rather than sent. `registrationPrices` omits any length below
-`minLabelLength`. A client MUST NOT show a quote for a label the registry
-refuses: a hashed query carries no length, so only the client can check it.
-
-Below v22, `RNAME` carries the bare record and nothing else, and every answer
-without one is `ERR NAME NOT_FOUND`. A v22 client reads such an answer as
-`registered` with no expiry, grace or reservation.
-
-From v22 a client MUST NOT read `ERR NAME NOT_FOUND` as "registrable": only
-`available` says that.
-
-A router that cannot state an answer completely MUST send `ERR NAME RESOLVER
-<detail>` rather than answer partially or guess. That covers a TLD with no
-registrar or price oracle, an unreachable chain, a timeout, a registration it
-could not date or resolve, and any status word it does not recognise.
-
-`nameRecord` MUST be a UTF-8 JSON object with the following schema:
-
-| Field | JSON type | Constraints |
-|---|---|---|
-| `name` | string | ≤ 255 bytes UTF-8 |
-| `nickname` | string | ≤ 255 bytes UTF-8; senders MUST emit the empty string `""` when unset |
-| `website` | string | ≤ 255 bytes UTF-8; same empty-string-when-unset rule |
-| `location` | string | ≤ 255 bytes UTF-8; same empty-string-when-unset rule |
-| `simplexContact` | array of strings | each a SimpleX contact link (primary first); empty array `[]` when unset |
-| `simplexChannel` | array of strings | each a SimpleX channel link (primary first); empty array `[]` when unset |
-| `eth` | string or null | ≤ 255 bytes UTF-8; senders MUST emit `null` when unset; receivers MUST also accept absent keys as unset |
-| `btc` | string or null | ≤ 255 bytes UTF-8; same null / absent rules |
-| `xmr` | string or null | ≤ 255 bytes UTF-8; same null / absent rules |
-| `dot` | string or null | ≤ 255 bytes UTF-8; same null / absent rules |
-| `owner` | string | `"0x"` followed by 40 lowercase hex characters (20 raw bytes) |
-| `resolver` | string | `"0x"` followed by 40 lowercase hex characters; the resolver contract address that produced the record |
-
-Testnet-vs-mainnet status is derived from the queried TLD, not from the record.
-
-Receivers MUST tolerate extra unknown fields; adding a required field is a
-breaking change requiring an SMP version bump. Receivers parse by key name, so
-peers MUST NOT rely on a byte-canonical form.
-
-The names router caps the resolver response it will accept
-(`resolver_max_response_bytes`, at most 16000 bytes) so the re-encoded `RNAME`
-stays within the SMP proxied transmission budget of 16224 bytes; a response over
-the cap is `ERR NAME RESOLVER`.
-
 ## Transport connection with the SMP router
 
 ### General transport protocol considerations

protocol/xftp.md

@@ -1,4 +1,4 @@
-Version 4, 2026-08-08
+Version 3, 2025-01-24
 
 # SimpleX File Transfer Protocol
 
@@ -50,12 +50,11 @@ The objective of SimpleX File Transfer Protocol (XFTP) is to facilitate the secu
 
 XFTP is implemented as an application level protocol on top of HTTP2 and TLS.
 
-This document describes XFTP protocol version 4. The version history:
+This document describes XFTP protocol version 3. The version history:
 
 - v1: initial version
 - v2: authenticated commands - added basic auth support for commands
 - v3: blocked files - added BLOCKED error type for policy violations
-- v4: server public information and entitlement proof in handshake, file storage time in FNEW and SIDS
 
 The protocol describes the set of commands that senders and recipients can send to XFTP routers to create, upload, download and delete data packets of several pre-defined sizes. XFTP routers SHOULD support packets of 4 sizes: 64KB, 256KB, 1MB and 4MB (1KB = 1024 bytes, 1MB = 1024KB).
 
@@ -330,7 +329,7 @@ Once TLS handshake is complete, client and router will exchange blocks of fixed
 
 ```abnf
 paddedRouterHello = <padded(routerHello, 16384)>
-routerHello = xftpVersionRange sessionIdentifier routerCerts signedRouterKey webIdentityProof serverInfo ignoredPart
+routerHello = xftpVersionRange sessionIdentifier routerCerts signedRouterKey ignoredPart
 xftpVersionRange = minXftpVersion maxXftpVersion
 minXftpVersion = xftpVersion
 maxXftpVersion = xftpVersion
@@ -339,27 +338,15 @@ sessionIdentifier = shortString
 routerCerts = length 1*routerCert ; NonEmpty list of certificates in chain
 routerCert = originalLength <x509encoded>
 signedRouterKey = originalLength <x509encoded> ; signed by router certificate
-webIdentityProof = shortString ; signature over the web client challenge and sessionIdentifier, empty when the client sent no challenge
-serverInfo = %s"0" / (%s"1" largeString) ; JSON server public information, sent when maxXftpVersion is 4 or above
 
 paddedClientHello = <padded(clientHello, 16384)>
-clientHello = xftpVersion keyHash entitlementProof ignoredPart
+clientHello = xftpVersion keyHash ignoredPart
 ; chosen XFTP protocol version - must be the maximum supported version
 ; within the range offered by the router
 
-entitlementProof = %s"0" / (%s"1" issuerKeyIndex bbsProof entitlement)
-; proof of the user entitlement, encoded from v4, bound to sessionIdentifier
-issuerKeyIndex = 2*2OCTET ; Word16 index of the issuer public key
-bbsProof = largeString
-entitlement = entExpires entName entExtra
-entExpires = shortString ; expiration as an ISO8601 UTC timestamp
-entName = shortString ; e.g. "supporter", "legend"
-entExtra = largeString ; opaque to the router
-
 xftpVersion = 2*2OCTET ; Word16 version number
 keyHash = shortString
 shortString = length length*OCTET
-largeString = originalLength *OCTET
 length = 1*1OCTET
 originalLength = 2*2OCTET
 ignoredPart = *OCTET
@@ -445,7 +432,7 @@ Routers SHOULD support basic auth with this command, to allow only router owners
 The syntax is:
 
 ```abnf
-register = %s"FNEW " fileInfo rcvPublicAuthKeys basicAuth fileStorageTime
+register = %s"FNEW " fileInfo rcvPublicAuthKeys basicAuth
 fileInfo = sndKey size digest
 sndKey = length x509encoded
 size = 4*4 OCTET ; Word32 big-endian
@@ -453,27 +440,19 @@ digest = length *OCTET
 rcvPublicAuthKeys = length 1*rcvPublicAuthKey
 rcvPublicAuthKey = length x509encoded
 basicAuth = "0" / "1" length *OCTET
-fileStorageTime = %s"0" / (%s"1" storageHours)
-; encoded from v4; absent or zero requests the maximum the router allows
-storageHours = 4*4 OCTET ; Word32 big-endian
 
 x509encoded = <binary X509 key encoding>
 
 length = 1*1 OCTET
 ```
 
-`fileStorageTime` requests how long the router stores the data packet. The router grants the smaller of the request and the maximum it allows for the entitlement presented in the handshake, and returns the granted expiration in `sndIds`.
-
 If the data packet is registered successfully, the router must send `sndIds` response with the sender's and recipients' data packet IDs:
 
 ```abnf
-sndIds = %s"SIDS " senderId recipientIds grantedStorageTime
+sndIds = %s"SIDS " senderId recipientIds
 senderId = length *OCTET
 recipientIds = length 1*recipientId
 recipientId = length *OCTET
-grantedStorageTime = %s"0" / (%s"1" grantedExpires) ; encoded from v4
-grantedExpires = %s"T" expiresAt
-expiresAt = 8*8 OCTET ; Int64 big-endian, seconds since epoch
 ```
 
 #### Add data packet recipients

protocol/xrcp.md

@@ -71,7 +71,7 @@ The session invitation contains this data:
 
 Host application decrypts (except the first session) and validates the invitation:
 - Session signature is valid.
-- Timestamp of a multicast announcement is not earlier than 3660 seconds before and not later than 3600 seconds after the current time of the host. The host ignores announcements outside of this interval and continues listening.
+- Timestamp is within some window from the current time.
 - Long-term key signature is valid.
 - Long-term CA and signature key are the same as in the first session.
 - Some version in the offered range is supported.
@@ -121,15 +121,9 @@ During TLS handshake, parties validate certificate chains against previously kno
 
 ### Session verification and protocol negotiation
 
-Once TLS session is established, both the host and controller devices present a "session security code" to the user. The session security code must be a digest of tlsunique channel binding. As it is computed as a digest of the TLS handshake for both the controller and the host, it will validate that the same TLS certificates are used on both sides, and that the same TLS session is established, mitigating the possibility of MITM attack in the connection.
+Once TLS session is established, both the host and controller devices present a "session security code" to the user who must match them (e.g., visually or via QR code scan) and confirm on the host device. The session security code must be a digest of tlsunique channel binding. As it is computed as a digest of the TLS handshake for both the controller and the host, it will validate that the same TLS certificates are used on both sides, and that the same TLS session is established, mitigating the possibility of MITM attack in the connection.
 
-To verify the session, the user matches the codes (e.g., visually or via QR code scan) and confirms the session on the host device.
-
-The user MUST verify the first session with a new controller. The host stores the long-term identity of the controller (CA fingerprint and Ed25519 public key) only after this session is verified. A controller with the stored long-term identity is a known controller.
-
-The user MAY verify the subsequent sessions with a known controller. In a session that the host confirms without user action, the controller is authenticated only by its long-term identity stored in the first session (see [Threat model](#threat-model)). SimpleX Chat mobile apps require user verification of sessions with known controllers only when the user enables the "Verify connections" option.
-
-Once the session is confirmed, the host sends HELLO block to the controller.
+Once the session is confirmed by the user, the host sends HELLO block to the controller.
 
 XRCP blocks inside TLS are padded to 16384 bytes.
 
@@ -311,7 +305,7 @@ to receive: (rcvKey', sk, nonce) = HKDF(rcvKey, "SimpleXSbChain", 88)
 - prevent host and controller devices from establishing the session
 
 *cannot:*
-- same as passive adversary, provided that user visually verified session code out-of-band in the first session with the controller.
+- same as passive adversary, provided that user visually verified session code out-of-band.
 
 #### An active adversary with the access to the network:
 
@@ -330,15 +324,6 @@ to receive: (rcvKey', sk, nonce) = HKDF(rcvKey, "SimpleXSbChain", 88)
 *cannot:*
 - connect to the host or make host connect to itself.
 
-#### An active adversary with the access to the network who also obtained the long-term private keys of a known controller (e.g., from the controller device):
-
-*can:*
-- make host connect to itself instead of the controller.
-- access any data of the controlled host application, within the capabilities of the provided API, if the host confirms the session without user action.
-
-*cannot:*
-- access host application data, provided that user visually verified session code out-of-band.
-
 #### Compromised controller device:
 
 *can:*

rfcs/2026-07-11-service-rpc.md

@@ -1,114 +0,0 @@
----
-Proposed: 2026-07-11
-Protocol: agent-protocol (new version)
-Depends on: 2026-07-12-address-pqdr-keys
----
-
-# One-off requests to service addresses
-
-Implementation plan: to follow, after this and the address-DR RFC are reviewed.
-
-## Problem
-
-Client applications need to interact with services, for example: badge issuance, directory requests, telemetry submissions, blockchain reads and writes, LLM calls. The only communication primitive available today is a duplex connection, so each of these interactions requires the full connection procedure - creating queues, key agreement, double ratchet initialization - and leaves persistent state on both sides: queues, ratchet state, connection records, message history.
-
-This is the wrong primitive for most service interactions:
-
-1. Cost. To send the first request to a not yet connected service, the client and the service exchange multiple commands across two servers. For "search the directory" all of it is overhead. The setup cost also creates an incentive to keep connections open, and a service with N users who used it once permanently holds N sets of queues and ratchet states.
-
-2. Privacy. A connection is a stable pairwise pseudonym. If a service were to use a duplex connection, it could link all requests made over it into a profile: search history in the directory, blockchain operations linked even when different on-chain keys are used, telemetry that becomes longitudinal tracking. The client also accumulates history that can be recovered from the device. Where continuity is needed, it can be provided in the application protocol (e.g., a token included in requests), without a transport-level identity.
-
-3. Encryption. Messages sent to contact addresses outside an established connection have a single layer of X25519 encryption, with no post-quantum protection and no forward secrecy. This is not acceptable for service requests.
-
-In-app service addresses should be stored as names resolving to links via the existing addressing layer (server host in the link authority, current link data retrieved with `LGET`), so that service links can be changed without redeploying the apps. Name resolution is already supported, and out of scope.
-
-## Security objectives
-
-1. Requests from the same client must not be linkable to each other by the service or by servers, and no long term state is created on either side in the transport layer.
-2. Post-quantum resistant end-to-end encryption of requests and replies.
-3. Reply authenticity must be verifiable against the link; substitution, replay, dropping or reordering of replies by servers must be detectable.
-4. A repeated request for the same operation must not be executed twice.
-
-## Solution
-
-A service address is an ordinary short-link contact address. The client sends one request to the address queue and receives replies in a reply queue it creates for the request. The double ratchet is established from the address's published keys (see the address-DR RFC): the request is the first ratchet message, and replies are subsequent ratchet messages. So a request-response exchange is a short-lived one-directional double ratchet connection, established from the first message and removed after the last.
-
-The exchange:
-
-1. Retrieve the address link data (`LGET`, via proxy when IP protection is needed): the root key, the identity key, the prekey with its id, and the KEM key.
-2. Create a reply queue (`NEW`, subscribed).
-3. Establish the sending ratchet from the published keys (`pqX3dhSnd`, `initSndRatchet`). Build the request: the reply queue, the requester's X3DH parameters, and the payload encrypted under the ratchet. Encrypt the whole request to the address queue and send it once (unauthenticated `SEND`, via proxy when IP protection is needed). There is no transport retry; a reply is the success signal, and a hard error fails the request.
-4. The service establishes the receiving ratchet from its private keys and the request's X3DH parameters (`pqX3dhRcv`, `initRcvRatchet`), decrypts the payload, and delivers it to the service application. To reply it creates a send connection with the ratchet and sends reply messages to the reply queue, each encrypted under the ratchet.
-5. The client decrypts and delivers each reply message to the application. The first reply message returns from the request; later reply messages are delivered through a callback the application registered. The exchange ends on a reply marked final. The client deletes the reply queue and the ratchet on the final message, the deadline, or when the application cancels.
-
-How this meets the objectives:
-
-1. Unlinkability: fresh X3DH keys and a fresh reply queue per request; the sender's IP address and session are protected by existing private routing; the reply queue and both ratchets are removed after the exchange; nothing is shared between two requests.
-2. Encryption: the double ratchet with its sntrup761 KEM, from the first message.
-3. Authenticity: the ratchet is established against the identity key committed by the link hash, so a decryptable reply proves it came from the address owner; the ratchet message numbering detects dropped and reordered replies. No separate signature is needed.
-4. Single execution: the service identifies a request by the hash of its decrypted payload and, within a fixed retention period, re-sends the stored replies for a repeated request without running the operation again.
-
-## Design
-
-Syntax uses [ABNF][1] with [case-sensitive strings extension][2]. `SMPQueueInfo`, `agentVersion`, and the X3DH and ratchet types are as in the [agent protocol](../protocol/agent-protocol.md) and `Crypto.Ratchet`.
-
-### Correlation and chat
-
-A reply is connected to its request by the reply queue: each request has its own reply queue, and every message in that queue is a reply to that request. The request hash (of the decrypted payload) is used only for idempotency. The application sets an id inside the payload to make two requests the same operation or different ones; the agent does not read it.
-
-Both ends are chat bots on the chat library over the agent. The chat library serializes a service command into the request payload and deserializes the responses; the agent transports them, establishes and removes the ratchet, and correlates by reply queue.
-
-### Request
-
-A new agent envelope, shaped like `AgentConfirmation` (X3DH parameters to establish the ratchet, plus a body encrypted under it):
-
-```abnf
-agentRequest = agentVersion %s"Q" replyQueues prekeyId sndE2EParams encRequest
-replyQueues = length 1*SMPQueueInfo ; the first is used; more are for redundancy
-prekeyId = shortString ; the published prekey used, from link data
-sndE2EParams = <requester Snd X3DH parameters, see address-DR RFC>
-encRequest = <ratchet-encrypted request payload>
-```
-
-The whole `agentRequest` is encrypted to the address queue with the per-queue layer and sent with `SEND`, as an invitation is today. The reply queue keys and the X3DH parameters are not visible to servers. A request must fit one message; a larger payload is an XFTP file description in the payload.
-
-The request hash is the SHA3-256 of the decrypted payload - the same bytes on both sides. There is no transport retry; a hard error (`AUTH`, `QUOTA`) fails the request, and the application decides whether to send a new one.
-
-### Replies
-
-The service creates a send connection with the ratchet and sends reply messages to the reply queue, each encrypted under the ratchet. A reply message uses a new agent envelope:
-
-```abnf
-agentResponse = agentVersion %s"P" final responses
-final = %s"T" / %s"F" ; T - no more reply messages follow
-responses = length 1*responseItem ; non-empty list of application responses
-responseItem = largeString ; opaque application response
-```
-
-Each message includes a list of responses, so responses known together are sent in one message and responses that become known over time are sent in separate messages. The message is encrypted and numbered by the ratchet, which authenticates it against the committed identity key and detects dropped or reordered messages; no separate signature is used.
-
-The first reply message returns from the request. Later reply messages are delivered through the callback the application registered with the request, while the process runs. The exchange ends on a message with `final = T`. The client deletes the reply queue and the ratchet on that message, on the deadline, or when the application cancels. Deleting the reply queue stops further replies. The transport keeps no exchange across a client restart; the application keeps its own state and sends a new request when it needs to.
-
-### Rejection
-
-The service refuses a request with the `AgentRejection` envelope from the [communicating rejection RFC](../../simplex-chat/docs/rfcs/2024-03-22-communicating-reject.md), sent to the reply queue under the ratchet, with an opaque application reason. The same envelope communicates refusal of a connection request, where today it is dropped silently. A rejection ends the exchange like a final reply.
-
-### Idempotency
-
-The service keeps, for a fixed retention period it chooses (1 to 24 hours, in service configuration, not in link data), the request hash, the ordered response messages it produced, and the reply queues and ratchets subscribed under that hash. A repeat request with the same hash does not reach the service application:
-
-- while the first request is being answered, the repeat establishes its own ratchet and reply queue, is added to the record, receives the responses already produced, and receives each later response too.
-- after the operation completed, the repeat receives the whole stored sequence of responses, re-encrypted under its own ratchet.
-
-The stored responses are the application response bytes, not the ratchet ciphertext, because a repeat establishes a new ratchet and the responses are re-encrypted for it. This gives single execution over at-least-once delivery. After the retention period a request with the same hash is a new operation and runs again.
-
-### Out of scope
-
-- Recovery across restart: the transport keeps no exchange across a client restart; the application persists its own state and sends a new request when it needs to.
-- Service-initiated messages: there is no standing channel; use a connection where the service must reach the client without a request.
-- Abuse protection beyond existing queue quotas: services can require application-level credentials (e.g., a badge) in the request payload; rate limiting is a separate discussion.
-- Scaling request reception: a single address queue bounds service throughput; distributing reception across multiple queues or relays (the existing `relays` field in contact link data) is a separate question, but it would fit well with name resolving to multiple addresses, both for redundancy, reliability and higher throughput.
-- Name resolution: existing addressing layer.
-
-[1]: https://tools.ietf.org/html/rfc5234
-[2]: https://tools.ietf.org/html/rfc7405

rfcs/2026-07-12-address-pqdr-keys.md

@@ -1,83 +0,0 @@
----
-Proposed: 2026-07-12
-Protocol: agent-protocol (new version)
----
-
-# Establishing the double ratchet from address data
-
-## Problem
-
-A contact address receives a connection request as an `AgentInvitation` message, encrypted only with the per-queue X25519 layer. The double ratchet is established later: the address owner joins the requester's connection request, generates its X3DH keys, and sends them in the confirmation. So the first message to an address - the invitation and the profile in it - is not under the double ratchet, and has no post-quantum protection.
-
-The cause is that the address owner's X3DH contribution is generated per request and sent in the confirmation, so it cannot exist before the requester's first message.
-
-## Solution
-
-Publish the address owner's X3DH contribution in the address link data, so a requester can establish the double ratchet in its first message. The requester runs the existing `pqX3dhSnd` against the published keys, initializes a sending ratchet, and encrypts its first message under it. The owner runs the existing `pqX3dhRcv` against its stored private keys and the requester's X3DH keys from the message, initializes a receiving ratchet, and decrypts it.
-
-Publish the owner's X3DH contribution - two X448 keys and an optional sntrup761 KEM key, with the e2e version range - as one bundle in the mutable contact user data, signed by the root key. The bundle is the existing `RcvE2ERatchetParamsUri` type that a one-time invitation already advertises, with an id for rotation.
-
-This is backward compatible. A requester that does not use the published bundle sends a current `AgentInvitation` with its own X3DH keys, and the owner does what it does today: generates fresh X3DH keys and sends them in the confirmation. The owner branches on whether the incoming message uses the published bundle.
-
-Three properties follow. The first message, including the profile, is under the double ratchet, which closes the profile gap and gives it post-quantum protection through the ratchet's sntrup761 KEM. A decryptable message proves the sender established X3DH against the root-signed keys, so it authenticates the address owner without a separate signature. And because the bundle is in mutable data, an existing address can advertise the double ratchet by updating its mutable data - no new link.
-
-## Design
-
-Syntax uses [ABNF][1] with [case-sensitive strings extension][2]. Key and ratchet types are as in `Crypto.Ratchet` and the [agent protocol](../protocol/agent-protocol.md); all DH keys are X448, the KEM is sntrup761.
-
-### Published keys in link data
-
-The owner's X3DH contribution is a ratchet-keys bundle appended to the mutable contact user data, signed by the root key. Nothing is added to the immutable fixed link data:
-
-```abnf
-userContactData =/ ratchetKeys ; appended, ignored by earlier versions
-ratchetKeys = %s"0" / (%s"1" ratchetKeyId e2eParams)
-ratchetKeyId = shortString ; identifies this bundle, changes on rotation, echoed in the request
-e2eParams = <RcvE2ERatchetParamsUri: e2e version range, two X448 keys, optional sntrup761 key>
-```
-
-`e2eParams` is the existing `RcvE2ERatchetParamsUri` - the same type a one-time invitation advertises - so a requester reads it, negotiates the concrete e2e version against its own range, and runs `pqX3dhSnd` against it, exactly as it does for an invitation. The KEM key is optional (the params' KEM field is a `Maybe`), controlled by the address's initial-keys mode, the same 3-way choice as invitations: advertise the KEM (post-quantum from the first message), advertise X448-only but still support post-quantum if the requester proposes its own KEM (one message later, avoiding the ~1158-byte key in link data), or no post-quantum.
-
-The owner keeps the private side of each bundle - the two X448 private keys and, when PQ is on, the KEM keypair - indexed by `ratchetKeyId`. On rotation with `LSET` it publishes a new bundle with a new `ratchetKeyId` and keeps the previous private keys for a window covering queue message retention, so a request that used a just-rotated bundle still decrypts. Because the bundle is in mutable data, an existing address advertises the double ratchet by updating its mutable data - no new link, no re-creation.
-
-### Request confirmation
-
-A requester that uses the published keys establishes the sending ratchet before its first message, so it sends that message as a confirmation, not an invitation - the same envelope a joining party sends in a connection. The confirmation gains an optional `ratchetKeyId` naming the bundle the requester used, so the owner selects the matching private keys:
-
-```abnf
-agentConfirmation =/ ratchetKeyId ; the bundle the requester used, echoed; absent on other confirmations
-```
-
-The confirmation holds the requester's Snd X3DH parameters (so the owner runs `pqX3dhRcv`) and, encrypted under the ratchet, the first message. A confirmation with `ratchetKeyId` on a contact address takes the published-key path; an `AgentInvitation`, as today, takes the current path where the owner generates fresh X3DH keys and returns them in its own confirmation. A connection-request URI is unchanged: it advertises the requester's Rcv parameters and must not include Snd parameters.
-
-### Establishing the ratchet
-
-Requester:
-
-1. Retrieve link data (`LGET`), read the bundle - its `ratchetKeyId` and `e2eParams` (`RcvE2ERatchetParamsUri`) - and negotiate the concrete e2e version against its own range.
-2. `generateSndE2EParams` for its own X3DH contribution - encapsulating to the bundle's KEM if it advertises one, or proposing its own KEM if the requester wants post-quantum and the bundle is X448-only.
-3. `pqX3dhSnd` against the bundle's parameters, then `initSndRatchet` - the sending ratchet.
-4. Encrypt the first message under the ratchet, and send a confirmation with its Snd parameters and `ratchetKeyId`.
-
-Owner:
-
-1. On a confirmation with `ratchetKeyId` on a contact address, select the private X3DH keys and, if any, KEM keypair by `ratchetKeyId` (current or a retained previous generation).
-2. `pqX3dhRcv` against the requester's Snd parameters with those private keys, then `initRcvRatchet` - the receiving ratchet. Decrypting the first message advances the ratchet and gives it a send side, so the owner can reply.
-3. Decrypt, and reply under the ratchet.
-
-A request whose `ratchetKeyId` is no longer retained cannot be decrypted; the owner does not learn the requester or its reply address, and the requester's attempt fails at its own timeout.
-
-### Authentication
-
-The ratchet-keys bundle is in the mutable link data, signed by the root key. A decryptable message proves the sender established X3DH against those root-signed keys: an SMP server cannot substitute them without forging the root signature (`decryptLinkData` verifies it), which is the X3DH anti-substitution property. Where a message today relies on a separate signature over its content for authenticity, this establishment provides it, and the signature is not needed. The root Ed25519 key is the address's signing identity; the X3DH keys are separate DH keys (X448), and X3DH is over crypto_box, so deniability is preserved. A malicious server can still serve an older but validly-signed bundle (rollback to a retired generation); this is bounded by the retention window and by the ratchet advancing after the first message, and a per-key signature would not prevent it. Reusing a bundle across requesters is consistent with the address already being a shared identifier.
-
-## Uses
-
-- Invitations to an address, and the profile in them, are under the double ratchet from the first message.
-- An existing address gains the double ratchet when the app updates its mutable link data (`LSET`, with the user's confirmation and current profile, combined with the full→short address migration); no new link is issued.
-- The service RPC (see the RPC RFC) establishes the ratchet this way to send the request as the first ratchet message.
-
-This RFC depends on nothing else here. It replaces the need for the PQ-queue RFC in the address case, because the ratchet provides post-quantum protection for the first message; the PQ-queue RFC remains for first messages that do not establish a ratchet.
-
-[1]: https://tools.ietf.org/html/rfc5234
-[2]: https://tools.ietf.org/html/rfc7405

rfcs/2026-07-12-queue-pq-encryption.md

@@ -1,42 +0,0 @@
----
-Proposed: 2026-07-12
-Protocol: smp-client (new version)
----
-
-# Post-quantum encryption of the SMP queue layer
-
-## Problem
-
-A message sent to a queue outside an established double ratchet has one layer of end-to-end encryption: NaCl crypto_box over an X25519 DH secret. The sender generates an ephemeral X25519 key, computes the secret with the recipient's per-queue DH key, and puts its ephemeral key in the message public header (`agentCbEncryptOnce`). This layer protects invitations, confirmations, and the profile sent with them.
-
-It is not post-quantum. An adversary that records this traffic and later has a quantum computer can recover the X25519 secret and decrypt it. The double ratchet adds a post-quantum KEM once it is established, but the first message to a queue, before the ratchet, has only X25519.
-
-This RFC adds post-quantum protection to the single-shot queue encryption itself, for cases that do not establish a double ratchet from the first message. Where a double ratchet is established from the first message (see the address-DR RFC), the ratchet provides post-quantum protection and this layer is not needed.
-
-## Solution
-
-Extend the single-shot queue encryption to a hybrid X25519 + sntrup761 scheme, in a new SMP client version. The recipient publishes a KEM encapsulation key alongside its per-queue DH key. The sender encapsulates to it, combines the KEM shared secret with the X25519 DH secret, and encrypts the body with the combined secret. The KEM ciphertext travels in the message public header next to the ephemeral X25519 key.
-
-Recording the traffic and breaking X25519 later is not sufficient: without breaking sntrup761 as well, the combined secret is not recoverable.
-
-## Design
-
-The message public header (`PubHeader`) gains a hybrid variant, selected by a version and a tag, so older senders and the empty-header case are unchanged:
-
-```abnf
-smpPubHeaderHybrid = smpClientVersion %s"2" senderPublicDhKey kemCiphertext
-senderPublicDhKey = length x509encoded ; sender ephemeral X25519 key
-kemCiphertext = largeString ; sntrup761 ciphertext, 1039 bytes
-```
-
-The secret combines both shared secrets, and the body is encrypted with NaCl secret_box (the DH-only path uses crypto_box today; the combined secret is no longer a plain DH result, so it is used as a secret_box key), padded to the same lengths:
-
-```
-secret = HKDF(dh(recipient key, sender ephemeral key) || KEM shared secret)
-```
-
-The recipient's KEM encapsulation key is distributed the same way its per-queue DH key is today - in the queue address for a connection request, and in link data for a short link. The KEM ciphertext is stored the same way the ephemeral DH key is: in the public header, readable by the destination server (and not by the proxy with proxied sending), which cannot derive the secret without the recipient's KEM private key.
-
-The recipient stores the computed secret the way the per-queue DH secret is stored on receiving the first message (`setRcvQueueConfirmedE2E`), and reuses it for later messages on the queue.
-
-Sizes: the KEM ciphertext is 1039 bytes and the encapsulation key 1158 bytes. Link data user data is padded to 13784 bytes, so the key fits with application data. This RFC is independent of the RPC, SSND, and address-DR RFCs.

rfcs/2026-07-12-smp-secure-send.md

@@ -1,55 +0,0 @@
----
-Proposed: 2026-07-12
-Protocol: smp (new version)
----
-
-# SSND: combined secure-and-send command
-
-## Problem
-
-Two SMP flows secure a messaging queue and then immediately send the first message to it, as two commands and two round trips:
-
-- The fast connection handshake: the joining party secures the queue with `SKEY`, then sends the confirmation with `SEND`.
-- Any first send to a sender-securable queue where the sender both secures it and delivers the first message.
-
-The two commands express one intent - "this is my key, and here is my first message" - so they can be one command and one round trip. The combination must be idempotent, because the first send is retried on network failure and a queue is often secured before the response is known. `SKEY` is already idempotent (a repeat with the same key succeeds). `SEND` is not, so a naive combination would deliver a duplicate message on retry.
-
-## Solution
-
-A new command `SSND` combines `SKEY` and `SEND` in one transmission, idempotent in both parts:
-
-- Key part: as `SKEY` - a repeat with the same key succeeds, a different key fails with `AUTH`.
-- Send part: the server keeps the hash of the message until it is acknowledged, and reports a repeat of the same message as delivered without delivering it again.
-
-The server-side hash covers the common case, when the retry arrives before the message is acknowledged. A retry that arrives after the acknowledgement is delivered as a duplicate and discarded by the receiving agent by message hash, as duplicate messages are discarded today.
-
-## Design
-
-Syntax uses [ABNF][1] with [case-sensitive strings extension][2]. `senderAuthPublicKey`, `msgFlags` and `smpEncMessage` are as in the [SMP protocol](../protocol/simplex-messaging.md).
-
-```abnf
-secureSend = %s"SSND " senderAuthPublicKey SP msgFlags SP smpEncMessage
-senderAuthPublicKey = length x509encoded
-```
-
-`SSND` is a sender command, authorized with the key it sets, and accepted only on messaging-mode queues (`QMMessaging`), where the sender can secure the queue. The server responds `OK` or `ERR`.
-
-Server processing:
-
-1. Secure the queue with the key, as `SKEY`. A repeat with the same key succeeds; a different key returns `AUTH`.
-2. If the queue holds an unacknowledged message whose stored hash equals the hash of this message, respond `OK` without storing it again.
-3. Otherwise store the message, keep its hash with the queue until the message is acknowledged, and deliver it.
-
-The stored hash is one value per not-yet-acknowledged queue message. It is removed when the message is acknowledged. All queue store backends (in-memory, journal, PostgreSQL) keep it.
-
-`SSND` composes with the proxy protocol without change: `proxySMPCommand` forwards any sender command through `PFWD`/`RFWD`, so `SSND` is proxied as `SEND` and `SKEY` are today.
-
-## Uses
-
-- The fast connection handshake replaces `SKEY` then the `SEND` confirmation with one `SSND`.
-- The service RPC response (see the RPC RFC) secures the reply queue and sends the first reply with one `SSND`.
-
-This RFC is independent of the RPC, PQ-queue, and address-DR RFCs and can be implemented on its own.
-
-[1]: https://tools.ietf.org/html/rfc5234
-[2]: https://tools.ietf.org/html/rfc7405

rfcs/2026-08-09-fast-queue-rotation.md

@@ -1,97 +0,0 @@
----
-Proposed: 2026-08-09
-Protocol: agent-protocol v8
-Diagram: ../protocol/diagrams/duplex-messaging/queue-rotation-fast.svg
----
-
-# Fast queue rotation
-
-## Problem
-
-In the current rotation the peer returns `QKEY` to the initiator over the initiator's current
-receiving queue. When that queue's server is unavailable the rotation cannot complete, so a client
-cannot move away from a failed server.
-
-## Solution
-
-Both the current rotation and v8 add a queue, deliver to both queues while the rotation is in
-progress, and remove the old queue; the recipient drops duplicates in both. The main difference is where
-the new queue's secret is established. In the current rotation it is established over the current
-queue, by `QKEY`, so it cannot complete when the current server is down. In v8 the peer establishes the
-new queue's secret over the new queue itself — a confirmation it sends on R' — so establishing the
-secret no longer depends on the current queue, and the rotation completes even when the current server
-is down.
-
-v8 also starts writing to both queues earlier: from the moment the queue is added, including the
-current queue's not-yet-delivered backlog. So the initiator adds the new queue with `QADD`; from that
-point the peer writes every message to both the current queue and R'. Once R' is secured the peer
-writes new messages to it alone, while the current queue delivers whatever was already scheduled on it
-and a final `QEND`, and is then removed. Because the recipient drops duplicates, neither the order of
-arrival nor which queue carries a message matters, provided each message arrives on at least one queue
-— with one exception, the confirmation, which is always the first message on R'. A dead new queue does
-not stop delivery either, because the current queue keeps delivering until R' is secured.
-
-Roles: A initiates (its receiving queue rotates; A receives on the new queue R'). B is the peer (B
-holds the sending queue to A, secures R', and delivers to both).
-
-Sequence:
-
-    A -> R'       : create new queue (messaging mode, SKEY allowed)
-    A -> S -> B   : QADD(R')          (over A's sending queue; A's current server untouched)
-    B             : from QADD, schedule every message and the current backlog on both queues
-    B -> current  : deliver the scheduled messages (R' holds its copies while securing)
-    B -> R'       : SKEY              (authorize B as sender)
-    B -> R'       : confirmation      (empty; establishes R' secret; first message on R')
-    B -> R'       : deliver R''s held copies (A dedups), then new messages to R' only
-    B -> current  : deliver the remaining tail, then QEND(current)
-    B -> R'       : QEND(current)
-    A             : on QEND, delete the current queue; keep receiving on R'
-
-## Confirmation
-
-The confirmation is the only message a recipient can read on a queue that is not yet secured, and it
-establishes the queue's shared secret without depending on the current queue. Because a data message
-that reached R' before the confirmation could not be read, the confirmation is the first message the
-peer sends on R', and the peer does not start ordinary delivery on R' until the confirmation has been
-sent.
-
-The confirmation body is empty: both parties already know each other, so no profile or reply queue is
-sent. It is sealed by the queue's box (keyed by the shared secret being established) and is not
-additionally encrypted with the double ratchet, so rotation does not advance the message ratchet.
-
-## Termination
-
-`QEND` names a queue to remove and is delivered on both queues. On receipt the recipient deletes the
-named queue; on send the peer removes its sending queue of that address. `QEND` is a general
-queue-removal message — the peer can remove either queue with it — so on the wire a rotation is the
-addition of a queue (`QADD`) and the later removal of the replaced one (`QEND`), each an ordinary
-operation on the queue set rather than a `QTEST`-style completion. Delivering `QEND` on the removed
-queue is best effort; the copy on the surviving queue removes it and reaches the recipient even when
-the removed server is dead.
-
-## Per-queue secret
-
-R' secret is a fresh Diffie-Hellman between the peer's queue key, sent in the confirmation header, and
-the initiator's R' key. It does not depend on any current queue, so redundancy of current queues is
-unaffected.
-
-## Compatibility
-
-Fast rotation runs only when the connection's agreed agent protocol version is 8 or higher; the peer
-chooses it. Otherwise the `QKEY`/`QUSE` exchange is used. `QEND` is defined at version 8 and is only
-sent during fast rotation, so peers below version 8 never receive it.
-
-    new A / new B  : fast (QADD, confirmation on R', QEND)
-    new A / old B  : slow (old B returns QKEY; new A keeps the QKEY/QUSE handling)
-    old A / new B  : slow (agreed version below 8; new B returns QKEY)
-    old A / old B  : slow
-
-The recipient does not choose by version; it reacts to whichever message arrives — `QKEY`, or a
-confirmation on R' followed later by `QEND`.
-
-## Dead current server
-
-The initiator keeps reading messages on the new queue and removes the current queue when `QEND`
-arrives there, without waiting for the current server. Nothing is lost, because every message is
-scheduled on the new queue; the only cleanup that a dead current server delays is the deletion
-of its queue, which is retried a bounded number of times and then abandoned.

rfcs/2026-08-22-xftp-file-storage-time.md

@@ -1,109 +0,0 @@
-# XFTP variable file storage time
-
-## Summary
-
-The server stores a storage time for each file. The sender sets it in the FNEW command. The client may present a proof of an entitlement in the handshake to raise the maximum storage time the server allows. The proof is bound to the TLS session, so it cannot be reused for another session.
-
-An entitlement belongs to the user, not to a file: the client presents it once per connection, and the server applies it to everything the client does in that session. The server can also vary other limits, such as throttling, by the entitlement.
-
-## Entitlement
-
-An entitlement is a name, an expiration, and an extra string. It is the disclosed content of a BBS proof: the holder's secret remains undisclosed, and the three fields are revealed. The server reads `entName` to select a maximum storage time, checks `entExpires`, and ignores `entExtra`; the interpretation of `entExtra` is out of scope here. The protocol references only the entitlement, never a badge; chat maps its own badge to an entitlement before it asks the agent to send.
-
-The proof discloses the entitlement and includes the issuer key index and the BBS proof. The holder's secret and the BBS signature remain with the sender and are never transmitted. The origin of the sender's signed entitlement, from the entitlement service, is out of scope here.
-
-```
-entitlement = entExpires entName entExtra
-entExpires = shortString     ; expiration as a UTCTime ISO8601 string
-entName = shortString        ; e.g. "supporter", "legend"
-entExtra = largeString       ; opaque, interpretation out of scope
-
-entitlementProof = issuerKeyIndex bbsProof entitlement
-issuerKeyIndex = 2*2 OCTET   ; Word16, network byte order
-bbsProof = largeString       ; BBS proof bytes
-```
-
-The presentation header that the BBS proof is generated over is not transmitted; the server takes it from the session (see [Binding](#binding)), which is what binds the proof.
-
-### Issuer keys
-
-Client apps and servers share one list of issuer public keys, indexed by `issuerKeyIndex`. The secret key for the current index is held by the service that issues entitlements, on conditions that are out of scope here, such as payment.
-
-The list holds eight keys so that the issuing service can rotate its current key without an app or server release: the next index is already known to every app and server. Each rotation consumes one index, so a new key has to be added to client apps and servers eventually, and released before the list is exhausted.
-
-## Storage time
-
-```
-fileStorageTime = %s"0" / (%s"1" storageHours)
-storageHours = 4*4 OCTET     ; Word32, network byte order
-```
-
-The storage time is an optional number of hours. Absent (`%s"0"`), or present as zero hours, requests the maximum the server allows for the presented entitlement, or the default maximum when no proof is present. A value above zero requests that number of hours, and the server grants the smaller of it and the maximum.
-
-## Handshake, new XFTP version
-
-The client handshake carries the entitlement proof.
-
-```
-clientHandshake = xftpVersion keyHash optEntitlementProof
-optEntitlementProof = %s"0" / (%s"1" entitlementProof)
-```
-
-`xftpVersion` and `keyHash` are defined by the current XFTP protocol. Version 3 and earlier encode no proof.
-
-The server verifies the proof when it accepts the handshake and keeps the result for the session; further handshakes on the same connection keep that result and verify nothing, so a session costs one verification however many handshakes it makes. A proof that names an entitlement the server does not configure, or one whose expiration passed 24 hours ago or more, is ignored without verification; a proof that fails to verify is logged. In each case the session gets the default maximum. The response is the same in every case, but only a configured, unlapsed name costs a verification, so the handshake latency tells the client which names the server configures.
-
-## Commands
-
-The new protocol version extends FNEW with the storage time.
-
-```
-fnew = %s"FNEW " fileInfo rcvKeys optBasicAuth fileStorageTime
-```
-
-`fileInfo`, `rcvKeys`, and `optBasicAuth` are defined by the current XFTP protocol. Version 3 and earlier encode no `fileStorageTime`, and the server applies the default storage time.
-
-## Responses
-
-FNEW extends the SIDS response with the granted storage.
-
-```
-sndIds = %s"SIDS " senderId rcvIds optGrantedStorageTime
-optGrantedStorageTime = %s"0" / (%s"1" grantedStorageTime)
-grantedStorageTime = grantedExpires
-grantedExpires = %s"T" expiresAt
-expiresAt = 8*8 OCTET        ; Int64, seconds since epoch (absolute UTC instant), network byte order
-```
-
-`grantedExpires` returns the absolute expiration — the same value stored for the file. The sum encoding retains a one-character prefix so further variants can be added. Version 3 and earlier omit `optGrantedStorageTime` entirely; a client decoding such a response reads it as absent. `senderId` and `rcvIds` are defined by the current XFTP protocol.
-
-## Binding
-
-The presentation header binds the proof to the TLS session, so a proof presented on any other session fails to verify.
-
-```
-presHeader = sessionId
-```
-
-`sessionId` is the TLS session identifier, the TLS unique channel binding. Both sides take it from the connection: the client has it once TLS is established, and the client checks that the identifier the server sends in its handshake matches.
-
-Binding to the session is what stops a proof being replayed by another client. A proof is not bound to a file, because the entitlement belongs to the user and authorises everything the client does in that session.
-
-BBS proofs of one credential are unlinkable, but every proof discloses the same `entExpires`, `entName` and `entExtra`, so a server can link the sessions of one holder by that triple, and two servers can correlate them. The issuing service decides how identifying it is: `entExpires` set to the same instant for everyone who buys in the same period, and an `entExtra` that carries nothing per-holder, make the holders of that period indistinguishable.
-
-## Maximum storage time
-
-The server configures a maximum storage time for each entitlement name, and a default maximum for requests with no proof. Each maximum is a number of hours. The server exits at startup if any name's maximum is below the default, so a proof never reduces the allowed time. The server honours an entitlement for 24 hours after its expiration; past that grace it is treated as no proof.
-
-If the requested time exceeds the maximum, the server stores the file for the maximum and does not reject the request. The expiration is rounded up to the hour, stored, and returned as `grantedExpires`.
-
-## Encoding primitives
-
-```
-shortString = length *OCTET   ; 0-255 bytes
-largeString = length2 *OCTET
-length = 1*1 OCTET
-length2 = 2*2 OCTET           ; Word16, network byte order
-```
-
-`senderId`, `rcvIds`, `fileInfo`, `sndKey`, `digest`, `rcvKeys`, `optBasicAuth`, and `sessionId` are defined by the current XFTP and SMP protocols.

scripts/main/simplex-servers-update

@@ -237,11 +237,11 @@ checks() {
     exit 1
   fi
 
-  mkdir -p "$path_conf_info" "$path_tmp_bin"
-
   check_versions
   check_distro
 
+  mkdir -p $path_conf_info $path_tmp_bin
+
   return 0
 }
 

scripts/resolver/.env.example

@@ -1,32 +0,0 @@
-# Copy to .env (next to docker-compose.yml) only to change a setting; without it
-# docker-compose.yml uses the values shown here.
-
-# ============================================================================
-# Chain settings.
-# ============================================================================
-
-# Ethereum network: mainnet (the SNRC `.testing` contracts live on mainnet)
-# or holesky (test). Mainnet full sync needs ~1 day and ~1.2 TB NVMe.
-NETWORK=mainnet
-
-# Beacon checkpoint-sync URL — used ONCE on first sync. Must expose the heavy
-# /eth/v2/debug/beacon/states/finalized endpoint (generic beacon APIs do not;
-# use a dedicated checkpoint provider). List: https://eth-clients.github.io/checkpoint-sync-endpoints/
-#   mainnet: https://mainnet-checkpoint-sync.attestant.io  (also beaconstate.info, sync-mainnet.beaconcha.in)
-#   holesky: https://checkpoint-sync.holesky.ethpandaops.io
-TRUSTED_NODE_URL=https://mainnet-checkpoint-sync.attestant.io
-
-# ============================================================================
-# Optional overrides — leave commented unless you need to change them.
-# ============================================================================
-
-# Nimbus NAT (default: any). For a stable public node set an explicit IP:
-#   NAT=extip:1.2.3.4        # your public IPv4: curl -s ifconfig.me
-
-# Resolver (defaults in the README's "Load and scaling" and "Logs" sections)
-#   SNRC_WORKERS=4                          # worker processes; default CPU count, at most 4
-#   SNRC_RPC_TIMEOUT=5                      # seconds per request to reth
-#   SNRC_LOG_FORMAT=json                    # default text (key=value)
-#   SNRC_LOG_COLOR=always                   # coloured `docker compose logs`
-#   SNRC_LOG_LEVEL=debug                    # also log health checks and idle connections
-#   SNRC_TRUSTED_PROXIES=172.16.0.0/12      # behind a reverse proxy on the host; see README

scripts/resolver/README.md

@@ -1,469 +0,0 @@
-# Self-hosted SNRC stack
-
-One `docker compose up` runs the self-hosted SimpleX Namespace (SNRC) backend
-against **Ethereum mainnet** (where the `.testing` contracts live):
-
-| # | Component | What it does |
-|---|---|---|
-| 1 | **reth + nimbus** | self-hosted Ethereum node (`--minimal` — enough for the resolver's `eth_call` at chain head) |
-| 2 | **resolver** | the REST resolver the smp-server's `[NAMES]` role queries (`snrc-resolve.py`) |
-
-## Requirements
-
-- **Docker** + Compose v2.
-- **≥ 300 GB NVMe SSD** for `reth --minimal` (~260 GB on mainnet; TLC, not QLC
-  — QLC stalls during sync) + **32 GB RAM**, fast multi-core CPU.
-- **~1 day** for the initial reth sync. The resolver returns errors until reth
-  has caught up — that's expected.
-- Firewall: open p2p ports `30303` (tcp/udp) and `9000` (tcp/udp).
-
-## 1. Configure
-
-The defaults in `docker-compose.yml` work as-is:
-
-```sh
-NETWORK=mainnet                                               # default
-TRUSTED_NODE_URL=https://mainnet-checkpoint-sync.attestant.io # default
-```
-
-To override these or NAT, copy `.env.example` to `.env` and edit it; `.env` is
-not committed.
-
-## 2. Run
-
-```sh
-cd scripts/resolver
-docker compose up -d
-docker compose logs -f reth resolver
-```
-
-`depends_on` handles ordering automatically (start node → start resolver).
-
-## 3. Wait for the node to sync
-
-```sh
-docker compose logs --tail=20 reth
-```
-
-This is the long pole (~1 day on mainnet). Until reth is synced the resolver
-returns `502`.
-
-## Verify
-
-Run these once the stack is up (the node-dependent ones pass after sync):
-
-**1. reth is reachable and reporting a block:**
-```sh
-curl -s -X POST http://127.0.0.1:8545 \
-  -H 'content-type: application/json' \
-  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' | jq
-```
-
-**2. resolver is healthy:**
-```sh
-curl -s http://127.0.0.1:8000/health | jq
-# → {"ok": true, "rpc": "http://reth:8545", "registries": {"testing": "0x…", "simplex": ""},
-#    "blockNumber": 23400000, "chainLagSeconds": 12}
-```
-
-`chainLagSeconds` is how far the node is behind the wall clock. A resolver that
-is reachable and answering can still be hours behind, and every name it reports
-is that stale. `null` means the node could not be reached.
-
-**3. resolver resolves a live name** (`foobar.testing` is a populated test name):
-```sh
-curl -s http://127.0.0.1:8000/resolve/foobar.testing | jq
-# → {"name":"foobar.testing","nickname":"Foo","simplexContact":["https://smp16.simplex.im/a#…"], … }
-```
-
-**4. the route your router will call** (check 3 passes on an older resolver too):
-```sh
-curl -s http://127.0.0.1:8000/v2/resolve/foobar.testing | jq
-# → {"type":"registered","expires":1780000000,"graceUntil":…,"nameRecord":{…}}
-```
-
-**Wire your smp-server:** in its `[NAMES]` section set
-`resolver_endpoint: http://127.0.0.1:8000` (no auth needed for loopback).
-
-## Ports (all loopback unless noted)
-
-| Service | Host | Purpose |
-|---|---|---|
-| reth JSON-RPC | `127.0.0.1:8545` | smp-server RPC |
-| reth p2p | `:30303` tcp/udp | Ethereum sync (open on firewall) |
-| nimbus p2p | `:9000` tcp/udp | beacon sync (open on firewall) |
-| nimbus REST | `127.0.0.1:5052` | beacon API |
-| **resolver** | `127.0.0.1:8000` | SNRC REST (`/v2/resolve`, `/resolve`, `/health`) |
-
-## Caveats
-
-- **All images track `:latest`** (reth, nimbus) — you get upstream fixes on each
-  `docker compose pull`; re-run the verify checks after pulling.
-- All ports bind to loopback; expose only what you put behind a TLS reverse proxy.
-
-## Teardown
-
-```sh
-docker compose down       # stop, keep all state
-docker compose down -v    # also wipe volumes → full re-sync
-```
-
-`down -v` wipes the chain data (full re-sync on the next `up`).
-
----
-
-## Resolver API reference
-
-The resolver (`snrc-resolve.py`, host `127.0.0.1:8000`) is also runnable
-standalone for local dev (no Docker), via [`uv`](https://docs.astral.sh/uv/):
-
-```sh
-uv run scripts/resolver/service/snrc-resolve.py  # defaults to local reth + mainnet .testing
-```
-
-Tests and lint run from `scripts/resolver/service`:
-
-```sh
-uv run pytest
-uv run ruff check .
-```
-
-Dependencies are locked in `uv.lock`; the Docker image installs exactly those.
-
-Three routes, versioned separately from the protocol so each only changes when
-its own shape does:
-
-| Route | Called by | Answers |
-|---|---|---|
-| `/v2/resolve/<query>` | routers from SMP v22 | a `NameRegistration` |
-| `/resolve/<name>` | routers before SMP v22 | a name record, flat |
-| `/health` | anyone | readiness |
-
-`/v1/resolve/<name>` is an alias for `/resolve/<name>`.
-
-### v2: `/v2/resolve/<query>`
-
-The body is the SMP protocol's `NameResponse`, which the router decodes as is
-and forwards; translating the registry's model to it is this resolver's job. It
-is the registration below, plus `lastBlockTs`, the timestamp of the block it was read
-at: a node that lags answers with names it has not seen registered yet.
-Its `type` is `registered`, `available` or `reserved`, and the fields each one
-carries are specified once, in the **Name response** section of
-[`protocol/simplex-messaging.md`](../../protocol/simplex-messaging.md). It is
-the wire format, so it is documented with the wire.
-
-Two things follow from that and are worth stating here. Expiry and grace belong
-to the registration, not to the record: `expires` and `graceUntil` sit beside
-`nameRecord`, not inside it. And a name nobody holds is not an error: it
-answers 200 with `type: available` and its price, so no status code from this
-route means "not registered".
-
-| Status | Meaning |
-|---|---|
-| 200 | a registration: `registered`, `available` or `reserved` |
-| 400 | `tldNotConfigured`, `notFullyQualified` |
-| 502 | `noPriceOracle`, `labelNotRecorded`, an unreadable status, or `upstreamError` |
-
-Error bodies carry `name` and a fixed `error` code to branch on. Only
-`upstreamError` adds a `message`; the v1 route always adds one.
-
-`labelNotRecorded` means the registrar holds the name but never recorded its
-label, so a hashed query cannot be answered with a name. See
-[Querying by labelhash](#querying-by-labelhash).
-
-A subname reports the expiry and grace of the 2LD above it, since that is what
-bounds its lifetime. A subname nobody created reports as not registered.
-
-### v1: `/resolve/<name>`
-
-What routers before SMP v22 call. Its shape is unrelated to v2's: the record is
-flat, and `status`, `expires`, `graceEnds`, `reasonCode` and `reason` sit
-alongside its fields.
-
-#### v1 response shape
-
-```jsonc
-{
-  "name": "foobar.testing",
-  "nickname": "Foo", "website": "https://foo.bar", "location": "",
-  "simplexContact": ["https://smp16.simplex.im/a#…", "https://smp11…"],  // primary first, fallbacks after
-  "simplexChannel": [],
-  "eth": null, "btc": "bc1q…", "xmr": "4ANz…", "dot": "139G…",
-  "owner": "0xd83b…", "resolver": "0x80fa…",
-  "status": "registered",      // registered | grace | expired | unregistered | unknown
-  "expires": 1780000000,       // Unix seconds; when the registration ends
-  "graceEnds": 1787776000,     // expires + GRACE_PERIOD; last moment the owner can renew
-  "reasonCode": null,          // set when the name is held back as well
-  "reason": null               // set when the name is held back as well
-}
-```
-
-`simplexContact`/`simplexChannel` are arrays (a name can advertise multiple SMP
-servers; clients try them in order). On-chain they're a single `;`-separated
-text record; the resolver splits/trims/drops-empties. Address encodings are
-canonical per chain (EIP-55 / bech32 / SS58 / Monero-base58). Subnames work
-identically (`bar.foobar.testing`).
-
-#### v1 registration status and expiry
-
-A response carries `status`, `expires` and `graceEnds` whenever the resolver
-read them, a successful resolve included, so a client that has just resolved a
-name already knows when it expires. Both timestamps are Unix seconds, and
-`null` when they could not be read.
-
-| `status` | Meaning |
-|---|---|
-| `registered` | live; `expires` is when that ends |
-| `grace` | lapsed, but only the previous owner may renew it, until `graceEnds` |
-| `expired` | lapsed and past grace; anyone may register it |
-| `unregistered` | never registered, and free to take |
-| `unknown` | no `SNRC_REGISTRAR_<TLD>` configured, so status could not be read |
-
-A reservation is orthogonal to the status: a name held back by the registry
-carries `reasonCode` and `reason` whether or not it is registered.
-
-`grace` and `expired` are told apart by the registrar's own `available(id)`
-rule, `expires + GRACE_PERIOD < now`. `GRACE_PERIOD` is read from the contract
-rather than assumed, and `now` is the latest block's timestamp rather than the
-host clock, which the registrar compares against too, so a machine with a wrong
-clock cannot misreport a registration. That rule alone is not enough: it also
-holds for a name nobody ever registered (`0 + GRACE_PERIOD < now`), so a zero
-expiry is what separates *never registered* from *registered and since
-released*.
-
-A subname reports the status of the 2LD above it, which is only as good as the
-name it sits under. A subname nobody created answers 404 `unregistered`.
-
-#### v1 errors
-
-Every non-2xx body carries two fields: `error` is a fixed code to branch on,
-and `message` is a sentence for a human. Match on `error`, never on `message`,
-which is free to change.
-
-```jsonc
-{"name": "nope.testing", "error": "unregistered",
- "message": "this name has never been registered",
- "status": "unregistered", "expires": null, "graceEnds": null}
-```
-
-The codes are `tldNotConfigured`, `notFullyQualified`, `unregistered`,
-`expired`, `noSuchRoute` and `upstreamError`. When the registration is what went
-wrong, `error` and `status` hold the same value, so one field is enough to read.
-
-`upstreamError` says only which exception type the RPC call raised. The text
-goes to the resolver's log instead, because `SNRC_RPC` can carry a provider key
-and the exception can carry the URL it failed on. It is also the answer
-when a registrar, controller or oracle address has no contract behind it: the
-empty reply is refused rather than read as zero, which would make every name
-look free.
-
-#### v1 status codes
-
-| Status | Meaning |
-|---|---|
-| 200 | resolved (`status` is `registered` or `grace`, or `unknown` when no registrar is configured) |
-| 400 | TLD not configured, or not a fully-qualified name |
-| 404 | `unregistered` |
-| 410 | `expired`: lapsed and past grace, so anyone may take it |
-| 502 | upstream RPC error / reth not synced |
-
-### Querying by labelhash
-
-A client asking whether a name is free is usually about to register it, and
-whoever runs the resolver could register it first. To avoid that, send the
-keccak hash of the label in ENS's `[<64 hex>]` form instead of the label:
-
-```sh
-# instead of /resolve/acme.testing
-curl -s "http://127.0.0.1:8000/resolve/[$(printf acme | keccak-256sum | cut -d' ' -f1)].testing"
-```
-
-namehash is `keccak(parent || keccak(label))`, so this reaches the same node and
-returns the same record. The registrar keys `nameExpires` and `reservedNames` on
-the labelhash too, so the status fields do not need the label either. The
-resolver learns the name only by guessing the label and hashing it.
-
-Only the second-level label is a registry key, and `status` decodes a bracket
-there at any depth. The record does not: a bracket is decoded only in a
-two-label name, so `sub.[<hash>].testing` is not a supported query. Subname
-labels stay text; a bracket label left of the 2LD is an ordinary label. Routers
-from v22 send every 2LD this way, so a registrable name normally never reaches
-this service.
-
-On v2, read `type`: only `available` means the name is free. On v1, read
-`status`: a name is free on `unregistered` (404) and on `expired` (410), every
-other status means somebody holds it, and a `reasonCode` means the registry will
-refuse it whatever the status says.
-
-The hash must be keccak-256. `openssl dgst -sha3-256` and `sha3sum` compute
-SHA3-256, a different function that returns 64 valid-looking hex characters
-pointing at the wrong node.
-
-The resolver lowercases the query before matching, so uppercase hex works too.
-Clients that refuse raw brackets in a path can percent-encode them as `%5B` and
-`%5D`.
-
-Brackets cannot collide with a real name: they are invalid in a normalised ENS
-name, and a `[<64 hex>]` label is 66 bytes against the registrar's
-`maxLabelLength` of 63. A plain `0x…` label is not treated as a hash, since that
-is an ordinary, registrable name.
-
-Only 2LDs can be queried this way, as only a 2LD can be raced for: subnames are
-created by the 2LD's owner. A bracket label in a subname is hashed as written,
-so it points at a node nobody can own. ENS tooling accepts the bracketed form at
-any depth; this resolver does not, on purpose.
-
-This hides interest in a name and nothing else: the registration itself is
-public, and commit-reveal covers that step. A short or well-known label is easy
-to guess by hashing candidates, and the reveal publishes the labelhash, so an
-operator who logged the query can match it to the name afterwards.
-
-### What a name costs
-
-The controller's `prices()` names the price oracle, so no extra configuration is
-needed beyond `SNRC_CONTROLLER_<TLD>`. A `SimplexPriceOracle` exposes its curve
-through `prices()`, in US cents per year, which is the unit this API carries.
-
-An ENS-shaped oracle exposes only `price1Letter()`..`price6Letter()`, in attoUSD
-per second, and charges a premium on a lapsed name that it does not expose. A
-quote from one is therefore only safe for a name that was never registered: an
-`expired` name gets no price rather than one below what the registrar charges.
-
-**Set `SNRC_CONTROLLER_<TLD>` wherever `SNRC_REGISTRAR_<TLD>` is.** Without a
-controller there is no oracle, so no name can be priced.
-
-**Upgrade this service before the routers that query it.** Routers from SMP v22
-call `/v2/resolve`, which an older resolver does not serve. Every name then
-answers `ERR NAME RESOLVER "HTTP 404"` until this service is upgraded, while
-`/health` still reports it as ready.
-
-### Why a name is reserved
-
-v2 carries the controller's reason as `reservedReason`. v1 carries the same word
-as `reasonCode`, plus `reason`, an English sentence for a human reading this API.
-Clients should branch on the word and phrase it themselves, in the user's
-language.
-
-| `reasonCode` | Meaning |
-|---|---|
-| `internal` | reserved for SimpleX |
-| `trademark` | reserved to protect a trademark |
-| `community` | reserved for the community |
-| `unknown` | a reason added to the contract after this resolver; still reserved |
-
-These are `SimplexController.Reason`, where 0 means not reserved. A controller
-from before the enum stores a boolean, whose `true` decodes as 1, which is why
-1 reads as `internal`, so nothing needs migrating.
-
-### Configuring addresses
-
-The resolver reads three contracts, each configured per TLD.
-
-The **registry** answers who owns a node, and `/resolve` reads the records from
-it. The **registrar** (ERC-721) holds `nameExpires` and `GRACE_PERIOD`, which
-is where every expiry field comes from, and `labelOf`, which is how a hashed
-query is answered with a name. A name registered without recording its label
-cannot answer one, and `/v2/resolve` refuses it rather than answer with a name
-the client will reject. With no registrar for a TLD, `/resolve` still works and
-reports `"status": "unknown"`. The **controller** holds
-`reservedNames`, which is where `reasonCode` comes from. With no controller a
-held-back name reads as not reserved, and no name can be priced.
-
-All three default to the mainnet `.testing` deployment. `.simplex` is unset
-until it is deployed.
-
-The controller default is the **proxy**, not `SimplexControllerImpl`. Storage
-lives in the proxy, so the implementation address answers nothing. The two
-deployment files use different names for that proxy:
-`deployments.mainnet.testing.json` records it under the ENS role name
-`ETHRegistrarController`, and `verification.mainnet.testing.json` calls it
-`SimplexControllerProxy`. Both are the same address, and it is the one used
-here.
-
-To override any of them, set `SNRC_REGISTRY_<TLD>`, `SNRC_REGISTRAR_<TLD>` or
-`SNRC_CONTROLLER_<TLD>` on the `resolver` service in `docker-compose.yml`, or
-as env vars when you run the script directly.
-
-### Load and scaling
-
-A lookup reads the chain in at most three rounds: the name's status and
-registry entries, then its record from its resolver, then prices for a name
-that is free. The node runs the calls of a JSON-RPC batch one after another,
-so the contract reads of a round go to the chain as one `eth_call` to
-[Multicall3](https://github.com/mds1/multicall). A node that is slow for a
-moment therefore delays a lookup a few times, not once per read. Connections to
-the node are kept open and reused.
-
-The resolver answers HTTP/1.1, so each smp-server keeps its connections to it
-open instead of connecting for every lookup. An idle connection is closed after
-60 s; the smp-server drops its own idle ones after 30 s, so it closes them first.
-
-Each access log line ends with how long the request took, so slow requests
-show up in `docker compose logs resolver`.
-
-| Variable | Default | Meaning |
-|---|---|---|
-| `SNRC_WORKERS` | CPU count, at most 4 | processes sharing the port. If one exits, the others stop, so the container restarts |
-| `SNRC_RPC_TIMEOUT` | `5` | seconds to wait for each request to the node; the smp-server gives up after 3 |
-| `SNRC_MULTICALL` | `0xcA11bde05977b3631167028862bE2a173976CA11` | Multicall3 address. If it does not answer, each round is sent as a plain batch and the log says so once |
-
-### Logs
-
-Each event is one line: UTC time, level, event name, then `key=value` fields.
-
-```
-2026-09-26T08:06:19.992Z INFO  request  client=203.0.113.7 worker=12 method=GET path=/v2/resolve/[4fdd…].testing status=200 bytes=701 ms=35
-2026-09-26T08:06:20.051Z WARN  upstream_error  name=foobar.testing error=ConnectionRefusedError message="[Errno 111] Connection refused"
-```
-
-| Event | Level | Meaning |
-|---|---|---|
-| `listening` | info | started: address, workers, RPC endpoint, registries, trusted proxies |
-| `request` | info | one answered request; `/health` only at debug |
-| `upstream_error` | warn | a read from the node failed; the caller got 502 |
-| `multicall_unavailable` | warn | Multicall3 did not answer; rounds are sent as plain batches |
-| `client_gone` | warn | the caller hung up before the answer, usually an smp-server past its timeout |
-| `http_error` | warn | a malformed request; idle keep-alive timeouts only at debug |
-| `request_failed` | error | a request failed unexpectedly, with its traceback |
-| `worker_exited`, `worker_failed` | error | a worker stopped; the others stop too, so the container restarts |
-| `stopping` | info | stopped by a signal |
-
-| Variable | Default | Meaning |
-|---|---|---|
-| `SNRC_LOG_FORMAT` | `text` | `text` (`key=value`) or `json`, one object per line |
-| `SNRC_LOG_COLOR` | `auto` | `auto` colours only a terminal, `always`, `never`. Docker output is no terminal, so set `always` for coloured `docker compose logs` |
-| `SNRC_LOG_LEVEL` | `info` | `debug` also logs health checks and idle connections closing |
-| `SNRC_TRUSTED_PROXIES` | none | addresses or CIDRs of reverse proxies whose `X-Forwarded-For` names the client |
-
-With docker compose, set these, and those in "Load and scaling", in `.env`
-(`.env.example` lists them).
-
-Behind a reverse proxy such as Caddy on the host, the resolver sees the Docker
-network's gateway rather than the client. Trust that gateway, and the logged
-client is the last `X-Forwarded-For` address no trusted proxy added, so a caller
-through the proxy cannot claim another address. Everything else on the host that
-connects to `127.0.0.1:8000`, the smp-servers included, arrives through the same
-gateway and could set the logged address too; run the proxy in the compose network
-and trust only its address to avoid that.
-
-The gateway is
-`docker network inspect resolver_default --format '{{(index .IPAM.Config 0).Gateway}}'`,
-with the network named after the compose project. It changes when the network is
-recreated, as `docker compose down` does, so trust one of, in `.env`:
-
-- `SNRC_TRUSTED_PROXIES=172.16.0.0/12`: Docker's default pools for compose
-  networks, whatever subnet it picks. This also trusts every other container on the
-  host. Add the gateway too if Docker ever gives the network a `192.168.x.x` one.
-- a pinned subnet, so the gateway stays fixed, and only that gateway:
-
-  ```yaml
-  networks:
-    default:
-      ipam:
-        config:
-          - subnet: 172.30.0.0/24
-            gateway: 172.30.0.1
-  ```
-
-  in `docker-compose.yml`, with `SNRC_TRUSTED_PROXIES=172.30.0.1`, on a subnet no
-  other network on the host uses.

scripts/resolver/docker-compose.yml

@@ -1,177 +0,0 @@
-services:
-  # One-shot setup (runs as root): generates /jwt/jwt.hex and chowns the
-  # nimbus-data volume to UID 1000 (the user Nimbus runs as inside its image).
-  # Without this chown Nimbus gets "Permission denied" on its data dir
-  # because docker creates fresh named volumes owned by root.
-  init:
-    image: alpine:latest
-    volumes:
-      - jwt:/jwt
-      - nimbus-data:/nimbus-data
-    command: >
-      sh -c '
-        set -e;
-        if [ ! -f /jwt/jwt.hex ]; then
-          apk add --no-cache openssl >/dev/null;
-          openssl rand -hex 32 | tr -d "\n" > /jwt/jwt.hex;
-          chmod 644 /jwt/jwt.hex;
-          echo "Generated /jwt/jwt.hex";
-        else
-          echo "jwt.hex already exists";
-        fi;
-        chown 1000:1000 /nimbus-data;
-        echo "Chowned /nimbus-data to 1000:1000";
-      '
-    restart: "no"
-
-  # One-shot: fetches a recent finalised checkpoint into the Nimbus data dir
-  # using the trustedNodeSync subcommand. Skipped if the data dir is already
-  # initialised, so subsequent compose-ups are no-ops.
-  nimbus-checkpoint-sync:
-    image: statusim/nimbus-eth2:multiarch-latest
-    depends_on:
-      init:
-        condition: service_completed_successfully
-    volumes:
-      - nimbus-data:/home/user/nimbus-eth2/build/data
-    entrypoint:
-      - sh
-      - -c
-      - |
-        if [ -d /home/user/nimbus-eth2/build/data/${NETWORK:-mainnet}/db ]; then
-          echo "Nimbus data dir already initialised — skipping checkpoint sync";
-          exit 0;
-        fi;
-        /home/user/nimbus-eth2/build/nimbus_beacon_node trustedNodeSync \
-          --network=${NETWORK:-mainnet} \
-          --data-dir=/home/user/nimbus-eth2/build/data/${NETWORK:-mainnet} \
-          --trusted-node-url=${TRUSTED_NODE_URL:-https://mainnet-checkpoint-sync.attestant.io} \
-          --backfill=false
-    restart: "no"
-
-  # One-shot: downloads a pre-synced snapshot from snapshots.reth.rs into the
-  # Reth data dir. Turns a multi-day from-scratch sync into a ~hour download.
-  # Skipped if the data dir is already initialised — re-runs are no-ops.
-  # Privacy note: snapshots.reth.rs sees this download (operator existence).
-  # Subsequent eth_call traffic stays local.
-  reth-snapshot-init:
-    image: ghcr.io/paradigmxyz/reth:latest
-    depends_on:
-      init:
-        condition: service_completed_successfully
-    volumes:
-      - reth-data:/data
-    entrypoint:
-      - sh
-      - -c
-      - |
-        if [ -f /data/.snapshot-done ] || [ -d /data/db ]; then
-          echo "Reth data already initialised — skipping snapshot download";
-          exit 0;
-        fi;
-        echo "Downloading Reth ${NETWORK:-mainnet} --minimal snapshot...";
-        reth download --datadir /data --chain ${NETWORK:-mainnet} --minimal && \
-        touch /data/.snapshot-done && \
-        echo "Snapshot download complete"
-    restart: "no"
-
-  reth:
-    image: ghcr.io/paradigmxyz/reth:latest
-    depends_on:
-      reth-snapshot-init:
-        condition: service_completed_successfully
-    volumes:
-      - reth-data:/data
-      - jwt:/jwt:ro
-    ports:
-      # JSON-RPC for smp-server. Bound to loopback — put Caddy in front for remote access.
-      - "127.0.0.1:8545:8545"
-      # p2p (Ethereum network). Open these on your firewall for sync.
-      - "30303:30303/tcp"
-      - "30303:30303/udp"
-    command: >
-      node
-      --datadir /data
-      --chain ${NETWORK:-mainnet}
-      --minimal
-      --authrpc.jwtsecret /jwt/jwt.hex
-      --authrpc.addr 0.0.0.0 --authrpc.port 8551
-      --http
-      --http.addr 0.0.0.0 --http.port 8545
-      --http.api eth,net
-      --rpc.gascap 50000000
-      --port 30303
-      --discovery.port 30303
-    restart: unless-stopped
-
-  nimbus:
-    image: statusim/nimbus-eth2:multiarch-latest
-    depends_on:
-      nimbus-checkpoint-sync:
-        condition: service_completed_successfully
-    volumes:
-      - nimbus-data:/home/user/nimbus-eth2/build/data
-      - jwt:/jwt:ro
-    ports:
-      - "9000:9000/tcp"
-      - "9000:9000/udp"
-      - "127.0.0.1:5052:5052"
-    command: >
-      --network=${NETWORK:-mainnet}
-      --data-dir=/home/user/nimbus-eth2/build/data/${NETWORK:-mainnet}
-      --el=http://reth:8551
-      --jwt-secret=/jwt/jwt.hex
-      --non-interactive
-      --rest --rest-address=0.0.0.0 --rest-port=5052
-      --nat=${NAT:-any}
-    restart: unless-stopped
-
-  # SNRC REST resolver. Talks to reth on the compose-internal network,
-  # exposes /resolve and /health on 127.0.0.1:8000 by default. The
-  # smp-server points its [NAMES] resolver_endpoint at this URL.
-  # To change the host port, edit the LEFT side of the port mapping below.
-  resolver:
-    image: snrc-resolve:local
-    build:
-      context: ./service
-      dockerfile: Dockerfile
-    depends_on:
-      # reth's `service_started` is sufficient — the resolver tolerates
-      # eth_call failures gracefully (returns 502 with the error body), so
-      # starting before reth has finished snapshot replay just yields a few
-      # 502s until the chain is queryable. The upstream reth image doesn't
-      # ship a HEALTHCHECK, so we can't gate on healthy.
-      reth:
-        condition: service_started
-    environment:
-      SNRC_RPC: http://reth:8545
-      SNRC_BIND: 0.0.0.0
-      # Registry addresses cascade through the script's own defaults
-      # (mainnet `.testing`; `.simplex` unconfigured). Set explicitly here
-      # only if you're deploying against a different network or contract.
-      # SNRC_REGISTRY_TESTING: 0x...
-      # SNRC_REGISTRY_SIMPLEX: 0x...
-      # Registrar and controller, same cascade. Without the registrar `status`
-      # is "unknown"; without the controller a reserved name is "unregistered".
-      # SNRC_REGISTRAR_TESTING: 0x...
-      # SNRC_REGISTRAR_SIMPLEX: 0x...
-      # SNRC_CONTROLLER_TESTING: 0x...
-      # SNRC_CONTROLLER_SIMPLEX: 0x...
-      # Set in .env (see .env.example); unset or empty means the resolver's default.
-      SNRC_WORKERS: ${SNRC_WORKERS:-}
-      SNRC_RPC_TIMEOUT: ${SNRC_RPC_TIMEOUT:-}
-      SNRC_LOG_FORMAT: ${SNRC_LOG_FORMAT:-}
-      SNRC_LOG_COLOR: ${SNRC_LOG_COLOR:-}
-      SNRC_LOG_LEVEL: ${SNRC_LOG_LEVEL:-}
-      SNRC_TRUSTED_PROXIES: ${SNRC_TRUSTED_PROXIES:-}
-    ports:
-      - "127.0.0.1:8000:8000"
-    # a line per lookup; the local driver keeps 5 rotated files of 20 MB
-    logging:
-      driver: local
-    restart: unless-stopped
-
-volumes:
-  reth-data:
-  nimbus-data:
-  jwt:

scripts/resolver/service/.dockerignore

@@ -1,5 +0,0 @@
-.venv
-**/__pycache__
-.pytest_cache
-.ruff_cache
-test_snrc_resolve.py

scripts/resolver/service/.python-version

@@ -1 +0,0 @@
-3.13

scripts/resolver/service/Dockerfile

@@ -1,49 +0,0 @@
-# syntax=docker/dockerfile:1.7
-# ---------- builder ----------
-# The official uv image (Astral) on top of a slim Python base installs the
-# locked dependencies into a portable .venv for the runtime stage.
-# Pinned to the uv that wrote uv.lock (its bookworm images stopped at 0.9.30), on the
-# runtime's Debian release, so the .venv's Python path exists there.
-FROM ghcr.io/astral-sh/uv:0.12.16-python3.13-trixie-slim AS builder
-
-ENV UV_LINK_MODE=copy \
-    UV_COMPILE_BYTECODE=1 \
-    UV_PYTHON_DOWNLOADS=never \
-    UV_NO_PROGRESS=1
-
-WORKDIR /app
-
-# Dependencies first (separate layer), exactly as locked: script edits don't bust this cache.
-COPY pyproject.toml uv.lock .python-version ./
-RUN --mount=type=cache,target=/root/.cache/uv \
-    uv sync --frozen --no-dev --no-install-project
-
-# Script is added after the dep layer for cache friendliness.
-COPY snrc-resolve.py ./
-
-# ---------- runtime ----------
-# Slim runtime — only the venv + script. No uv, no apt.
-FROM python:3.13-slim-trixie AS runtime
-
-ENV PYTHONUNBUFFERED=1 \
-    PYTHONDONTWRITEBYTECODE=1 \
-    PATH="/app/.venv/bin:$PATH"
-
-# Non-root user (matches resolver privacy posture: it has no need for root).
-RUN groupadd --system --gid 10001 snrc && \
-    useradd  --system --uid 10001 --gid snrc --no-create-home --shell /usr/sbin/nologin snrc
-
-WORKDIR /app
-# owned by root, so the service cannot rewrite its own code
-COPY --from=builder /app /app
-
-USER snrc:snrc
-
-EXPOSE 8000
-
-# Liveness check hits the script's own /health route. Three failures mark the
-# container unhealthy; compose does not restart it for that, but `docker ps` shows it.
-HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
-    CMD ["python", "-c", "import urllib.request, sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3).status == 200 else 1)"]
-
-ENTRYPOINT ["python", "snrc-resolve.py"]

scripts/resolver/service/pyproject.toml

@@ -1,22 +0,0 @@
-[project]
-name = "snrc-resolve"
-version = "0.1.0"
-description = "SimpleX Namespace (SNRC) resolver — REST API over ENS-shaped Ethereum registries"
-readme = "README.md"
-requires-python = ">=3.11"
-license = "AGPL-3.0-only"
-dependencies = [
-  "eth-hash[pycryptodome]>=0.7",
-]
-
-[dependency-groups]
-dev = [
-  "pytest>=8",
-  "ruff>=0.6",
-]
-
-[tool.uv]
-package = false
-
-[tool.ruff.lint]
-select = ["E9", "F", "B"]

scripts/resolver/service/uv.lock

@@ -1,156 +0,0 @@
-version = 1
-revision = 3
-requires-python = ">=3.11"
-
-[[package]]
-name = "colorama"
-version = "0.4.6"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
-]
-
-[[package]]
-name = "eth-hash"
-version = "0.8.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/3c/f5/c67fc24f2f676aa9b7ab29679d44f113f314c817207cd4319353356f62da/eth_hash-0.8.0.tar.gz", hash = "sha256:b009752b620da2e9c7668014849d1f5fadbe4f138603f1871cc5d4ca706896b1", size = 12225, upload-time = "2026-03-25T16:36:55.099Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/87/87/b36792150ca0b28e4df683a34be15a61461ca0e349e5b5cf3ec8f694edb9/eth_hash-0.8.0-py3-none-any.whl", hash = "sha256:523718a51b369ab89866b929a5c93c52978cd866ea309192ad980dd8271f9fac", size = 7965, upload-time = "2026-03-25T16:36:54.205Z" },
-]
-
-[package.optional-dependencies]
-pycryptodome = [
-    { name = "pycryptodome" },
-]
-
-[[package]]
-name = "iniconfig"
-version = "2.3.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
-]
-
-[[package]]
-name = "packaging"
-version = "26.3"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },
-]
-
-[[package]]
-name = "pluggy"
-version = "1.6.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
-]
-
-[[package]]
-name = "pycryptodome"
-version = "3.23.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/8e/a6/8452177684d5e906854776276ddd34eca30d1b1e15aa1ee9cefc289a33f5/pycryptodome-3.23.0.tar.gz", hash = "sha256:447700a657182d60338bab09fdb27518f8856aecd80ae4c6bdddb67ff5da44ef", size = 4921276, upload-time = "2025-05-17T17:21:45.242Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/04/5d/bdb09489b63cd34a976cc9e2a8d938114f7a53a74d3dd4f125ffa49dce82/pycryptodome-3.23.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:0011f7f00cdb74879142011f95133274741778abba114ceca229adbf8e62c3e4", size = 2495152, upload-time = "2025-05-17T17:20:20.833Z" },
-    { url = "https://files.pythonhosted.org/packages/a7/ce/7840250ed4cc0039c433cd41715536f926d6e86ce84e904068eb3244b6a6/pycryptodome-3.23.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:90460fc9e088ce095f9ee8356722d4f10f86e5be06e2354230a9880b9c549aae", size = 1639348, upload-time = "2025-05-17T17:20:23.171Z" },
-    { url = "https://files.pythonhosted.org/packages/ee/f0/991da24c55c1f688d6a3b5a11940567353f74590734ee4a64294834ae472/pycryptodome-3.23.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4764e64b269fc83b00f682c47443c2e6e85b18273712b98aa43bcb77f8570477", size = 2184033, upload-time = "2025-05-17T17:20:25.424Z" },
-    { url = "https://files.pythonhosted.org/packages/54/16/0e11882deddf00f68b68dd4e8e442ddc30641f31afeb2bc25588124ac8de/pycryptodome-3.23.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eb8f24adb74984aa0e5d07a2368ad95276cf38051fe2dc6605cbcf482e04f2a7", size = 2270142, upload-time = "2025-05-17T17:20:27.808Z" },
-    { url = "https://files.pythonhosted.org/packages/d5/fc/4347fea23a3f95ffb931f383ff28b3f7b1fe868739182cb76718c0da86a1/pycryptodome-3.23.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d97618c9c6684a97ef7637ba43bdf6663a2e2e77efe0f863cce97a76af396446", size = 2309384, upload-time = "2025-05-17T17:20:30.765Z" },
-    { url = "https://files.pythonhosted.org/packages/6e/d9/c5261780b69ce66d8cfab25d2797bd6e82ba0241804694cd48be41add5eb/pycryptodome-3.23.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9a53a4fe5cb075075d515797d6ce2f56772ea7e6a1e5e4b96cf78a14bac3d265", size = 2183237, upload-time = "2025-05-17T17:20:33.736Z" },
-    { url = "https://files.pythonhosted.org/packages/5a/6f/3af2ffedd5cfa08c631f89452c6648c4d779e7772dfc388c77c920ca6bbf/pycryptodome-3.23.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:763d1d74f56f031788e5d307029caef067febf890cd1f8bf61183ae142f1a77b", size = 2343898, upload-time = "2025-05-17T17:20:36.086Z" },
-    { url = "https://files.pythonhosted.org/packages/9a/dc/9060d807039ee5de6e2f260f72f3d70ac213993a804f5e67e0a73a56dd2f/pycryptodome-3.23.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:954af0e2bd7cea83ce72243b14e4fb518b18f0c1649b576d114973e2073b273d", size = 2269197, upload-time = "2025-05-17T17:20:38.414Z" },
-    { url = "https://files.pythonhosted.org/packages/f9/34/e6c8ca177cb29dcc4967fef73f5de445912f93bd0343c9c33c8e5bf8cde8/pycryptodome-3.23.0-cp313-cp313t-win32.whl", hash = "sha256:257bb3572c63ad8ba40b89f6fc9d63a2a628e9f9708d31ee26560925ebe0210a", size = 1768600, upload-time = "2025-05-17T17:20:40.688Z" },
-    { url = "https://files.pythonhosted.org/packages/e4/1d/89756b8d7ff623ad0160f4539da571d1f594d21ee6d68be130a6eccb39a4/pycryptodome-3.23.0-cp313-cp313t-win_amd64.whl", hash = "sha256:6501790c5b62a29fcb227bd6b62012181d886a767ce9ed03b303d1f22eb5c625", size = 1799740, upload-time = "2025-05-17T17:20:42.413Z" },
-    { url = "https://files.pythonhosted.org/packages/5d/61/35a64f0feaea9fd07f0d91209e7be91726eb48c0f1bfc6720647194071e4/pycryptodome-3.23.0-cp313-cp313t-win_arm64.whl", hash = "sha256:9a77627a330ab23ca43b48b130e202582e91cc69619947840ea4d2d1be21eb39", size = 1703685, upload-time = "2025-05-17T17:20:44.388Z" },
-    { url = "https://files.pythonhosted.org/packages/db/6c/a1f71542c969912bb0e106f64f60a56cc1f0fabecf9396f45accbe63fa68/pycryptodome-3.23.0-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:187058ab80b3281b1de11c2e6842a357a1f71b42cb1e15bce373f3d238135c27", size = 2495627, upload-time = "2025-05-17T17:20:47.139Z" },
-    { url = "https://files.pythonhosted.org/packages/6e/4e/a066527e079fc5002390c8acdd3aca431e6ea0a50ffd7201551175b47323/pycryptodome-3.23.0-cp37-abi3-macosx_10_9_x86_64.whl", hash = "sha256:cfb5cd445280c5b0a4e6187a7ce8de5a07b5f3f897f235caa11f1f435f182843", size = 1640362, upload-time = "2025-05-17T17:20:50.392Z" },
-    { url = "https://files.pythonhosted.org/packages/50/52/adaf4c8c100a8c49d2bd058e5b551f73dfd8cb89eb4911e25a0c469b6b4e/pycryptodome-3.23.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:67bd81fcbe34f43ad9422ee8fd4843c8e7198dd88dd3d40e6de42ee65fbe1490", size = 2182625, upload-time = "2025-05-17T17:20:52.866Z" },
-    { url = "https://files.pythonhosted.org/packages/5f/e9/a09476d436d0ff1402ac3867d933c61805ec2326c6ea557aeeac3825604e/pycryptodome-3.23.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c8987bd3307a39bc03df5c8e0e3d8be0c4c3518b7f044b0f4c15d1aa78f52575", size = 2268954, upload-time = "2025-05-17T17:20:55.027Z" },
-    { url = "https://files.pythonhosted.org/packages/f9/c5/ffe6474e0c551d54cab931918127c46d70cab8f114e0c2b5a3c071c2f484/pycryptodome-3.23.0-cp37-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:aa0698f65e5b570426fc31b8162ed4603b0c2841cbb9088e2b01641e3065915b", size = 2308534, upload-time = "2025-05-17T17:20:57.279Z" },
-    { url = "https://files.pythonhosted.org/packages/18/28/e199677fc15ecf43010f2463fde4c1a53015d1fe95fb03bca2890836603a/pycryptodome-3.23.0-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:53ecbafc2b55353edcebd64bf5da94a2a2cdf5090a6915bcca6eca6cc452585a", size = 2181853, upload-time = "2025-05-17T17:20:59.322Z" },
-    { url = "https://files.pythonhosted.org/packages/ce/ea/4fdb09f2165ce1365c9eaefef36625583371ee514db58dc9b65d3a255c4c/pycryptodome-3.23.0-cp37-abi3-musllinux_1_2_i686.whl", hash = "sha256:156df9667ad9f2ad26255926524e1c136d6664b741547deb0a86a9acf5ea631f", size = 2342465, upload-time = "2025-05-17T17:21:03.83Z" },
-    { url = "https://files.pythonhosted.org/packages/22/82/6edc3fc42fe9284aead511394bac167693fb2b0e0395b28b8bedaa07ef04/pycryptodome-3.23.0-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:dea827b4d55ee390dc89b2afe5927d4308a8b538ae91d9c6f7a5090f397af1aa", size = 2267414, upload-time = "2025-05-17T17:21:06.72Z" },
-    { url = "https://files.pythonhosted.org/packages/59/fe/aae679b64363eb78326c7fdc9d06ec3de18bac68be4b612fc1fe8902693c/pycryptodome-3.23.0-cp37-abi3-win32.whl", hash = "sha256:507dbead45474b62b2bbe318eb1c4c8ee641077532067fec9c1aa82c31f84886", size = 1768484, upload-time = "2025-05-17T17:21:08.535Z" },
-    { url = "https://files.pythonhosted.org/packages/54/2f/e97a1b8294db0daaa87012c24a7bb714147c7ade7656973fd6c736b484ff/pycryptodome-3.23.0-cp37-abi3-win_amd64.whl", hash = "sha256:c75b52aacc6c0c260f204cbdd834f76edc9fb0d8e0da9fbf8352ef58202564e2", size = 1799636, upload-time = "2025-05-17T17:21:10.393Z" },
-    { url = "https://files.pythonhosted.org/packages/18/3d/f9441a0d798bf2b1e645adc3265e55706aead1255ccdad3856dbdcffec14/pycryptodome-3.23.0-cp37-abi3-win_arm64.whl", hash = "sha256:11eeeb6917903876f134b56ba11abe95c0b0fd5e3330def218083c7d98bbcb3c", size = 1703675, upload-time = "2025-05-17T17:21:13.146Z" },
-]
-
-[[package]]
-name = "pygments"
-version = "2.21.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/49/2e/ced460408999b33da6b31b0021b0f37d329e202d4169aeb164493778f25b/pygments-2.21.0.tar.gz", hash = "sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c", size = 5005329, upload-time = "2026-08-17T08:02:48.824Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", hash = "sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", size = 1250147, upload-time = "2026-08-17T08:02:44.912Z" },
-]
-
-[[package]]
-name = "pytest"
-version = "9.1.1"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
-    { name = "colorama", marker = "sys_platform == 'win32'" },
-    { name = "iniconfig" },
-    { name = "packaging" },
-    { name = "pluggy" },
-    { name = "pygments" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
-]
-
-[[package]]
-name = "ruff"
-version = "0.16.9"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/96/bf/c935ca98e73fe8ce65b87ef08a280c0c1e85295d569228c15e87d8fdfaf1/ruff-0.16.9.tar.gz", hash = "sha256:12b625c6cfba78d285d9f48eda5f053374f1e53cb10ef17342a383750db99161", size = 4948764, upload-time = "2026-09-24T20:37:49.416Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/0d/26/df51322b52ee1ada7eff2d071ea09d11d5c2d1dcc9f02594f5785c4d1635/ruff-0.16.9-py3-none-linux_armv6l.whl", hash = "sha256:95e6f022090368ab3b824c36276839c53b2adf1a3f4c09fefc33dfc400f6da96", size = 10082922, upload-time = "2026-09-24T20:37:13.045Z" },
-    { url = "https://files.pythonhosted.org/packages/a5/27/7bf51f5a7aa375e9f339280a303aab44ca75dc1525f1cdc5991761685b0f/ruff-0.16.9-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:a5f27be168556594a86d2f415db0cf43f5291917849318f873c7e2791f7a8c67", size = 10236360, upload-time = "2026-09-24T20:37:16.049Z" },
-    { url = "https://files.pythonhosted.org/packages/b6/63/09659283f92f02dff45809da194a70da2f688d87c55d8875c4fae3536072/ruff-0.16.9-py3-none-macosx_11_0_arm64.whl", hash = "sha256:1632eb1d6197f33bd00b1acbc5b71009e89a8895c158e2d2b03a834fac964ab6", size = 9892940, upload-time = "2026-09-24T20:37:17.957Z" },
-    { url = "https://files.pythonhosted.org/packages/24/58/98de1b72ec172f5f8f1731236fe21585b3998bf7dfe9fcc44ae9ba626012/ruff-0.16.9-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b3f951b14d865d5952c89d40a5ca07e87abe24fa5453299878411e127748fb1c", size = 10032114, upload-time = "2026-09-24T20:37:19.942Z" },
-    { url = "https://files.pythonhosted.org/packages/c8/7d/f1e17c54ab59d4bad1dce8ee3e22a7a1d0ef4745240decacdcf3832b5bb2/ruff-0.16.9-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:447fc07e1573afff7cb02803462b12b6c8ece7cf10e2cd78565fa6d7a1c0bf8d", size = 9910227, upload-time = "2026-09-24T20:37:21.872Z" },
-    { url = "https://files.pythonhosted.org/packages/34/19/436f647a65075bbd3bab2668b3bdaa5120559b294694018cdcefabbbf30b/ruff-0.16.9-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8a3e039a6a40ed976c491722b60e0ae4a4aa1a86057f540ee7a37a5d19ae9120", size = 10547484, upload-time = "2026-09-24T20:37:24.229Z" },
-    { url = "https://files.pythonhosted.org/packages/03/59/38430a6bc2f6d8095447ac39625cf8b6e9344a47e6c26225c8ba1bff3ffb/ruff-0.16.9-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:4684dded7db60aa57cb118fa158630f5feade4af5782903b6053484bdf9bd129", size = 11412367, upload-time = "2026-09-24T20:37:26.307Z" },
-    { url = "https://files.pythonhosted.org/packages/c8/bd/bbb6d7fc7f208c8b8c50dd5dc8206e4cfdb1e7a8fb852606a3adf370c880/ruff-0.16.9-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d29c934357e45642fda2f34c0b1f4025b4a6c01e15e4bf0016879d60078a142c", size = 10869787, upload-time = "2026-09-24T20:37:28.35Z" },
-    { url = "https://files.pythonhosted.org/packages/bc/b8/9c543074918061abbefc3bd139bee22de35abedb00dde0f2d27288838962/ruff-0.16.9-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a21713e629d3e5bdb2f5c2def1cc7f04f47fa8e1a7eb0571b4a28e1da64bc728", size = 10406494, upload-time = "2026-09-24T20:37:30.624Z" },
-    { url = "https://files.pythonhosted.org/packages/35/7a/5a8851bd146e7ccf8fd4b003f6c75c11f8fbdb0e60673b45097986b6bf41/ruff-0.16.9-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:7baa24ef5fc8e77aa93879e1d3f43754a01ae488e869f1ae30cf431afd4d2452", size = 10590083, upload-time = "2026-09-24T20:37:32.439Z" },
-    { url = "https://files.pythonhosted.org/packages/87/f0/4c3467188f23f806960b46fa76575a7cd0514c9ba90562650b71efc96980/ruff-0.16.9-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:a41aac6230aadfaa133bdfa1614488531ffa3e0837567ae04c0da2058a9c0f9e", size = 10119151, upload-time = "2026-09-24T20:37:34.581Z" },
-    { url = "https://files.pythonhosted.org/packages/15/34/5a4def5adea572ce6aea0bb64f21f928ee317b80e0db747d7979b01d7261/ruff-0.16.9-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:c2529fb5896d49115b0e9aa8f887490b34bbe76baf879ec2264ac59406869ce7", size = 9911544, upload-time = "2026-09-24T20:37:36.796Z" },
-    { url = "https://files.pythonhosted.org/packages/62/5d/d15ebea7499eef9373318c0ee6ca127832927c6529731f6d48e18dce7ca9/ruff-0.16.9-py3-none-musllinux_1_2_i686.whl", hash = "sha256:41e3870277694177429b56406d65dfbdb2c2802c52b715edaf6a0b829c69d4ee", size = 10269884, upload-time = "2026-09-24T20:37:38.857Z" },
-    { url = "https://files.pythonhosted.org/packages/d1/56/c5d3cd119ded7a3c7aba0e961b69cb3df701c91662c98ad694467d060ce1/ruff-0.16.9-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:8adbe4e58af167f767d7b2ba5e83c42e878350796cf78c2f5e14ab9903a92588", size = 10749366, upload-time = "2026-09-24T20:37:41.042Z" },
-    { url = "https://files.pythonhosted.org/packages/ac/fe/734ec7527029ac757ecf821f143c9f3fcf69149c21f53a044a899b430f5a/ruff-0.16.9-py3-none-win32.whl", hash = "sha256:0e1dbc2073624dee6618d41d0098690a7244654af746704b64759e12b6b6b385", size = 10152355, upload-time = "2026-09-24T20:37:43.025Z" },
-    { url = "https://files.pythonhosted.org/packages/14/21/26e4643629b3ebb44f0a06f9c9a53058d63d989415f63a9a3c28e2ee7f22/ruff-0.16.9-py3-none-win_amd64.whl", hash = "sha256:6bd40fec8cd4c8a3d4dd589bd8ad4e6320c13c29234159bfd959a40d529d597b", size = 10592965, upload-time = "2026-09-24T20:37:44.944Z" },
-    { url = "https://files.pythonhosted.org/packages/51/60/5fb1a39dbb5ae314d5f59bc7348a63c1d5c20f3cd83914c4b5cb0be31d2d/ruff-0.16.9-py3-none-win_arm64.whl", hash = "sha256:ed1a252039200f57a59eebc063b54beabea67bfbaaca0eeaa7f54b5fbcda2284", size = 10458649, upload-time = "2026-09-24T20:37:46.882Z" },
-]
-
-[[package]]
-name = "snrc-resolve"
-version = "0.1.0"
-source = { virtual = "." }
-dependencies = [
-    { name = "eth-hash", extra = ["pycryptodome"] },
-]
-
-[package.dev-dependencies]
-dev = [
-    { name = "pytest" },
-    { name = "ruff" },
-]
-
-[package.metadata]
-requires-dist = [{ name = "eth-hash", extras = ["pycryptodome"], specifier = ">=0.7" }]
-
-[package.metadata.requires-dev]
-dev = [
-    { name = "pytest", specifier = ">=8" },
-    { name = "ruff", specifier = ">=0.6" },
-]

scripts/resolver/tools/ens-lookup.py

@@ -1,168 +0,0 @@
-#!/usr/bin/env python3
-"""Resolve an ENS name via local Reth (the same shape SNRC will use).
-
-Usage:
-  ./ens-lookup.py                     # defaults to simplexchat.eth
-  ./ens-lookup.py vitalik.eth
-  ./ens-lookup.py corevo.eth
-
-Requires:  pip install --break-system-packages 'eth-hash[pycryptodome]'
-"""
-
-import base64
-import json
-import sys
-from urllib.request import Request, urlopen
-
-from eth_hash.auto import keccak
-
-RPC = "http://127.0.0.1:8545"
-# ENS Registry (current, post-2020 migration)
-ENS_REGISTRY = "0x00000000000C2E074eC69A0dFb2997BA6C7d2e1e"
-
-
-def rpc(method, params):
-    body = json.dumps({"jsonrpc": "2.0", "method": method, "params": params, "id": 1}).encode()
-    req = Request(RPC, data=body, headers={"Content-Type": "application/json"})
-    res = json.loads(urlopen(req, timeout=15).read())
-    if "error" in res:
-        raise RuntimeError(res["error"])
-    return res["result"]
-
-
-def namehash(name: str) -> bytes:
-    """ENS namehash — recursive keccak256 over reversed labels."""
-    node = b"\x00" * 32
-    if name:
-        for label in reversed(name.split(".")):
-            node = keccak(node + keccak(label.encode()))
-    return node
-
-
-def selector(signature: str) -> str:
-    return "0x" + keccak(signature.encode())[:4].hex()
-
-
-def eth_call(to: str, data: str) -> str:
-    return rpc("eth_call", [{"to": to, "data": data}, "latest"])
-
-
-def decode_address(hex_data: str) -> str:
-    return "0x" + hex_data[-40:]
-
-
-def decode_bytes(hex_data: str) -> bytes:
-    raw = bytes.fromhex(hex_data[2:] if hex_data.startswith("0x") else hex_data)
-    if len(raw) < 64:
-        return b""
-    length = int.from_bytes(raw[32:64], "big")
-    return raw[64:64 + length]
-
-
-def encode_text_call(node: bytes, key: str) -> str:
-    """ABI-encode text(bytes32 node, string key). String arg is dynamic:
-    offset (=0x40) + length + right-padded data."""
-    sel = selector("text(bytes32,string)")
-    head = node.hex() + (0x40).to_bytes(32, "big").hex()
-    key_bytes = key.encode()
-    body = len(key_bytes).to_bytes(32, "big").hex() + key_bytes.hex()
-    # right-pad to 32-byte boundary
-    pad = (-len(key_bytes)) % 32
-    body += "00" * pad
-    return sel + head + body
-
-
-def text(resolver: str, node: bytes, key: str) -> str:
-    raw = decode_bytes(eth_call(resolver, encode_text_call(node, key)))
-    return raw.decode("utf-8", errors="replace") if raw else ""
-
-
-# Common ENS text keys (ENSIP-5). Resolvers may return empty for any of these.
-TEXT_KEYS = [
-    "url",
-    "avatar",
-    "description",
-    "email",
-    "notice",
-    "keywords",
-    "com.twitter",
-    "com.github",
-    "com.discord",
-    "org.telegram",
-    "io.keybase",
-    "xyz.farcaster",
-]
-
-
-def decode_contenthash(raw: bytes) -> str:
-    """ENS contenthash → human-readable URI (best-effort)."""
-    if not raw:
-        return "(empty)"
-    # Multicodec prefixes:
-    #   0xe301 = ipfs-ns + dag-pb (CIDv0/v1)
-    #   0xe501 = ipns-ns
-    #   0xe40101701b... = swarm
-    if raw[:2] == b"\xe3\x01":
-        cid_bytes = raw[2:]
-        # Base32 lowercase + 'b' prefix per CIDv1 spec
-        b32 = base64.b32encode(cid_bytes).decode().lower().rstrip("=")
-        return f"ipfs://b{b32}"
-    if raw[:2] == b"\xe5\x01":
-        cid_bytes = raw[2:]
-        b32 = base64.b32encode(cid_bytes).decode().lower().rstrip("=")
-        return f"ipns://b{b32}"
-    return "0x" + raw.hex()
-
-
-def main():
-    name = sys.argv[1] if len(sys.argv) > 1 else "simplexchat.eth"
-
-    print(f"  name:       {name}")
-    node = namehash(name)
-    print(f"  namehash:   0x{node.hex()}")
-
-    # 1. Ask the registry which resolver is responsible for this name
-    resolver_data = selector("resolver(bytes32)") + node.hex()
-    resolver_raw = eth_call(ENS_REGISTRY, resolver_data)
-    resolver = decode_address(resolver_raw)
-    print(f"  resolver:   {resolver}")
-    if resolver == "0x0000000000000000000000000000000000000000":
-        print("  → no resolver set for this name")
-        return
-
-    node_hex = node.hex()
-
-    # 2. Ask the resolver for the address
-    try:
-        addr = decode_address(eth_call(resolver, selector("addr(bytes32)") + node_hex))
-        print(f"  address:    {addr}")
-    except Exception as e:
-        print(f"  address:    (error: {e})")
-
-    # 3. Ask the resolver for the content hash (IPFS pointer)
-    try:
-        ch = decode_bytes(eth_call(resolver, selector("contenthash(bytes32)") + node_hex))
-        print(f"  contenthash: {decode_contenthash(ch)}")
-    except Exception as e:
-        print(f"  contenthash: (not supported: {e})")
-
-    # 4. Owner from the registry
-    try:
-        owner = decode_address(eth_call(ENS_REGISTRY, selector("owner(bytes32)") + node_hex))
-        print(f"  owner:      {owner}")
-    except Exception as e:
-        print(f"  owner:      (error: {e})")
-
-    # 5. Text records (EIP-634). Print only the non-empty ones.
-    print("  text records:")
-    for key in TEXT_KEYS:
-        try:
-            v = text(resolver, node, key)
-            if v:
-                print(f"    {key:<16s} {v}")
-        except Exception as e:
-            print(f"    {key:<16s} (error: {e})")
-
-
-if __name__ == "__main__":
-    main()

scripts/resolver/tools/progress.py

@@ -1,246 +0,0 @@
-#!/usr/bin/env python3
-"""Sync progress for the Reth + Nimbus stack.
-
-Usage:
-  ./progress.py           # continuous (Ctrl-C to exit, auto-exits when synced)
-  ./progress.py --once    # single snapshot
-
-Requires Nimbus REST port exposed at 127.0.0.1:5052 (add --rest flag in compose).
-"""
-
-import json
-import sys
-import time
-from collections import deque
-from datetime import timedelta
-from urllib.error import URLError
-from urllib.request import Request, urlopen
-
-RETH = "http://127.0.0.1:8545"
-NIMBUS = "http://127.0.0.1:5052"
-INTERVAL = 5
-WINDOW = 60
-BAR_W = 40
-
-# ANSI helpers
-def c(s, code): return f"\033[{code}m{s}\033[0m"
-GREEN, YELLOW, RED, DIM, BOLD = "32", "33", "31", "2;37", "1"
-
-
-def rpc(method):
-    body = json.dumps({"jsonrpc": "2.0", "method": method, "params": [], "id": 1}).encode()
-    req = Request(RETH, data=body, headers={"Content-Type": "application/json"})
-    return json.loads(urlopen(req, timeout=5).read())["result"]
-
-
-def get_reth():
-    try:
-        r = rpc("eth_syncing")
-        try:
-            peers = int(rpc("net_peerCount"), 16)
-        except Exception:
-            peers = -1            # net namespace not exposed
-        if r is False:
-            head = int(rpc("eth_blockNumber"), 16)
-            return {"state": "synced", "current": head, "target": head, "peers": peers,
-                    "stage": None, "stages": {}, "err": None}
-        current = int(r["currentBlock"], 16)
-        highest = int(r["highestBlock"], 16)
-        # Build stage map (name -> block).
-        stages = {s["name"]: int(s["block"], 16) for s in r.get("stages", [])}
-        active_stages = {k: v for k, v in stages.items() if v > 0}
-        # Headers download phase: nothing has progressed yet.
-        if current == 0 and highest == 0 and not active_stages:
-            return {"state": "headers", "current": 0, "target": 0, "peers": peers,
-                    "stage": "Headers", "stages": stages, "err": None}
-        # Derive progress from the stages pipeline.
-        # Bottleneck (rate-limiting stage) = stage with lowest non-zero block.
-        # Target = leading stage block (typically Headers = chain tip).
-        # Reth's top-level currentBlock/highestBlock are unreliable during initial
-        # sync (often 0 until execution stage runs), so prefer stages-derived values.
-        if active_stages:
-            bottleneck = min(active_stages, key=active_stages.get)
-            stage_current = active_stages[bottleneck]
-            stage_target = max(stages.values()) if stages else 0
-            # Trust the stages-derived values if highest is unset or stages tip is higher.
-            if highest <= 0 or stage_target > highest:
-                current = stage_current
-                highest = stage_target
-            elif current <= 0:
-                current = stage_current
-        else:
-            bottleneck = None
-        return {"state": "syncing", "current": current, "target": highest,
-                "peers": peers, "stage": bottleneck, "stages": stages, "err": None}
-    except URLError as e:
-        return {"state": "down", "current": 0, "target": 0, "peers": 0,
-                "stage": None, "stages": {}, "err": str(e.reason)}
-    except Exception as e:
-        return {"state": "error", "current": 0, "target": 0, "peers": 0,
-                "stage": None, "stages": {}, "err": str(e)}
-
-
-def get_nimbus():
-    try:
-        d = json.loads(urlopen(f"{NIMBUS}/eth/v1/node/syncing", timeout=5).read())["data"]
-        peers_d = json.loads(urlopen(f"{NIMBUS}/eth/v1/node/peer_count", timeout=5).read())["data"]
-        head = int(d["head_slot"])
-        dist = int(d["sync_distance"])
-        peers = int(peers_d.get("connected", "0"))
-        return {"state": "synced" if not d["is_syncing"] else "syncing",
-                "current": head, "target": head + dist, "peers": peers,
-                "optimistic": bool(d.get("is_optimistic", False)),
-                "el_offline": bool(d.get("el_offline", False)),
-                "err": None}
-    except URLError as e:
-        return {"state": "down", "current": 0, "target": 0, "peers": 0,
-                "optimistic": False, "el_offline": False, "err": str(e.reason)}
-    except Exception as e:
-        return {"state": "error", "current": 0, "target": 0, "peers": 0,
-                "optimistic": False, "el_offline": False, "err": str(e)}
-
-
-def format_num(n): return f"{n:,}"
-
-
-def format_eta(seconds):
-    if seconds is None: return "?"
-    if seconds < 0: return "?"
-    if seconds < 60: return f"{int(seconds)}s"
-    if seconds < 3600:
-        return f"{int(seconds // 60)}m {int(seconds % 60)}s"
-    if seconds < 86400:
-        return f"{int(seconds // 3600)}h {int((seconds % 3600) // 60)}m"
-    return f"{int(seconds // 86400)}d {int((seconds % 86400) // 3600)}h"
-
-
-def rate_per_sec(history):
-    if len(history) < 2: return None
-    t0, c0 = history[0]
-    t1, c1 = history[-1]
-    if t1 <= t0: return None
-    return (c1 - c0) / (t1 - t0)
-
-
-def eta_seconds(history, target):
-    r = rate_per_sec(history)
-    if r is None or r <= 0: return None
-    remaining = target - history[-1][1]
-    if remaining <= 0: return 0
-    return remaining / r
-
-
-def progress_bar(pct):
-    pct = max(0.0, min(100.0, pct))
-    filled = int(pct / 100 * BAR_W)
-    return c("█" * filled, GREEN) + c("░" * (BAR_W - filled), DIM)
-
-
-def peers_label(peers):
-    if peers < 0:
-        return c("· peers unknown (enable net namespace)", DIM)
-    return c(f"· {peers} peers", DIM)
-
-
-def stages_summary(stages):
-    """One-line view: stages that have progressed, with their block numbers."""
-    if not stages:
-        return ""
-    advanced = [(n, b) for n, b in stages.items() if b > 0]
-    if not advanced:
-        return c("  stages: all 0 (headers downloading)", DIM)
-    advanced.sort(key=lambda kv: kv[1], reverse=True)
-    parts = [f"{n}={format_num(b)}" for n, b in advanced[:4]]
-    return c("  stages: " + ", ".join(parts), DIM)
-
-
-def render_one(name, x, hist):
-    state = x["state"]
-    peers = x.get("peers", 0)
-    extras = []
-    if name == "Nimbus":
-        if x.get("optimistic"):
-            extras.append(c("(optimistic head — Reth not yet verifying)", YELLOW))
-        if x.get("el_offline"):
-            extras.append(c("⚠ EL OFFLINE", RED))
-    if state == "synced":
-        out = [f"  {c(name, BOLD):<14s} {c('✓ synced', GREEN)}      {c(format_num(x['current']), BOLD)}  {peers_label(peers)}"]
-    elif state == "headers":
-        out = [
-            f"  {c(name, BOLD):<14s} {c('⧗ headers', YELLOW)}    {c('downloading initial chain', DIM)}  {peers_label(peers)}",
-            f"                 {c('(per-block progress unavailable until headers validated — see docker logs)', DIM)}",
-        ]
-    elif state == "syncing" and x["target"] <= 0:
-        out = [f"  {c(name, BOLD):<14s} {c('⧗ syncing', YELLOW)}    {c('waiting for fork-choice', DIM)}  {peers_label(peers)}"]
-    elif state == "syncing":
-        pct = x["current"] / x["target"] * 100
-        r = rate_per_sec(hist)
-        eta = eta_seconds(hist, x["target"])
-        rate_s = f"{format_num(int(r))} /s" if r and r > 0 else c("stalled", RED)
-        eta_s = format_eta(eta) if eta is not None else "?"
-        stage = x.get("stage")
-        stage_s = c(f"[{stage}]", DIM) if stage else ""
-        out = [
-            f"  {c(name, BOLD):<14s} {c('⧗ syncing', YELLOW)}    {format_num(x['current'])} / {format_num(x['target'])}  {stage_s}  {peers_label(peers)}",
-            f"                 {progress_bar(pct)} {c(f'{pct:6.2f}%', BOLD)}",
-            f"                 {c(rate_s, DIM)}   ETA {c(eta_s, BOLD)}",
-        ]
-    else:
-        out = [
-            f"  {c(name, BOLD):<14s} {c('✗ ' + state, RED)}",
-            f"                 {c(x.get('err') or '', DIM)}",
-        ]
-    # Reth-only: stages summary
-    if name == "Reth" and x.get("stages"):
-        out.append(f"               {stages_summary(x['stages'])}")
-    for e in extras:
-        out.append(f"                 {e}")
-    return out
-
-
-def render(reth, nimbus, reth_hist, nim_hist):
-    print("\033[2J\033[H", end="")
-    width = 64
-    title = f"Reth + Nimbus sync"
-    ts = time.strftime("%H:%M:%S")
-    print()
-    print(f"  {c(title, BOLD)}   {c(ts, DIM)}")
-    print(f"  {c('─' * width, DIM)}")
-    print()
-    for line in render_one("Reth", reth, reth_hist):
-        print(line)
-    print()
-    for line in render_one("Nimbus", nimbus, nim_hist):
-        print(line)
-    print()
-    win_s = (len(reth_hist) - 1) * INTERVAL if len(reth_hist) > 1 else 0
-    print(f"  {c(f'window {win_s}s · refresh {INTERVAL}s · Ctrl-C to exit', DIM)}")
-    print()
-
-
-def main():
-    once = "--once" in sys.argv
-    reth_hist = deque(maxlen=WINDOW)
-    nim_hist = deque(maxlen=WINDOW)
-    try:
-        while True:
-            r = get_reth()
-            n = get_nimbus()
-            now = time.time()
-            if r["target"] > 0 or r["state"] == "syncing":
-                reth_hist.append((now, r["current"]))
-            if n["target"] > 0 or n["state"] == "syncing":
-                nim_hist.append((now, n["current"]))
-            render(r, n, reth_hist, nim_hist)
-            if once:
-                break
-            if r["state"] == "synced" and n["state"] == "synced":
-                print(f"  {c('✓ all synced.', GREEN)}\n")
-                break
-            time.sleep(INTERVAL)
-    except KeyboardInterrupt:
-        print()
-
-
-if __name__ == "__main__":
-    main()

scripts/smp-server-digitalocean-droplet/files/opt/simplex/initialize_server.sh

@@ -22,7 +22,7 @@ smp-server --version
 
 # Initialize server
 ip_address=$(curl ifconfig.me)
-smp-server init -l --disable-web --ip $ip_address
+smp-server init -l --ip $ip_address
 
 # Server fingerprint
 fingerprint=$(cat /etc/opt/simplex/fingerprint)

scripts/smp-server-digitalocean-droplet/files/opt/simplex/on_login.sh

@@ -12,11 +12,6 @@ Check SMP server status with: systemctl status smp-server
 To keep this server secure, the UFW firewall is enabled.
 All ports are BLOCKED except 22 (SSH), 443 (HTTPS), 5223 (SMP server).
 
-Embedded HTTPS web is disabled because this image does not provision
-/etc/opt/simplex/web.crt or /etc/opt/simplex/web.key. To enable it, provision
-those files, uncomment WEB https/cert/key in /etc/opt/simplex/smp-server.ini,
-and restart smp-server.
-
 ********************************************************************************
 To stop seeing this message delete line - bash /opt/simplex/on_login.sh - from /root/.bashrc
 EOF

scripts/smp-server-linode.sh

@@ -75,9 +75,6 @@ init_opts=()
 
 [[ $ENABLE_STORE_LOG == "on" ]] && init_opts+=(-l)
 
-# This script does not provision /etc/opt/simplex/web.crt or web.key.
-init_opts+=(--disable-web)
-
 ip_address=$(curl ifconfig.me)
 init_opts+=(--ip $ip_address)
 
@@ -114,11 +111,6 @@ Check SMP server status with: systemctl status smp-server
 To keep this server secure, the UFW firewall is enabled.
 All ports are BLOCKED except 22 (SSH), 443 (HTTPS), 5223 (SMP server).
 
-Embedded HTTPS web is disabled because this script does not provision
-/etc/opt/simplex/web.crt or /etc/opt/simplex/web.key. To enable it, provision
-those files, uncomment WEB https/cert/key in /etc/opt/simplex/smp-server.ini,
-and restart smp-server.
-
 ********************************************************************************
 To stop seeing this message delete line - bash /opt/simplex/on_login.sh - from /root/.bashrc
 EOF2

simplexmq.cabal

@@ -1,7 +1,7 @@
-cabal-version: 3.0
+cabal-version: 1.12
 
 name:           simplexmq
-version:        7.1.0.10
+version:        6.5.2.0
 synopsis:       SimpleXMQ message broker
 description:    This package includes <./docs/Simplex-Messaging-Server.html server>,
                 <./docs/Simplex-Messaging-Client.html client> and
@@ -16,7 +16,7 @@ homepage:       https://github.com/simplex-chat/simplexmq#readme
 author:         simplex.chat
 maintainer:     [email protected]
 copyright:      2020-2022 simplex.chat
-license:        AGPL-3.0-only
+license:        AGPL-3
 license-file:   LICENSE
 build-type:     Simple
 extra-source-files:
@@ -24,13 +24,6 @@ extra-source-files:
     CHANGELOG.md
     cbits/sha512.h
     cbits/sntrup761.h
-    cbits/blst/**/*.c
-    cbits/blst/**/*.h
-    cbits/blst/**/*.s
-    cbits/blst/**/*.S
-    cbits/blst/**/*.asm
-    cbits/libbbs/**/*.c
-    cbits/libbbs/**/*.h
     apps/common/Web/static/index.html
     apps/common/Web/static/link.html
     apps/common/Web/static/media/apk_icon.png
@@ -89,11 +82,6 @@ flag server_postgres
   manual: True
   default: False
 
-flag commoncrypto
-  description: On Apple platforms, use SecRandomCopyBytes (Security.framework) for libbbs randomness. getentropy is a non-public symbol on iOS and triggers App Store rejection (ITMS-90338).
-  manual: True
-  default: False
-
 library
   exposed-modules:
       Simplex.FileTransfer.Agent
@@ -134,8 +122,6 @@ library
       Simplex.Messaging.Crypto.File
       Simplex.Messaging.Crypto.Lazy
       Simplex.Messaging.Crypto.Ratchet
-      Simplex.Messaging.Crypto.BBS
-      Simplex.Messaging.Crypto.Entitlement
       Simplex.Messaging.Crypto.SNTRUP761
       Simplex.Messaging.Crypto.SNTRUP761.Bindings
       Simplex.Messaging.Crypto.SNTRUP761.Bindings.Defines
@@ -144,7 +130,6 @@ library
       Simplex.Messaging.Crypto.ShortLink
       Simplex.Messaging.Encoding
       Simplex.Messaging.Encoding.String
-      Simplex.Messaging.Names.Record
       Simplex.Messaging.Notifications.Client
       Simplex.Messaging.Notifications.Protocol
       Simplex.Messaging.Notifications.Transport
@@ -153,11 +138,9 @@ library
       Simplex.Messaging.Protocol
       Simplex.Messaging.Protocol.Types
       Simplex.Messaging.Server.Expiration
-      Simplex.Messaging.Server.Information
       Simplex.Messaging.Server.QueueStore.Postgres.Config
       Simplex.Messaging.Server.QueueStore.QueueInfo
       Simplex.Messaging.ServiceScheme
-      Simplex.Messaging.SimplexName
       Simplex.Messaging.Session
       Simplex.Messaging.SystemTime
       Simplex.Messaging.TMap
@@ -192,10 +175,6 @@ library
           Simplex.Messaging.Agent.Store.Postgres.Migrations.M20251230_strict_tables
           Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260410_receive_attempts
           Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260411_service_certs
-          Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260712_address_dr_rpc
-          Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260823_snd_files_entitlement
-          Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260919_ratchet_verify_codes
-          Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260929_ratchet_indexes
   else
       exposed-modules:
           Simplex.Messaging.Agent.Store.SQLite
@@ -248,10 +227,6 @@ library
           Simplex.Messaging.Agent.Store.SQLite.Migrations.M20251230_strict_tables
           Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260410_receive_attempts
           Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260411_service_certs
-          Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260712_address_dr_rpc
-          Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260823_snd_files_entitlement
-          Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260919_ratchet_verify_codes
-          Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260929_ratchet_indexes
           Simplex.Messaging.Agent.Store.SQLite.Util
   if flag(client_postgres) || flag(server_postgres)
       exposed-modules:
@@ -276,15 +251,16 @@ library
           Simplex.Messaging.Server.CLI
           Simplex.Messaging.Server.Control
           Simplex.Messaging.Server.Env.STM
+          Simplex.Messaging.Server.Information
           Simplex.Messaging.Server.Main
           Simplex.Messaging.Server.Main.GitCommit
           Simplex.Messaging.Server.Main.Init
           Simplex.Messaging.Server.Web
           Simplex.Messaging.Server.MsgStore
+          Simplex.Messaging.Server.MsgStore.Journal
+          Simplex.Messaging.Server.MsgStore.Journal.SharedLock
           Simplex.Messaging.Server.MsgStore.STM
           Simplex.Messaging.Server.MsgStore.Types
-          Simplex.Messaging.Server.Names
-          Simplex.Messaging.Server.Names.HttpResolver
           Simplex.Messaging.Server.NtfStore
           Simplex.Messaging.Server.Prometheus
           Simplex.Messaging.Server.QueueStore
@@ -324,33 +300,11 @@ library
   ghc-options: -Weverything -Wno-missing-exported-signatures -Wno-missing-import-lists -Wno-missed-specialisations -Wno-all-missed-specialisations -Wno-unsafe -Wno-safe -Wno-missing-local-signatures -Wno-missing-kind-signatures -Wno-missing-deriving-strategies -Wno-monomorphism-restriction -Wno-prepositive-qualified-module -Wno-implicit-prelude -Wno-missing-safe-haskell-mode -Wno-missing-export-lists -Wno-partial-fields -Wcompat -Werror=incomplete-record-updates -Werror=incomplete-patterns -Werror=incomplete-uni-patterns -Werror=missing-home-modules -Werror=missing-methods -Werror=tabs -Wredundant-constraints -Wincomplete-record-updates -Wunused-type-patterns -O2
   include-dirs:
       cbits
-      cbits/blst/bindings
-      cbits/blst/src
-      cbits/libbbs/include
-      cbits/libbbs/src
-  cc-options: -D__BLST_PORTABLE__
-  if flag(commoncrypto)
-    cc-options: -DBBS_CRYPTO_CC
-    frameworks: Security
   c-sources:
       cbits/sha512.c
       cbits/sntrup761.c
-      cbits/blst/src/server.c
-      cbits/libbbs/src/bbs.c
-      cbits/libbbs/src/bbs_ciphersuites.c
-      cbits/libbbs/src/bbs_util.c
-      cbits/libbbs/src/compat-string.c
-      cbits/libbbs/src/sha256.c
-      cbits/libbbs/src/shake256.c
-  asm-sources:
-      cbits/blst/build/assembly.S
   extra-libraries:
       crypto
-  if os(windows)
-    c-sources:
-        cbits/getentropy_win.c
-    extra-libraries:
-        bcrypt
   build-depends:
       aeson ==2.2.*
     , asn1-encoding ==0.9.*
@@ -401,10 +355,7 @@ library
     build-depends:
         case-insensitive ==1.2.*
       , hashable ==1.4.*
-      , http-client >=0.7 && <0.8
-      , http-client-tls >=0.3 && <0.4
       , ini ==0.4.1
-      , network-uri >=2.6 && <2.7
       , optparse-applicative >=0.15 && <0.17
       , process ==1.6.*
       , temporary ==1.3.*
@@ -538,7 +489,6 @@ test-suite simplexmq-test
       AgentTests.EqInstances
       AgentTests.FunctionalAPITests
       AgentTests.MigrationTests
-      AgentTests.ResolveNameTests
       AgentTests.ServerChoice
       AgentTests.ShortLinkTests
       CLITests
@@ -555,12 +505,9 @@ test-suite simplexmq-test
       CoreTests.VersionRangeTests
       FileDescriptionTests
       RemoteControl
-      NamesResolverServer
-      RSLVTests
       ServerTests
       SMPAgentClient
       SMPClient
-      SMPNamesTests
       SMPProxyTests
       Util
       XFTPAgent
@@ -641,8 +588,6 @@ test-suite simplexmq-test
     , unliftio
     , unliftio-core
     , unordered-containers
-    , wai
-    , warp
     , yaml
   default-language: Haskell2010
   if flag(server_postgres)

smp-web/.gitignore

@@ -0,0 +1,3 @@
+node_modules
+dist
+dist-test

spec/README.md

@@ -0,0 +1,93 @@
+# Specifications
+
+Disclaimer: specifications in this folder and its subfolders are generated by LLM. While they were reviewed and appear to be correct, they may contain mistakes, and should not be considered a protocol specification.
+
+## Structure
+
+Spec has two levels:
+
+### `spec/modules/` — Per-module documentation
+
+Mirrors the `src/Simplex/` directory structure exactly. Each `.hs` file has a corresponding `.md` file at the same relative path. Contains only information that is **not obvious from reading the code** and cannot fit in a one-line source comment:
+
+- Non-obvious behavior (subtle invariants, ordering dependencies, concurrency assumptions)
+- Usage considerations (when to use X vs Y, common mistakes, caller obligations)
+- Relationships to other modules not visible from imports
+- Security notes specific to this module
+
+**Not included**: type signatures, code snippets, function-by-function prose that restates the source. If reading the code tells you everything, the module doc says so briefly.
+
+Function references use fully qualified names with markdown links:
+```
+[Simplex.Messaging.Server.subscribeServiceMessages](./modules/Simplex/Messaging/Server.md#subscribeServiceMessages)
+```
+
+Source code links back via comments:
+```haskell
+-- spec: spec/modules/Simplex/Messaging/Server.md#subscribeServiceMessages
+subscribeServiceMessages :: ...
+```
+
+### `spec/` root — Topic documentation
+
+Cross-module documentation that follows a feature, mechanism, or concern across the entire stack. Topics answer "how does X work end-to-end?" rather than "what does this file do?"
+
+Topics reference module docs rather than restating implementation details. They focus on:
+- End-to-end data flow across modules
+- Cross-cutting security analysis and invariants
+- Design rationale, risks, test gaps
+- Version gates and compatibility concerns
+
+Some topics may migrate to `product/` if they are primarily about user-visible behavior and guarantees rather than implementation mechanics.
+
+### `spec/security-invariants.md` — All security invariants
+
+Cross-referenced from both module docs and topic docs.
+
+## Conventions
+
+Module doc entry format:
+```
+## functionName
+**Purpose**: ...
+**Calls**: [Module.a](./modules/path.md#a), [Module.b](./modules/path.md#b)
+**Called by**: [Module.c](./modules/path.md#c)
+**Invariant**: SI-XX
+**Security**: ...
+```
+
+## Index
+
+### Architecture
+
+Component topology and message flow diagrams for each layer:
+
+- [routers.md](routers.md) — Layer 1: SMP, XFTP, NTF routers
+- [clients.md](clients.md) — Layer 2: protocol client libraries
+- [agent.md](agent.md) — Layer 3: connection manager
+
+### Topics
+
+Cross-cutting concerns that span multiple modules:
+
+- [topics/transport.md](topics/transport.md) — TLS, HTTP/2, WebSocket transport layers
+- [topics/patterns.md](topics/patterns.md) — Exception handling, encoding, compression, TMap
+- [topics/subscriptions.md](topics/subscriptions.md) — Queue subscriptions and delivery
+- [topics/notifications.md](topics/notifications.md) — Push notification flow
+- [topics/xftp.md](topics/xftp.md) — File transfer protocol
+- [topics/client-services.md](topics/client-services.md) — Service certificates for bulk operations
+- [topics/encryption.md](topics/encryption.md) — Encryption layers (TODO)
+
+### Agent internals
+
+- [agent/infrastructure.md](agent/infrastructure.md) — Workers, store, operation suspension
+- [agent/connections.md](agent/connections.md) — Connection lifecycle and states
+- [agent/xrcp.md](agent/xrcp.md) — Remote control protocol
+
+### Modules
+
+See `spec/modules/` — mirrors `src/Simplex/` structure.
+
+### Security
+
+- [security-invariants.md](security-invariants.md) — All security invariants

spec/TOPICS.md

@@ -0,0 +1,69 @@
+# Topic Candidates
+
+> Cross-cutting patterns noticed during module documentation. Each entry may become a topic doc in `spec/` after all module docs are complete.
+
+- **Exception handling strategy**: `catchOwn`/`catchAll`/`tryAllErrors` pattern (defined in Util.hs) used across router, client, and agent modules. The three-category classification (synchronous, own-async, cancellation) and when to use which catch variant is not obvious from any single call site.
+
+- **Padding schemes**: Three different padding formats across the codebase — Crypto.hs uses 2-byte Word16 length prefix (max ~65KB), Crypto/Lazy.hs uses 8-byte Int64 prefix (file-sized), and both use '#' fill character. Ratchet header padding uses fixed sizes (88 or 2310 bytes). All use `pad`/`unPad` but with incompatible formats. The relationship between padding, encryption, and message size limits spans Crypto, Lazy, Ratchet, and the protocol layer.
+
+- **NaCl construction variants**: crypto_box, secret_box, and KEM hybrid secret all use the same XSalsa20+Poly1305 core (Crypto.hs `xSalsa20`), but with different key sources (DH, symmetric, SHA3_256(DH||KEM)). The lazy streaming variant (Lazy.hs) adds prepend-tag vs tail-tag placement. File.hs wraps lazy streaming with handle-based I/O. Full picture requires reading Crypto.hs, Lazy.hs, File.hs, and SNTRUP761.hs together.
+
+- **Transport encryption layering**: Three encryption layers overlap — TLS (Transport.hs), optional block encryption via sbcHkdf chains (Transport.hs tPutBlock/tGetBlock), and SMP protocol-level encryption. Block encryption is disabled for proxy connections (already encrypted), and absent for NTF protocol. The interaction of these layers with proxy version downgrade logic spans Transport.hs, Client.hs, and the SMP proxy module.
+
+- **Certificate chain trust model**: ChainCertificates (Shared.hs) defines 0–4 cert chain semantics, used by both Client.hs (validateCertificateChain) and Server.hs (validateClientCertificate, SNI credential switching). The 4-length case skipping index 2 (operator cert) and the FQHN-disabled x509validate are decisions that span the entire transport security model.
+
+- **SMP proxy protocol flow**: The PRXY/PFWD/RFWD proxy protocol involves Client.hs (proxySMPCommand with 10 error scenarios, forwardSMPTransmission with sessionSecret encryption), Protocol.hs (command types, version-dependent encoding), Transport.hs (proxiedSMPRelayVersion cap, proxyServer flag disabling block encryption). The double encryption (client-relay via PFWD + proxy-relay via RFWD), combined timeout (tcpConnect + tcpTimeout), nonce/reverseNonce pairing, and version downgrade logic are not visible from any single module.
+
+- **Service certificate subscription model**: Service subscriptions (SUBS/NSUBS) and per-queue subscriptions (SUB/NSUB) coexist with complex state transitions. Client/Agent.hs manages dual active/pending subscription maps with session-aware cleanup. Protocol.hs defines useServiceAuth (only NEW/SUB/NSUB). Client.hs implements authTransmission with dual signing (entity key over cert hash + transmission, service key over transmission only). Transport.hs handles the service certificate handshake extension (v16+). The full subscription lifecycle — from DBService credentials through handshake to service subscription to disconnect/reconnect — spans all four modules.
+
+- **Two agent layers**: Client/Agent.hs ("small agent") is used only in routers — SMP proxy and notification router — to manage client connections to other SMP routers. Agent.hs + Agent/Client.hs ("big agent") is used in client applications. Both manage SMP client connections with subscription tracking and reconnection, but the big agent adds the full messaging agent layer (connections, double ratchet, file transfer). When documenting Agent/Client.hs, Client/Agent.hs should be reviewed for shared patterns and differences.
+
+- **Handshake protocol family**: SMP (Transport.hs), NTF (Notifications/Transport.hs), and XFTP (FileTransfer/Transport.hs) all have handshake protocols with the same structure (version negotiation + session binding + key exchange) but different feature sets. NTF is a strict subset. XFTP doesn't use the TLS handshake at all (HTTP2 layer). The shared types (THandle, THandleParams, THandleAuth) mean changes to the handshake infrastructure affect all three protocols.
+
+- **Router subscription architecture**: The SMP router's subscription model spans Server.hs (serverThread split-STM lifecycle, tryDeliverMessage sync/async, ProhibitSub/ServerSub state machine), Env/STM.hs (SubscribedClients TVar-of-Maybe continuity, Client three-queue architecture), and Client/Agent.hs (small agent dual subscription model). The interaction between service subscriptions, direct queue subscriptions, notification subscriptions, and the serverThread subQ processing is not visible from any single module.
+
+- **Duplex connection handshake**: The SMP duplex connection procedure (standard 10-step and fast 7-step) spans Agent.hs (orchestration, state machine), Agent/Protocol.hs (message types: AgentConfirmation/AgentConnInfoReply/AgentInvitation/HELLO, queue status types), Client.hs (SMP command dispatch), Protocol.hs (SMP-level KEY/SKEY commands). The handshake involves two-layer encryption (per-queue E2E + double ratchet), version-dependent paths (v2+ duplex, v6+ sender auth key, v7+ ratchet on confirmation, v9+ fast handshake with SKEY), and the asymmetry between initiating and accepting parties (different message types, different confirmation processing). The protocol spec (`agent-protocol.md`) defines the procedure but the implementation details — error handling, state persistence across restarts, race conditions between confirmation and message delivery — are only visible by reading the code across these modules.
+
+- **Connection links**: Full connection links (URI format with `#/?` query parameters) and binary-encoded links (`Encoding` instances) serve different contexts — URIs for out-of-band sharing, binary for agent-to-agent messages. Each has independent version-conditional encoding with different backward-compat rules (URI parser adjusts agent version ranges for old contact links, binary parser patches `queueMode` for forward compat). The `VersionI`/`VersionRangeI` typeclasses convert between `SMPQueueInfo` (versioned, in confirmations) and `SMPQueueUri` (version-ranged, in links). Full picture requires Agent/Protocol.hs, Protocol.hs, and agent-protocol.md.
+
+- **Short links**: Short links are a compact representation for sharing via URLs, not a replacement for full connection links — both are used. Short links store encrypted link data on the router and encode only a router hostname, link type character, and key hash in the URL. The link data lifecycle (creation, encryption with key derivation, owner chain-of-trust validation, mutable user data updates) spans Agent/Protocol.hs (types, serialization, owner validation, router shortening/restoration), Agent.hs (link creation and resolution API), and the router-side link storage. The `FixedLinkData`/`ConnLinkData` split (immutable vs mutable), `OwnerAuth` chain validation, and `PreparedLinkParams` pre-computation are not visible from any single module.
+
+- **Agent worker framework**: `getAgentWorker` (lifecycle, restart rate limiting, crash recovery) + `withWork`/`withWork_`/`withWorkItems` (task retrieval with doWork flag atomics) defined in Agent/Client.hs, consumed by Agent.hs (async commands, message delivery), NtfSubSupervisor.hs (notification workers), FileTransfer/Agent.hs (XFTP workers), and simplex-chat. The framework separates two concerns: worker lifecycle (create-or-reuse, fork async, rate-limit restarts, escalate to CRITICAL) and task pattern (get next task, do task, as separate parameters). The doWork TMVar flag choreography (clear before query to prevent race) and the work-item-error vs store-error distinction are not obvious from any single consumer.
+
+- **Agent operation suspension**: Five `AgentOpState` TVars (RcvNetwork, MsgDelivery, SndNetwork, Database, NtfNetwork) with a cascade ordering: ending RcvNetwork suspends MsgDelivery, ending MsgDelivery suspends SndNetwork + Database, ending SndNetwork suspends Database. `beginAgentOperation` retries if suspended, `endAgentOperation` decrements and cascades. All DB access goes through `withStore` which brackets with AODatabase. This ensures graceful shutdown propagates through dependent operations. Defined in Agent/Client.hs, used by Agent.hs subscriber and worker loops.
+
+- **Queue rotation protocol**: Four agent messages (QADD → QKEY → QUSE → QTEST) on top of SMP commands, with asymmetric state machines on receiver side (`RcvSwitchStatus`: 4 states) and sender side (`SndSwitchStatus`: 2 states). Receiver initiates, creates new queue, sends QADD. Sender responds with QKEY. Receiver sends QUSE. Sender sends QTEST to complete. State types in Agent/Protocol.hs, orchestration in Agent.hs, queue creation/deletion in Agent/Client.hs. Protocol spec in agent-protocol.md. The fast variant (v9+ SMP with SKEY) skips the KEY command step.
+
+- **Outside-STM lookup pattern**: Multiple modules use the pattern of looking up TVar references outside STM (via readTVarIO/TM.lookupIO), then reading/modifying the TVar contents inside STM. This avoids transaction re-evaluation from unrelated map changes. Used in: Server.hs (serverThread client lookup, tryDeliverMessage subscriber lookup), Env/STM.hs (deleteSubcribedClient), Client/Agent.hs (removeClientAndSubs, reconnectSMPClient). The safety invariant is that the outer map entries (TVars) are never removed — only their contents change.
+
+- **NTF token lifecycle**: Token registration (TNEW) → verification push → NTConfirmed → TVFY → NTActive, with idempotent re-registration (DH secret check), TRPL (device token replacement reusing DH key), status repair for stuck tokens, and `PPApnsNull` test tokens suppressing stats. The lifecycle spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md) (command handling, verification push delivery), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) (conditional status updates, duplicate registration cleanup), [Types.hs](modules/Simplex/Messaging/Notifications/Types.md) (NtfTknStatus state machine), and [Env.hs](modules/Simplex/Messaging/Notifications/Server/Env.md) (push client lazy initialization).
+
+- **NTF push delivery pipeline**: Bounded TBQueue (`pushQ`) creates backpressure → `ntfPush` thread reads → `checkActiveTkn` gates PNMessage (but not PNVerification or PNCheckMessages) → APNS delivery with single retry on connection errors (new push client on retry) → PPTokenInvalid marks token NTInvalid. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md), [APNS.hs](modules/Simplex/Messaging/Notifications/Server/Push/APNS.md) (DER JWT signing, HTTP/2 serializing queue, fire-and-forget connection), [Env.hs](modules/Simplex/Messaging/Notifications/Server/Env.md) (push client caching with race tolerance).
+
+- **NTF service subscription model**: Service-level subscriptions (SUBS/NSUBS on SMP) vs individual queue subscriptions, with fallback from service to individual when `CAServiceUnavailable`. Service credentials are lazily generated per SMP router with 25h backdating and ~2700yr validity. XOR hash triggers on PostgreSQL maintain subscription aggregate counts. Subscription status tracking uses `ntf_service_assoc` flag to distinguish service-associated from individually-subscribed queues. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md) (subscriber thread, service fallback), [Env.hs](modules/Simplex/Messaging/Notifications/Server/Env.md) (lazy credential generation, Weak ThreadId subscriber cleanup), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) (XOR hash triggers, batch status updates, cursor-based pagination).
+
+- **NTF startup resubscription**: `resubscribe` runs as detached `forkIO` (not in `raceAny_` group), uses `mapConcurrently` across SMP routers, each with `subscribeLoop` using 100x database batch multiplier and cursor-based pagination. `ExitCode` exceptions from `exitFailure` on DB error propagate to main thread despite `forkIO`. `getServerNtfSubscriptions` claims subscriptions by batch-updating to `NSPending`. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md).
+
+- **XFTP file upload pipeline**: Agent-side encryption (streaming 64KB blocks, fixed-size padding) → chunk size selection (75% threshold algorithm) → per-router data packet creation with ID collision retry (3 attempts) → recipient registration (recursive batching up to `maxRecipients` per FADD) → per-router data packet upload (command + data in single HTTP/2 streaming request) → file description generation (cross-product: M chunks × R replicas × N recipients → N descriptions). Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md) (worker orchestration, description generation), [Client.hs](modules/Simplex/FileTransfer/Client.md) (upload protocol), [Server.hs](modules/Simplex/FileTransfer/Server.md) (quota reservation with rollback, skipCommitted idempotency), [Crypto.hs](modules/Simplex/FileTransfer/Crypto.md) (streaming encryption with embedded header), [Description.hs](modules/Simplex/FileTransfer/Description.md) (validation, first-replica-only digest optimization).
+
+- **XFTP file download pipeline**: Description parsing (ValidFileDescription validation, YAML or web URI) → per-router data packet download with ephemeral DH key pair per download (forward secrecy) → size and digest verification before decryption → streaming decryption with auth tag verification (output deleted on failure) → redirect resolution (depth-1 chain: decrypt redirect YAML, validate size/digest, download actual file). Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md) (worker orchestration, redirect handling), [Client.hs](modules/Simplex/FileTransfer/Client.md) (ephemeral DH, size-proportional timeout), [Client/Main.hs](modules/Simplex/FileTransfer/Client/Main.md) (web URI decoding, parallel download with router grouping), [Crypto.hs](modules/Simplex/FileTransfer/Crypto.md) (dual decrypt paths, auth tag deletion), [Description.hs](modules/Simplex/FileTransfer/Description.md) (redirect file descriptions).
+
+- **XFTP handshake state machine**: Three-state session-cached handshake (`No entry` → `HandshakeSent` → `HandshakeAccepted`) per HTTP/2 session. Web clients use `xftp-web-hello` header and challenge-response identity proof; native clients use standard ALPN. SNI presence gates CORS headers, web serving, and SESSION error for unrecognized connections. Key reuse on re-hello preserves existing DH keys. Spans [Server.hs](modules/Simplex/FileTransfer/Server.md) (handshake logic, CORS, web serving), [Client.hs](modules/Simplex/FileTransfer/Client.md) (ALPN selection, cert chain validation), [Transport.hs](modules/Simplex/FileTransfer/Transport.md) (block size, version).
+
+- **XFTP storage lifecycle**: Quota reservation via atomic `stateTVar` before upload → rollback on failure (subtract + delete partial data packet) → stored data packet deleted before store cleanup (crash risk: store references missing data packet) → `RoundedSystemTime 3600` for privacy-preserving expiration timestamps → expiration with configurable throttling (100ms between data packets) → startup storage reconciliation (override stats from live store). Spans [Server.hs](modules/Simplex/FileTransfer/Server.md), [Server/Store.hs](modules/Simplex/FileTransfer/Server/Store.md), [Server/Env.hs](modules/Simplex/FileTransfer/Server/Env.md), [Server/StoreLog.hs](modules/Simplex/FileTransfer/Server/StoreLog.md) (error-resilient replay, compaction).
+
+- **XFTP worker architecture**: Five worker types in three categories: rcv (per-router data packet download + local decryption), snd (local prepare/encrypt + per-router data packet upload), del (per-router data packet delete). TMVar-based connection sharing with async retry on temporary errors, permanent error cleanup (put Left + delete from TMap). `withRetryIntervalLimit` caps consecutive retries; exhausted temporary errors silently abandon work cycle (chunk stays pending). `assertAgentForeground` dual check (throw if inactive + wait if backgrounded) gates every data packet operation. Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md), [Client/Agent.hs](modules/Simplex/FileTransfer/Client/Agent.md).
+
+- **SessionVar protocol client lifecycle**: Protocol client connections (SMP, NTF, XFTP) use a lazy singleton pattern: `getSessVar` atomically checks TMap → `newProtocolClient` fills TMVar on success/failure → `waitForProtocolClient` reads with timeout. Error caching via `persistErrorInterval` prevents connection storms (failed connections cache the error with expiry; callers receive cached error without reconnecting). `removeSessVar` uses monotonic `sessionVarId` compare-and-swap to prevent stale disconnect callbacks from removing newer clients. SMP has additional complexity: `SMPConnectedClient` wraps client with per-connection proxied relay map, `updateClientService` synchronizes service credentials post-connect, disconnect callback moves subscriptions to pending with session-ID matching. XFTP always uses `NRMBackground` timing regardless of caller request. Spans [Session.md](modules/Simplex/Messaging/Session.md), [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (lifecycle, disconnect callbacks, reconnection workers), [Agent.md](modules/Simplex/Messaging/Agent.md) (subscriber loop consuming events).
+
+- **Dual-backend agent store**: The agent store (~3700 lines in AgentStore.hs) compiles for both SQLite and PostgreSQL via `#if defined(dbPostgres)` CPP guards. Key behavioral differences: PostgreSQL uses `FOR UPDATE` row locking on reads preceding writes (SQLite relies on single-writer model); PostgreSQL uses `IN ?` with `In` wrapper for batch queries (SQLite falls back to per-row `forM` loops); PostgreSQL uses `constraintViolation` (SQLite checks `SQL.ErrorConstraint`); `createWithRandomId'` uses savepoints on PostgreSQL (failed statement aborts entire transaction without them). One known bug: `checkConfirmedSndQueueExists_` uses `#if defined(dpPostgres)` (typo: `dp` not `db`), so the `FOR UPDATE` clause is never included on any backend. Spans [AgentStore.md](modules/Simplex/Messaging/Agent/Store/AgentStore.md), [SQLite.md](modules/Simplex/Messaging/Agent/Store/SQLite.md).
+
+- **Deferred message encryption**: Message bodies are NOT encrypted at enqueue time. `enqueueMessageB` advances the ratchet header and validates padding, but stores only the body reference (`sndMsgBodyId`) and encryption key. Actual encryption (`rcEncryptMsg`) happens at delivery time in `runSmpQueueMsgDelivery`. This enables body deduplication via `VRValue`/`VRRef` — identical bodies (common for group messages) share one database row, but each connection's delivery encrypts independently with its own ratchet. Confirmation and ratchet key messages bypass deferred encryption (pre-encrypted at enqueue time). Spans [Agent.md](modules/Simplex/Messaging/Agent.md) (enqueue + delivery), [AgentStore.md](modules/Simplex/Messaging/Agent/Store/AgentStore.md) (`snd_message_bodies` storage).
+
+- **NTF agent subscription lifecycle**: The agent-side notification subscription system uses a supervisor-worker architecture with three worker pools (NTF router, SMP router, token deletion). `NSCCreate` triggers a four-way partition (`partitionQueueSubActions`): new sub, reset sub (credential mismatch or null action), continue SMP work, continue NTF work. Workers coordinate with the supervisor via `updated_by_supervisor` flag — workers only update local fields when the flag is set, preventing overwrite of supervisor decisions. The null-action sentinel (`workerErrors` sets action to NULL on permanent failure) bridges worker failure recovery to supervisor-driven re-creation. `retrySubActions` uses a shrinking TVar — each iteration only retries subs with temporary errors, so batches get smaller over time. `rescheduleWork` handles time-scheduled health checks by forking a sleep thread that re-signals `doWork`. Spans [NtfSubSupervisor.md](modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) (supervisor, worker pools), [AgentStore.md](modules/Simplex/Messaging/Agent/Store/AgentStore.md) (updated_by_supervisor, null-action sentinel), [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (worker framework).
+
+- **Session-aware SMP subscription management**: SMP queue subscriptions are tracked per transport session with session-ID validation at multiple points. `subscribeQueues` groups queues by transport session, subscribes concurrently, then validates `activeClientSession` post-RPC — if the client was replaced during the RPC, results are discarded and converted to temporary errors for retry. `removeClientAndSubs` (disconnect cleanup) only demotes subscriptions whose session ID matches the disconnecting client. Batch UP notifications are accumulated across transmissions and deduplicated against already-active subscriptions. When ALL results are temporary errors and no connections were already active, the SMP client is closed to force fresh connection. `maxPending` throttles concurrent pending subscriptions with STM retry backpressure. Spans [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (subscription state, session validation), [Agent.md](modules/Simplex/Messaging/Agent.md) (subscriber loop, processSMPTransmissions, UP accumulation).
+
+- **Agent message envelope**: Agent messages use a two-layer format — outer `AgentMsgEnvelope` (version + type tag C/M/I/R + payload) and inner `AgentMessage` (after double-ratchet decryption, tags I/D/R/M + AMessage). Tag characters deliberately overlap between layers (disambiguated by context). `AgentInvitation` uses only per-queue E2E encryption (no ratchet established yet); `AgentRatchetKey` uses per-queue E2E (can't use ratchet to renegotiate ratchet); `AgentConfirmation` uses double ratchet. PQ support *shrinks* message size budgets (ratchet header + reply link grow with SNTRUP761 keys). `AEvent` is a GADT indexed by `AEntity` — prevents file events on connection entities at the type level. Spans [Agent/Protocol.md](modules/Simplex/Messaging/Agent/Protocol.md) (types, encoding, size budgets), [Agent.md](modules/Simplex/Messaging/Agent.md) (four e2e key states dispatch, message processing).
+
+- **Ratchet synchronization protocol**: When the double ratchet gets out of sync (backup restoration, message loss), both parties exchange `AgentRatchetKey` messages with fresh DH keys. Role determination uses hash-ordering: `rkHash(k1, k2)` is computed by both sides — the party with the lower hash initializes the receiving ratchet, the other initializes sending and sends EREADY. This breaks symmetry when both parties simultaneously initiate. State machine: `RSOk`/`RSAllowed`/`RSRequired` → generate keys + reply; `RSStarted` → use stored keys; `RSAgreed` → error (reset to `RSRequired`). EREADY carries `lastExternalSndId` so the peer knows which messages used the old ratchet. `checkRatchetKeyHashExists` prevents processing the same key twice. Successful message decryption resets sync state to `RSOk` (the recovery signal). Spans [Agent.md](modules/Simplex/Messaging/Agent.md) (newRatchetKey, ereadyMsg, resetRatchetSync), [Agent/Protocol.md](modules/Simplex/Messaging/Agent/Protocol.md) (AgentRatchetKey type, cryptoErrToSyncState classification).

spec/agent.md

@@ -0,0 +1,191 @@
+# Agent Architecture
+
+The SimpleX Agent is the Layer 3 connection manager. It builds duplex encrypted connections on top of Layer 2 client libraries. This document shows its internal architecture: component topology and message processing flows.
+
+For usage and API overview, see [docs/AGENT.md](../docs/AGENT.md). For protocol specifications, see [Agent Protocol](../protocol/agent-protocol.md), [PQDR](../protocol/pqdr.md).
+
+**Split-phase encryption**: Message sending separates ratchet advancement (API thread, serialized) from body encryption (delivery worker, parallel). This prevents ratchet lock contention across queues while maintaining correct message ordering. See [infrastructure.md](agent/infrastructure.md#message-delivery).
+
+**Worker taxonomy**: Three worker families handle background operations - delivery workers (per send queue), async command workers (per connection), and NTF workers (per server). All use the same create-or-reuse pattern with restart rate limiting. See [infrastructure.md](agent/infrastructure.md#worker-framework).
+
+**Suspension cascade**: Operations drain in dependency order: `AORcvNetwork` → `AOMsgDelivery` → `AOSndNetwork` → `AODatabase`. Suspending receive processing cascades through to database access, ensuring clean shutdown. See [infrastructure.md](agent/infrastructure.md#operation-suspension-cascade).
+
+---
+
+**Module specs**: [Agent](modules/Simplex/Messaging/Agent.md) · [Agent Client](modules/Simplex/Messaging/Agent/Client.md) · [Agent Protocol](modules/Simplex/Messaging/Agent/Protocol.md) · [Store Interface](modules/Simplex/Messaging/Agent/Store/Interface.md) · [NtfSubSupervisor](modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) · [XFTP Agent](modules/Simplex/FileTransfer/Agent.md) · [Ratchet](modules/Simplex/Messaging/Crypto/Ratchet.md)
+
+### Agent components
+
+![Agent components](diagrams/agent.svg)
+
+### Message receive flow
+
+```mermaid
+sequenceDiagram
+    participant R as SMP Router
+
+    box Agent
+        participant SC as smpClients<br>(ProtocolClient pool)
+        participant MQ as msgQ<br>(TBQueue)
+        participant S as subscriber
+        participant St as Store
+        participant SQ as subQ<br>(TBQueue)
+    end
+
+    participant App as Application
+
+    R->>SC: MSG (encrypted packet)
+    SC->>MQ: write batch
+
+    S->>MQ: read batch
+    S->>S: withConnLock<br>(serialize per connection)
+    S->>St: load ratchet state<br>(lockConnForUpdate)
+    S->>S: agentRatchetDecrypt<br>(double ratchet)
+    S->>S: checkMsgIntegrity<br>(sequence + hash chain)
+    S->>St: store received message,<br>update ratchet
+    S->>SQ: write AEvt (MSG + metadata)
+
+    App->>SQ: read event
+
+    Note over App: application processes message
+
+    App->>S: ackMessage (agentMsgId)
+    Note over S,R: ACK is async<br>(enqueued as internal command)
+    S->>SC: ACK
+    SC->>R: ACK
+```
+
+### Message send flow
+
+```mermaid
+sequenceDiagram
+    participant App as Application
+
+    box Agent
+        participant API as sendMessage
+        participant St as Store
+        participant DW as deliveryWorker<br>(per send queue)
+        participant SC as smpClients<br>(ProtocolClient pool)
+    end
+
+    participant R as SMP Router
+
+    App->>API: sendMessage<br>(connId, body)
+    API->>St: agentRatchetEncryptHeader<br>(advance ratchet, store<br>encrypt key + pending message)
+    API->>DW: signal doWork (TMVar)
+    API->>App: return msgId
+
+    DW->>St: getPendingQueueMsg
+    DW->>DW: rcEncryptMsg<br>(encrypt body with stored key)
+    DW->>DW: encode AgentMsgEnvelope
+    DW->>SC: sendAgentMessage<br>(per-queue encrypt + SEND)
+    SC->>R: SEND (encrypted packet)
+    R->>SC: OK
+
+    DW->>St: delete pending message
+    DW->>App: SENT msgId (via subQ)
+```
+
+### Connection establishment flow
+
+```mermaid
+sequenceDiagram
+    participant A as Alice (initiator)
+
+    box Agent A
+        participant AA as Agent
+    end
+
+    participant SMP as SMP Router
+
+    box Agent B
+        participant AB as Agent
+    end
+
+    participant B as Bob (joiner)
+
+    A->>AA: createConnection
+    AA->>SMP: NEW<br>(Alice's receive queue)
+    SMP->>AA: queue ID + keys
+    AA->>A: invitation URI<br>(queue address + DH keys)
+
+    Note over A,B: invitation passed out-of-band<br>(QR code, link)
+
+    B->>AB: joinConnection<br>(invitation)
+    AB->>AB: initSndRatchet<br>(PQ X3DH key agreement)
+    AB->>SMP: SKEY (sender auth on<br>Alice's queue)
+    AB->>SMP: NEW<br>(Bob's receive queue)
+    SMP->>AB: queue ID
+    AB->>SMP: SEND confirmation to<br>Alice's queue (Bob's queue<br>address + ratchet keys)
+
+    SMP->>AA: MSG (confirmation)
+    AA->>AA: initRcvRatchet<br>(PQ X3DH key agreement),<br>decrypt confirmation
+    AA->>A: CONF (request approval)
+    A->>AA: allowConnection(confId)
+    AA->>SMP: KEY (register sender key<br>on Alice's rcv queue)
+    AA->>SMP: SKEY (sender auth on<br>Bob's queue)
+    AA->>SMP: SEND reply to Bob's queue<br>(Alice's connection info)
+
+    SMP->>AB: MSG (reply)
+    AB->>SMP: KEY (register sender key<br>on Bob's rcv queue)
+    AB->>SMP: SEND HELLO to Alice
+
+    SMP->>AA: MSG (HELLO)
+    AA->>SMP: SEND HELLO to Bob
+    AA->>A: CON (connected)
+
+    SMP->>AB: MSG (HELLO)
+    AB->>B: CON (connected)
+```
+
+### File delivery flow (XFTP)
+
+```mermaid
+sequenceDiagram
+    participant SA as Sender App
+
+    box Sender Agent
+        participant S as xftpSnd workers
+        participant SS as Store
+    end
+
+    participant XFTP as XFTP Routers
+    participant SMP as SMP Router
+
+    box Receiver Agent
+        participant RS as Store
+        participant R as xftpRcv workers
+    end
+
+    participant RA as Receiver App
+
+    SA->>S: xftpSendFile(file)
+    S->>S: encrypt file<br>(XSalsa20-Poly1305, random key + nonce)
+    S->>S: split into chunks<br>(fixed sizes: 64KB - 4MB)
+    S->>SS: store SndFile + chunks
+
+    loop each chunk
+        S->>XFTP: FNEW (create data packet)
+        XFTP->>S: sender ID + recipient IDs
+        S->>XFTP: FPUT (upload encrypted chunk)
+    end
+
+    S->>S: assemble FileDescription<br>(chunk locations, replicas,<br>encryption key + nonce)
+    S->>SA: SFDONE<br>(sender + recipient descriptions)
+
+    Note over SA,RA: recipient description sent as<br>SMP message (encrypted, via double ratchet)
+
+    SA->>SMP: description in A_MSG
+    SMP->>RA: description in MSG
+
+    RA->>R: xftpReceiveFile<br>(description)
+    R->>RS: store RcvFile + chunks
+
+    loop each chunk (parallel per server)
+        R->>XFTP: FGET (per-recipient auth key)
+        XFTP->>R: encrypted chunk stream
+    end
+
+    R->>R: stream chunks through<br>stateful decrypt (key + nonce),<br>verify auth tag at end
+    R->>RA: RFDONE<br>(decrypted file path)
+```

spec/agent/connections.md

@@ -0,0 +1,232 @@
+# Agent Connections
+
+Duplex connection lifecycle: establishment, queue rotation, ratchet synchronization, and message integrity. These cross-module flows span the Agent, protocol client, and store layers.
+
+For per-module details: [Agent](../modules/Simplex/Messaging/Agent.md) · [Agent Protocol](../modules/Simplex/Messaging/Agent/Protocol.md) · [Ratchet](../modules/Simplex/Messaging/Crypto/Ratchet.md) · [Store Interface](../modules/Simplex/Messaging/Agent/Store/Interface.md). For the component diagram, see [agent.md](../agent.md). For protocol specification, see [Agent Protocol](../../protocol/agent-protocol.md) and [PQDR](../../protocol/pqdr.md).
+
+- [Design constraints](#design-constraints)
+- [Connection establishment](#connection-establishment)
+- [Queue rotation](#queue-rotation)
+- [Ratchet synchronization](#ratchet-synchronization)
+- [Message envelope hierarchy](#message-envelope-hierarchy)
+- [Integrity chain](#integrity-chain)
+
+---
+
+## Design constraints
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/Client.hs](../../src/Simplex/Messaging/Agent/Client.hs)
+
+Two properties of the protocol drive much of the agent's complexity:
+
+**TOFU retry safety**: Queues and links are secured via trust-on-first-use - the router accepts the first key presented (SKEY, KEY) and rejects any subsequent different key. If a network call succeeds but the response is lost, the client must retry with the *same* key, or the router will reject it. This means all cryptographic keys must be generated and persisted *before* the network call that uses them. The agent's pervasive store-then-execute pattern (`enqueueCommand` persists to DB, then worker executes with stored keys) exists primarily to satisfy this constraint.
+
+**Network asymmetry**: After a client sends a message to a queue, the peer's response can arrive at the agent before the originating API call returns to the application. The application must already know the connection exists when it receives the event, otherwise it gets handshake events for an unknown connection. This drives split-phase APIs where the connection is registered locally before any network call.
+
+Together, these constraints explain why the agent separates key generation from network operations, why commands are persisted before execution, and why connection creation is split into prepare + create phases.
+
+---
+
+## Connection establishment
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/Protocol.hs](../../src/Simplex/Messaging/Agent/Protocol.hs)
+
+### Split-phase connection creation
+
+Connection creation is split into two phases to satisfy both design constraints:
+
+**`prepareConnectionLink`** (no network, no database): generates root Ed25519 signing key pair and queue-level X25519 DH keys. Derives a short link key as `SHA3_256` of the encoded fixed link data. Returns the connection link URI and `PreparedLinkParams` in memory. The application can now embed the link in link data (e.g., for short link resolution) before the queue exists.
+
+**`createConnectionForLink`** (single network call): uses the prepared parameters to create the queue on the router with NEW (root signing key as owner auth). The sender ID is deterministically derived from the correlation nonce (`SMP.EntityId $ B.take 24 $ C.sha3_384 corrId`), so a lost response can be retried - the router validates the same sender ID.
+
+Without split-phase, the application would need to create the queue first, get the link, then update the queue with link data containing the link - requiring an extra round-trip.
+
+### Standard handshake
+
+The connection establishment flow is shown in [agent.md](../agent.md#connection-establishment-flow). The key non-obvious details:
+
+**Ratchet initialization is asymmetric**: The initiator (Alice) generates X3DH key parameters during `newRcvConnSrv` and stores them via `createRatchetX3dhKeys`, but does not initialize any ratchet yet. The *receiving* ratchet is only initialized later in `smpConfirmation` via `initRcvRatchet` when the confirmation arrives with the responder's parameters. The responder (Bob) initializes a *sending* ratchet during `startJoinInvitation` via `initSndRatchet`. The names `RcvE2ERatchetParams`/`SndE2ERatchetParams` are historical - what matters is that the responder initializes first (sending direction), and the initiator initializes second (receiving direction) using the responder's parameters.
+
+**Confirmation decryption proves key agreement**: In `smpConfirmation`, the initiator creates a fresh receiving ratchet from the responder's parameters and immediately uses it to decrypt the confirmation body. If decryption fails, the entire confirmation is rejected - there is no state where a connection has mismatched ratchets.
+
+**HELLO exchange completes the handshake**: After `allowConnection`, both sides have duplex queues but haven't confirmed liveness. The responder (Bob) sends the first HELLO (with `notification = True` in MsgFlags), triggered by `ICDuplexSecure`. The initiator (Alice) receives it and sends her own HELLO back (also with `notification = True`). The initiator emits `CON` in the *delivery callback* of her HELLO (her rcvQueue is already Active from receiving Bob's HELLO). The responder emits `CON` when he *receives* the initiator's reply HELLO (his sndQueue is already Active from his own HELLO delivery). There are exactly two HELLO messages.
+
+### Contact URI async path
+
+For contact URIs (`joinConnectionAsync` with `CRContactUri`), the join is enqueued as an async command. The connection record is created locally (NewConnection state) before the network call, satisfying the network asymmetry constraint. The background worker then creates the receive queue, sends the invitation, and processes the handshake.
+
+### PQ key agreement
+
+PQ support is negotiated via version numbers: `agentVersion >= pqdrSMPAgentVersion && e2eVersion >= pqRatchetE2EEncryptVersion`. When both sides support PQ, the KEM public key travels in the confirmation body (too large for invitation URI). The responder encapsulates, producing `(kemCiphertext, kemSharedKey)`, and the hybrid key is derived via HKDF-SHA512 over the concatenation of three X3DH shared secrets plus the KEM shared secret, with info string `"SimpleXX3DH"`.
+
+**PQ support is monotonic**: once enabled for a connection (`PQSupport PQSupportOn`), it cannot be downgraded. This affects header padding size (88 bytes without PQ vs 2310 bytes with PQ).
+
+### Connection type state machine
+
+```
+NewConnection
+  +-> RcvConnection (initiator, after newRcvConnSrv)
+  |     +-> DuplexConnection (after allowConnection + connectReplyQueues)
+  |     +-> ContactConnection (contact address case)
+  +-> SndConnection (responder, before reply queue created)
+  |     +-> DuplexConnection (after reply queue created)
+  +-> ContactConnection (short link / contact address)
+```
+
+---
+
+## Queue rotation
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/Store.hs](../../src/Simplex/Messaging/Agent/Store.hs)
+
+Queue rotation replaces a receive queue with a new one on a different router, providing forward secrecy for the transport layer. The protocol uses a 4-message handshake.
+
+### Protocol sequence
+
+Rotation is initiated by the switching party calling `switchConnectionAsync` (client API), which sends QADD. The peer responds to QADD by creating a new send queue and replying with QKEY. Preconditions: connection must be duplex, no switch already in progress, ratchet must not be syncing.
+
+```
+Receiver (switching party)              Sender (peer)
+    |                                       |
+    |-- QADD (new queue address) ---------->|
+    |                                       |-- creates SndQueue to new address
+    |<--------- QKEY (sender auth key) ----|
+    |                                       |
+    |-- secures new queue (ICQSecure) ----->|
+    |-- QUSE (start using new queue) ----->|
+    |                                       |-- switches delivery to new queue
+    |<--------- QTEST (on new queue) ------|
+    |                                       |
+    |-- deletes old queue (ICQDelete) ---->|
+```
+
+### State machines
+
+**Receiver (RcvQueue switch)**: `RSSwitchStarted` -> `RSSendingQADD` -> `RSSendingQUSE` -> `RSReceivedMessage`. The switch becomes non-abortable at `RSSendingQUSE` - by this point the sender may have already deleted the old queue, so aborting would break the connection. `canAbortRcvSwitch` enforces this.
+
+**Sender (SndQueue switch)**: creates new SndQueue on QADD, sends QKEY, marks old as `SSSendingQKEY`. On QUSE: sends QTEST *only to the new queue*, marks as `SSSendingQTEST`. Completes when QTEST delivery succeeds.
+
+### Consecutive rotation handling
+
+`dbReplaceQueueId` tracks which old queue a new one replaces. Each new queue stores `dbReplaceQueueId = Just oldQueueId`. When QADD is processed, send queues whose `dbReplaceQueueId` points to the current queue's `dbQueueId` are found and deleted in bulk. This handles consecutive rotation requests - only the latest rotation survives.
+
+### Old queue deletion
+
+Three triggers delete the old queue:
+1. **Sender-side**: QTEST delivery succeeds - old queue removed from `smpDeliveryWorkers` (worker thread stops)
+2. **Receiver-side**: first message arrives on new queue - receiver marks old queue for deletion via `ICQDelete`
+3. **Abort cleanup**: `abortConnectionSwitch` explicitly deletes new queues created during a failed switch attempt
+
+---
+
+## Ratchet synchronization
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/Protocol.hs](../../src/Simplex/Messaging/Agent/Protocol.hs)
+
+When double ratchet state becomes desynchronized (e.g., one side restores from backup), the agent can re-establish the ratchet without breaking the connection.
+
+### State machine
+
+```
+RSOk (synchronized)
+  |
+  v  (crypto error detected)
+RSAllowed / RSRequired
+  |
+  v  (synchronizeRatchet called)
+RSStarted (waiting for peer)
+  |
+  v  (peer responds with own keys)
+RSAgreed (both exchanged keys)
+  |
+  v  (ratchet recreated, EREADY sent/received)
+RSOk
+```
+
+**Send prohibition**: `ratchetSyncSendProhibited` returns `True` for `RSRequired`, `RSStarted`, and `RSAgreed`. This blocks *all* messages including queue rotation messages - preventing state corruption while the ratchet is being re-established.
+
+### Key exchange protocol
+
+1. Initiator calls `synchronizeRatchet`, which generates new X3DH keys and sends them in an `AgentRatchetKey` envelope (discriminant `'R'`). State becomes `RSStarted`.
+2. Peer receives the ratchet key in `newRatchetKey`. If peer hasn't started sync, it generates own keys and sends a reply `AgentRatchetKey`.
+3. Both sides now have each other's keys. State becomes `RSAgreed`.
+
+### Hash-ordered role assignment
+
+Both parties compute `rkHash = SHA256(pubKeyBytes k1 || pubKeyBytes k2)` for their own keys. The party with the *smaller* hash initializes the receiving ratchet (`pqX3dhRcv`); the party with the larger hash initializes the sending ratchet (`pqX3dhSnd`) and sends `EREADY`. This deterministic tie-breaking avoids a separate negotiation round.
+
+### EREADY completion
+
+`EREADY` carries `lastExternalSndId` - the ID of the last message the sender received from the peer before switching ratchets. The receiving party uses this to know when the old ratchet's messages are exhausted and the new ratchet is fully active. Until EREADY arrives, messages may arrive encrypted with either the old or new ratchet.
+
+### Error recovery
+
+- **Crypto error during decrypt**: `cryptoErrToSyncState` classifies the error and sets state to `RSAllowed` or `RSRequired`. Client is notified via `RSYNC`.
+- **Successful decrypt during non-RSOk state**: if state is not `RSStarted` (which means sync is actively in progress), reset to `RSOk`. A successful message proves the ratchets are synchronized.
+- **Duplicate handling**: `rkHash` of received keys is checked against stored hashes to prevent reprocessing the same ratchet key message.
+
+---
+
+## Message envelope hierarchy
+
+**Source**: [Agent/Protocol.hs](../../src/Simplex/Messaging/Agent/Protocol.hs)
+
+Messages use three nesting levels, each adding a layer of structure:
+
+### Level 1: AgentMsgEnvelope (transport)
+
+Four variants with single-character discriminants:
+
+| Variant | Disc. | Encryption | When |
+|---------|-------|-----------|------|
+| `AgentConfirmation` | `'C'` | Per-queue E2E (outer) + double ratchet (inner `encConnInfo`) | Connection handshake |
+| `AgentMsgEnvelope` | `'M'` | Double ratchet | Normal messages |
+| `AgentInvitation` | `'I'` | Per-queue E2E only | Contact URI join |
+| `AgentRatchetKey` | `'R'` | Per-queue E2E only | Ratchet sync |
+
+`AgentMsgEnvelope` is fully double-ratchet encrypted. `AgentConfirmation` uses per-queue E2E for the outer envelope but also contains `encConnInfo` which is double-ratchet encrypted (the ratchet is initialized during confirmation processing). `AgentInvitation` and `AgentRatchetKey` use only per-queue E2E - the double ratchet is either not yet established or being replaced.
+
+### Level 2: AgentMessage (application)
+
+Inside the decrypted envelope:
+- `AgentConnInfo` / `AgentConnInfoReply` - connection info during handshake (double-ratchet encrypted inside `encConnInfo`)
+- `AgentRatchetInfo` - ratchet sync payload (not double-ratchet encrypted)
+- `AgentMessage APrivHeader AMessage` - user and control messages (double-ratchet encrypted)
+
+The private header (`APrivHeader`) carries `sndMsgId` and `prevMsgHash` for the integrity chain.
+
+### Level 3: AMessage (semantic)
+
+Message types with 1-2 character discriminants:
+- User messages: `HELLO_`, `A_MSG_`, `A_RCVD_`, `A_QCONT_`, `EREADY_`
+- Queue rotation: `QADD_`, `QKEY_`, `QUSE_`, `QTEST_`
+
+### ACK semantics
+
+- **User messages** (`A_MSG_`): NOT auto-ACKed. Agent returns `ACKPending`; application must call `ackMessage`.
+- **Receipts** (`A_RCVD`): returns `ACKPending` when valid receipts are present (application must ACK after processing); auto-ACKed only when all receipts fail.
+- **Other control messages** (HELLO, QADD, QKEY, QUSE, QTEST, EREADY, A_QCONT): auto-ACKed by the agent.
+- **Error during processing**: `handleNotifyAck` sends `ERR` to the application but still ACKs to the router, preventing re-delivery of a message that will fail again.
+
+---
+
+## Integrity chain
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/Protocol.hs](../../src/Simplex/Messaging/Agent/Protocol.hs)
+
+Each message in a connection commits to the previous message via two mechanisms:
+
+1. **External sender ID** (`lastExternalSndId`): monotonically increasing counter per connection
+2. **Previous message hash** (`prevMsgHash`): SHA256 of the previous message body
+
+`checkMsgIntegrity` produces one of five outcomes:
+
+| Outcome | Condition |
+|---------|-----------|
+| `MsgOk` | Sequential ID and matching hash |
+| `MsgBadId` | ID from the past (less than previous) |
+| `MsgDuplicate` | Same ID as previous |
+| `MsgSkipped` | Gap in IDs (messages lost) |
+| `MsgBadHash` | Sequential ID but hash mismatch |
+
+**Non-rejecting semantics**: the agent does NOT reject messages with integrity failures. The result is reported to the application via `MsgMeta.integrity`. The application decides the policy - warn, ignore, or terminate the connection.

spec/agent/infrastructure.md

@@ -0,0 +1,211 @@
+# Agent Infrastructure
+
+The Agent's internal machinery: worker lifecycle, command dispatch, message delivery, subscription tracking, operation suspension, protocol client management, and dual-backend store. These cross-module patterns are not visible from any single module spec.
+
+This document covers the "big agent" (`Agent.hs` + `Agent/Client.hs`) used in client applications. The "small agent" (`SMPClientAgent`) used in routers is documented in [clients.md](../clients.md).
+
+For per-module details: [Agent](../modules/Simplex/Messaging/Agent.md) · [Agent Client](../modules/Simplex/Messaging/Agent/Client.md) · [Store Interface](../modules/Simplex/Messaging/Agent/Store/Interface.md) · [NtfSubSupervisor](../modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) · [XFTP Agent](../modules/Simplex/FileTransfer/Agent.md). For the component diagram, see [agent.md](../agent.md).
+
+- [Worker framework](#worker-framework)
+- [Async command processing](#async-command-processing)
+- [Message delivery](#message-delivery)
+- [Subscription tracking](#subscription-tracking)
+- [Operation suspension cascade](#operation-suspension-cascade)
+- [SessionVar lifecycle](#sessionvar-lifecycle)
+- [Dual-backend store](#dual-backend-store)
+
+---
+
+## Worker framework
+
+**Source**: [Agent/Client.hs](../../src/Simplex/Messaging/Agent/Client.hs), [Agent/Env/SQLite.hs](../../src/Simplex/Messaging/Agent/Env/SQLite.hs) (Worker type)
+
+All agent background processing - async commands, message delivery, notification workers, XFTP workers - uses a shared worker infrastructure defined in `Agent/Client.hs`.
+
+**Create-or-reuse**: `getAgentWorker` atomically checks a `TMap` for an existing worker keyed by the work item (connection+server, send queue address, etc.). If absent, creates a new `Worker` with a unique monotonic `workerId` from `workerSeq` and inserts it. If present and `hasWork=True`, signals the existing worker via `tryPutTMVar doWork ()`.
+
+**Fork and run**: `runWorkerAsync` uses bracket on the worker's `action` TMVar. If the taken value is `Nothing`, the worker is idle - start it. If `Just _`, it's already running - put it back and return. The `action` TMVar holds `Just (Weak ThreadId)` to avoid preventing GC of the worker thread.
+
+**Task retrieval race prevention**: `withWork` clears the `doWork` flag *before* calling `getWork` (not after). This prevents a race: query finds nothing → another thread adds work + signals → worker clears flag (losing the signal). By clearing first, any signal that arrives during the query is preserved.
+
+**Error classification**: `withWork` distinguishes two failure modes:
+- *Work-item error* (`isWorkItemError`): the task itself is broken (likely recurring). Worker stops and sends `CRITICAL False`.
+- *Other error*: any non-work-item error (e.g., transient database issue). Worker re-signals `doWork` and reports `INTERNAL` (retry may succeed).
+
+**Restart rate limiting**: On worker exit, `restartOrDelete` checks the `restarts` counter against `maxWorkerRestartsPerMin`. Under the limit: reset action, re-signal, restart. Over the limit: delete the worker from the map and send `CRITICAL True` (escalation to the application). A restart only proceeds if the `workerId` in the map still matches the current worker - a stale restart from a replaced worker is a no-op.
+
+**Consumers**: Four families use this framework:
+- Async command workers - keyed by `(ConnId, Maybe SMPServer)`, in `asyncCmdWorkers` TMap
+- Delivery workers - keyed by `SndQAddr`, in `smpDeliveryWorkers` TMap, paired with a `TMVar ()` retry lock
+- NTF workers - three pools (`ntfWorkers` per NTF server, `ntfSMPWorkers` per SMP server, `ntfTknDelWorkers` for token deletion) in `NtfSubSupervisor`
+- XFTP workers - three worker types (rcv, snd, del) with TMVar-based connection sharing
+
+---
+
+## Async command processing
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/Protocol.hs](../../src/Simplex/Messaging/Agent/Protocol.hs) (command types), [Agent/Store.hs](../../src/Simplex/Messaging/Agent/Store.hs) (internal command types)
+
+Async commands handle state transitions that require network calls but shouldn't block the API thread: securing queues, deleting old queues during rotation, acknowledging messages. The dispatch loop `runCommandProcessing` runs one worker per `(ConnId, Maybe SMPServer)` key.
+
+**Enqueueing**: API functions call `enqueueCommand`, which persists the command to the `commands` table (crash-safe) and spawns/wakes the worker via `getAsyncCmdWorker`. On agent startup, `resumeAllCommands` fetches all pending commands grouped by connection+server and signals their workers.
+
+**Command types**: Two categories share the same dispatch loop:
+- *Client commands* (`AClientCommand`): `NEW`, `JOIN`, `LET` (allow connection), `ACK`, `LSET`/`LGET` (set/get connection link data), `SWCH` (switch queue), `DEL`. Triggered by application API calls.
+- *Internal commands* (`AInternalCommand`): `ICAck` (ack to router), `ICAckDel` (ack + delete local message), `ICAllowSecure`/`ICDuplexSecure` (secure after confirmation), `ICQSecure` (secure queue during switch), `ICQDelete` (delete old queue after switch), `ICDeleteConn` (delete connection), `ICDeleteRcvQueue` (delete specific receive queue). Generated *during* message processing to handle state transitions asynchronously.
+
+**Retry and movement**: `tryMoveableCommand` wraps execution with `withRetryInterval`. On `temporaryOrHostError`, it retries with backoff. Individual command handlers can return `CCMoved` (e.g., when a queue has moved to a different router) after updating the command's server field in the store - `tryMoveableCommand` then exits cleanly, letting the moved command be picked up by the appropriate worker.
+
+**Locking**: State-sensitive commands use `tryWithLock` / `tryMoveableWithLock`, which acquire `withConnLock` before execution. This serializes operations on the same connection, preventing races between concurrent command processing and message receipt.
+
+**Event overflow**: Events are written directly to `subQ` if there is room. When `subQ` is full, events overflow into a local `pendingCmds` list and are flushed to `subQ` after the command completes, providing backpressure handling.
+
+---
+
+## Message delivery
+
+**Source**: [Agent.hs](../../src/Simplex/Messaging/Agent.hs), [Agent/RetryInterval.hs](../../src/Simplex/Messaging/Agent/RetryInterval.hs)
+
+Message delivery uses a split-phase encryption design: the ratchet advances in the API thread (serialized), while the actual body encryption happens in the per-queue delivery worker (parallel). This avoids ratchet lock contention across queues.
+
+**Phase 1 - API thread** (`enqueueMessageB`):
+1. Encode the agent message with `internalSndId` + `prevMsgHash` (for the receiver's integrity chain)
+2. Call `agentRatchetEncryptHeader` - advances the double ratchet, produces a message encryption key (MEK), padded length, and PQ encryption status
+3. Store `SndMsg` with `SndMsgPrepData` (MEK, paddedLen, sndMsgBodyId) in the database
+4. Create `SndMsgDelivery` record for each send queue
+5. `submitPendingMsg` - increments `msgDeliveryOp.opsInProgress` (for suspension tracking) and signals delivery workers via `getDeliveryWorker`
+
+**Phase 2 - delivery worker** (`runSmpQueueMsgDelivery`):
+1. `throwWhenNoDelivery` - kills the worker thread if the queue's address has been removed from `smpDeliveryWorkers` (prevents delivery to queues replaced during switch)
+2. `getPendingQueueMsg` - fetches the next pending message from the store, resolving the `sndMsgBodyId` reference into the actual message body and constructing `PendingMsgPrepData`
+3. Re-encode the message with `internalSndId`/`prevMsgHash`, then `rcEncryptMsg` to encrypt with the stored MEK (no ratchet access needed)
+4. `sendAgentMessage` - per-queue encrypt + SEND to the router
+
+**Connection info messages** (`AM_CONN_INFO`, `AM_CONN_INFO_REPLY`) skip split-phase encryption entirely - they are sent as per-queue E2E encrypted confirmation bodies via `sendConfirmation` (encrypted with `agentCbEncrypt`, not with the double ratchet).
+
+**Retry with dual intervals**: Delivery uses `withRetryLock2`, which maintains two retry interval states (slow and fast) but only one wait is active at a time. A background thread sleeps for the current interval, then signals the delivery worker via `tryPutTMVar`. When the router sends `QCONT` (queue buffer cleared), the agent calls `tryPutTMVar retryLock ()` to wake the delivery thread immediately, avoiding unnecessary delay.
+
+**Error handling**:
+- `SMP QUOTA` - switch to slow retry, don't penalize (backpressure from router)
+- `SMP AUTH` - permanent failure: for data messages, notify and delete; for handshake messages, report connection error; for queue-switch messages, report queue error
+- `temporaryOrHostError` - retry with backoff
+- Other errors - report to application, delete command
+
+---
+
+## Subscription tracking
+
+**Source**: [Agent/TSessionSubs.hs](../../src/Simplex/Messaging/Agent/TSessionSubs.hs), [Agent/Client.hs](../../src/Simplex/Messaging/Agent/Client.hs)
+
+The agent tracks per-queue subscription state in `TSessionSubs` (defined in `Agent/TSessionSubs.hs`), keyed by `SMPTransportSession = (UserId, SMPServer, Maybe ByteString)` where the `ByteString` carries the entity ID in entity-session mode or `Nothing` in shared mode. Each transport session holds:
+
+```
+SessSubs
+├── subsSessId    :: TVar (Maybe SessionId)  -- TLS session ID
+├── activeSubs    :: TMap RecipientId RcvQueueSub
+├── pendingSubs   :: TMap RecipientId RcvQueueSub
+├── activeServiceSub  :: TVar (Maybe ServiceSub)
+└── pendingServiceSub :: TVar (Maybe ServiceSub)
+```
+
+**State machine**: Subscriptions move between three states:
+
+- **Pending → Active**: After subscription RPC succeeds, `addActiveSub'` promotes the queue - but only if the returned session ID matches the stored TLS session ID (`Just sessId == sessId'`). On mismatch (TLS reconnected between RPC send and response), the subscription is silently added to pending instead. No exception - concurrent resubscription paths handle this naturally.
+
+- **Active → Pending**: When `setSessionId` is called with a *different* session ID (TLS reconnect), all active subscriptions are atomically demoted to pending. Session ID is updated to the new value.
+
+- **Pending → Removed**: `failSubscriptions` moves permanently-failed queues (non-temporary SMP errors) to `removedSubs` - a separate `TMap` in `AgentClient`, not part of `TSessionSubs`. The removal is tracked for diagnostic reporting via `getSubscriptions`.
+
+**Service-associated queues**: Queues with `serviceAssoc=True` are *not* added to `activeSubs` individually. Instead, the service subscription's count is incremented and its `idsHash` XOR-accumulates the queue's hash. The router tracks individual queues via the service subscription; the agent only tracks the aggregate. Consequence: `hasActiveSub(rId)` returns `False` for service-associated queues - callers must check the service subscription separately.
+
+**Disconnect cleanup** (`smpClientDisconnected`):
+1. `removeSessVar` with CAS check (monotonic `sessionVarId` prevents stale callbacks from removing newer clients)
+2. `setSubsPending` - demote active→pending, filtered by matching `SessionId` only
+3. Delete proxied relay sessions created by this client
+4. Fire `DISCONNECT`, `DOWN` (affected connections), `SERVICE_DOWN` (if service sub existed)
+5. Release GET locks for affected queues
+6. Resubscribe: either spawn `resubscribeSMPSession` worker (entity-session mode) or directly resubscribe queues and services (other modes)
+
+**Resubscription worker**: Per-transport-session worker with exponential backoff. Loops until `pendingSubs` and `pendingServiceSub` are both empty. Uses `waitForUserNetwork` with bounded wait - proceeds even without network (prevents indefinite blocking). Worker self-cleans via `removeSessVar` on exit.
+
+**UP event deduplication**: After a batch subscription RPC, `UP` events are emitted only for connections that were *not* already in `activeSubs` before the batch. This prevents duplicate notifications for already-subscribed connections.
+
+---
+
+## Operation suspension cascade
+
+**Source**: [Agent/Client.hs](../../src/Simplex/Messaging/Agent/Client.hs)
+
+Five `AgentOpState` TVars track in-flight operations for graceful shutdown. Each holds `{opSuspended :: Bool, opsInProgress :: Int}`.
+
+**Cascade ordering**:
+```
+AONtfNetwork  (independent - no cascading)
+
+AORcvNetwork → AOMsgDelivery → AOSndNetwork → AODatabase
+```
+
+**Mechanics**: `endAgentOperation` decrements `opsInProgress`. If the count reaches zero and the operation is suspended, it calls the cascade action: `AORcvNetwork` suspends `AOMsgDelivery`, which suspends `AOSndNetwork`, which suspends `AODatabase`. At the leaf (`AODatabase`), `notifySuspended` writes `SUSPENDED` to `subQ` and sets `agentState = ASSuspended`.
+
+**Blocking**: `beginAgentOperation` blocks (STM `retry`) while `opSuspended == True`. This means new operations of a suspended type cannot start - they wait until the operation is resumed. `agentOperationBracket` provides structured bracketing (begin on entry, end on exit).
+
+**Two wait modes**:
+- `waitWhileSuspended` - blocks only during `ASSuspended`, proceeds during `ASSuspending` (allows in-flight operations to complete)
+- `waitUntilForeground` - blocks during both `ASSuspending` and `ASSuspended` (stricter, for operations that need full foreground)
+
+**Usage**: `withStore` brackets all database access with `AODatabase`. Message delivery uses `AOSndNetwork` + `AOMsgDelivery`. Receive processing uses `AORcvNetwork`. This ensures that suspending receive processing cascades through delivery to database, and nothing touches the database after all operations drain.
+
+---
+
+## SessionVar lifecycle
+
+**Source**: [Agent/Client.hs](../../src/Simplex/Messaging/Agent/Client.hs)
+
+Protocol client connections (SMP, XFTP, NTF) use a lazy singleton pattern via `SessionVar` - a `TMVar` in a `TMap` keyed by transport session.
+
+**Connection**: `getSessVar` atomically checks the TMap. Returns `Left newVar` (absent - caller must connect) or `Right existingVar` (present - wait for result). `newProtocolClient` wraps the connection attempt: on success, fills the TMVar with `Right client` and writes `CONNECT` event; on failure, fills with `Left (error, maybeRetryTime)` and re-throws.
+
+**Error caching**: Failed connections cache the error with an expiry timestamp based on `persistErrorInterval`. Future attempts during the interval immediately receive the cached error without reconnecting - this prevents connection storms when a router is down. When `persistErrorInterval == 0`, the SessionVar is removed immediately on failure (fresh connection on next attempt).
+
+**Compare-and-swap**: Each SessionVar has a monotonic `sessionVarId` from `workerSeq`. `removeSessVar` only removes if the `sessionVarId` matches the current map entry. This prevents a stale disconnect callback (from an old client) from removing a newer client that connected after the old one disconnected.
+
+**Service credential synchronization** (`updateClientService`): On SMP reconnect, the agent reconciles service credentials between client and router state - updating, creating, or removing service associations as needed. Router version downgrade (router loses service support) triggers client-side service deletion.
+
+**XFTP special case**: `getProtocolServerClient` ignores the caller's `NetworkRequestMode` parameter for XFTP, always using `NRMBackground` timing. XFTP connections always use background retry timing regardless of the caller's request.
+
+---
+
+## Dual-backend store
+
+**Source**: [Agent/Store/SQLite.hs](../../src/Simplex/Messaging/Agent/Store/SQLite.hs), [Agent/Store/Postgres.hs](../../src/Simplex/Messaging/Agent/Store/Postgres.hs), [Agent/Store/AgentStore.hs](../../src/Simplex/Messaging/Agent/Store/AgentStore.hs)
+
+The agent supports SQLite and PostgreSQL via CPP compilation flags (`#if defined(dbPostgres)`). Three wrapper modules (`Interface.hs`, `Common.hs`, `DB.hs`) re-export the appropriate backend. A single binary compiles with one active backend.
+
+**Key behavioral differences**:
+
+| Aspect | SQLite | PostgreSQL |
+|--------|--------|------------|
+| Row locking | Single-writer model (no locking needed) | `FOR UPDATE` on reads preceding writes |
+| Batch queries | Per-row `forM` loops | `IN ?` with `In` wrapper |
+| Constraint violations | `SQL.ErrorConstraint` pattern match | `constraintViolation` function |
+| Transaction savepoints | Not needed | Used in `createWithRandomId'` (failed statement aborts entire transaction without them) |
+| Busy/locked errors | `ErrorBusy`/`ErrorLocked` → `SEDatabaseBusy` → `CRITICAL True` | All SQL errors → `SEInternal` |
+
+**Store access bracketing**: `withStore` wraps all database operations with `agentOperationBracket AODatabase`, connecting the store to the suspension cascade. `withStoreBatch` / `withStoreBatch'` run multiple operations in a single transaction with per-operation error catching.
+
+**Known bug**: `checkConfirmedSndQueueExists_` uses `#if defined(dpPostgres)` (typo - should be `dbPostgres`), so the `FOR UPDATE` clause is never included on either backend.
+
+### Migration framework
+
+**Source**: [Agent/Store/Migrations.hs](../../src/Simplex/Messaging/Agent/Store/Migrations.hs), [Agent/Store/Shared.hs](../../src/Simplex/Messaging/Agent/Store/Shared.hs)
+
+Migrations are Haskell modules under `Agent/Store/SQLite/Migrations/` and `Agent/Store/Postgres/Migrations/`. Each has `up` SQL and optional `down` SQL.
+
+**Key behaviors**:
+
+- `migrationsToRun` compares app migrations against the `migrations` table by name. Divergent histories (app has `[a,b]`, DB has `[a,c]`) produce `MTREDifferent` error - manual intervention required.
+- Each migration runs in its own transaction with the `migrations` insert *before* the schema change - failure rolls back both.
+- Downgrades require all intermediate migrations to have `down` SQL; missing any produces `MTRENoDown`.
+- `MigrationConfirmation` controls whether upgrades/downgrades auto-apply, prompt, or error.
+
+**Special case**: `m20220811_onion_hosts` triggers `updateServers` to expand host entries with Tor addresses - this is data migration, not just schema.

spec/agent/xrcp.md

@@ -0,0 +1,99 @@
+# XRCP - Cross-Device Remote Control
+
+XRCP enables a desktop application to control a mobile device over the local network. The protocol establishes an encrypted session between two devices using TLS, post-quantum hybrid key exchange, and optional multicast discovery.
+
+This document covers the cross-module flows that are not visible from individual module specs. For message formats and cryptographic operations, see [protocol/xrcp.md](../../protocol/xrcp.md). For per-module details: [Client](../modules/Simplex/RemoteControl/Client.md) · [Invitation](../modules/Simplex/RemoteControl/Invitation.md) · [Discovery](../modules/Simplex/RemoteControl/Discovery.md) · [Types](../modules/Simplex/RemoteControl/Types.md).
+
+**Terminology note**: in the code, "host" is the mobile device (being controlled) and "ctrl" is the desktop (controlling). The protocol spec uses the reverse convention - "host" serves, "controller" connects. This document uses the code convention.
+
+- [Session handshake flow](#session-handshake-flow)
+- [KEM hybrid key exchange](#kem-hybrid-key-exchange)
+- [Multicast discovery](#multicast-discovery)
+- [Block framing and padding](#block-framing-and-padding)
+
+---
+
+## Session handshake flow
+
+**Source**: [RemoteControl/Client.hs](../../src/Simplex/RemoteControl/Client.hs), [RemoteControl/Discovery.hs](../../src/Simplex/RemoteControl/Discovery.hs)
+
+The handshake spans `Client.connectRCHost` (controller side, despite the name), `Client.connectRCCtrl` (host side), `Invitation.mkInvitation`, and `Discovery.startTLSServer`. The full sequence:
+
+1. **Controller starts TLS server**: generates ephemeral session keys + DH keys, creates a signed invitation containing the CA fingerprint and identity key, starts a TLS server on an ephemeral port. The TLS hook `onNewHandshake` enforces single-session - a second connection attempt is rejected by checking whether the session TMVar is already filled.
+
+2. **Invitation delivery**: the invitation reaches the host either out-of-band (QR code scan for first pairing) or via encrypted multicast announcement (subsequent sessions - see [Multicast discovery](#multicast-discovery)).
+
+3. **Host connects via TLS**: `connectRCCtrl` establishes a TLS connection. Both sides validate certificate chains. On the controller side, `onClientCertificate` explicitly checks for a 2-certificate chain (leaf + CA root) and validates the host's CA fingerprint against `KnownHostPairing.hostFingerprint` (or stores it on first pairing). On the host side, the controller's CA fingerprint is validated against `RCCtrlPairing.ctrlFingerprint` in `updateCtrlPairing`.
+
+4. **User confirmation barrier**: after TLS connects, both sides extract the TLS channel binding (`tlsUniq`) as a session code. The application displays this code on both devices for the user to verify. On the host side, `confirmCtrlSession` uses a double `putTMVar` - the first put signals the decision (accept/reject), the second blocks until the session thread acknowledges the value, ensuring `confirmCtrlSession` does not return prematurely.
+
+5. **Hello exchange** (asymmetric encryption):
+  - Host sends `RCHostEncHello` (`prepareHostHello`): DH public key in plaintext + encrypted body containing the KEM encapsulation key, CA fingerprint, and app info. Encrypted with `cbEncrypt` (classical DH secret).
+  - Controller decrypts the hello, performs KEM encapsulation (see [KEM hybrid key exchange](#kem-hybrid-key-exchange)), derives the hybrid session key, initializes a chain via `sbcInit`, and sends `RCCtrlEncHello` (`prepareHostSession`) encrypted with a key derived from the chain (`sbcHkdf` + `sbEncrypt`).
+  - The asymmetry is deliberate: at the time the host sends its hello, KEM hasn't completed yet, so only classical DH encryption is available. After the controller encapsulates, both sides have the hybrid key.
+
+6. **Chain key initialization**: both sides call `sbcInit` with the hybrid key to derive send/receive chain keys. The host explicitly **swaps** the key pair (`swap` call in `prepareCtrlSession`, which runs on the host side despite its name) - both sides derive keys in the same order from `sbcInit`, but have opposite send/receive roles, so the host must reverse them. The controller does not swap.
+
+7. **Error path**: if KEM encapsulation fails, the controller sends `RCCtrlEncError` (a variant of `RCCtrlEncHello`) encrypted with the DH key (not the hybrid key, which doesn't exist yet). The host can decrypt the error because it has the DH secret from step 5. Note: this error path is not yet fully implemented in the code.
+
+---
+
+## KEM hybrid key exchange
+
+**Source**: [RemoteControl/Client.hs](../../src/Simplex/RemoteControl/Client.hs)
+
+The session key combines classical Diffie-Hellman with SNTRUP761 (lattice-based KEM) via `SHA3_256(dhSecret || kemSharedKey)` (`kemHybridSecret` in `Crypto/SNTRUP761.hs`). This provides protection against quantum computers while maintaining classical security as a fallback.
+
+The KEM public key is too large for a QR code invitation, so it travels in the encrypted hello body. Fresh KEM keys are generated every session - no KEM state is cached between sessions.
+
+1. Host generates a fresh KEM key pair (`prepareHostHello`), puts the KEM public key in the host hello body
+2. Controller decrypts hello with DH secret, extracts KEM public key
+3. Controller encapsulates (`sntrup761Enc`): produces `(kemCiphertext, kemSharedKey)`
+4. Controller derives hybrid key: `SHA3_256(dhSecret || kemSharedKey)`
+5. Controller sends `kemCiphertext` in the ctrl hello body (`RCCtrlEncHello`)
+6. Host decapsulates `kemCiphertext` (`sntrup761Dec`) to recover `kemSharedKey`, derives the same hybrid key
+
+The KEM exchange is identical for first and subsequent sessions. The only difference between sessions is how the invitation is delivered (QR code vs multicast) and whether TLS fingerprints are stored for the first time or verified against known pairings.
+
+`updateKnownHost` (called in `prepareHostSession` on the controller) updates the stored host DH public key (`hostDhPubKey` in `KnownHostPairing`) - this is used for encrypting multicast announcements in subsequent sessions, not for KEM.
+
+**Key rotation and `prevDhPrivKey`**: when the host updates its DH key pair for a new session, it retains the previous private key in `RCCtrlPairing.prevDhPrivKey`. This is critical for multicast - during the transition window, the controller may send announcements encrypted with the old public key. `findRCCtrlPairing` tries decryption with both the current and previous DH keys. Without this fallback, key rotation would break multicast discovery.
+
+---
+
+## Multicast discovery
+
+**Source**: [RemoteControl/Client.hs](../../src/Simplex/RemoteControl/Client.hs), [RemoteControl/Invitation.hs](../../src/Simplex/RemoteControl/Invitation.hs), [RemoteControl/Discovery.hs](../../src/Simplex/RemoteControl/Discovery.hs)
+
+For subsequent sessions (after initial QR pairing), the controller announces its presence via UDP multicast so the host can connect without scanning a new QR code. The flow spans `Client.announceRC`, `Client.discoverRCCtrl`, `Client.findRCCtrlPairing`, `Invitation.signInvitation`/`verifySignedInvitation`, and `Discovery.withListener`/`withSender`.
+
+**Announcement creation** (`announceRC`):
+
+1. The invitation is signed with a dual-signature chain: the session key signs the invitation URI, then the identity key signs the concatenation `URI + "&ssig=" + sessionSignature`. This chain means a compromised session key alone cannot forge a valid identity-signed announcement - the identity key must also be compromised.
+2. The signed invitation is encrypted with a DH shared secret between the host's known DH public key and the controller's ephemeral DH private key.
+3. The encrypted packet is padded to 900 bytes (privacy: all announcements are indistinguishable by size).
+4. Sent 60 times at 1-second intervals to multicast group `224.0.0.251:5227`.
+5. Runs as a cancellable async task - cancelled in `connectRCHost` after `prepareHostSession` returns, once the session is established.
+
+**Listener and discovery** (`discoverRCCtrl`):
+
+1. Host calls `joinMulticast` to subscribe to the multicast group. A shared `TMVar Int` counter tracks active listeners - OS-level `IP_ADD_MEMBERSHIP` is only issued on 0→1 transition, `IP_DROP_MEMBERSHIP` on 1→0. This prevents duplicate syscalls when multiple listeners are active.
+2. For each received packet, `findRCCtrlPairing` iterates over known pairings and tries decryption with the current DH key, falling back to `prevDhPrivKey` if present.
+3. After successful decryption, the invitation's `dh` field is verified against the announcement's `dhPubKey` to prevent relay attacks.
+4. The source IP address is checked against the invitation's `host` field - prevents re-broadcasting a legitimate announcement from a different host.
+5. Dual signatures are verified: session signature first, then identity signature.
+6. 30-second timeout on the entire discovery process (`RCENotDiscovered` on expiry).
+
+---
+
+## Block framing and padding
+
+**Source**: [RemoteControl/Client.hs](../../src/Simplex/RemoteControl/Client.hs), [RemoteControl/Types.hs](../../src/Simplex/RemoteControl/Types.hs)
+
+XRCP uses three padding sizes at different protocol layers:
+
+- **16,384 bytes** - XRCP block size for all session messages (hello, commands, responses). Matches SMP's block size. Hides message content size variation within the TLS session.
+- **12,288 bytes** - hello body padding within the 16,384-byte block, after encryption overhead.
+- **900 bytes** - multicast announcement padding. Constrained by typical UDP MTU to avoid fragmentation.
+
+All padding uses the standard `pad`/`unPad` format (2-byte length prefix + `#` fill). The fixed sizes ensure that an observer monitoring network traffic cannot distinguish different XRCP operations by packet size.

spec/clients.md

@@ -0,0 +1,179 @@
+# Client Architecture
+
+SimpleX clients are the Layer 2 libraries that connect to routers. This document shows their internal architecture: component topology and command processing flows.
+
+For deployment and usage, see [docs/CLIENT.md](../docs/CLIENT.md). For protocol specifications, see [SMP](../protocol/simplex-messaging.md), [XFTP](../protocol/xftp.md), [Push Notifications](../protocol/push-notifications.md).
+
+---
+
+## SMP Client (ProtocolClient)
+
+**Four threads**: Send and receive threads are separate to allow backpressure - a slow receiver doesn't block sending. The process thread decouples parsing from delivery, preventing a slow consumer from stalling the receive loop. The monitor thread provides application-level keepalive beyond TCP - detecting protocol-level stalls. See [transport.md](topics/transport.md#connection-management).
+
+**Correlation ID lifecycle**: IDs are generated before send and removed on response OR timeout. Removal on timeout prevents unbounded growth of `sentCommands` when the router is unresponsive.
+
+**Module specs**: [Client](modules/Simplex/Messaging/Client.md) · [Protocol](modules/Simplex/Messaging/Protocol.md) · [Transport](modules/Simplex/Messaging/Transport.md) · [Crypto](modules/Simplex/Messaging/Crypto.md)
+
+Generic protocol client used for both SMP and NTF connections. Manages a single TLS connection with multiplexed command/response matching via correlation IDs.
+
+### SMP Client components
+
+![SMP Client components](diagrams/smp-client.svg)
+
+### Command/result flow
+
+```mermaid
+sequenceDiagram
+    participant C as Caller<br>(Agent / router)
+
+    box ProtocolClient
+        participant SC as sentCommands<br>(TMap CorrId Request)
+        participant SQ as sndQ
+        participant S as send thread
+        participant R as receive thread
+        participant RQ as rcvQ
+        participant P as process thread
+    end
+
+    participant Router as SMP Router
+
+    C->>SC: mkTransmission<br/>(generate CorrId, create Request<br/>with empty responseVar)
+    C->>SQ: write (Request, encoded command)
+    S->>SQ: read
+    S-->>S: check pending flag (drop if timed out)
+    S->>Router: tPutLog (transmit bytes)
+
+    Router->>R: tGetClient (receive batch)
+    R->>RQ: write transmissions
+
+    P->>RQ: read
+    P->>SC: lookup CorrId
+    alt command response (CorrId matches, pending)
+        P->>SC: remove CorrId + fill responseVar (TMVar)
+    else expired response (CorrId matches, already timed out)
+        P->>C: write to msgQ (STResponse)
+    else server event (empty CorrId)
+        P->>C: write to msgQ (STEvent)
+    end
+
+    Note over C: getResponse: takeTMVar with timeout
+```
+
+---
+
+## SMPClientAgent
+
+**Dual consumers**: Used by both SMP router (for proxy connections to relays) and NTF router (for NSUB subscriptions to SMP routers). Same connection pooling and reconnection logic, different command sets.
+
+**Session ID gating**: Subscription responses are validated against the current TLS session ID. A response from a stale session (connection dropped and reconnected between send and receive) is discarded rather than corrupting state. See [infrastructure.md](agent/infrastructure.md#subscription-tracking).
+
+**Module specs**: [Client Agent](modules/Simplex/Messaging/Client/Agent.md)
+
+Connection manager that multiplexes multiple ProtocolClient connections. Tracks subscriptions, handles reconnection with backoff, and forwards server messages and connection events upward. Used by SMP router (proxying) and NTF router (subscriptions).
+
+### SMPClientAgent components
+
+![SMPClientAgent components](diagrams/smp-client-agent.svg)
+
+### Connection lifecycle
+
+```mermaid
+sequenceDiagram
+    participant C as Consumer<br>(router / app)
+
+    box
+        participant A as SMPClientAgent
+        participant PC as ProtocolClient
+    end
+
+    participant Router as SMP Router
+
+    C->>A: getSMPServerClient'' (server)
+    alt client exists in smpClients
+        A->>C: return existing client
+    else no client
+        A->>PC: connectClient (create new ProtocolClient)
+        PC->>Router: TLS handshake
+        A->>A: register disconnect handler
+        A->>C: return new client
+    end
+
+    C->>A: subscribeQueuesNtfs (queueIds)
+    A->>A: add to pendingQueueSubs
+    A->>PC: sendProtocolCommands (SUB batch)
+    PC->>Router: SUB commands
+    Router->>PC: OK responses
+    A->>A: move pending → activeQueueSubs
+    A->>C: CASubscribed (via agentQ)
+
+    Note over Router: connection drops
+
+    PC->>A: disconnect handler fires
+    A->>A: filter by SessionId (only remove subs matching disconnected session)
+    A->>A: move active → pending (queue subs + service subs)
+    A->>C: CAServiceDisconnected (via agentQ, if service sub existed)
+    A->>C: CADisconnected (via agentQ, if queue subs existed)
+    A->>A: spawn smpSubWorker (retry with backoff)
+    A->>PC: reconnect + resubscribe pending subs
+    A->>C: CAConnected + CASubscribed (via agentQ)
+```
+
+---
+
+## XFTP Client
+
+**No subscriptions**: File operations complete independently - no persistent server-side state to track. This allows XFTPClient to be a thin wrapper with no threads of its own.
+
+**Module specs**: [Client](modules/Simplex/FileTransfer/Client.md) · [Protocol](modules/Simplex/FileTransfer/Protocol.md) · [HTTP/2 Client](modules/Simplex/Messaging/Transport/HTTP2/Client.md)
+
+Stateless wrapper around HTTP2Client. XFTPClient adds no threads of its own. Serialization and multiplexing happen inside HTTP2Client's internal request queue and process thread.
+
+### XFTP Client components
+
+![XFTP Client components](diagrams/xftp-client.svg)
+
+### Packet delivery flow
+
+```mermaid
+sequenceDiagram
+    participant C as Caller<br>(Agent / app)
+    participant X as XFTPClient
+    participant H as HTTP2Client
+    participant Router as XFTP Router
+
+    C->>X: createXFTPChunk (FNEW)
+    X->>H: HTTP/2 POST (encoded command)
+    H->>Router: request
+    Router->>H: response (sender ID + recipient IDs)
+    H->>X: decode response
+    X->>C: return IDs
+
+    C->>X: uploadXFTPChunk (FPUT + file data)
+    X->>H: HTTP/2 POST (streaming body)
+    H->>Router: request with file stream
+    Router->>H: OK
+    H->>X: OK
+    X->>C: return OK
+
+    C->>X: downloadXFTPChunk (FGET + ephemeral DH key)
+    X->>H: HTTP/2 POST (command)
+    H->>Router: request
+    Router->>H: streaming response (server DH key + nonce + encrypted data)
+    H->>X: streaming body
+    X->>X: compute DH secret, decrypt + save to file
+    X->>C: return ()
+```
+
+---
+
+## NTF Client
+
+**Module specs**: [Client](modules/Simplex/Messaging/Notifications/Client.md) · [Protocol](modules/Simplex/Messaging/Notifications/Protocol.md)
+
+Type alias for ProtocolClient - same architecture as SMP Client:
+
+```haskell
+type NtfClient = ProtocolClient NTFVersion ErrorType NtfResponse
+```
+
+Same threads (send, receive, process, monitor), same queues (sndQ, rcvQ, sentCommands, msgQ), same command/response flow. Different command types: TNEW, TVFY, TCHK, TRPL, TDEL, TCRN, SNEW, SCHK, SDEL, PING.

spec/diagrams/agent.svg

@@ -0,0 +1,288 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 780" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+    <marker id="arr-x" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#c62828" />
+    </marker>
+  </defs>
+
+  <!-- ===== ROW 0: APPLICATION (external) ===== -->
+  <rect x="350" y="10" width="180" height="34" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="440" y="32" text-anchor="middle">Application</text>
+
+  <!-- subQ queue box (right of Application) -->
+  <rect x="560" y="10" width="120" height="34" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="620" y="25" text-anchor="middle" font-weight="bold">subQ</text>
+  <text x="620" y="39" text-anchor="middle" font-size="9">(TBQueue)</text>
+
+  <!-- subQ -> Application -->
+  <line x1="560" y1="27" x2="532" y2="27"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- ===== subQ BUS: three stubs up from columns → horizontal → up → into subQ ===== -->
+
+  <!-- Three vertical stubs from column tops up to bus at y=142 -->
+  <line x1="172" y1="158" x2="172" y2="142" stroke="#333" />
+  <line x1="483" y1="158" x2="483" y2="142" stroke="#333" />
+  <line x1="788" y1="158" x2="788" y2="142" stroke="#333" />
+
+  <!-- Bus: horizontal across, right margin up, into subQ -->
+  <polyline points="172,142 940,142 940,27 682,27"
+            fill="none" stroke="#333" marker-end="url(#arr)" />
+
+  <!-- ===== App -> API arrows (down) ===== -->
+  <line x1="400" y1="44" x2="170" y2="76"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <line x1="440" y1="44" x2="480" y2="76"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <line x1="480" y1="44" x2="786" y2="76"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- ===== ROW 1: API FUNCTIONS ===== -->
+
+  <!-- SMP API -->
+  <rect x="25" y="76" width="290" height="50" rx="4"
+        fill="#f4f4f4" stroke="#999" />
+  <text x="170" y="94" text-anchor="middle" font-size="10">sendMessage, createConnection</text>
+  <text x="170" y="109" text-anchor="middle" font-size="10">joinConnection, subscribe...</text>
+
+  <!-- XFTP API -->
+  <rect x="335" y="76" width="290" height="50" rx="4"
+        fill="#f4f4f4" stroke="#999" />
+  <text x="480" y="94" text-anchor="middle" font-size="10">xftpSendFile</text>
+  <text x="480" y="109" text-anchor="middle" font-size="10">xftpReceiveFile...</text>
+
+  <!-- NTF API -->
+  <rect x="641" y="76" width="290" height="50" rx="4"
+        fill="#f4f4f4" stroke="#999" />
+  <text x="786" y="94" text-anchor="middle" font-size="10">registerNtfToken</text>
+  <text x="786" y="109" text-anchor="middle" font-size="10">toggleConnectionNtfs...</text>
+
+  <!-- ===== ROW 2: THREE PROTOCOL COLUMNS ===== -->
+
+  <!-- ====== SMP COLUMN ====== -->
+  <rect x="20" y="158" width="305" height="260" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="30" y="174" fill="#888" font-size="10">SMP</text>
+
+  <!-- msgQ queue box (inside SMP column, left side) -->
+  <rect x="35" y="180" width="110" height="30" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="90" y="194" text-anchor="middle" font-weight="bold" font-size="10">msgQ</text>
+  <text x="90" y="206" text-anchor="middle" font-size="8">(TBQueue)</text>
+
+  <!-- subscriber (green, singleton, right side) -->
+  <rect x="160" y="180" width="150" height="44" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="235" y="198" text-anchor="middle">subscriber</text>
+  <text x="235" y="212" text-anchor="middle" font-size="9" fill="#666">(reads msgQ)</text>
+
+  <!-- msgQ -> subscriber -->
+  <line x1="147" y1="198" x2="158" y2="198"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- SMP worker pools label -->
+  <text x="35" y="246" fill="#888" font-size="9">worker pools (on-demand, one per queue / conn+server / session)</text>
+
+  <!-- delivery worker -->
+  <rect x="35" y="256" width="128" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="99" y="273" text-anchor="middle" font-size="11">delivery</text>
+  <text x="99" y="286" text-anchor="middle" font-size="8" fill="#666">(per send queue)</text>
+
+  <!-- asyncCmd worker -->
+  <rect x="175" y="256" width="135" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="242" y="273" text-anchor="middle" font-size="11">asyncCmd</text>
+  <text x="242" y="286" text-anchor="middle" font-size="8" fill="#666">(per conn + server)</text>
+
+  <!-- smpSub worker -->
+  <rect x="35" y="308" width="128" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="99" y="325" text-anchor="middle" font-size="11">smpSub</text>
+  <text x="99" y="338" text-anchor="middle" font-size="8" fill="#666">(per session)</text>
+
+  <!-- ====== XFTP COLUMN ====== -->
+  <rect x="335" y="158" width="296" height="260" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="345" y="174" fill="#888" font-size="10">XFTP</text>
+
+  <!-- XFTP worker pools label -->
+  <text x="345" y="246" fill="#888" font-size="9">worker pools (on-demand, one per server)</text>
+
+  <!-- xftpRcv worker -->
+  <rect x="350" y="256" width="130" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="415" y="273" text-anchor="middle" font-size="11">xftpRcv</text>
+  <text x="415" y="286" text-anchor="middle" font-size="8" fill="#666">(per server + local)</text>
+
+  <!-- xftpSnd worker -->
+  <rect x="490" y="256" width="130" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="555" y="273" text-anchor="middle" font-size="11">xftpSnd</text>
+  <text x="555" y="286" text-anchor="middle" font-size="8" fill="#666">(per server + local)</text>
+
+  <!-- xftpDel worker -->
+  <rect x="350" y="308" width="130" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="415" y="325" text-anchor="middle" font-size="11">xftpDel</text>
+  <text x="415" y="338" text-anchor="middle" font-size="8" fill="#666">(per server)</text>
+
+  <!-- ====== NTF COLUMN ====== -->
+  <rect x="641" y="158" width="295" height="260" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="651" y="174" fill="#888" font-size="10">NTF</text>
+
+  <!-- ntfSubQ queue box (inside NTF column, left side) -->
+  <rect x="656" y="180" width="88" height="30" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="700" y="194" text-anchor="middle" font-weight="bold" font-size="10">ntfSubQ</text>
+  <text x="700" y="206" text-anchor="middle" font-size="8">(TBQueue)</text>
+
+  <!-- ntfSupervisor (green, singleton, right side) -->
+  <rect x="756" y="180" width="168" height="44" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="840" y="198" text-anchor="middle">ntfSupervisor</text>
+  <text x="840" y="212" text-anchor="middle" font-size="9" fill="#666">(reads ntfSubQ)</text>
+
+  <!-- ntfSubQ -> ntfSupervisor -->
+  <line x1="746" y1="198" x2="754" y2="198"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- NTF API -> ntfSubQ -->
+  <line x1="700" y1="126" x2="700" y2="178"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- NTF worker pools label -->
+  <text x="656" y="246" fill="#888" font-size="9">worker pools (on-demand, one per server)</text>
+
+  <!-- ntfWorkers -->
+  <rect x="656" y="256" width="125" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="718" y="273" text-anchor="middle" font-size="11">ntfWorkers</text>
+  <text x="718" y="286" text-anchor="middle" font-size="8" fill="#666">(per NTF server)</text>
+
+  <!-- ntfSMP worker -->
+  <rect x="793" y="256" width="128" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="857" y="273" text-anchor="middle" font-size="11">ntfSMP</text>
+  <text x="857" y="286" text-anchor="middle" font-size="8" fill="#666">(per SMP server)</text>
+
+  <!-- ntfTknDel worker -->
+  <rect x="656" y="308" width="125" height="38" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="718" y="325" text-anchor="middle" font-size="11">ntfTknDel</text>
+  <text x="718" y="338" text-anchor="middle" font-size="8" fill="#666">(per NTF server)</text>
+
+  <!-- ===== CROSS-PROTOCOL: asyncCmd -> ntfSubQ (ICQDelete) ===== -->
+  <polyline points="310,272 500,195 654,195"
+            fill="none" stroke="#c62828" stroke-dasharray="4,3" marker-end="url(#arr-x)" />
+  <text x="420" y="218" font-size="9" fill="#c62828">ntfSubQ (queue rotation)</text>
+
+  <!-- ===== SHARED SINGLETON THREADS ===== -->
+  <text x="30" y="448" fill="#888" font-size="9">shared singletons (all green run in raceAny_)</text>
+  <rect x="320" y="436" width="135" height="24" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="387" y="452" text-anchor="middle" font-size="10">cleanupManager</text>
+
+  <rect x="470" y="436" width="135" height="24" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="537" y="452" text-anchor="middle" font-size="10">logServersStats</text>
+
+  <!-- ===== CROSS-PROTOCOL: ntfSMP -> smpClients ===== -->
+  <polyline points="857,294 857,482 170,482 170,558"
+            fill="none" stroke="#c62828" stroke-dasharray="4,3" marker-end="url(#arr-x)" />
+  <text x="500" y="478" font-size="9" fill="#c62828">ntfSMP uses smpClients</text>
+
+  <!-- ===== ROW 3: CENTRAL STATE ===== -->
+  <rect x="25" y="494" width="450" height="46" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="250" y="514" text-anchor="middle" font-weight="bold">Store</text>
+  <text x="250" y="530" text-anchor="middle" font-size="10">(SQLite / Postgres)</text>
+
+  <rect x="490" y="494" width="445" height="46" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="712" y="514" text-anchor="middle" font-weight="bold">Operation State</text>
+  <text x="712" y="530" text-anchor="middle" font-size="10">(5-op suspension cascade)</text>
+
+  <!-- store access: representative dashed line -->
+  <line x1="242" y1="294" x2="220" y2="492"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="248" y="400" font-size="9" fill="#555">store</text>
+
+  <!-- ===== ROW 4: PROTOCOL CLIENT POOLS ===== -->
+  <rect x="25" y="558" width="290" height="46" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="170" y="578" text-anchor="middle">smpClients</text>
+  <text x="170" y="594" text-anchor="middle" font-size="9" fill="#666">(TMap SMPTransportSession)</text>
+
+  <rect x="335" y="558" width="290" height="46" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="480" y="578" text-anchor="middle">xftpClients</text>
+  <text x="480" y="594" text-anchor="middle" font-size="9" fill="#666">(TMap XFTPTransportSession)</text>
+
+  <rect x="641" y="558" width="290" height="46" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="786" y="578" text-anchor="middle">ntfClients</text>
+  <text x="786" y="594" text-anchor="middle" font-size="9" fill="#666">(TMap NtfTransportSession)</text>
+
+  <!-- ===== ROW 5: EXTERNAL ROUTERS ===== -->
+  <text x="170" y="630" text-anchor="middle" font-size="9" fill="#999">SMP Routers</text>
+  <line x1="170" y1="604" x2="170" y2="620"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <text x="480" y="630" text-anchor="middle" font-size="9" fill="#999">XFTP Routers</text>
+  <line x1="480" y1="604" x2="480" y2="620"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <text x="786" y="630" text-anchor="middle" font-size="9" fill="#999">NTF Routers</text>
+  <line x1="786" y1="604" x2="786" y2="620"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- ===== smpClients -> msgQ (left margin) ===== -->
+  <polyline points="25,581 8,581 8,195 33,195"
+            fill="none" stroke="#333" marker-end="url(#arr)" />
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="20" y="654" width="920" height="60" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="35" y="668" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="55" y="678" font-size="10">on-demand worker</text>
+
+  <rect x="185" y="668" width="14" height="11" rx="2"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="205" y="678" font-size="10">singleton thread</text>
+
+  <rect x="325" y="668" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="345" y="678" font-size="10">queue / state</text>
+
+  <rect x="445" y="668" width="14" height="11" rx="2"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="465" y="678" font-size="10">external connection</text>
+
+  <rect x="595" y="668" width="14" height="11" rx="2"
+        fill="#f4f4f4" stroke="#999" />
+  <text x="615" y="678" font-size="10">API entry point</text>
+
+  <line x1="730" y1="673" x2="770" y2="673"
+        stroke="#c62828" stroke-dasharray="4,3" marker-end="url(#arr-x)" />
+  <text x="780" y="677" font-size="10">cross-protocol</text>
+
+  <text x="35" y="704" font-size="10" fill="#666">
+    Solid arrows: TBQueue flow.  Dashed grey: store access.  Dashed red: cross-protocol link.  All threads and pools write events to subQ.
+  </text>
+
+</svg>

spec/diagrams/ntf-router.svg

@@ -0,0 +1,210 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 830 610" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+  </defs>
+
+  <!-- ===== PER-CLIENT GROUP (left) ===== -->
+  <rect x="25" y="40" width="300" height="170" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="55" fill="#888" font-size="10">per client (raceAny_)</text>
+
+  <!-- network in -->
+  <text x="13" y="78" font-size="9" fill="#999">net</text>
+  <line x1="12" y1="82" x2="40" y2="82"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- receive -->
+  <rect x="42" y="66" width="78" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="81" y="88" text-anchor="middle">receive</text>
+
+  <!-- rcvQ -->
+  <line x1="120" y1="83" x2="158" y2="83"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="139" y="77" text-anchor="middle" font-size="10" fill="#555">rcvQ</text>
+
+  <!-- client -->
+  <rect x="160" y="66" width="78" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="199" y="88" text-anchor="middle">client</text>
+
+  <!-- sndQ -->
+  <line x1="238" y1="83" x2="268" y2="83"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="253" y="77" text-anchor="middle" font-size="10" fill="#555">sndQ</text>
+
+  <!-- send -->
+  <rect x="270" y="66" width="48" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="294" y="88" text-anchor="middle">send</text>
+
+  <!-- network out -->
+  <text x="326" y="88" font-size="9" fill="#999">net</text>
+
+  <!-- commands label -->
+  <text x="52" y="120" font-size="10" fill="#666">TNEW, TVFY, TCHK, TRPL, TDEL, TCRN</text>
+  <text x="52" y="134" font-size="10" fill="#666">SNEW, SCHK, SDEL, PING</text>
+
+  <!-- client -> store (down) -->
+  <line x1="199" y1="100" x2="199" y2="278"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="211" y="230" font-size="10" fill="#555">store</text>
+
+  <!-- client -> pushQ (PNVerification on TNEW/TRPL) -->
+  <line x1="160" y1="100" x2="50" y2="388"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="62" y="260" font-size="9" fill="#555">pushQ</text>
+
+  <!-- ===== SMP CLIENT AGENT GROUP (right) ===== -->
+  <rect x="370" y="40" width="435" height="220" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="380" y="55" fill="#888" font-size="10">SMP Client Agent (connects to SMP routers)</text>
+
+  <!-- SMP routers external label -->
+  <text x="620" y="38" font-size="9" fill="#999">SMP routers</text>
+  <line x1="620" y1="40" x2="620" y2="62"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- SMPClientAgent -->
+  <rect x="530" y="64" width="220" height="34" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="640" y="86" text-anchor="middle">SMPClientAgent</text>
+
+  <!-- msgQ arrow: SMPClientAgent -> receiveSMP -->
+  <line x1="580" y1="98" x2="510" y2="128"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="530" y="112" font-size="10" fill="#555">msgQ</text>
+
+  <!-- agentQ arrow: SMPClientAgent -> receiveAgent -->
+  <line x1="690" y1="98" x2="690" y2="128"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="702" y="117" font-size="10" fill="#555">agentQ</text>
+
+  <!-- ntfSubscriber/receiveSMP -->
+  <rect x="390" y="130" width="195" height="34" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="487" y="152" text-anchor="middle" font-size="11">ntfSubscriber/receiveSMP</text>
+
+  <!-- ntfSubscriber/receiveAgent -->
+  <rect x="605" y="130" width="185" height="34" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="697" y="152" text-anchor="middle" font-size="11">receiveAgent</text>
+
+  <!-- race_ label between them -->
+  <text x="597" y="144" font-size="9" fill="#888">race_</text>
+
+  <!-- receiveSMP -> pushQ (down) -->
+  <line x1="487" y1="164" x2="487" y2="195"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="500" y="183" font-size="10" fill="#555">pushQ</text>
+
+  <!-- receiveAgent -> store (down) -->
+  <line x1="697" y1="164" x2="697" y2="278"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="709" y="230" font-size="10" fill="#555">store</text>
+
+  <!-- per-SMP-server subscriber -->
+  <rect x="390" y="200" width="200" height="44" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="490" y="219" text-anchor="middle" font-size="11">runSMPSubscriber</text>
+  <text x="490" y="236" text-anchor="middle" font-size="9" fill="#666">(one per SMP router)</text>
+
+  <!-- subscriberSubQ label -->
+  <text x="408" y="196" font-size="9" fill="#555">subscriberSubQ</text>
+
+  <!-- ===== CENTRAL STORE ===== -->
+  <rect x="100" y="280" width="620" height="50" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="400" y="301" text-anchor="middle" font-weight="bold">tokens / subscriptions / tokenLastNtfs</text>
+  <text x="400" y="319" text-anchor="middle" font-size="10">(PostgreSQL)</text>
+
+  <!-- ===== PUSH DELIVERY ===== -->
+  <rect x="25" y="355" width="780" height="90" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="370" fill="#888" font-size="10">push delivery pipeline</text>
+
+  <!-- pushQ hub: store -> ntfPush -->
+  <line x1="400" y1="330" x2="400" y2="388"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="412" y="365" font-size="10" fill="#555">pushQ</text>
+
+  <!-- ntfPush -->
+  <rect x="350" y="390" width="110" height="34" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="405" y="412" text-anchor="middle">ntfPush</text>
+
+  <!-- APNS provider -->
+  <line x1="460" y1="407" x2="528" y2="407"
+        stroke="#333" marker-end="url(#arr)" />
+  <rect x="530" y="390" width="135" height="34" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="597" y="412" text-anchor="middle">APNS provider</text>
+
+  <!-- periodicNtfsThread -->
+  <rect x="45" y="390" width="175" height="34" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="132" y="412" text-anchor="middle" font-size="11">periodicNtfsThread</text>
+
+  <!-- periodic -> pushQ -->
+  <line x1="220" y1="407" x2="348" y2="407"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="284" y="401" font-size="10" fill="#555">pushQ</text>
+
+  <!-- periodic reads store -->
+  <line x1="132" y1="390" x2="132" y2="330"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="117" y="365" font-size="9" fill="#555">reads</text>
+
+  <!-- ===== OPTIONAL ===== -->
+  <rect x="25" y="462" width="780" height="45" rx="6"
+        fill="none" stroke="#bbb" stroke-dasharray="4,3" />
+  <text x="35" y="477" fill="#bbb" font-size="10">optional</text>
+
+  <rect x="45" y="484" width="128" height="18" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="109" y="497" text-anchor="middle" font-size="9">logServerStats</text>
+
+  <rect x="190" y="484" width="108" height="18" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="244" y="497" text-anchor="middle" font-size="9">prometheus</text>
+
+  <rect x="315" y="484" width="108" height="18" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="369" y="497" text-anchor="middle" font-size="9">controlPort</text>
+
+  <rect x="440" y="484" width="108" height="18" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="494" y="497" text-anchor="middle" font-size="9">resubscribe</text>
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="25" y="525" width="780" height="55" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="40" y="540" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="60" y="550" font-size="10">per-client thread</text>
+
+  <rect x="195" y="540" width="14" height="11" rx="2"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="215" y="550" font-size="10">singleton thread</text>
+
+  <rect x="350" y="540" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="370" y="550" font-size="10">storage</text>
+
+  <rect x="445" y="540" width="14" height="11" rx="2"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="465" y="550" font-size="10">external connection</text>
+
+  <text x="40" y="572" font-size="10" fill="#666">
+    Solid arrows: TBQueue connections.  Dashed: store access.
+  </text>
+
+</svg>

spec/diagrams/smp-client-agent.svg

@@ -0,0 +1,143 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 740 400" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+  </defs>
+
+  <!-- ===== CONSUMER (top left) ===== -->
+  <text x="15" y="35" font-size="9" fill="#999">consumer</text>
+  <text x="15" y="48" font-size="9" fill="#999">(NTF router /</text>
+  <text x="15" y="61" font-size="9" fill="#999"> SMP proxy /</text>
+  <text x="15" y="74" font-size="9" fill="#999"> application)</text>
+
+  <!-- ===== agentQ (upper right) ===== -->
+  <rect x="360" y="15" width="210" height="35" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="465" y="33" text-anchor="middle" font-weight="bold">agentQ</text>
+  <text x="465" y="46" text-anchor="middle" font-size="10">(TBQueue SMPClientAgentEvent)</text>
+
+  <!-- consumer <- agentQ -->
+  <line x1="360" y1="32" x2="80" y2="32"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- agentQ events note -->
+  <text x="580" y="22" font-size="9" fill="#666">CAConnected / Disconnected</text>
+  <text x="580" y="33" font-size="9" fill="#666">CASubscribed / SubError</text>
+  <text x="580" y="44" font-size="9" fill="#666">CAServiceDisconnected</text>
+  <text x="580" y="55" font-size="9" fill="#666">CAServiceSubscribed / SubError</text>
+  <text x="580" y="66" font-size="9" fill="#666">CAServiceUnavailable</text>
+
+  <!-- ===== msgQ (upper left, below agentQ) ===== -->
+  <rect x="130" y="55" width="195" height="35" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="227" y="73" text-anchor="middle" font-weight="bold">msgQ</text>
+  <text x="227" y="86" text-anchor="middle" font-size="10">(TBQueue, server messages)</text>
+
+  <!-- consumer <- msgQ -->
+  <line x1="130" y1="72" x2="80" y2="72"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- ===== AGENT CORE ===== -->
+  <rect x="25" y="100" width="700" height="82" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="113" fill="#888" font-size="10">SMPClientAgent (connection manager)</text>
+
+  <!-- smpClients -->
+  <rect x="45" y="120" width="170" height="48" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="130" y="141" text-anchor="middle" font-weight="bold">smpClients</text>
+  <text x="130" y="159" text-anchor="middle" font-size="9">(TMap SMPServer SMPClientVar)</text>
+
+  <!-- smpClients -> msgQ (server messages forwarded up) -->
+  <line x1="145" y1="120" x2="145" y2="92"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- smpSubWorkers -->
+  <rect x="250" y="120" width="130" height="48" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="315" y="141" text-anchor="middle" font-size="11">smpSubWorkers</text>
+  <text x="315" y="159" text-anchor="middle" font-size="9" fill="#666">(one per server)</text>
+
+  <!-- workers -> smpClients (reconnect, dashed) -->
+  <line x1="250" y1="144" x2="215" y2="144"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="222" y="138" font-size="8" fill="#555">reconnect</text>
+
+  <!-- subscription tracking -->
+  <rect x="415" y="118" width="275" height="56" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="552" y="134" text-anchor="middle" font-size="10">activeQueueSubs / pendingQueueSubs</text>
+  <text x="552" y="146" text-anchor="middle" font-size="9" fill="#666">(TMap SMPServer (TMap QueueId ...))</text>
+  <text x="552" y="159" text-anchor="middle" font-size="10">activeServiceSubs / pendingServiceSubs</text>
+  <text x="552" y="171" text-anchor="middle" font-size="9" fill="#666">(TMap SMPServer (TVar (Maybe ...)))</text>
+
+  <!-- subscription state -> agentQ (connection events forwarded up) -->
+  <line x1="465" y1="118" x2="465" y2="52"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- ===== PROTOCOL CLIENTS (bottom) ===== -->
+  <rect x="25" y="198" width="700" height="72" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="213" fill="#888" font-size="10">ProtocolClient connections (one per SMP Router)</text>
+
+  <!-- Client 1 -->
+  <rect x="45" y="220" width="140" height="40" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="115" y="238" text-anchor="middle">ProtocolClient</text>
+  <text x="115" y="253" text-anchor="middle" font-size="10">→ SMP Router A</text>
+
+  <!-- Client 2 -->
+  <rect x="210" y="220" width="140" height="40" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="280" y="238" text-anchor="middle">ProtocolClient</text>
+  <text x="280" y="253" text-anchor="middle" font-size="10">→ SMP Router B</text>
+
+  <!-- Client N -->
+  <rect x="375" y="220" width="140" height="40" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="445" y="238" text-anchor="middle">ProtocolClient</text>
+  <text x="445" y="253" text-anchor="middle" font-size="10">→ SMP Router N</text>
+
+  <!-- dots -->
+  <text x="530" y="245" font-size="14" fill="#888">...</text>
+
+  <!-- smpClients -> clients -->
+  <line x1="100" y1="168" x2="100" y2="218"
+        stroke="#333" marker-end="url(#arr)" />
+  <line x1="185" y1="168" x2="265" y2="218"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- ===== NOTES ===== -->
+  <text x="45" y="292" font-size="10" fill="#666">getSMPServerClient'': get or create client</text>
+  <text x="45" y="306" font-size="10" fill="#666">connectClient: create ProtocolClient, register disconnect handler</text>
+  <text x="45" y="320" font-size="10" fill="#666">on disconnect: filter by SessionId, move active to pending, notify agentQ, spawn worker</text>
+  <text x="45" y="334" font-size="10" fill="#666">worker: retry connect with backoff, resubscribe pending subs</text>
+  <text x="45" y="348" font-size="10" fill="#666">subscribeQueuesNtfs / subscribeServiceNtfs: subscribe + track state</text>
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="25" y="365" width="700" height="30" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="40" y="374" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="60" y="384" font-size="10">ProtocolClient</text>
+
+  <rect x="185" y="374" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="205" y="384" font-size="10">state / queue</text>
+
+  <rect x="310" y="374" width="14" height="11" rx="2"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="330" y="384" font-size="10">background worker</text>
+
+  <text x="480" y="384" font-size="10" fill="#666">
+    Solid arrows: TBQueue flow.  Dashed: reconnection.
+  </text>
+
+</svg>

spec/diagrams/smp-client.svg

@@ -0,0 +1,170 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 380" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+  </defs>
+
+  <!-- ===== PER-CONNECTION GROUP ===== -->
+  <rect x="100" y="15" width="435" height="250" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="110" y="30" fill="#888" font-size="10">per connection (raceAny_: any exit tears down all threads)</text>
+
+  <!-- ===== MONITOR (optional, above sndQ area) ===== -->
+  <rect x="275" y="38" width="88" height="24" rx="4"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="319" y="55" text-anchor="middle" font-size="11">monitor</text>
+  <text x="370" y="54" font-size="9" fill="#bbb">optional</text>
+
+  <!-- monitor -> API (PING via sendProtocolCommand) -->
+  <line x1="300" y1="62" x2="265" y2="74"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="305" y="73" font-size="9" fill="#555">PING</text>
+
+  <!-- ===== ROW 1: SEND PATH ===== -->
+
+  <!-- caller label -->
+  <text x="15" y="92" font-size="9" fill="#999">caller</text>
+  <text x="10" y="105" font-size="9" fill="#999">(Agent /</text>
+  <text x="10" y="118" font-size="9" fill="#999"> router)</text>
+
+  <!-- caller -> API -->
+  <line x1="55" y1="98" x2="143" y2="98"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- API functions block -->
+  <rect x="145" y="76" width="165" height="42" rx="4"
+        fill="#f4f4f4" stroke="#999" />
+  <text x="227" y="93" text-anchor="middle" font-size="10">sendProtocolCommand</text>
+  <text x="227" y="107" text-anchor="middle" font-size="10">mkTransmission</text>
+
+  <!-- API -> sndQ -->
+  <line x1="310" y1="97" x2="335" y2="97"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- sndQ -->
+  <rect x="337" y="82" width="88" height="34" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="381" y="98" text-anchor="middle" font-weight="bold">sndQ</text>
+  <text x="381" y="111" text-anchor="middle" font-size="9">(TBQueue, 64)</text>
+
+  <!-- sndQ -> send -->
+  <line x1="425" y1="97" x2="448" y2="97"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- send thread -->
+  <rect x="450" y="84" width="60" height="28" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="480" y="103" text-anchor="middle">send</text>
+
+  <!-- send -> Router (exits per-connection box) -->
+  <line x1="510" y1="97" x2="543" y2="97"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- Router label (outside box) -->
+  <text x="548" y="88" font-size="9" fill="#999">SMP</text>
+  <text x="548" y="101" font-size="9" fill="#999">Router</text>
+  <text x="548" y="114" font-size="9" fill="#999">(TLS)</text>
+
+  <!-- ===== ROW 2: RECEIVE PATH + SENTCOMMANDS ===== -->
+
+  <!-- API -> sentCommands (insert CorrId + Request) -->
+  <line x1="227" y1="118" x2="227" y2="148"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="235" y="138" font-size="9" fill="#555">insert</text>
+
+  <!-- sentCommands -->
+  <rect x="145" y="150" width="165" height="38" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="227" y="167" text-anchor="middle" font-weight="bold">sentCommands</text>
+  <text x="227" y="182" text-anchor="middle" font-size="9">(TMap CorrId Request)</text>
+
+  <!-- sentCommands -> caller (responseVar, dashed) -->
+  <line x1="145" y1="169" x2="55" y2="125"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="62" y="152" font-size="9" fill="#555">responseVar</text>
+  <text x="62" y="164" font-size="9" fill="#555">(TMVar)</text>
+
+  <!-- Router -> receive (enters per-connection box) -->
+  <line x1="543" y1="163" x2="510" y2="163"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- receive thread -->
+  <rect x="450" y="150" width="60" height="28" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="480" y="169" text-anchor="middle">receive</text>
+
+  <!-- receive -> rcvQ -->
+  <line x1="450" y1="163" x2="425" y2="163"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- rcvQ -->
+  <rect x="337" y="148" width="88" height="34" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="381" y="164" text-anchor="middle" font-weight="bold">rcvQ</text>
+  <text x="381" y="177" text-anchor="middle" font-size="9">(TBQueue, 64)</text>
+
+  <!-- ===== ROW 3: PROCESS ===== -->
+
+  <!-- rcvQ -> process -->
+  <line x1="370" y1="182" x2="300" y2="216"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- process -> sentCommands (match CorrId, dashed upward) -->
+  <line x1="245" y1="218" x2="245" y2="190"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="252" y="208" font-size="9" fill="#555">match</text>
+
+  <!-- process thread -->
+  <rect x="195" y="220" width="110" height="28" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="250" y="239" text-anchor="middle">process</text>
+
+  <!-- ===== msgQ (outside per-connection box) ===== -->
+
+  <!-- process -> msgQ -->
+  <line x1="250" y1="248" x2="250" y2="278"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="258" y="268" font-size="9" fill="#555">events</text>
+
+  <!-- msgQ -->
+  <rect x="175" y="280" width="160" height="33" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="255" y="296" text-anchor="middle" font-weight="bold">msgQ (optional)</text>
+  <text x="255" y="308" text-anchor="middle" font-size="9">(TBQueue, server events)</text>
+
+  <!-- msgQ -> consumer -->
+  <line x1="175" y1="296" x2="55" y2="296"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <text x="65" y="290" font-size="9" fill="#999">to Agent / SMPClientAgent</text>
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="25" y="325" width="590" height="48" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="40" y="337" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="60" y="347" font-size="10">thread</text>
+
+  <rect x="130" y="337" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="150" y="347" font-size="10">queue / state</text>
+
+  <rect x="270" y="337" width="14" height="11" rx="2"
+        fill="#f4f4f4" stroke="#999" />
+  <text x="290" y="347" font-size="10">API entry point</text>
+
+  <rect x="410" y="337" width="14" height="11" rx="2"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="430" y="347" font-size="10">optional</text>
+
+  <text x="40" y="365" font-size="10" fill="#666">
+    Solid arrows: TBQueue flow.  Dashed: STM lookups / TMVar responses.
+  </text>
+
+</svg>

spec/diagrams/smp-router.svg

@@ -0,0 +1,193 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 590" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+  </defs>
+
+  <!-- ===== PER-CLIENT GROUP ===== -->
+  <rect x="25" y="38" width="570" height="210" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="53" fill="#888" font-size="10">per client connection (raceAny_: any exit tears down connection)</text>
+
+  <!-- receive -->
+  <rect x="45" y="68" width="90" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="90" y="90" text-anchor="middle">receive</text>
+
+  <!-- rcvQ arrow: receive -> client -->
+  <line x1="135" y1="85" x2="198" y2="85"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="166" y="79" text-anchor="middle" font-size="10" fill="#555">rcvQ</text>
+
+  <!-- client -->
+  <rect x="200" y="68" width="100" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="250" y="90" text-anchor="middle">client</text>
+
+  <!-- sndQ arrow: client -> send -->
+  <line x1="300" y1="78" x2="388" y2="78"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="344" y="72" text-anchor="middle" font-size="10" fill="#555">sndQ</text>
+
+  <!-- send -->
+  <rect x="390" y="62" width="70" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="425" y="84" text-anchor="middle">send</text>
+
+  <!-- msgQ arrow: client -> sendMsg -->
+  <line x1="300" y1="93" x2="388" y2="113"
+        stroke="#333" marker-end="url(#arr)" />
+  <text x="336" y="112" text-anchor="middle" font-size="10" fill="#555">msgQ</text>
+
+  <!-- sendMsg -->
+  <rect x="390" y="102" width="90" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="435" y="124" text-anchor="middle">sendMsg</text>
+
+  <!-- network in label + arrow -->
+  <text x="13" y="81" font-size="9" fill="#999">net</text>
+  <line x1="12" y1="85" x2="45" y2="85"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- network out arrows -->
+  <line x1="460" y1="79" x2="498" y2="79"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <line x1="480" y1="119" x2="498" y2="119"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <text x="504" y="100" font-size="9" fill="#999">net</text>
+
+  <!-- client -> QueueStore (down-left) -->
+  <line x1="228" y1="102" x2="130" y2="150"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- client -> MsgStore (down-right) -->
+  <line x1="278" y1="102" x2="395" y2="150"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- QueueStore -->
+  <rect x="45" y="150" width="155" height="48" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="122" y="171" text-anchor="middle" font-weight="bold">QueueStore</text>
+  <text x="122" y="189" text-anchor="middle" font-size="10">(STM or Postgres)</text>
+
+  <!-- MsgStore -->
+  <rect x="320" y="150" width="155" height="48" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="397" y="171" text-anchor="middle" font-weight="bold">MsgStore</text>
+  <text x="397" y="189" text-anchor="middle" font-size="10">(STM or Postgres)</text>
+
+  <!-- StoreLog under QueueStore only (queues are logged, msgs stored directly) -->
+  <rect x="45" y="208" width="155" height="18" rx="3"
+        fill="#f5f5f5" stroke="#ccc" />
+  <text x="122" y="221" text-anchor="middle" font-size="9" fill="#888">StoreLog (optional)</text>
+
+  <!-- NtfStore under MsgStore (in-memory notification queue, read by deliverNtfs/expireNtfs) -->
+  <rect x="320" y="208" width="155" height="18" rx="3"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="397" y="221" text-anchor="middle" font-size="9" fill="#888">NtfStore (STM TMap)</text>
+
+  <!-- subQ arrow: from client bottom-center, through gap between stores, to serverThread -->
+  <!-- Gap between stores: x=200..320 -->
+  <line x1="250" y1="102" x2="250" y2="296"
+        stroke="#4285f4" stroke-width="1.5" stroke-dasharray="5,3"
+        marker-end="url(#arr)" />
+  <text x="262" y="238" font-size="10" fill="#4285f4">subQ</text>
+
+  <!-- ===== SINGLETON GROUP ===== -->
+  <rect x="25" y="258" width="810" height="195" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="273" fill="#888" font-size="10">singleton threads (one instance each, all in raceAny_)</text>
+
+  <!-- serverThread (SMP subs) -->
+  <rect x="45" y="288" width="175" height="46" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="132" y="308" text-anchor="middle">serverThread</text>
+  <text x="132" y="324" text-anchor="middle" font-size="10">(SMP subscriptions)</text>
+
+  <!-- serverThread (NTF subs) -->
+  <rect x="240" y="288" width="175" height="46" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="327" y="308" text-anchor="middle">serverThread</text>
+  <text x="327" y="324" text-anchor="middle" font-size="10">(NTF subscriptions)</text>
+
+  <!-- pendingEvents: serverThread -> sendPendingEvts -->
+  <line x1="195" y1="334" x2="230" y2="355"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="176" y="352" font-size="9" fill="#555">pendingEvents</text>
+
+  <!-- deliverNtfs -->
+  <rect x="45" y="352" width="128" height="30" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="109" y="372" text-anchor="middle" font-size="11">deliverNtfs</text>
+
+  <!-- sendPendingEvts -->
+  <rect x="195" y="352" width="148" height="30" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="269" y="372" text-anchor="middle" font-size="11">sendPendingEvts</text>
+
+  <!-- expireMessages -->
+  <rect x="365" y="352" width="138" height="30" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="434" y="372" text-anchor="middle" font-size="11">expireMessages</text>
+
+  <!-- expireNtfs -->
+  <rect x="45" y="396" width="115" height="30" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="102" y="416" text-anchor="middle" font-size="11">expireNtfs</text>
+
+  <!-- proxyAgent -->
+  <rect x="180" y="396" width="118" height="30" rx="4"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="239" y="416" text-anchor="middle" font-size="11">proxyAgent</text>
+
+  <!-- ===== OPTIONAL (nested inside singleton, right side) ===== -->
+  <rect x="610" y="278" width="215" height="130" rx="6"
+        fill="none" stroke="#bbb" stroke-dasharray="4,3" />
+  <text x="620" y="293" fill="#bbb" font-size="10">optional</text>
+
+  <!-- logServerStats -->
+  <rect x="625" y="302" width="140" height="24" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="695" y="319" text-anchor="middle" font-size="10">logServerStats</text>
+
+  <!-- prometheus -->
+  <rect x="625" y="336" width="140" height="24" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="695" y="353" text-anchor="middle" font-size="10">prometheus</text>
+
+  <!-- controlPort -->
+  <rect x="625" y="370" width="140" height="24" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="695" y="387" text-anchor="middle" font-size="10">controlPort</text>
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="25" y="472" width="810" height="60" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="40" y="486" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="60" y="496" font-size="10">per-client thread</text>
+
+  <rect x="195" y="486" width="14" height="11" rx="2"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="215" y="496" font-size="10">singleton thread</text>
+
+  <rect x="350" y="486" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="370" y="496" font-size="10">storage</text>
+
+  <rect x="445" y="486" width="14" height="11" rx="2"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="465" y="496" font-size="10">optional</text>
+
+  <text x="40" y="522" font-size="10" fill="#666">
+    Solid arrows: TBQueue connections.  Dashed blue: subQ linking per-client to singleton threads.
+  </text>
+
+</svg>

spec/diagrams/xftp-client.svg

@@ -0,0 +1,80 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 620 280" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+  </defs>
+
+  <!-- ===== CLIENT WRAPPER ===== -->
+  <rect x="120" y="40" width="380" height="130" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="130" y="55" fill="#888" font-size="10">per connection (XFTPClient adds no threads; serialization in HTTP2Client)</text>
+
+  <!-- caller label -->
+  <text x="15" y="95" font-size="9" fill="#999">caller</text>
+  <text x="10" y="108" font-size="9" fill="#999">(Agent/</text>
+  <text x="10" y="121" font-size="9" fill="#999"> router)</text>
+
+  <!-- caller -> XFTPClient -->
+  <line x1="55" y1="100" x2="138" y2="100"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- XFTPClient -->
+  <rect x="140" y="68" width="150" height="48" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="215" y="89" text-anchor="middle" font-weight="bold">XFTPClient</text>
+  <text x="215" y="107" text-anchor="middle" font-size="10">sendXFTPCommand</text>
+
+  <!-- XFTPClient -> HTTP2Client -->
+  <line x1="290" y1="92" x2="328" y2="92"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- HTTP2Client -->
+  <rect x="330" y="68" width="150" height="48" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="405" y="89" text-anchor="middle" font-weight="bold">HTTP2Client</text>
+  <text x="405" y="107" text-anchor="middle" font-size="10">(TLS + HTTP/2 streams)</text>
+
+  <!-- HTTP2Client -> network -->
+  <line x1="480" y1="92" x2="518" y2="92"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <text x="525" y="88" font-size="9" fill="#999">XFTP</text>
+  <text x="525" y="101" font-size="9" fill="#999">Router</text>
+  <text x="525" y="114" font-size="9" fill="#999">(HTTP/2)</text>
+
+  <!-- Handshake state -->
+  <rect x="140" y="130" width="150" height="32" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="215" y="151" text-anchor="middle" font-size="10">thParams (negotiated)</text>
+
+  <!-- upload/download note -->
+  <text x="330" y="143" font-size="10" fill="#666">uploads: streaming request body</text>
+  <text x="330" y="157" font-size="10" fill="#666">downloads: ephemeral DH + streaming</text>
+  <text x="330" y="171" font-size="10" fill="#666">  response body (per-chunk forward secrecy)</text>
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="25" y="195" width="570" height="70" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="40" y="210" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="60" y="220" font-size="10">client wrapper</text>
+
+  <rect x="195" y="210" width="14" height="11" rx="2"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="215" y="220" font-size="10">external connection</text>
+
+  <rect x="385" y="210" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="405" y="220" font-size="10">state</text>
+
+  <text x="40" y="248" font-size="10" fill="#666">
+    XFTPClient adds no threads. HTTP2Client has internal reqQ + process thread.
+  </text>
+
+</svg>

spec/diagrams/xftp-router.svg

@@ -0,0 +1,130 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 740 460" font-family="monospace" font-size="12">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#333" />
+    </marker>
+    <marker id="arr-g" viewBox="0 0 10 10" refX="9" refY="5"
+            markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#999" />
+    </marker>
+  </defs>
+
+  <!-- ===== REQUEST HANDLING (no persistent threads) ===== -->
+  <rect x="25" y="38" width="510" height="280" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="53" fill="#888" font-size="10">per request (inline HTTP/2 callback, no spawned threads)</text>
+
+  <!-- network in -->
+  <text x="13" y="78" font-size="9" fill="#999">net</text>
+  <line x1="12" y1="82" x2="45" y2="82"
+        stroke="#999" marker-end="url(#arr-g)" />
+
+  <!-- HTTP/2 Transport -->
+  <rect x="45" y="66" width="190" height="34" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="140" y="88" text-anchor="middle">HTTP/2 handler</text>
+
+  <!-- Handshake state -->
+  <rect x="280" y="60" width="240" height="48" rx="4"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="400" y="78" text-anchor="middle" font-size="11">Handshake State (per session)</text>
+  <text x="400" y="96" text-anchor="middle" font-size="10" fill="#666">None -> Sent -> Accepted</text>
+
+  <!-- sessions TMap link -->
+  <line x1="235" y1="80" x2="278" y2="80"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+  <text x="256" y="74" text-anchor="middle" font-size="9" fill="#555">sessions</text>
+
+  <!-- HTTP/2 handler -> Command Processing -->
+  <line x1="140" y1="100" x2="140" y2="118"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- Command Processing -->
+  <rect x="45" y="120" width="475" height="36" rx="4"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="282" y="143" text-anchor="middle">Command Processing (FNEW, FADD, FPUT, FGET, FACK, FDEL, PING)</text>
+
+  <!-- Command Processing -> FileStore -->
+  <line x1="160" y1="156" x2="130" y2="180"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- Command Processing -> Disk Storage -->
+  <line x1="380" y1="156" x2="390" y2="180"
+        stroke="#333" marker-end="url(#arr)" />
+
+  <!-- FileStore -->
+  <rect x="45" y="182" width="180" height="48" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="135" y="203" text-anchor="middle" font-weight="bold">FileStore</text>
+  <text x="135" y="221" text-anchor="middle" font-size="10">(TMap in STM)</text>
+
+  <!-- Disk Storage -->
+  <rect x="300" y="182" width="220" height="48" rx="4"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="410" y="203" text-anchor="middle" font-weight="bold">Disk Storage</text>
+  <text x="410" y="221" text-anchor="middle" font-size="10">filesPath / base64(senderId)</text>
+
+  <!-- Quota note -->
+  <text x="410" y="246" text-anchor="middle" font-size="9" fill="#888">quota-managed via usedStorage TVar</text>
+
+  <!-- StoreLog -->
+  <rect x="45" y="260" width="180" height="18" rx="3"
+        fill="#f5f5f5" stroke="#ccc" />
+  <text x="135" y="273" text-anchor="middle" font-size="9" fill="#888">StoreLog (append-only)</text>
+
+  <!-- FileStore -> StoreLog -->
+  <line x1="135" y1="230" x2="135" y2="258"
+        stroke="#555" stroke-dasharray="3,2" marker-end="url(#arr)" />
+
+  <!-- network out (command result) -->
+  <line x1="520" y1="138" x2="548" y2="138"
+        stroke="#999" marker-end="url(#arr-g)" />
+  <text x="555" y="143" font-size="9" fill="#999">net</text>
+
+  <!-- ===== BACKGROUND THREADS ===== -->
+  <rect x="25" y="335" width="690" height="50" rx="6"
+        fill="none" stroke="#888" stroke-dasharray="6,3" />
+  <text x="35" y="350" fill="#888" font-size="10">background threads (singleton, in raceAny_)</text>
+
+  <!-- expireFiles -->
+  <rect x="45" y="358" width="118" height="22" rx="3"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="104" y="374" text-anchor="middle" font-size="10">expireFiles</text>
+
+  <!-- logServerStats -->
+  <rect x="180" y="358" width="128" height="22" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="244" y="374" text-anchor="middle" font-size="10">logServerStats</text>
+
+  <!-- prometheus -->
+  <rect x="325" y="358" width="118" height="22" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="384" y="374" text-anchor="middle" font-size="10">prometheus</text>
+
+  <!-- controlPort -->
+  <rect x="460" y="358" width="118" height="22" rx="3"
+        fill="#f5f5f5" stroke="#bbb" />
+  <text x="519" y="374" text-anchor="middle" font-size="10">controlPort</text>
+
+  <!-- ===== LEGEND ===== -->
+  <rect x="25" y="400" width="690" height="50" rx="4"
+        fill="none" stroke="#ddd" />
+
+  <rect x="40" y="414" width="14" height="11" rx="2"
+        fill="#e8f0fe" stroke="#4285f4" />
+  <text x="60" y="424" font-size="10">request handler (no threads)</text>
+
+  <rect x="250" y="414" width="14" height="11" rx="2"
+        fill="#fef7e0" stroke="#f9ab00" />
+  <text x="270" y="424" font-size="10">storage</text>
+
+  <rect x="350" y="414" width="14" height="11" rx="2"
+        fill="#fce8e6" stroke="#ea4335" />
+  <text x="370" y="424" font-size="10">per-session state</text>
+
+  <rect x="500" y="414" width="14" height="11" rx="2"
+        fill="#e6f4ea" stroke="#34a853" />
+  <text x="520" y="424" font-size="10">background thread</text>
+
+</svg>

spec/modules/NOTES.md

@@ -0,0 +1,155 @@
+# Design Notes
+
+Non-bug observations from module specs that are worth tracking. These remain documented in their respective module specs — this file serves as an index.
+
+## Backend Observations
+
+### N-01: SNotifier path doesn't cache
+
+**Location**: `Simplex.Messaging.Server.QueueStore.Postgres` — `getQueues_` SNotifier branch
+**Description**: The SRecipient path caches loaded queues via `cacheRcvQueue` with double-check locking. The SNotifier path does NOT cache — it uses a stale TMap snapshot and `maybe (mkQ False rId qRec) pure`, so concurrent loads for the same notifier can create duplicate ephemeral queue objects. Functionally correct but wasteful.
+**Module spec**: [QueueStore/Postgres.md](Simplex/Messaging/Server/QueueStore/Postgres.md)
+
+### N-02: assertUpdated error conflation
+
+**Location**: `Simplex.Messaging.Server.QueueStore.Postgres` — `assertUpdated`
+**Description**: `assertUpdated` returns `AUTH` for zero-rows-affected. This is the same error code used for "not found" (via `readQueueRecIO`) and "duplicate" (via `handleDuplicate`). The actual cause — stale cache, deleted queue, or constraint violation — is indistinguishable in logs.
+**Module spec**: [QueueStore/Postgres.md](Simplex/Messaging/Server/QueueStore/Postgres.md)
+
+## Design Characteristics
+
+### N-03: RCVerifiedInvitation constructor exported
+
+**Location**: `Simplex.RemoteControl.Invitation` — `RCVerifiedInvitation`
+**Description**: `RCVerifiedInvitation` is a newtype with constructor exported via `(..)`. It can be constructed without calling `verifySignedInvitation`, bypassing signature verification. The trust boundary is conventional, not enforced by the type system. `connectRCCtrl` accepts only `RCVerifiedInvitation`.
+**Module spec**: [RemoteControl/Invitation.md](Simplex/RemoteControl/Invitation.md)
+
+### N-04: smpEncode Word16 silent truncation
+
+**Location**: `Simplex.Messaging.Encoding` — `Encoding Word16` instance
+**Description**: `smpEncode` for ByteString uses a 1-byte length prefix. Maximum encodable length is 255 bytes. Longer values silently wrap via `w2c . fromIntegral`. Callers must ensure ByteStrings fit or use `Large`.
+**Module spec**: [Encoding.md](Simplex/Messaging/Encoding.md)
+
+### N-05: writeIORef for period stats — not atomic
+
+**Location**: `Simplex.Messaging.Server.Stats` — `setPeriodStats`
+**Description**: Uses `writeIORef` (not atomic). Only safe during router startup when no other threads are running. If called concurrently, period data could be corrupted.
+**Module spec**: [Server/Stats.md](Simplex/Messaging/Server/Stats.md)
+
+### N-06: setStatsByServer orphans old TVars
+
+**Location**: `Simplex.Messaging.Notifications.Server.Stats` — `setStatsByServer`
+**Description**: Builds a fresh `Map Text (TVar Int)` in IO, then atomically replaces the TMap's root TVar. Old per-router TVars are not reused — any other thread holding a reference from a prior `TM.lookupIO` would modify an orphaned counter. Called at startup, but lacks the explicit "not thread safe" comment.
+**Module spec**: [Notifications/Server/Stats.md](Simplex/Messaging/Notifications/Server/Stats.md)
+
+### N-07: Lazy.unPad doesn't validate data length
+
+**Location**: `Simplex.Messaging.Crypto.Lazy` — `unPad` / `splitLen`
+**Description**: `splitLen` does not validate that the remaining data is at least `len` bytes — `LB.take len` silently returns a shorter result. The source comment notes this is intentional to avoid consuming all lazy chunks for validation.
+**Module spec**: [Crypto/Lazy.md](Simplex/Messaging/Crypto/Lazy.md)
+
+### N-08: Batched commands have no timeout-based expiry
+
+**Location**: `Simplex.Messaging.Client` — `sendBatch`
+**Description**: Batched commands are written with `Nothing` as the request parameter — the send thread skips the `pending` flag check. Individual commands have timeout-based expiry. If the router stops returning results, batched commands can block the send queue indefinitely.
+**Module spec**: [Client.md](Simplex/Messaging/Client.md)
+
+### N-09: Postgres MsgStore nanosecond precision
+
+**Location**: `Simplex.Messaging.Server.MsgStore.Postgres` — `toMessage`
+**Description**: `MkSystemTime ts 0` constructs timestamps with zero nanoseconds. Only whole seconds are stored. Messages read from Postgres have coarser timestamps than STM/Journal stores. Not a practical issue — timestamps are typically rounded to hours or days.
+**Module spec**: [Server/MsgStore/Postgres.md](Simplex/Messaging/Server/MsgStore/Postgres.md)
+
+### N-10: MsgStore Postgres — error stubs crash at runtime
+
+**Location**: `Simplex.Messaging.Server.MsgStore.Postgres` — multiple `MsgStoreClass` methods
+**Description**: Multiple `MsgStoreClass` methods are `error "X not used"`. Required by the type class but not applicable to Postgres. Calling any at runtime crashes. Safe because Postgres overrides the relevant default methods, but a new caller using the wrong method would crash with no compile-time warning.
+**Module spec**: [Server/MsgStore/Postgres.md](Simplex/Messaging/Server/MsgStore/Postgres.md)
+
+### N-11: strP default assumes base64url for all types
+
+**Location**: `Simplex.Messaging.Encoding.String` — `StrEncoding` class default
+**Description**: The `MINIMAL` pragma allows defining only `strDecode` without `strP`. The default `strP = strDecode <$?> base64urlP` assumes input is base64url-encoded for any type. A new `StrEncoding` instance that defines only `strDecode` for non-base64 data would get a broken parser.
+**Module spec**: [Encoding/String.md](Simplex/Messaging/Encoding/String.md)
+
+## Silent Behaviors
+
+Intentional design choices that are correct but non-obvious. A code modifier who doesn't know these could introduce bugs.
+
+### N-12: Service signing silently skipped on empty authenticator
+
+**Location**: `Simplex.Messaging.Client` — service signature path
+**Description**: The service signature is only added when the entity authenticator is non-empty. If authenticator generation fails silently (returns empty bytes), service signing is silently skipped.
+**Module spec**: [Client.md](Simplex/Messaging/Client.md)
+
+### N-13: stmDeleteNtfToken — nonexistent token indistinguishable from empty
+
+**Location**: `Simplex.Messaging.Notifications.Server.Store` — `stmDeleteNtfToken`
+**Description**: If the token ID doesn't exist in the `tokens` map, the registration-cleanup branch is skipped and the function returns an empty list. The caller cannot distinguish "deleted a token with no subscriptions" from "token never existed."
+**Module spec**: [Notifications/Server/Store.md](Simplex/Messaging/Notifications/Server/Store.md)
+
+### N-14: createCommand silently drops commands for deleted connections
+
+**Location**: `Simplex.Messaging.Agent.Store.AgentStore` — `createCommand`
+**Description**: When `createCommand` encounters a constraint violation (the referenced connection was already deleted), it logs the error and returns successfully. Commands targeting deleted connections are silently dropped.
+**Module spec**: [Agent/Store/AgentStore.md](Simplex/Messaging/Agent/Store/AgentStore.md)
+
+### N-15: Redirect chain loading errors silently swallowed
+
+**Location**: `Simplex.Messaging.Agent.Store.AgentStore`
+**Description**: When loading redirect chains, errors loading individual redirect files are silently swallowed via `either (const $ pure Nothing) (pure . Just)`. Prevents a corrupt redirect from blocking access to the main file.
+**Module spec**: [Agent/Store/AgentStore.md](Simplex/Messaging/Agent/Store/AgentStore.md)
+
+### N-16: BLOCKED encoded as AUTH for old XFTP clients
+
+**Location**: `Simplex.FileTransfer.Protocol` — `encodeProtocol`
+**Description**: If the protocol version is below `blockedFilesXFTPVersion`, a `BLOCKED` result is encoded as `AUTH` instead. The blocking information (reason) is permanently lost for these clients.
+**Module spec**: [FileTransfer/Protocol.md](Simplex/FileTransfer/Protocol.md)
+
+### N-17: restore_messages three-valued logic with implicit default
+
+**Location**: `Simplex.Messaging.Server.Main` — INI config
+**Description**: The `restore_messages` INI setting has three-valued logic: explicit "on" → restore, explicit "off" → skip, missing → inherits from `enable_store_log`. This implicit default is not captured in the type system — callers see `Maybe Bool`.
+**Module spec**: [Server/Main.md](Simplex/Messaging/Server/Main.md)
+
+### N-18: Stats format migration permanently loses precision
+
+**Location**: `Simplex.Messaging.Server.Stats` — `strP` for `ServerStatsData`
+**Description**: The parser handles multiple format generations. Old format `qDeleted=` is read as `(value, 0, 0)`. `qSubNoMsg` is parsed and discarded. `subscribedQueues` is parsed but replaced with empty data. Data loaded from old formats is coerced — precision is permanently lost.
+**Module spec**: [Server/Stats.md](Simplex/Messaging/Server/Stats.md)
+
+### N-19: resubscribe exceptions silently lost
+
+**Location**: `Simplex.Messaging.Notifications.Server` — `resubscribe`
+**Description**: `resubscribe` is launched via `forkIO` before `raceAny_` starts — not part of the `raceAny_` group. Most exceptions are silently lost per `forkIO` semantics. `ExitCode` exceptions are special-cased by GHC's runtime and do propagate.
+**Module spec**: [Notifications/Server.md](Simplex/Messaging/Notifications/Server.md)
+
+### N-20: closeSMPClientAgent worker cancellation is fire-and-forget
+
+**Location**: `Simplex.Messaging.Client.Agent` — `closeSMPClientAgent`
+**Description**: Executes in order: set `active = False`, close all client connections, swap workers map to empty and fork cancellation threads. Cancel threads use `uninterruptibleCancel` but are fire-and-forget — the function may return before all workers are cancelled.
+**Module spec**: [Client/Agent.md](Simplex/Messaging/Client/Agent.md)
+
+### N-21: APNS unknown 410 reasons trigger retry instead of permanent failure
+
+**Location**: `Simplex.Messaging.Notifications.Server.Push.APNS`
+**Description**: Unknown 410 (Gone) reasons fall through to `PPRetryLater`, while unknown 400 and 403 reasons fall through to `PPResponseError`. An unexpected APNS 410 reason string triggers retry rather than permanent failure.
+**Module spec**: [Notifications/Server/Push/APNS.md](Simplex/Messaging/Notifications/Server/Push/APNS.md)
+
+### N-22: NTInvalid/NTExpired tokens can create subscriptions
+
+**Location**: `Simplex.Messaging.Notifications.Protocol` — token status permissions
+**Description**: Token status `NTInvalid` allows subscription commands (SNEW, SCHK, SDEL). A TODO comment explains: invalidation can happen after verification, and existing subscriptions should remain manageable. `NTExpired` is also permitted.
+**Module spec**: [Notifications/Protocol.md](Simplex/Messaging/Notifications/Protocol.md)
+
+### N-23: removeInactiveTokenRegistrations doesn't clean up empty inner maps
+
+**Location**: `Simplex.Messaging.Notifications.Server.Store` — `stmRemoveInactiveTokenRegistrations`
+**Description**: `stmDeleteNtfToken` checks whether inner TMap is empty after removal and cleans up the outer key. `stmRemoveInactiveTokenRegistrations` does not — surviving active tokens' registrations remain, but empty inner maps can persist.
+**Module spec**: [Notifications/Server/Store.md](Simplex/Messaging/Notifications/Server/Store.md)
+
+### N-24: cbNonce silently truncates or pads
+
+**Location**: `Simplex.Messaging.Crypto` — `cbNonce`
+**Description**: If the input is longer than 24 bytes, it is silently truncated. If shorter, it is silently padded. No error is raised. Callers must ensure correct length.
+**Module spec**: [Crypto.md](Simplex/Messaging/Crypto.md)

spec/modules/README.md

@@ -0,0 +1,197 @@
+# How to Document a Module
+
+> Read this before writing any module doc. It defines what goes in, what stays out, and why.
+
+## Purpose
+
+Module docs exist for one reason: to capture knowledge that **cannot be obtained by reading the source code**. If reading the `.hs` file tells you everything you need to know, the module doc should be brief or empty.
+
+These docs are an investment — their value compounds over time as multiple people (and LLMs) work on the code. Optimize for long-term value, not for looking thorough today.
+
+## Process
+
+**Read every line of the source file.** The non-obvious filter applies to what you *write*, not to what you *read*. Without reading each line, you will produce documentation from inferences rather than facts. Many non-obvious behaviors only become visible when you see a specific line of code and recognize that its implications would surprise a reader who doesn't have the surrounding context.
+
+## File structure
+
+Module docs mirror `src/Simplex/` exactly. Same subfolder structure, `.hs` replaced with `.md`:
+
+```
+src/Simplex/Messaging/Server.hs    →  spec/modules/Simplex/Messaging/Server.md
+src/Simplex/Messaging/Crypto.hs    →  spec/modules/Simplex/Messaging/Crypto.md
+src/Simplex/FileTransfer/Agent.hs  →  spec/modules/Simplex/FileTransfer/Agent.md
+```
+
+## What to include
+
+### 1. Non-obvious behavior
+Things that would surprise a competent Haskell developer reading the code for the first time:
+- Subtle invariants maintained across function calls
+- Ordering dependencies ("must call X before Y because...")
+- Concurrency assumptions ("this TVar is only written from thread Z")
+- Implicit contracts between caller and callee not captured by types
+
+### 2. Usage considerations
+- When to use function X vs function Y
+- Common mistakes callers make
+- Caller obligations not enforced by the type system
+- Performance characteristics that affect usage decisions
+
+### 3. Cross-module relationships
+- Dependencies on other modules' behavior not visible from import lists
+- Assumptions about how other modules use this one
+- Coordination patterns (e.g., "Server.hs reads this TVar, Agent.hs writes it")
+
+### 4. Security notes
+- Trust boundaries this module enforces or relies on
+- What happens if inputs are malicious
+- Which functions are security-critical and why (reference SI-XX invariants)
+
+### 5. Design rationale
+- Why the code is structured this way (when not obvious)
+- Alternatives considered and rejected
+- Known limitations and their justification
+
+## Non-obvious threshold
+
+The guiding principle: **non-obvious state machines and flows require documentation; standard things don't.**
+
+Document:
+- Multi-step protocols and negotiation flows (e.g., KEM propose/accept round-trips)
+- Monotonic or irreversible state transitions (e.g., PQ support can only be enabled, never disabled)
+- Silent error behaviors (e.g., `verify` returns `False` on algorithm mismatch instead of an error)
+- Design rationale for non-standard choices (e.g., why byte-reverse a nonce, why hash-then-encrypt for authenticators)
+
+Do NOT document:
+- Standard algorithm properties (e.g., Ed25519 public key derivable from private key)
+- Well-known protocol mechanics (e.g., HKDF usage per RFC 5869, deterministic nonce derivation in double ratchet)
+- Implementation details that follow directly from the type signatures
+
+## What NOT to include
+
+- **Type signatures** — the code has them
+- **Code snippets** — if you're pasting code, you're making a stale copy
+- **Function-by-function prose that restates the implementation** — "this function takes X and returns Y by doing Z" adds nothing
+- **Line numbers** — they're brittle and break on every edit
+- **Comments that fit in one line in source** — put those in the source file instead as `-- spec:` comments
+- **Verbatim quotes of source comments** — reference them instead: "See comment on `functionName`." Then add only what the comment doesn't cover (cross-module implications, what breaks if violated). If the source comment says everything, the function doesn't need a doc entry.
+- **Tables that reproduce code structure** — if the information is self-evident from reading the code's pattern matching or type definitions, it doesn't belong in the doc (e.g., per-command credential requirements, version-conditional encoding branches)
+
+## Format
+
+Each module doc has a header, then entries for functions/types that need documentation.
+
+```markdown
+# Module.Name
+
+> One-line description of what this module does.
+
+**Source**: [`Path/To/Module.hs`](relative link to source)
+
+## Overview
+
+[Only if the module's purpose or architecture is non-obvious.
+Skip for simple modules.]
+
+## functionName
+
+**Purpose**: [What this does that isn't obvious from the name and type]
+**Calls**: [Qualified.Name.a](link), [Qualified.Name.b](link)
+**Called by**: [Qualified.Name.c](link)
+**Invariant**: SI-XX
+**Security**: [What this function ensures for the threat model]
+
+[Free-form notes about non-obvious behavior, gotchas, etc.]
+
+## anotherFunction
+
+...
+```
+
+**For trivial modules** (< 100 LOC, no non-obvious behavior):
+
+```markdown
+# Module.Name
+
+> One-line description.
+
+**Source**: [`Path/To/Module.hs`](relative link to source)
+
+No non-obvious behavior. See source.
+```
+
+This is valuable — it confirms someone looked and found nothing to document.
+
+## Linking conventions
+
+### Module doc → protocol docs
+When a module implements or is governed by a protocol specification in `protocol/`, link to it near the top of the module doc (after the overview). Do not duplicate protocol content — just reference it:
+```markdown
+**Protocol spec**: [`protocol/pqdr.md`](../../../../protocol/pqdr.md) — Post-quantum resistant augmented double ratchet algorithm.
+```
+
+This is especially important for modules in transport, protocol, client, server, and agent layers where behavior is defined by the protocol spec rather than being self-evident from the code.
+
+### Module doc → other module docs
+Use fully qualified names as link text:
+```markdown
+[Simplex.Messaging.Server.subscribeServiceMessages](./Simplex/Messaging/Server.md#subscribeServiceMessages)
+```
+
+### Module doc → topic docs
+```markdown
+See [rcv-services](../rcv-services.md) for the end-to-end service subscription flow.
+```
+
+### Source → module doc
+
+Add `-- spec:` comments as part of the module documentation work — when you document something non-obvious, add the link in source at the same time. Two levels:
+
+**Module-level** (below the module declaration): when the Overview section has value.
+```haskell
+module Simplex.Messaging.Util (...) where
+-- spec: spec/modules/Simplex/Messaging/Util.md
+```
+
+**Function-level** (above the function): when that function has a doc entry worth pointing to.
+```haskell
+-- spec: spec/modules/Simplex/Messaging/Util.md#catchOwn
+-- Catches all exceptions except async cancellations (misleading name)
+catchOwn :: ...
+```
+
+Only add `-- spec:` comments where the module doc actually says something the code doesn't. Don't add links to "No non-obvious behavior" docs or to entries that merely restate the source.
+
+## Topic candidate tracking
+
+While documenting modules, you will notice cross-cutting patterns — behaviors that span multiple modules and can't be understood from any single one. Note these in `spec/TOPICS.md` for later. Don't write the topic doc during module work; just record:
+
+```markdown
+- **Queue rotation**: Agent.hs initiates, Client.hs sends commands, Server.hs processes,
+  Protocol.hs defines types. End-to-end flow not obvious from any single module.
+```
+
+## Quality bar
+
+Before finishing a module doc, ask:
+1. Does every entry document something NOT in the source code?
+2. Would removing any entry lose information? If not, remove it.
+3. Are cross-module relationships captured that imports alone don't reveal?
+4. Are security-critical functions flagged with invariant IDs?
+5. Is this doc short enough that someone will actually read it?
+
+If any answer reveals a problem, fix it and repeat from question 1. Only finish when a full pass produces no changes.
+
+## Terminology — the spec as translation boundary
+
+The protocol documents (`protocol/overview-tjr.md`, `protocol/simplex-messaging.md`, `protocol/agent-protocol.md`) define the canonical terminology. Code uses different names for some of the same concepts. The spec is where the translation happens.
+
+The most important distinction: SimpleX protocol routers are referred to as "servers" in code. The term "server" was adopted historically because SimpleX routers were implemented as Linux-based software that is deployed in the same way as servers. But the similarity is entirely formal. Functionally, servers serve responses to the requests of their users - that is why the term "server" was adopted for computers and software that provide Internet services. SimpleX protocol routers don't serve responses - they route packets between endpoints, and they have no concept of a user. Functionally they are similar to Internet Protocol routers, but with a resource-based addressing scheme. Further, SimpleX protocol routers are hardware and software agnostic. SimpleX protocols are open and documented, so they can be implemented in any language and run on a different architecture. For example, [SimpleGo](https://simplego.dev) is a prototype implementation of the SimpleX protocol stack in C for a microcontroller architecture.
+
+**The rule**: use protocol terms for concepts, code terms for identifiers. Write "router" when describing the network node's role, `SMPServer` or `Server.hs` when referencing code. Similarly, "router identity" for the concept (called "server key hash" or "fingerprint" in code). When the distinction matters, bridge explicitly: "the SMP router (implemented by the `Server` module)" or "the `SMPServer` type (representing a router address)."
+
+## Exclusions
+
+- **Individual migration files** (M20XXXXXX_*.hs): Self-describing SQL. No per-migration docs.
+- **Auto-generated files** (GitCommit.hs): Skip.
+- **Pure boilerplate** (Prometheus.hs metrics, Web/Embedded.hs static files): Document only if non-obvious.

spec/modules/Simplex/FileTransfer/Agent.md

@@ -0,0 +1,90 @@
+# Simplex.FileTransfer.Agent
+
+> XFTP agent: worker-based file send/receive/delete with retry, encryption, redirect chains, and file description generation.
+
+**Source**: [`FileTransfer/Agent.hs`](../../../../src/Simplex/FileTransfer/Agent.hs)
+
+## Terminology
+
+The agent splits a **file** into **chunks** determined by the chunking algorithm. Each chunk is stored on an XFTP router as a **data packet** — the router has no concept of files or chunks, only directly addressable data packets. This document uses "chunk" for the agent's internal tracking and "data packet" when referring to what is transferred to/from or stored on routers.
+
+## Architecture
+
+The XFTP agent uses five worker types organized in three categories:
+
+| Worker | Key (router) | Purpose |
+|--------|-------------|---------|
+| `xftpRcvWorker` | `Just server` | Download data packets from a specific XFTP router |
+| `xftpRcvLocalWorker` | `Nothing` | Decrypt completed downloads locally |
+| `xftpSndPrepareWorker` | `Nothing` | Encrypt files and create data packets on routers |
+| `xftpSndWorker` | `Just server` | Upload data packets to a specific XFTP router |
+| `xftpDelWorker` | `Just server` | Delete data packets from a specific XFTP router |
+
+Workers are created on-demand via `getAgentWorker` and keyed by router address. The local workers (keyed by `Nothing`) handle CPU-bound operations that don't require network access.
+
+## Non-obvious behavior
+
+### 1. startXFTPWorkers vs startXFTPSndWorkers
+
+`startXFTPWorkers` starts all three worker categories (rcv, snd, del). `startXFTPSndWorkers` starts only snd workers. This distinction exists because receiving and deleting require a full agent context, while sending can operate with a partial setup (used when the agent is in send-only mode).
+
+### 2. Download completion triggers local worker
+
+When `downloadFileChunk` determines that all chunks are received (`all chunkReceived chunks`), it calls `getXFTPRcvWorker True c Nothing` to wake the local decryption worker. The `True` parameter signals that work is available. Without this, the local worker would sleep until the next `waitForWork` check.
+
+### 3. Decryption verifies both digest and size before decrypting
+
+`decryptFile` first computes the total size of all encrypted chunk files, then their SHA-512 digest. If either mismatches the expected values, it throws an error *before* starting decryption. This prevents wasting CPU on corrupted or tampered downloads.
+
+### 4. Redirect chain with depth limit
+
+When a received file has a `redirect`, the local worker:
+1. Decrypts the redirect file (a YAML file description)
+2. Validates the inner description's size and digest against `RedirectFileInfo`
+3. Registers the inner file's chunks and starts downloading them
+
+The redirect chain is implicitly limited to depth 1: `createRcvFileRedirect` creates the destination file entry with `redirect = Nothing`, and `updateRcvFileRedirect` does not update the redirect column. So even if the decoded inner description contains a redirect field, the database record for the destination file has no redirect, preventing further chaining.
+
+### 5. Decrypting worker resumes from RFSDecrypting
+
+If the agent restarts while a file is in `RFSDecrypting` status, the local worker detects this and deletes the partially-decrypted output file before restarting decryption. This prevents corrupted output from a previous incomplete decryption attempt.
+
+### 6. Encryption worker resumes from SFSEncrypting
+
+Similarly, `prepareFile` checks `status /= SFSEncrypted` and deletes the partial encrypted file if status is `SFSEncrypting`. This allows clean restart of interrupted encryption.
+
+### 7. Redirect files must be single-chunk
+
+`encryptFileForUpload` for redirect files calls `singleChunkSize` instead of `prepareChunkSizes`. If the redirect file description doesn't fit in a single chunk, it throws `FILE SIZE`. This ensures redirect files are atomic — they either download completely or not at all.
+
+### 8. addRecipients recursive batching
+
+During upload, `addRecipients` recursively calls itself if a data packet needs more recipients than `xftpMaxRecipientsPerRequest`. Each iteration sends an FADD command for up to `maxRecipients` new recipients, accumulates the results, and recurses until all recipients are registered.
+
+### 9. File description generation cross-product
+
+`createRcvFileDescriptions` (in both `Agent.hs` and `Client/Main.hs`) performs a cross-product transformation: M chunks × R replicas × N recipients → N file descriptions, each containing M chunks with R replicas. The `addRcvChunk` accumulator builds a `Map rcvNo (Map chunkNo FileChunk)` to correctly distribute replicas across recipient descriptions.
+
+### 10. withRetryIntervalLimit caps consecutive retries
+
+`withRetryIntervalLimit maxN` allows at most `maxN` total attempts (initial attempt at `n=0` plus `maxN-1` retries). When all attempts are exhausted for temporary errors, the operation is silently abandoned for this work cycle — the chunk remains in pending state and may be retried on the next cycle. Only permanent errors (handled by `retryDone`) mark the file as errored.
+
+### 11. Retry distinguishes temporary from permanent errors
+
+`retryOnError` checks `temporaryOrHostError`: temporary/host errors trigger retry with exponential backoff; permanent errors (AUTH, SIZE, etc.) immediately mark the file as failed. On host errors during retry, a warning notification is sent to the client.
+
+### 12. Delete workers skip files older than rcvFilesTTL
+
+`runXFTPDelWorker` uses `rcvFilesTTL` (not a dedicated delete TTL) to filter pending deletions. Data packets older than this TTL would already be expired on the router, so attempting deletion is pointless. This reuses the receive TTL as a proxy for router-side expiration.
+
+### 13. closeXFTPAgent atomically swaps worker maps
+
+`closeXFTPAgent` uses `swapTVar workers M.empty` to atomically replace each worker map with an empty map, then cancels all retrieved workers. This prevents races where a new worker could be inserted between reading and clearing the map.
+
+### 14. assertAgentForeground dual check
+
+`assertAgentForeground` both throws if the agent is inactive (`throwWhenInactive`) and blocks until it's in the foreground (`waitUntilForeground`). This is called before every chunk operation to ensure the agent isn't suspended or backgrounded during file transfers.
+
+### 15. Per-router stats tracking
+
+Every data packet download, upload, and delete operation increments per-router statistics (`downloads`, `uploads`, `deletions`, `downloadAttempts`, `uploadAttempts`, `deleteAttempts`, and error variants). Size-based stats (`downloadsSize`, `uploadsSize`) track throughput in kilobytes.

spec/modules/Simplex/FileTransfer/Client.md

@@ -0,0 +1,37 @@
+# Simplex.FileTransfer.Client
+
+> XFTP client: connection management, handshake, data packet upload/download with forward secrecy.
+
+**Source**: [`FileTransfer/Client.hs`](../../../../src/Simplex/FileTransfer/Client.hs)
+
+## Non-obvious behavior
+
+### 1. ALPN-based handshake version selection
+
+`getXFTPClient` checks the ALPN result after TLS negotiation:
+- **`xftpALPNv1` or `httpALPN11`**: performs v1 handshake with key exchange (`httpALPN11` is used for web port connections)
+- **No ALPN or unrecognized**: uses legacy v1 transport parameters without handshake
+
+### 2. Router certificate chain validation
+
+`xftpClientHandshakeV1` validates the router's identity by checking that the CA fingerprint from the certificate chain matches the expected `keyHash` from the router address. The router signs an authentication public key (X25519) with its long-term key. The client verifies this signature against the certificate chain, then extracts the X25519 key for HMAC-based command authentication. This authentication key is distinct from the per-download ephemeral DH keys.
+
+### 3. Ephemeral DH key pair per download
+
+`downloadXFTPChunk` generates a fresh X25519 key pair for each data packet download. The public key is sent with the FGET command; the router returns its own ephemeral key. The derived shared secret encrypts the data packet in transit. This provides forward secrecy — compromising a past DH key doesn't decrypt other downloads.
+
+### 4. Size-proportional download timeout
+
+`downloadXFTPChunk` calculates the timeout as `baseTimeout + (sizeInKB * perKbTimeout)`, where `baseTimeout` is the base TCP timeout and `perKbTimeout` is a per-kilobyte timeout from the network config. Larger data packets get proportionally more time. This prevents premature timeouts on large data packets over slow connections.
+
+### 5. prepareChunkSizes threshold algorithm
+
+`prepareChunkSizes` selects data packet sizes using a 75% threshold: if the remaining payload exceeds 75% of the next larger size, it uses the larger size. Otherwise, it uses the smaller size. `singleChunkSize` returns `Just size` only if the payload fits in a single data packet (used for redirect files which must be single-packet).
+
+### 6. Upload sends data packet after command block
+
+`uploadXFTPChunk` sends the FPUT command and data packet body in the same streaming HTTP/2 request: the protocol command block is sent first, followed immediately by the raw encrypted data via `hSendFile`. The command result (`FROk` or error) is received only after both the command and data have been fully sent. This is a single HTTP/2 round trip, not a two-phase interaction.
+
+### 7. Empty corrId as nonce
+
+`sendXFTPCommand` uses `""` (empty bytestring) as the correlation ID for all commands. XFTP is strictly request-response within a single HTTP/2 stream, so correlation IDs are unnecessary. The empty value is passed to `C.cbNonce` to produce a constant nonce for command authentication (HMAC/signing), not encryption — XFTP authenticates commands but does not encrypt them within the TLS tunnel.

spec/modules/Simplex/FileTransfer/Client/Agent.md

@@ -0,0 +1,27 @@
+# Simplex.FileTransfer.Client.Agent
+
+> XFTP client: router connection management with TMVar-based sharing, async retry, and connection lifecycle.
+
+**Source**: [`FileTransfer/Client/Agent.hs`](../../../../../src/Simplex/FileTransfer/Client/Agent.hs)
+
+## Non-obvious behavior
+
+### 1. TMVar-based connection sharing
+
+`getXFTPServerClient` first checks the `TMap XFTPServer (TMVar (Either XFTPClientAgentError XFTPClient))`. If no entry exists, it atomically inserts an empty `TMVar` and initiates connection. Other threads requesting the same router block on `readTMVar` until the connection is established or fails. This prevents duplicate connections to the same router.
+
+### 2. Async retry on temporary errors
+
+When `newXFTPClient` encounters a temporary error, it launches an async retry loop that attempts reconnection with backoff. The `TMVar` remains in the map but is empty until the retry succeeds. Other threads waiting on `readTMVar` block until either the retry succeeds or a permanent error occurs.
+
+### 3. Permanent error cleanup
+
+On permanent error, `newXFTPClient` puts the `Left error` into the `TMVar` (unblocking waiters) AND deletes the entry from the `TMap`. This means the next caller will see no entry and create a fresh connection attempt, rather than reading a stale error. Waiters that already read the `Left` receive the error.
+
+### 4. Connection timeout
+
+`waitForXFTPClient` wraps `readTMVar` in a timeout. If the connection establishment takes too long (e.g., router unreachable and retry loop is slow), the caller gets a timeout error rather than blocking indefinitely. The underlying connection attempt continues in the background.
+
+### 5. closeXFTPServerClient removes from TMap
+
+Closing a router client deletes its entry from the TMap, so the next request will establish a fresh connection. This is called on connection errors during data packet operations to force reconnection.

spec/modules/Simplex/FileTransfer/Client/Main.md

@@ -0,0 +1,43 @@
+# Simplex.FileTransfer.Client.Main
+
+> XFTP CLI client: send, receive, delete files with parallel chunk operations and web URI encoding.
+
+**Source**: [`FileTransfer/Client/Main.hs`](../../../../../src/Simplex/FileTransfer/Client/Main.hs)
+
+## Non-obvious behavior
+
+### 1. Web URI encoding: base64url(deflate(YAML))
+
+`encodeWebURI` compresses the YAML-encoded file description with raw DEFLATE, then base64url-encodes the result. `decodeWebURI` reverses this. The compressed description goes in the URL fragment (after `#`), which is never sent to the router — the file description stays client-side.
+
+### 2. CLI receive accepts both file paths and URLs
+
+`getInputFileDescription` checks if the input starts with `http://` or `https://`. If so, it extracts the URL fragment, decodes it via `decodeWebURI`, and uses the resulting file description. Otherwise, it reads a YAML file from disk. This allows receiving files via web links without a browser.
+
+### 3. Redirect chain depth limited to 1
+
+`receive` tracks a `depth` parameter starting at 1. After following one redirect, `depth` becomes 0. A second redirect throws "Redirect chain too long". This prevents infinite redirect loops from malicious file descriptions.
+
+### 4. Parallel data packet uploads with router grouping
+
+`uploadFile` groups data packets by router via `groupAllOn`, then uses `pooledForConcurrentlyN 16` to process up to 16 router-groups concurrently. Within each group, data packets are uploaded sequentially (`mapM`). Errors from any upload are collected and the first one is thrown.
+
+### 5. Random router selection
+
+`getXFTPServer` selects a random router from the provided list for each chunk. With a single router, it's deterministic. With multiple routers, it uses `StdGen` in a TVar for thread-safe random selection via `stateTVar`.
+
+### 6. withReconnect nests retry with reconnection
+
+`withReconnect` wraps `withRetry` twice: the outer retry reconnects to the router, and the inner operation runs against the connection. On failure, the router connection is explicitly closed before retrying, forcing a fresh connection on the next attempt.
+
+### 7. withRetry rejects zero retries
+
+`withRetry' 0` returns an "internal: no retry attempts" error. `withRetry' 1` executes the action once without retry. This off-by-one convention means `retryCount = 3` (the default) gives 3 total attempts (1 initial + 2 retries).
+
+### 8. File description auto-deletion prompt
+
+After successful receive or delete, `removeFD` either auto-deletes the file description (if `--yes` flag) or prompts the user. This prevents accidental reuse of one-time file descriptions — each receive consumes the description by ACKing data packets on the router.
+
+### 9. Sender description uses first replica's router
+
+`createSndFileDescription` takes the router from the first replica of each chunk for the sender's `FileChunkReplica`. This reflects the current limitation that each data packet is uploaded to exactly one router — the sender description records that single router.

spec/modules/Simplex/FileTransfer/Crypto.md

@@ -0,0 +1,31 @@
+# Simplex.FileTransfer.Crypto
+
+> File encryption and decryption with streaming, padding, and auth tag verification.
+
+**Source**: [`FileTransfer/Crypto.hs`](../../../../src/Simplex/FileTransfer/Crypto.hs)
+
+## Non-obvious behavior
+
+### 1. Embedded file header in encrypted stream
+
+`encryptFile` prepends the `FileHeader` (containing filename and optional `fileExtra`) to the plaintext before encryption. A total data size field (8 bytes, `fileSizeLen`) is prepended before the header, encoding the combined size of header + file content. The decryptor uses this to distinguish real data from padding. The recipient must parse the header after decryption to recover the original filename — the header is not transmitted separately.
+
+### 2. Fixed-size padding hides actual file size
+
+The encrypted output is padded to `encSize` (the sum of data packet sizes). Since data packet sizes are fixed powers of 2 (64KB, 256KB, 1MB, 4MB), the encrypted file size reveals only which size bucket the file falls into, not the actual size. The encryption streams data with `LC.sbEncryptChunk` in a loop, pads the remaining space, then manually appends the auth tag via `LC.sbAuth`. This manual streaming approach (rather than using the all-at-once `LC.sbEncryptTailTag`) is necessary because encryption is interleaved with file I/O.
+
+### 3. Dual decrypt paths: single-chunk vs multi-chunk
+
+`decryptChunks` takes different paths based on chunk count:
+- **Single chunk**: reads the entire file into memory via `LB.readFile`, decrypts in-memory with `LC.sbDecryptTailTag`
+- **Multiple chunks**: opens the destination file for writing and streams through each chunk file with `LC.sbDecryptChunkLazy` (lazy bytestring variant), verifying the auth tag from the final chunk
+
+The single-chunk path avoids file handle management overhead for small files.
+
+### 4. Auth tag failure deletes output file
+
+In the multi-chunk streaming path, if `BA.constEq` detects an auth tag mismatch after decrypting all chunks, the partially-written output file is deleted before returning `FTCEInvalidAuthTag`. This prevents consumers from using a file whose integrity is unverified.
+
+### 5. Streaming encryption uses 64KB blocks
+
+`encryptFile` reads plaintext in 65536-byte blocks (`LC.sbEncryptChunk`), regardless of the XFTP data packet size. These are encryption blocks within a single continuous stream — not to be confused with XFTP data packets which are much larger (64KB–4MB).

spec/modules/Simplex/FileTransfer/Description.md

@@ -0,0 +1,43 @@
+# Simplex.FileTransfer.Description
+
+> File description: YAML encoding/decoding, validation, URI format, and replica optimization. A file description maps a file's chunks to data packets stored on XFTP routers — each chunk corresponds to one data packet, and each data packet may have multiple replicas on different routers.
+
+**Source**: [`FileTransfer/Description.hs`](../../../../src/Simplex/FileTransfer/Description.hs)
+
+## Non-obvious behavior
+
+### 1. ValidFileDescription non-exported constructor
+
+`ValidFileDescription` is a newtype with a non-exported data constructor (`ValidFD`), but the module exports a bidirectional pattern synonym `ValidFileDescription` that can be used as a constructor. Despite this, `validateFileDescription` provides the canonical validation path, checking:
+- Chunk numbers are sequential starting from 1
+- Total chunk sizes equal the declared file size
+
+Note: an empty chunk list with size 0 passes validation — there is no explicit "at least one chunk" check.
+
+### 2. First-replica-only digest and chunkSize
+
+When encoding chunks to YAML via `unfoldChunksToReplicas`, the `digest` and non-default `chunkSize` fields are only included on the first replica of each chunk. Subsequent replicas of the same chunk omit these fields. `foldReplicasToChunks` reconstructs them by carrying forward the digest/size from the first replica. If replicas have conflicting digests or sizes, validation fails.
+
+### 3. Default chunkSize elision
+
+The top-level `FileDescription` has a `chunkSize` field. Individual chunk replicas only serialize their `chunkSize` if it differs from this default. This saves space in the common case where most chunks are the same size (only the last chunk may be smaller).
+
+### 4. YAML encoding groups replicas by router
+
+`groupReplicasByServer` groups all data packet replicas by their router, producing `FileServerReplica` records. This is the serialization format — replicas are organized by router, not by chunk. The parser (`foldReplicasToChunks`) reverses this grouping back to per-chunk replica lists.
+
+### 5. FileDescriptionURI uses query-string encoding
+
+`FileDescriptionURI` serializes file descriptions into a compact query-string format (key=value pairs separated by `&`) with `QEscape` encoding for binary values. This is distinct from the YAML format used for file-based descriptions. The URI format is designed for embedding in links.
+
+### 6. QR code size limit
+
+`qrSizeLimit = 1002` bytes limits the maximum size of a file description URI that can be encoded as a QR code. Descriptions exceeding this limit cannot be shared via QR code and require alternative transport.
+
+### 7. Soft and hard file size limits
+
+Two limits exist: `maxFileSize = 1GB` (soft limit, checked by CLI client) and `maxFileSizeHard = 5GB` (hard limit, checked during agent-side encryption). The soft limit is a user-facing guard; the hard limit prevents resource exhaustion during encryption.
+
+### 8. Redirect file descriptions
+
+A `FileDescription` can contain a `redirect` field pointing to another file's metadata (`RedirectFileInfo` with size and digest). The outer description downloads an encrypted YAML data packet that, once decrypted, yields the actual `FileDescription` for the real file. This adds one level of indirection for privacy — the routers hosting the redirect data packet don't know the actual file's routers.

spec/modules/Simplex/FileTransfer/Protocol.md

@@ -0,0 +1,36 @@
+# Simplex.FileTransfer.Protocol
+
+> XFTP protocol types, commands, command results, and credential verification.
+
+**Source**: [`FileTransfer/Protocol.hs`](../../../../src/Simplex/FileTransfer/Protocol.hs)
+
+## Non-obvious behavior
+
+### 1. Asymmetric credential checks by command
+
+`checkCredentials` enforces different rules per command:
+- **FNEW**: requires `auth` (signature) but must NOT have a `fileId` — the sender key from the command body is used for verification
+- **PING**: must have NEITHER `auth` NOR `fileId` — actively rejects their presence
+- **All others** (FADD, FPUT, FDEL, FGET, FACK): require both `fileId` AND auth key
+
+This asymmetry means FNEW and PING bypass the standard entity-lookup path entirely — they are handled as separate `XFTPRequest` constructors (`XFTPReqNew`, `XFTPReqPing`).
+
+### 2. BLOCKED result downgraded to AUTH for old clients
+
+`encodeProtocol` checks the protocol version: if `v < blockedFilesXFTPVersion`, a `BLOCKED` result is encoded as `AUTH` instead. This prevents old clients that don't understand `BLOCKED` from receiving an unknown error type. The blocking information is silently lost for these clients.
+
+### 3. Single-transmission batch enforcement
+
+`xftpDecodeTServer` calls `xftpDecodeTransmission` which rejects batches containing more than one transmission. Despite using the batch framing format (length-prefixed), XFTP requires exactly one command per request. This differs from SMP where true batching is supported.
+
+### 4. xftpEncodeBatch1 always uses batch framing
+
+Even for single transmissions, `xftpEncodeBatch1` wraps the encoded transmission in batch format (1-byte count prefix + 2-byte length-prefixed transmission). There is no "non-batch" mode in XFTP — all protocol messages use the batch wire format regardless of the negotiated version.
+
+### 5. FileParty GADT partitions command space
+
+Commands are indexed by `FileParty` (`SFSender` / `SFRecipient`) at the type level via `FileCmd`. This ensures at compile time that sender commands (FNEW, FADD, FPUT, FDEL) and recipient commands (FGET, FACK, PING) cannot be confused. The router pattern-matches on `SFileParty` to determine which index (sender vs recipient) to look up in the file store.
+
+### 6. Empty corrId and implicit session ID
+
+`sendXFTPCommand` in the client uses an empty bytestring as `corrId`. This empty value is passed to `C.cbNonce` to produce a constant nonce for command authentication (HMAC/signing). With `implySessId = False` in the default XFTP transport setup, the session ID is not prepended to entity IDs during parsing. Session identity is provided by the TLS connection itself.

spec/modules/Simplex/FileTransfer/Server.md

@@ -0,0 +1,87 @@
+# Simplex.FileTransfer.Server
+
+> XFTP router: HTTP/2 request handling, handshake state machine, data packet operations, and statistics.
+
+**Source**: [`FileTransfer/Server.hs`](../../../../src/Simplex/FileTransfer/Server.hs)
+
+## Architecture
+
+The XFTP router runs several concurrent threads via `raceAny_`:
+
+| Thread | Purpose |
+|--------|---------|
+| `runServer` | HTTP/2 router accepting data packet transfer requests |
+| `expireFiles` | Periodic data packet expiration with throttling |
+| `logServerStats` | Periodic stats flush to CSV |
+| `savePrometheusMetrics` | Periodic Prometheus metrics dump |
+| `runCPServer` | Control port for admin commands |
+
+See [spec/routers.md](../../routers.md) for component and sequence diagrams.
+
+## Non-obvious behavior
+
+### 1. Three-state handshake with session caching
+
+The router maintains a `TMap SessionId Handshake` with three states:
+- **No entry**: first request — for non-SNI or `xftp-web-hello` requests, `processHello` generates DH key pair and sends router handshake; for SNI requests without `xftp-web-hello`, returns `SESSION` error
+- **`HandshakeSent pk`**: router hello sent, waiting for client handshake with version negotiation
+- **`HandshakeAccepted thParams`**: handshake complete, subsequent requests use cached params
+
+Web clients can re-send hello (`xftp-web-hello` header) even in `HandshakeSent` or `HandshakeAccepted` states — the router reuses the existing private key rather than generating a new one.
+
+### 2. Web identity proof via challenge-response
+
+When a web client sends a hello with a non-empty body, the router parses an `XFTPClientHello` containing a `webChallenge`. The router signs `challenge <> sessionId` with its long-term key and includes the signature in the handshake result. This proves router identity to web clients that cannot verify TLS certificates directly.
+
+### 3. skipCommitted drains request body on re-upload
+
+If `receiveServerFile` detects the data packet is already uploaded (`filePath` TVar is `Just`), it cannot simply ignore the request body — the HTTP/2 client would block waiting for the router to consume it. Instead, `skipCommitted` reads and discards the entire body in `fileBlockSize` increments, returning `FROk` when complete. This makes FPUT idempotent from the client's perspective.
+
+### 4. Atomic quota reservation with rollback
+
+`receiveServerFile` uses `stateTVar` to atomically check and reserve storage quota before receiving the data packet. If the upload fails (timeout, size mismatch, IO error), the reserved size is subtracted from `usedStorage` and the partial data packet is deleted on the router. This prevents failed uploads from permanently consuming quota.
+
+### 5. retryAdd generates new IDs on collision
+
+`createFile` and `addRecipient` use `retryAdd` which generates a random ID and makes up to 3 total attempts (initial + 2 retries) on `DUPLICATE_` errors. This handles the astronomically unlikely case of random ID collision without requiring uniqueness checking before insertion.
+
+### 6. Timing attack mitigation on entity lookup
+
+`verifyXFTPTransmission` calls `dummyVerifyCmd` (imported from SMP router) when a data packet entity is not found. This equalizes result timing to prevent attackers from distinguishing "entity doesn't exist" from "signature invalid" based on latency.
+
+### 7. BLOCKED vs EntityOff distinction
+
+When `verifyXFTPTransmission` reads `fileStatus`:
+- `EntityActive` → proceed with command
+- `EntityBlocked info` → return `BLOCKED` with blocking reason
+- `EntityOff` → return `AUTH` (same as entity-not-found)
+
+`EntityOff` is treated identically to missing entities for information-hiding purposes.
+
+### 8. blockServerFile deletes the stored data packet
+
+Despite the name suggesting it only marks a data packet as blocked, `blockServerFile` also deletes the stored data packet from disk via `deleteOrBlockServerFile_`. The `deleted = True` parameter to `blockFile` in the store adjusts `usedStorage`. A blocked data packet returns `BLOCKED` errors on access but has no data on disk.
+
+### 9. Stats restore overrides counts from live store
+
+`restoreServerStats` loads stats from the backup file but overrides `_filesCount` and `_filesSize` with values computed from the live file store (TMap size and `usedStorage` TVar). If the backup values differ, warnings are logged. This handles cases where data packets were expired or deleted while the router was down.
+
+### 10. Data packet expiration with configurable throttling
+
+`expireServerFiles` accepts an optional `itemDelay` (100ms when called from the periodic thread, `Nothing` at router startup). Between each data packet check, `threadDelay itemDelay` prevents expiration from monopolizing IO. At startup, data packets are expired without delay to clean up quickly.
+
+### 11. Stats log aligns to wall-clock midnight
+
+`logServerStats` computes an `initialDelay` to align the first stats flush to `logStatsStartTime` (default 0 = midnight UTC). If the target time already passed today, it adds 86400 seconds for the next day. Subsequent flushes use exact `logInterval` cadence.
+
+### 12. Stored data packet deleted before store cleanup
+
+`deleteOrBlockServerFile_` removes the stored data packet first, then runs the STM store action. If the process crashes between these two operations, the store will reference a data packet that no longer exists on disk. The next access would return `AUTH` (data packet not found on disk), and eventual expiration would clean the store entry.
+
+### 13. SNI-dependent CORS and web serving
+
+CORS headers require both `sniUsed = True` and `addCORSHeaders = True` in the transport config. Static web page serving is enabled when `sniUsed = True`. Non-SNI connections (direct TLS without hostname) skip both CORS and web serving. This separates the web-facing and protocol-facing behaviors of the same router port.
+
+### 14. Control port data packet operations use recipient index
+
+`CPDelete` and `CPBlock` commands look up data packets via `getFile fs SFRecipient fileId`, meaning the control port takes a recipient ID, not a sender ID. This is the ID visible to recipients and contained in data packet descriptions.

spec/modules/Simplex/FileTransfer/Server/Env.md

@@ -0,0 +1,24 @@
+# Simplex.FileTransfer.Server.Env
+
+> XFTP router environment: configuration, storage quota tracking, and request routing.
+
+**Source**: [`FileTransfer/Server/Env.hs`](../../../../../src/Simplex/FileTransfer/Server/Env.hs)
+
+## Non-obvious behavior
+
+### 1. Startup storage accounting with quota warning
+
+`newXFTPServerEnv` computes `usedStorage` by summing data packet sizes from the in-memory store at startup. If the computed usage exceeds the configured `fileSizeQuota`, a warning is logged but the router still starts. This allows the router to come up even if it's over quota (e.g., after a quota reduction), relying on expiration to reclaim space.
+
+### 2. XFTPRequest ADT separates new data packets from commands
+
+`XFTPRequest` has three constructors:
+- `XFTPReqNew`: data packet creation (carries `FileInfo`, recipient keys, optional basic auth)
+- `XFTPReqCmd`: command on an existing data packet (carries file ID, `FileRec`, and the command)
+- `XFTPReqPing`: health check
+
+This separation occurs after credential verification in `Server.hs`. `XFTPReqNew` bypasses entity lookup entirely since the data packet doesn't exist yet.
+
+### 3. fileTimeout for upload deadline
+
+`fileTimeout` in `XFTPServerConfig` sets the maximum time allowed for a single data packet upload (FPUT). The router wraps the receive operation in `timeout fileTimeout`. Default is 5 minutes (for 4MB chunks). This prevents slow or stalled uploads from holding router resources indefinitely.

spec/modules/Simplex/FileTransfer/Server/Main.md

@@ -0,0 +1,28 @@
+# Simplex.FileTransfer.Server.Main
+
+> XFTP router CLI: INI configuration parsing, TLS setup, and default constants.
+
+**Source**: [`FileTransfer/Server/Main.hs`](../../../../../src/Simplex/FileTransfer/Server/Main.hs)
+
+## Non-obvious behavior
+
+### 1. Key router constants
+
+| Constant | Value | Purpose |
+|----------|-------|---------|
+| `fileIdSize` | 16 bytes | Random data packet/recipient ID length |
+| `fileTimeout` | 5 minutes | Maximum upload duration per chunk |
+| `logStatsInterval` | 86400s (daily) | Stats CSV flush interval |
+| `logStatsStartTime` | 0 (midnight UTC) | First stats flush time-of-day |
+
+### 2. allowedChunkSizes defaults to all four sizes
+
+If not configured, `allowedChunkSizes` defaults to `[kb 64, kb 256, mb 1, mb 4]`. The INI file can restrict this to a subset, controlling which chunk sizes the router accepts.
+
+### 3. Storage quota from INI with unit parsing
+
+`fileSizeQuota` is parsed from the INI `[STORE_LOG]` section using `FileSize` parsing, which accepts byte values with optional unit suffixes (KB, MB, GB). Absence means unlimited quota (`Nothing`).
+
+### 4. Dual TLS credential support
+
+The router supports both primary TLS credentials (`caCertificateFile`/`certificateFile`/`privateKeyFile`) and optional HTTP-specific credentials (`httpCaCertificateFile`/etc.). When HTTP credentials are present, the router uses `defaultSupportedParamsHTTPS` which enables broader TLS compatibility for web clients.

spec/modules/Simplex/FileTransfer/Server/Stats.md

@@ -0,0 +1,19 @@
+# Simplex.FileTransfer.Server.Stats
+
+> XFTP router statistics: IORef-based counters with backward-compatible persistence.
+
+**Source**: [`FileTransfer/Server/Stats.hs`](../../../../../src/Simplex/FileTransfer/Server/Stats.hs)
+
+## Non-obvious behavior
+
+### 1. setFileServerStats is not thread safe
+
+`setFileServerStats` directly writes to IORefs without synchronization. It is explicitly intended for router startup only (restoring from backup file), before any concurrent threads are running.
+
+### 2. Backward-compatible parsing
+
+The `strP` parser uses `opt` for newer fields, defaulting missing fields to 0. This allows reading stats files from older router versions that don't include fields like `filesBlocked` or `fileDownloadAcks`.
+
+### 3. PeriodStats for download tracking
+
+`filesDownloaded` uses `PeriodStats` (not a simple `IORef Int`) to track unique data packet downloads over time periods (day/week/month). This enables the CSV stats log to report distinct data packets downloaded per period, not just total download count.

spec/modules/Simplex/FileTransfer/Server/Store.md

@@ -0,0 +1,39 @@
+# Simplex.FileTransfer.Server.Store
+
+> STM-based in-memory file store with dual indices, storage accounting, and privacy-preserving expiration.
+
+**Source**: [`FileTransfer/Server/Store.hs`](../../../../../src/Simplex/FileTransfer/Server/Store.hs)
+
+## Non-obvious behavior
+
+### 1. Dual-index lookup by sender and recipient
+
+The file store maintains two indices: `files :: TMap SenderId FileRec` (by sender ID) and `recipients :: TMap RecipientId (SenderId, RcvPublicAuthKey)` (by recipient ID, storing the sender ID and the recipient's public auth key). `getFile` dispatches on `SFileParty`: sender lookups use `files` directly, recipient lookups use `recipients` to find the `SenderId` then look up the `FileRec` in `files`. This means recipient operations require two TMap lookups.
+
+### 2. addRecipient checks both inner Set and global TMap
+
+`addRecipient` first checks the per-file `recipientIds` Set for duplicates, then inserts into the global `recipients` TMap. If either has a collision, it returns `DUPLICATE_`. The dual check is necessary because the Set tracks per-file membership while the TMap enforces global uniqueness of recipient IDs.
+
+### 3. Storage accounting on upload completion
+
+`setFilePath` adds the data packet size to `usedStorage` and records the file path in the `filePath` TVar. However, during normal FPUT handling, `Server.hs` does NOT call `setFilePath` — it directly writes `filePath` via `writeTVar`. The quota reservation in `Server.hs` (`stateTVar` on `usedStorage`) is the sole `usedStorage` increment during upload. `setFilePath` IS called during store log replay (`StoreLog.hs`), where it increments `usedStorage`; `newXFTPServerEnv` then overwrites with the correct value computed from the live store.
+
+### 4. deleteFile removes all recipients atomically
+
+`deleteFile` atomically removes the sender entry from `files`, all recipient entries from the global `recipients` TMap, and unconditionally subtracts the data packet size from `usedStorage` (regardless of whether the data packet was actually uploaded). The entire operation runs in a single STM transaction.
+
+### 5. RoundedSystemTime for privacy-preserving expiration
+
+Data packet timestamps use `RoundedFileTime` which is `RoundedSystemTime 3600` — system time rounded to 1-hour precision. This means data packets created within the same hour have identical timestamps. An observer with access to the store cannot determine exact data packet creation times, only the hour.
+
+### 6. expiredFilePath returns path only if expired
+
+`expiredFilePath` returns `STM (Maybe (Maybe FilePath))`. The outer `Maybe` is `Nothing` when the data packet doesn't exist or isn't expired; the inner `Maybe` is the file path (present only if the data packet was uploaded). The expiration check adds `fileTimePrecision` (one hour) to the creation timestamp before comparing, providing a grace period. The caller uses the inner path to decide whether to also delete the stored data packet.
+
+### 7. ackFile removes single recipient
+
+`ackFile` removes a specific recipient from both the global `recipients` TMap and the per-file `recipientIds` Set. Unlike `deleteFile` which removes the entire data packet, `ackFile` only removes one recipient's access. The data packet and other recipients remain intact.
+
+### 8. blockFile conditional storage adjustment
+
+`blockFile` takes a `deleted :: Bool` parameter. When `True` (data packet blocked with physical deletion), it subtracts the data packet size from `usedStorage`. When `False` (block without deletion), storage is unchanged. This allows blocking without physical deletion for audit purposes. Currently, both the router's `blockServerFile` and the store log replay path pass `True`.

spec/modules/Simplex/FileTransfer/Server/StoreLog.md

@@ -0,0 +1,33 @@
+# Simplex.FileTransfer.Server.StoreLog
+
+> Append-only store log for XFTP router data packet operations with error-resilient replay and compaction.
+
+**Source**: [`FileTransfer/Server/StoreLog.hs`](../../../../../src/Simplex/FileTransfer/Server/StoreLog.hs)
+
+## Non-obvious behavior
+
+### 1. Error-resilient replay
+
+`readFileStore` parses the store log line-by-line. Lines that fail to parse or fail to process (e.g., referencing a nonexistent sender ID) are logged as errors but do not halt replay. The store is reconstructed from whatever valid entries exist. This allows the router to recover from partial log corruption.
+
+### 2. Sender ID validation on recipient writes
+
+`writeFileStore` during compaction validates that each recipient's sender ID in the `recipients` TMap matches the `senderId` of the corresponding `FileRec`. This guards against in-memory state corruption (e.g., if a bug caused the `recipients` TMap and `FileRec.recipientIds` to get out of sync), not log corruption — the validation happens before writing the compacted log.
+
+### 3. Backward-compatible status parsing
+
+`AddFile` log entries include an `EntityStatus` field. The parser uses `<|> pure EntityActive` as a fallback, defaulting to `EntityActive` when the status field is missing. This allows reading store logs from older router versions that didn't record entity status.
+
+### 4. Compaction on restart
+
+`readFileStore` replays the full log to rebuild the in-memory store. The caller (in `Server/Env.hs`) then writes a fresh, compacted store log containing only the current state. This eliminates deleted entries and redundant operations, keeping the log size proportional to active state rather than total history.
+
+### 5. Log entry types track operation lifecycle
+
+Six log entry types capture the complete data packet lifecycle:
+- `AddFile`: data packet creation with sender ID, file info, timestamp, and status
+- `AddRecipients`: recipient registration (batched as `NonEmpty FileRecipient`) with sender ID association
+- `PutFile`: upload completion with file path
+- `DeleteFile`: data packet deletion by sender ID
+- `AckFile`: single recipient acknowledgment
+- `BlockFile`: data packet blocking with blocking info

spec/modules/Simplex/FileTransfer/Transport.md

@@ -0,0 +1,23 @@
+# Simplex.FileTransfer.Transport
+
+> XFTP protocol types, version negotiation, and encrypted file streaming with integrity verification.
+
+**Source**: [`FileTransfer/Transport.hs`](../../../../src/Simplex/FileTransfer/Transport.hs)
+
+## xftpClientHandshakeStub — XFTP doesn't use TLS handshake
+
+`xftpClientHandshakeStub` always fails with `throwE TEVersion`. The source comment states: "XFTP protocol does not use this handshake method." The XFTP handshake is performed at the HTTP/2 layer — `XFTPServerHandshake` and `XFTPClientHandshake` are sent as HTTP/2 request/response bodies (see `FileTransfer/Client.hs` and `FileTransfer/Server.hs`).
+
+## receiveSbFile — constant-time auth tag verification
+
+`receiveSbFile` validates the authentication tag using `BA.constEq` (constant-time byte comparison). The auth tag is collected from the stream after all file data — if the file data ends mid-chunk, the remaining bytes of that chunk are used first, and a follow-up read provides the rest of the tag if needed.
+
+## receiveFile_ — two-phase integrity verification
+
+File reception has two verification phases:
+1. **During receive**: either size checking (plaintext via `hReceiveFile`) or auth tag validation (encrypted via `receiveSbFile`)
+2. **After receive**: `LC.sha256Hash` of the entire received file is compared to `chunkDigest`
+
+## sendEncFile — auth tag appended after all chunks
+
+`sendEncFile` streams encrypted chunks via `LC.sbEncryptChunk`, then sends `LC.sbAuth sbState` (the authentication tag) as a final frame when the remaining size reaches zero.

spec/modules/Simplex/FileTransfer/Types.md

@@ -0,0 +1,27 @@
+# Simplex.FileTransfer.Types
+
+> Agent-side file transfer types: receive/send file records, status state machines, and chunk/replica structures. Chunks are the agent's view of file pieces; each chunk maps to a data packet on an XFTP router.
+
+**Source**: [`FileTransfer/Types.hs`](../../../../src/Simplex/FileTransfer/Types.hs)
+
+## Non-obvious behavior
+
+### 1. Receive file status state machine
+
+`RcvFileStatus` progresses: `RFSReceiving` → `RFSReceived` → `RFSDecrypting` → `RFSComplete`, with `RFSError` as a terminal state reachable from any non-complete state. The `RFSReceived` → `RFSDecrypting` transition is significant: all chunks are downloaded but decryption hasn't started. The local worker (server=Nothing) picks up files in `RFSReceived` status.
+
+### 2. Send file status state machine
+
+`SndFileStatus` progresses: `SFSNew` → `SFSEncrypting` → `SFSEncrypted` → `SFSUploading` → `SFSComplete`, with `SFSError` as terminal. The prepare worker handles `SFSNew` → `SFSEncrypted` (including retry from `SFSEncrypting`), while per-router upload workers handle `SFSUploading` → `SFSComplete`.
+
+### 3. Encrypted file path convention
+
+`sndFileEncPath` constructs the path as `prefixPath </> "xftp.encrypted"`. This is a convention shared between the agent (`Agent.hs`) and this module — both must agree on where the encrypted intermediate file lives relative to the prefix directory.
+
+### 4. FileHeader fileExtra for future extension
+
+`FileHeader` contains `fileName` and an optional `fileExtra :: Maybe Text` field. Currently unused (`Nothing` in all callers), it provides a forward-compatible extension point embedded in the encrypted file header without requiring protocol version changes.
+
+### 5. authTagSize = 16 bytes
+
+`authTagSize` is defined as `fromIntegral C.authTagSize` (16 bytes). This is the AES-GCM authentication tag appended to the encrypted file stream. It is included in the payload size calculation (`payloadSize = fileSize' + fileSizeLen + authTagSize`), which is then passed to `prepareChunkSizes` to determine data packet allocation.

spec/modules/Simplex/Messaging/Agent.md

@@ -0,0 +1,334 @@
+# Simplex.Messaging.Agent
+
+> Orchestration layer: duplex connection lifecycle, message processing dispatch, queue rotation, ratchet synchronization, and async command framework.
+
+**Source**: [`Agent.hs`](../../../../../src/Simplex/Messaging/Agent.hs)
+
+**See also**: [Agent/Client.md](./Agent/Client.md) — the infrastructure layer (AgentClient, worker framework, protocol client lifecycle, subscription state, operation suspension).
+
+**Protocol spec**: [`agent-protocol.md`](../../../../protocol/agent-protocol.md) — duplex connection procedure, agent message syntax.
+
+## Overview
+
+This module is the top-level SimpleX agent, consumed by simplex-chat and other client applications. It passes specific worker bodies, task queries, and handler logic into the frameworks defined in [Agent/Client.hs](./Agent/Client.md), and implements the orchestration policies: duplex handshake, queue rotation, ratchet synchronization, message integrity validation.
+
+### Agent startup — backgroundMode
+
+`getSMPAgentClient_` accepts a `backgroundMode` flag that fundamentally changes agent capabilities:
+- **Normal mode** (`backgroundMode = False`): starts four threads raced via `raceAny_` — `subscriber` (main event loop), `runNtfSupervisor` (notification management), `cleanupManager` (garbage collection), `logServersStats` (statistics). Also restores persisted router statistics. If any thread crashes, all are cancelled; statistics are saved in a `finally` block.
+- **Background mode** (`backgroundMode = True`): starts only the `subscriber` thread. No cleanup, no notifications, no stats persistence. Used when the agent needs minimal receive-only operation.
+
+Thread crashes are caught by the `run` wrapper: if the agent is still active (`acThread` is set), the exception is reported as `CRITICAL True` to `subQ`. If the agent is being disposed, crashes are silently ignored.
+
+### Service + entity session mode prohibition
+
+Service certificates and entity transport session mode (`TSMEntity`) are mutually exclusive. This is checked in four places: `getSMPAgentClient_`, `setNetworkConfig`, `createUser'`, `setUserService'`. If violated, throws `CMD PROHIBITED`. The constraint exists because service certificates associate multiple queues under one identity, which contradicts entity session mode's goal of preventing queue correlation.
+
+## Split-phase connection creation
+
+`prepareConnectionLink` and `createConnectionForLink` separate link preparation (key generation, link formatting — no network) from queue creation (single network call). This prevents the race where a link is published before the queue exists on the router.
+
+**Sender ID derivation.** The sender ID is deterministic: `SMP.EntityId $ B.take 24 $ C.sha3_384 corrId` where `corrId` is a random nonce. `createConnectionForLink` validates `actualSndId == sndId` — if the router returns a different sender ID, the connection is rejected. See source comment: "the remaining 24 bytes are reserved, possibly for notifier ID in the new notifications protocol."
+
+**PQ restriction.** `IKUsePQ` is prohibited for prepared links — throws `CMD PROHIBITED`. PQ keys are too large for the short link format.
+
+## Subscriber loop — processSMPTransmissions
+
+The subscriber thread reads batches from `msgQ` (filled by SMP protocol clients) and dispatches to `processSMPTransmissions`. Each batch is processed within `agentOperationBracket c AORcvNetwork waitUntilActive`, tying into the operation suspension cascade.
+
+**Batch UP notification accumulation.** Successful subscription confirmations (`processSubOk`) append to a shared `upConnIds` TVar across the batch. A single `UP` event is emitted after all transmissions are processed, not per-transmission. Similarly, `serviceRQs` accumulates service-associated receive queues for batch processing via `processRcvServiceAssocs`.
+
+**Double validation for subscription results.** `isPendingSub` checks two conditions atomically: the queue must be in the pending map AND the client session must still be active (`activeClientSession`). If either fails, the result is counted as ignored (statistics only). This handles the race where a subscription result arrives after reconnection.
+
+**SUB result piggybacking MSG.** When a SUB result arrives as `Right [email protected] {}`, the connection is marked UP (via `processSubOk`) AND the MSG is processed. The UP notification happens even if the MSG processing fails — the connection is up regardless.
+
+**subQ overflow to pendingMsgs.** `processSMP` writes events to `subQ` (bounded TBQueue) but when full, events go into a `pendingMsgs` TVar. After processing, pending messages are drained in reverse order (LIFO). This prevents the message processing thread from blocking on a full queue, which would stall the entire SMP client.
+
+**END/ENDS session validation.** Both check `activeClientSession` before removing subscriptions. If the session doesn't match (stale disconnect), the event is logged but ignored.
+
+## Message processing — processSMP
+
+`processSMP` dispatches on the SMP message type within a per-connection lock (`withConnLock`).
+
+### Four e2e key states
+
+The MSG handler discriminates on `(e2eDhSecret, e2ePubKey_)`:
+
+- `(Nothing, Just key)` — **Handshake**: computes DH, decrypts with per-queue E2E. Dispatches to `smpConfirmation` or `smpInvitation`.
+- `(Just dh, Nothing)` — **Established**: normal message flow. Dispatches to `AgentRatchetKey` or `AgentMsgEnvelope`.
+- `(Just dh, Just _)` — **Repeated confirmation**: only AgentConfirmation is accepted (ACK for previous one failed), everything else is rejected.
+- `(Nothing, Nothing)` — **Error**: no keys at all.
+
+### ACK semantics
+
+ACK is NOT automatic for `A_MSG` — the function returns `ACKPending` and the user must call `ackMessage`. ACK IS automatic for all control messages (HELLO, QADD, QKEY, QUSE, QTEST, EREADY, A_RCVD).
+
+`handleNotifyAck` wraps the MSG processing: if any error occurs, it sends `ERR` to the client but still ACKs the SMP message. This prevents a processing error from causing infinite re-delivery.
+
+### agentClientMsg — transactional message processing
+
+Performs ratchet decryption, message parsing, and integrity checking inside a single `withStore` transaction with `lockConnForUpdate`. This serializes all message processing for a given connection, preventing concurrent ratchet state modifications. Returns the pre-decryption ratchet state (`rcPrev`) alongside the message — needed by `ereadyMsg` to decide whether to send EREADY.
+
+### Additional queue status transitions on message receipt
+
+When receiving an `AgentMsgEnvelope` on a non-Active queue, the queue is set to Active. For primary queues during rotation (`dbReplaceQueueId` is set), the new queue is set as primary and the old queue is scheduled for deletion via `ICQDelete`. This is how the receiving side completes queue rotation — any message on the new queue triggers cleanup of the old one.
+
+### Duplicate message handling
+
+Three paths for `A_DUPLICATE` errors:
+
+1. **Stored and user-acked**: `getLastMsg` finds it with `userAck = True` → `ackDel`.
+2. **Stored, A_MSG, not user-acked**: re-notify the user with `MSG` event and return `ACKPending`. The user may not have seen the original.
+3. **Not stored or non-A_MSG**: `checkDuplicateHash` verifies the encrypted hash exists in the DB. If not, re-throws (real decryption failure, not duplicate).
+
+For crypto errors (`A_CRYPTO`): if the encrypted hash already exists, suppressed (duplicate). If not, `notifySync` classifies via `cryptoErrToSyncState` (RSAllowed or RSRequired) and updates the connection's ratchet sync state.
+
+### resetRatchetSync on successful decryption
+
+When a double-ratchet message is successfully decrypted and the connection's ratchet sync state is not `RSOk` or `RSStarted`, the state is reset to `RSOk` and `RSYNC RSOk` is notified. Successful message delivery is the recovery signal for ratchet desynchronization.
+
+### updateConnVersion — monotonic upgrade
+
+Every received `AgentMsgEnvelope` triggers `updateConnVersion`. If the message's agent version is higher than the current agreed version and compatible, the agreed version is upgraded. Versions only increase. `safeVersionRange` handles the case where the sender's version exceeds the receiver's maximum — creates a range from `minVersion` to the sender's version.
+
+## Duplex handshake
+
+See [agent-protocol.md](../../../../protocol/agent-protocol.md) for the protocol description. Implementation-specific details:
+
+### Initiating party (RcvConnection)
+
+Receives AgentConfirmation with `e2eEncryption = Just sndParams`. Initializes the receive ratchet from the sender's E2E parameters. **v7+ (ratchetOnConfSMPAgentVersion)**: creates the ratchet immediately on confirmation processing, not later on `allowConnection`. See source comment on `processConf` — this supports decrypting messages that may arrive before `allowConnection` is called. The ratchet creation, E2E secret setup, and confirmation storage all happen in one `withStore` transaction.
+
+### Accepting party (DuplexConnection)
+
+Receives AgentConfirmation with `e2eEncryption = Nothing` and `AgentConnInfo` (not `AgentConnInfoReply`). The ratchet was already initialized during `joinConnection`. If `senderKey` is present, enqueues `ICDuplexSecure` (queue needs securing with SKEY). If absent (sender already secured via LKEY), sends `CON` immediately and sets the queue Active.
+
+### HELLO exchange
+
+HELLO is processed in `helloMsg`. The key dispatch is on `sndStatus`:
+- `sndStatus == Active`: this side already sent HELLO, so receiving HELLO means both sides are connected → emit `CON`.
+- Otherwise: this side hasn't sent HELLO yet → enqueue HELLO reply.
+
+HELLO is not used in fast duplex connection (v9+ SMP with SKEY).
+
+### startJoinInvitation — retry-safe ratchet creation
+
+When retrying a join (existing `SndQueue`), `startJoinInvitation` tries to get the existing ratchet via `getSndRatchet` before creating a new one. If the ratchet exists, it reuses it. If not (error), it logs a non-blocking error via `nonBlockingWriteTBQueue` and creates a fresh ratchet. This prevents a retry from corrupting an already-established ratchet. The same pattern appears in `mkJoinInvitation` for contact URI joins.
+
+### PQ support negotiation
+
+PQ support is the AND of four conditions: the local client's PQ preference, the peer's agent version (>= `pqdrSMPAgentVersion`), the E2E encryption version (>= `pqRatchetE2EEncryptVersion`), and the connection's current PQ support. This negotiation happens at `joinConn` and `smpConfirmation` time via `versionPQSupport_` and `pqSupportAnd`.
+
+## Queue rotation
+
+Four agent messages implement queue rotation. See [agent-protocol.md](../../../../protocol/agent-protocol.md#rotating-messaging-queue) for the protocol. Implementation-specific details:
+
+**QADD** (processed by sender in `qAddMsg`): Creates a new `SndQueue` with DH key exchange. Deletes any previous pending replacement (`delSqs` partitioned by `dbReplaceQId`). Responds with `QKEY`. The replacement chain means consecutive rotation requests are handled correctly — only the latest survives.
+
+**QKEY** (processed by recipient in `qKeyMsg`): Validates queue is `New` or `Confirmed` and switch status is `RSSendingQADD`. Enqueues `ICQSecure` for async processing.
+
+**QUSE** (processed by sender in `qUseMsg`): Marks new queue `Secured`. Sends `QTEST` **only to the new queue**.
+
+**QTEST** (no handler in processSMP): Any message on the new queue triggers old queue deletion via `dbReplaceQueueId` logic. QTEST exists only to ensure at least one message traverses the new queue.
+
+**Sender-side completion in delivery handler.** When `AM_QTEST_` is successfully sent in `runSmpQueueMsgDelivery`, the old send queue is removed from the connection: pending messages are deleted, the queue record is removed, and the old queue's delivery worker is deleted from `smpDeliveryWorkers` (stopping its thread). This happens inside `withConnLockNotify` to prevent deadlock with the subscriber.
+
+**ICQDelete error tolerance.** In `runCommandProcessing`, if deleting the old receive queue fails with a permanent error (e.g., queue already gone on router), `finalizeSwitch` still runs — the local switch completes. Only temporary errors prevent completion.
+
+**Ratchet sync guard**: All four message handlers check `ratchetSyncSendProhibited` before proceeding.
+
+## Ratchet synchronization — newRatchetKey
+
+When an `AgentRatchetKey` message is received, `newRatchetKey` handles ratchet re-establishment.
+
+### Hash-ordering for initialization role
+
+Both parties generate key pairs and exchange them. The party whose `rkHash(k1, k2)` is **lower** (lexicographic comparison) initializes the **receiving** ratchet; the other initializes **sending** and sends EREADY. This breaks the symmetry when both parties simultaneously request ratchet sync.
+
+### State machine
+
+- `RSOk`, `RSAllowed`, `RSRequired` → **receiving client**: generate new keys, send `AgentRatchetKey` reply, then proceed with hash-ordering.
+- `RSStarted` → **initiating client**: use keys already stored (from `synchronizeRatchet'`), proceed with hash-ordering.
+- `RSAgreed` → **error**: sets state to `RSRequired`, throws `RATCHET_SYNC`. Handles the edge case where both parties initiate simultaneously and one has completed.
+
+### Deduplication
+
+`checkRatchetKeyHashExists` prevents processing the same ratchet key twice. The hash is stored atomically before processing begins.
+
+### EREADY
+
+Sent when the ratchet was initialized as receiving (`rcSnd` is `Nothing` in the pre-decryption ratchet state). Carries `lastExternalSndId` so the other party knows which messages were sent with the old ratchet.
+
+## Message integrity — checkMsgIntegrity
+
+Sequential external sender ID + previous message hash chain. Five outcomes: `MsgOk` (sequential + hashes match), `MsgBadId` (ID from the past), `MsgDuplicate` (same ID), `MsgSkipped` (gap in sequence), `MsgBadHash` (sequential but hashes differ).
+
+The integrity result is delivered to the client application via `MsgMeta`. The agent does not reject messages with integrity failures — it reports them and continues processing. The client decides the policy.
+
+## Async command processing — runCommandProcessing
+
+Uses the worker framework from [Agent/Client.hs](./Agent/Client.md#worker-framework). Keyed by `(connId, server)` — each connection/server combination gets its own command worker. Uses `AOSndNetwork` for operation suspension.
+
+### Internal commands
+
+- **ICAllowSecure**: User-initiated handshake completion (from `allowConnection`). On DuplexConnection (SKEY retry), if the error is temporary and the send queue's server differs from the command's server, the command is **moved** to the correct server queue via `updateCommandServer` + `getAsyncCmdWorker`. Returns `CCMoved` instead of `CCCompleted`.
+- **ICDuplexSecure**: Automatic handshake completion (from receiving AgentConnInfo with senderKey). Secures queue and sends HELLO.
+- **ICQSecure / ICQDelete**: Queue rotation — secure the new queue (KEY) and delete the old queue.
+- **ICAck / ICAckDel**: Send ACK to the router, optionally deleting the internal message record.
+- **ICDeleteConn**: No longer used, but may exist in old databases — cleaned up by deleting the command record.
+- **ICDeleteRcvQueue**: Queue cleanup during rotation.
+
+### Retry semantics
+
+`tryMoveableCommand` wraps execution with `withRetryInterval`: waits for `waitWhileSuspended` and `waitForUserNetwork`, then executes. Temporary/host errors trigger retry via `retrySndOp`. On success, the command is deleted. On permanent error, the error is notified and the command is deleted. `retrySndOp` separates `endAgentOperation`/`beginAgentOperation` into separate `atomically` blocks — see source comment: if `beginAgentOperation` blocks, `SUSPENDED` won't be sent.
+
+### withConnLockNotify — deadlock prevention
+
+Returns `Maybe ATransmission` and writes to `subQ` **after** releasing the lock. This prevents deadlock: if the lock holder writes to a full `subQ` while the subscriber thread needs the lock to process a message, both block indefinitely.
+
+## Message delivery — runSmpQueueMsgDelivery
+
+Per-queue delivery loop. Each `SndQueue` has its own worker keyed by queue address in `smpDeliveryWorkers`, paired with a `TMVar ()` retry lock (via `getAgentWorker'`).
+
+### Deferred encryption
+
+Message bodies are NOT encrypted at enqueue time. `enqueueMessageB` advances the ratchet header (`agentRatchetEncryptHeader`) and validates padding (`rcCheckCanPad`), but stores only the body reference (`sndMsgBodyId`) and encryption key (`encryptKey`, `paddedLen`). The actual message body encoding (`encodeAgentMsgStr`) and encryption (`rcEncryptMsg`) happen at delivery time. This allows the same body to be shared across multiple send queues via `sndMsgBodyId` — each delivery encrypts independently with its connection's ratchet.
+
+For confirmation and ratchet key messages (AM_CONN_INFO, AM_CONN_INFO_REPLY, AM_RATCHET_INFO), the body is pre-encrypted and stored in `msgBody` directly — no deferred encryption.
+
+### Per-message-type error handling
+
+**QUOTA**: Checks `internalTs` against `quotaExceededTimeout`. If the message is older than the timeout, expires it and all subsequent expired messages in the queue (via `getExpiredSndMessages` → bulk `MERRS` notification). If not expired, sends `MWARN` and retries with `RISlow`. For confirmation messages (AM_CONN_INFO/AM_CONN_INFO_REPLY), QUOTA is treated as `NOT_AVAILABLE`.
+
+**AUTH**: Per message type:
+- `AM_CONN_INFO` / `AM_CONN_INFO_REPLY` / `AM_RATCHET_INFO`: connection error `NOT_AVAILABLE`
+- `AM_HELLO_` with receive queue (initiating party): `NOT_AVAILABLE`. Without receive queue (joining party): `NOT_ACCEPTED`.
+- `AM_A_MSG_` / `AM_A_RCVD_` / `AM_QCONT_` / `AM_EREADY_`: delete message and notify `MERR`.
+- Queue rotation messages (`AM_QADD_` through `AM_QTEST_`): queue error with descriptive string.
+
+**Timeout/network errors**: message-type-aware timeout — `AM_HELLO_` uses `helloTimeout`, all others use `messageTimeout`. If expired, uses `notifyDelMsgs` which expires the current message AND fetches all expired messages for the queue in bulk. If `serverHostError`, sends `MWARN` before retrying. Non-host temporary errors retry silently.
+
+### Delivery success handling
+
+On successful send, per message type:
+- `AM_CONN_INFO` with `senderCanSecure` (fast handshake): sends `CON` + sets status `Active`.
+- `AM_CONN_INFO` without `senderCanSecure`: sets status `Confirmed` only.
+- `AM_CONN_INFO_REPLY`: sets status `Confirmed`.
+- `AM_HELLO_`: sets status `Active`. If receive queue exists AND its status is `Active`, sends `CON` (accepting party in v2).
+- `AM_A_MSG_`: sends `SENT msgId proxySrv_` to notify the client.
+- `AM_QKEY_`: re-reads connection and sends `SWITCH QDSnd SPConfirmed`.
+- `AM_QTEST_`: see "Sender-side completion" under Queue rotation above.
+- All other types: no notification.
+
+After success, the delivery record is deleted. For `AM_A_MSG_`, `keepForReceipt = True` — the record is kept until a receipt is received.
+
+### withRetryLock2 — external retry signaling
+
+The delivery loop uses `withRetryLock2` which combines the standard retry interval with `qLock` (the `TMVar ()` paired with the worker). When `A_QCONT` is received, the handler puts `()` into the retry lock, causing the retry to fire immediately instead of waiting for the backoff interval. See `continueSending` in `processSMP`.
+
+### submitPendingMsg — operation counting
+
+`submitPendingMsg` increments `opsInProgress` on `msgDeliveryOp` BEFORE spawning the delivery worker. This means the operation is counted even before the worker starts, ensuring the suspension cascade waits for all enqueued deliveries.
+
+## Batch message sending — sendMessagesB_
+
+### MsgReq grouping contract
+
+Messages to the same connection must be contiguous in the traversable, with only the first having a non-empty `connId`. Subsequent messages for the same connection must have empty `connId`. This is validated by `addConnId` which rejects duplicate `connId` values and empty first `connId`. The `getConn_` function uses a `TVar prev` to cache the last connection lookup, avoiding redundant database reads.
+
+### Connection locking
+
+`withConnLocks` takes locks for ALL connections in the batch before processing. This prevents concurrent sends to the same connection from interleaving ratchet state updates.
+
+### PQ support monotonic upgrade
+
+When `pqEnc == PQEncOn` but the connection has `pqSupport == PQSupportOff`, PQ support is upgraded via `setConnPQSupport`. PQ support can only be enabled, never disabled. The upgrade IDs are accumulated via `mapAccumL` and applied in a single batch database write.
+
+### VRValue/VRRef — database body deduplication
+
+VRValue/VRRef deduplication operates at the **database body storage** level, not encryption. `enqueueMessageB` tracks an `IntMap (Maybe Int64, AMessage)` mapping integer indices to database body IDs (`sndMsgBodyId`):
+
+- `VRValue (Just i) body`: stores the body in `snd_message_bodies`, records the `sndMsgBodyId`, and associates it with index `i` for future reference.
+- `VRRef i`: looks up index `i` to get the previously stored `sndMsgBodyId`, and creates a new `snd_messages` record linked to the same body.
+
+Encryption is NOT deduplicated — each connection's ratchet header is independently advanced at enqueue time, and each delivery encrypts the body independently. The optimization is purely about avoiding redundant database storage of identical message bodies (common for group messages).
+
+### Error propagation constraint
+
+When a connection type is wrong (e.g., SndConnection, NewConnection), the error is returned per-message but the batch continues. See source comment: "we can't fail here, as it may prevent delivery of subsequent messages that reference the body of the failed message." If a VRValue message fails, subsequent VRRef messages that reference it would break.
+
+## Subscription management
+
+### subscribeConnections_
+
+Partitions connections by type. SndConnection with `Confirmed` status returns success (it's not subscribed, just waiting). SndConnection with `Active` status returns `CONN SIMPLEX` (can't subscribe a send-only connection). After subscribing queues, resumes delivery workers for connections with pending deliveries (via `getConnectionsForDelivery`).
+
+**Multi-queue result combining.** For connections with multiple receive queues, results are combined using a priority system: Active+Success (1) > Active+Error (2) > non-Active+Success (3) > non-Active+Error (4). The highest-priority (lowest number) result is used. This ensures that if at least one Active queue subscribes successfully, the connection reports success.
+
+### subscribeAllConnections'
+
+**Active user priority.** If `activeUserId_` is provided, that user's subscriptions are processed first (`sortOn`).
+
+**Service subscription with fallback.** Service subscriptions are attempted first. If a service subscription fails with `SSErrorServiceId` or zero subscribed queues, the queues are unassociated from the service and subscribed individually. If the error is a client-level error (not a service-specific error), the same fallback applies.
+
+**Pending throttle.** `maxPending` limits concurrent pending subscriptions. The counter is incremented inside the database transaction (before leaving `withStore'`) and decremented in a `finally` block. When the count exceeds the limit, `subscribeUserServer` blocks in STM via `retry`.
+
+### resubscribeConnections'
+
+Filters out connections that already have active subscriptions (via `hasActiveSubscription`). For store errors, returns `True` for `isActiveConn` — this causes the error to be processed by `subscribeConnections_` which will report it.
+
+## Notification token lifecycle
+
+`registerNtfToken'` is a complex state machine. Key non-obvious behavior: on `NTF AUTH` error during token operations, the token is removed and re-registered from scratch (see `withToken` catch of `NTF AUTH`). Device token changes trigger `replaceToken`, which attempts an in-place replacement; if that fails with a permanent error, the token is removed and recreated.
+
+## Cleanup manager
+
+Runs periodically with a `cleanupStepInterval` delay BETWEEN each cleanup operation (not just between cycles). This prevents cleanup from monopolizing database access.
+
+Additional cleanup not previously mentioned:
+- **Expired receive message hashes**: `deleteRcvMsgHashesExpired`
+- **Expired send messages**: `deleteSndMsgsExpired`
+- **Expired ratchet key hashes**: `deleteRatchetKeyHashesExpired`
+- **Expired notification tokens**: `deleteExpiredNtfTokensToDelete`
+- **Expired send chunk replicas**: `deleteDeletedSndChunkReplicasExpired`
+
+## Agent suspension
+
+`suspendAgent` has two modes:
+- **Immediate** (`maxDelay = 0`): sets `ASSuspended` and suspends all operations immediately.
+- **Gradual** (`maxDelay > 0`): sets `ASSuspending` and triggers the cascade (NtfNetwork independent; RcvNetwork → MsgDelivery → SndNetwork → Database). A timeout thread fires after `maxDelay` and forces suspension of sending and database if still suspending.
+
+`foregroundAgent` resumes in reverse order: database → sending → delivery → receiving → notifications.
+
+## connectReplyQueues — background duplex upgrade
+
+Used during async command processing to complete the duplex handshake. Handles two cases:
+- **Fresh connection** (`sq_ = Nothing`): upgrades `RcvConnection` to `DuplexConnection`.
+- **SKEY retry** (`sq_ = Just sq`): connection is already duplex. See source comment: "in case of SKEY retry the connection is already duplex."
+
+## secureConfirmQueue vs secureConfirmQueueAsync
+
+- **secureConfirmQueue** (synchronous): secures queue and sends confirmation directly via network. Used in `joinConnection`.
+- **secureConfirmQueueAsync** (asynchronous): secures queue, stores confirmation, submits to delivery worker. Used in `allowConnection` (via `ICAllowSecure`).
+
+Both call `agentSecureSndQueue`, which returns `initiatorRatchetOnConf` — whether the initiator's ratchet should be created on confirmation (v7+ behavior). When the queue was already secured (retry), returns the same flag without re-securing.
+
+## smpConfirmation — version compatibility
+
+The confirmation handler accepts messages where the agent version or client version is either within the configured range OR at-or-below the already-agreed version. See source comment. This means a downgraded client can still complete in-progress handshakes.
+
+## smpInvitation — contact address handling
+
+Invitation messages received on a contact address are passed through even if version-incompatible. See source comment. The client application sees `REQ` with `PQSupportOff` when incompatible.
+
+## ackMessage' — receipt sending
+
+After ACKing a message, if the user provides receipt info (`rcptInfo_`), a receipt message (`A_RCVD`) is enqueued. Receipts are only allowed for `AM_A_MSG_` type. If the user ACKs without receipt info and the message already has a receipt with `MROk` status, the corresponding sent message is deleted from the database — it's confirmed delivered.
+
+## acceptContactAsync' — rollback on failure
+
+See source comment. Unlike the synchronous `acceptContact'` which takes a lock first, `acceptContactAsync'` marks the invitation as accepted before joining. On failure, `unacceptInvitation` rolls back. The comment notes this could be improved with an invitation lock map.
+
+## prepareConnectionToJoin — race prevention
+
+See source comment. Creates a connection record without queues, returning a `ConnId`. The caller saves this ID before the peer can send a confirmation. Without this, the sequence "joinConnection → peer sends confirmation → caller saves ConnId" could result in the confirmation arriving before the caller has the ID.

spec/modules/Simplex/Messaging/Agent/Client.md

@@ -0,0 +1,300 @@
+# Simplex.Messaging.Agent.Client
+
+> Agent infrastructure layer: protocol client lifecycle, worker framework, subscription management, operation suspension, and concurrency primitives.
+
+**Source**: [`Agent/Client.hs`](../../../../../../src/Simplex/Messaging/Agent/Client.hs)
+
+**See also**: [Agent.hs](./Agent.md) — the orchestration layer that consumes these primitives.
+
+## Overview
+
+This module defines `AgentClient`, the central state container for the SimpleX agent, and all reusable infrastructure that Agent.hs and other consumers (NtfSubSupervisor.hs, FileTransfer/Agent.hs, simplex-chat) build upon. It covers:
+
+- **Protocol client lifecycle**: lazy singleton connections to SMP/NTF/XFTP routers via `SessionVar` pattern, with disconnect callbacks and reconnection workers
+- **Worker framework**: `getAgentWorker` (lifecycle, restart rate limiting, crash recovery) + `withWork`/`withWork_`/`withWorkItems` (task retrieval with doWork flag atomics)
+- **Subscription state**: active/pending/removed queues, session-aware cleanup on disconnect, batch subscription RPCs with post-hoc session validation
+- **Operation suspension**: five `AgentOpState` TVars with cascade ordering for graceful shutdown
+- **Concurrency primitives**: per-connection locks, transport session batching, proxy routing
+
+The module is consumed by Agent.hs (which passes specific worker bodies, task queries, and handler logic into these frameworks) and by external consumers that reuse the worker and protocol client infrastructure.
+
+## AgentClient — central state container
+
+`AgentClient` has ~43 fields, almost all TVars or TMaps. Key architectural groupings:
+
+- **Event queues**: `subQ` (events to client application), `msgQ` (messages from SMP routers)
+- **Protocol client pools**: `smpClients`, `ntfClients`, `xftpClients` — all are TMaps of `TransportSession` → `SessionVar`, implementing lazy singletons via `getSessVar`
+- **Subscription tracking**: `currentSubs` (TSessionSubs, active+pending per transport session), `removedSubs` (failed subscriptions with errors), `subscrConns` (set of connection IDs currently subscribed)
+- **Worker pools**: `smpDeliveryWorkers`, `asyncCmdWorkers` — TMaps keyed by work address/connection. `smpSubWorkers` — TMaps keyed by transport session for resubscription.
+- **Operation states**: `ntfNetworkOp`, `rcvNetworkOp`, `msgDeliveryOp`, `sndNetworkOp`, `databaseOp`
+- **Locking**: `connLocks`, `invLocks`, `deleteLock`, `getMsgLocks`, `clientNoticesLock`
+- **Service state**: `useClientServices` (per-user boolean controlling whether service certificates are used)
+- **Proxy routing**: `smpProxiedRelays` (maps destination transport session → proxy router used)
+- **Network state**: `userNetworkInfo`, `userNetworkUpdated`, `useNetworkConfig` (slow/fast pair)
+
+All TVars are initialized in `newAgentClient`. The `active` TVar is the global kill switch — `closeAgentClient` sets it to `False`, and all protocol client getters check it first.
+
+## Protocol client lifecycle — SessionVar singleton pattern
+
+Protocol client connections (SMP, NTF, XFTP) use a lazy singleton pattern implemented by [Session.hs](../../../Session.md):
+
+1. **`getSessVar`** atomically checks the TMap. Returns `Left newVar` if absent (caller must connect), `Right existingVar` if present (caller waits for the TMVar).
+2. **`newProtocolClient`** wraps the connection attempt. On success, fills the `sessionVar` TMVar with `Right client` and writes a `CONNECT` event to `subQ`. On failure, fills with `Left (error, maybeRetryTime)` and re-throws.
+3. **`waitForProtocolClient`** reads the TMVar with a timeout. If the stored error has an expiry time that has passed, it removes the SessionVar and retries from scratch — this is the `persistErrorInterval` retry mechanism.
+
+### Error caching with persistErrorInterval
+
+When `newProtocolClient` fails and `persistErrorInterval > 0`, the error is cached with an expiry timestamp (`Just ts`). Future connection attempts during the interval immediately receive the cached error from `waitForProtocolClient` without attempting a connection. When `persistErrorInterval == 0`, the SessionVar is removed immediately on failure, so the next attempt starts a fresh connection. This prevents connection storms to unreachable routers.
+
+### SessionVar compare-and-swap
+
+`removeSessVar` (Session.hs) only removes a SessionVar from the map if its `sessionVarId` matches the current entry. The `sessionVarId` is a monotonically increasing counter from `workerSeq`. This prevents a stale disconnection callback from removing a *new* client that was created after the old one disconnected. Without this, the sequence "client A disconnects → client B connects → client A's callback runs" would incorrectly remove client B.
+
+### SMP connection — service credentials and session setup
+
+`smpConnectClient` connects an SMP client, with two important post-connection steps:
+
+1. **Session ID registration**: `SS.setSessionId` records the TLS session ID in `currentSubs`, linking the transport session to the actual TLS connection for later session validation.
+
+2. **Service credential synchronization** (`updateClientService`): After connecting, compares client-side and router-side service state. Four cases:
+   - Both have service and IDs match → update DB (no-op if same)
+   - Both have service but IDs differ → update DB and remove old queue-service associations
+   - Client has service, router doesn't → delete client service (handles router version downgrade)
+   - Router has service, client doesn't → log error (should not happen in normal flow)
+
+On connection failure, `smpConnectClient` triggers `resubscribeSMPSession` before re-throwing the error. This ensures pending subscriptions get retry logic even when the initial connection attempt fails.
+
+### SMP disconnect callback
+
+`smpClientDisconnected` is the most complex disconnect handler (NTF/XFTP have simpler versions that remove the SessionVar and write a `DISCONNECT` event):
+
+1. `removeSessVar` atomically removes the client if still current
+2. If `active`, moves active subscriptions to pending (only those matching the disconnecting client's `sessionId` — see next section)
+3. Removes proxied relay sessions that this client created
+4. Fires `DISCONNECT`, `DOWN`, and `SERVICE_DOWN` events for affected connections
+5. Releases GET locks for affected queues
+6. Triggers resubscription (see below)
+
+**Resubscription mode switching**: The disconnect handler chooses between two resubscription paths based on whether the session mode matches the entity presence: `(mode == TSMEntity) == isJust cId`. When they match, it calls `resubscribeSMPSession` which handles both service and queue resubscription in a single worker. When they don't match (e.g., entity-mode session disconnects but there's also a shared session), it separately resubscribes the service and queues, because they belong to different transport sessions.
+
+### Session-aware subscription cleanup
+
+`removeClientAndSubs` (inside `smpClientDisconnected`) uses `SS.setSubsPending` with the disconnecting client's `sessionId`. Only subscriptions whose session ID matches the disconnecting client are moved to pending. If a new client already connected and made its own subscriptions active, those are *not* disturbed. This prevents the race: "old client disconnects → new client subscribes → old client's cleanup incorrectly demotes new client's subscriptions."
+
+## ProtocolServerClient typeclass
+
+Unifies SMP/NTF/XFTP client management with associated types:
+- `Client msg` — the connected client type (SMP wraps in `SMPConnectedClient` with proxied relay map; NTF and XFTP use the raw protocol client)
+- `ProtoClient msg` — the underlying protocol client for logging/closing
+
+SMP is special: `SMPConnectedClient` bundles the protocol client with `proxiedRelays :: TMap SMPServer ProxiedRelayVar`, a per-connection map of relay sessions for proxy routing.
+
+XFTP is special in a different way: its `getProtocolServerClient` ignores the `NetworkRequestMode` parameter and always uses `NRMBackground` for `waitForProtocolClient`. This means XFTP connections always use background timing regardless of the caller's request mode.
+
+## Worker framework
+
+Defined here, consumed by Agent.hs, NtfSubSupervisor.hs, FileTransfer/Agent.hs, and simplex-chat. Two separable parts:
+
+### getAgentWorker — lifecycle management
+
+Creates or reuses a worker for a given key. Workers are stored in a TMap keyed by their work address.
+
+- **Create-or-reuse**: atomically checks the map. If absent, creates a new `Worker` (with `doWork` TMVar pre-filled with `()`). If present and `hasWork=True`, signals the existing worker.
+- **Fork**: `runWorkerAsync` takes the `action` TMVar. If `Nothing` (worker idle), it starts work. If `Just weakThreadId` (worker running), it puts the value back and returns. This bracket ensures at-most-one concurrent execution.
+- **Restart rate limiting**: on worker exit (success or error), `restartOrDelete` checks `restartCount` against `maxWorkerRestartsPerMin`. If under the limit, resets `action` to `Nothing` (idle), signals `hasWorkToDo`, and reports `INTERNAL` error. If over the limit, deletes the worker from the map and sends a `CRITICAL True` error. The restart only happens if the worker's `workerId` still matches the map entry — a stale restart from a replaced worker silently no-ops.
+
+`getAgentWorker'` is the generic version with custom worker wrapper — used by `smpDeliveryWorkers` which pairs each Worker with a `TMVar ()` retry lock.
+
+### withWork / withWork_ / withWorkItems — task retrieval
+
+Takes `getWork` (fetch next task) and `action` (process it) as separate parameters. The consumer's worker body loops: `waitForWork doWork` → `withWork doWork getTask handleTask`.
+
+**Critical: doWork flag race prevention.** `noWorkToDo` (clearing the flag) happens BEFORE `getWork` (querying for tasks), not after. This prevents the race where: (1) worker queries, finds nothing, (2) another thread adds work and sets the flag, (3) worker clears the flag — losing the signal. By clearing first, any concurrent signal after the query will be preserved.
+
+**Error classification**: `withWork_` distinguishes work-item errors from store errors:
+- **Work item error** (`isWorkItemError`): the worker stops and sends `CRITICAL False`. The next iteration would likely produce the same error, so stopping prevents infinite loops.
+- **Store error**: the flag is re-set and an `INTERNAL` error is reported. The assumption is that store errors are transient (e.g., DB busy) and retrying may succeed.
+
+`withWorkItems` handles batched work — a list of items where some may have individual errors. If all items are work-item errors, the worker stops. If only some are, the worker continues with the successful items and reports errors via `ERRS` event.
+
+### runWorkerAsync — at-most-one execution
+
+Uses a bracket on the `action` TMVar:
+- `takeTMVar action` — blocks if another thread is starting the worker (TMVar empty during start)
+- If the taken value is `Nothing` — worker is idle, start it. Store `Just weakThreadId` in the TMVar via `forkIO`.
+- If `Just _` — worker is already running, put it back and return.
+
+The `Weak ThreadId` in `action` is a weak reference — it doesn't prevent the worker thread from being garbage collected. It is used by `cancelWorker`, which calls `deRefWeak` to get the thread ID and kills it; if the thread was already GC'd, the kill is a no-op. The primary lifecycle management is through the `restartOrDelete` chain in `getAgentWorker'`, not the weak reference.
+
+### throwWhenNoDelivery — delivery worker self-termination
+
+Delivery workers call `throwWhenNoDelivery` to check if their entry still exists in the `smpDeliveryWorkers` map. If the worker was removed (delivery complete), it throws `ThreadKilled` to terminate the worker thread. This is distinct from `throwWhenInactive` (which checks global `active` state) — it allows individual workers to be stopped without shutting down the entire agent.
+
+## Operation suspension cascade
+
+Five `AgentOpState` TVars track whether each operation category is suspended and how many operations are in-flight:
+
+```
+AONtfNetwork  (independent)
+AORcvNetwork  → AOMsgDelivery → AOSndNetwork → AODatabase
+```
+
+The cascade means:
+- `endAgentOperation AORcvNetwork` suspends `AOMsgDelivery`, which cascades to `AOSndNetwork` → `AODatabase`
+- `endAgentOperation AOMsgDelivery` suspends `AOSndNetwork` → `AODatabase`
+- `endAgentOperation AOSndNetwork` suspends `AODatabase`
+- Each leaf in the cascade calls `notifySuspended` (writes `SUSPENDED` to `subQ`, sets `agentState` to `ASSuspended`)
+
+**`beginAgentOperation`** retries (blocks in STM) if the operation is suspended. This provides backpressure: new operations wait until the operation is resumed.
+
+**`agentOperationBracket`** wraps an operation with begin/end. It takes a `check` function that runs before `beginAgentOperation` — typically `throwWhenInactive`, which throws `ThreadKilled` if the agent is inactive. All database access goes through `withStore` which brackets with `AODatabase`. This ensures graceful shutdown propagates: suspending `AORcvNetwork` eventually suspends all downstream operations, and `notifySuspended` only fires when all in-flight operations have completed.
+
+**`waitWhileSuspended`** vs **`waitUntilForeground`**: `waitWhileSuspended` proceeds during `ASSuspending` (allowing in-flight operations to complete), while `waitUntilForeground` blocks during both `ASSuspending` and `ASSuspended`.
+
+**`waitForUserNetwork`**: bounded wait for network — if the network doesn't come online within `userNetworkInterval`, proceeds anyway. Uses `registerDelay` for the timeout.
+
+## Subscription management
+
+### subscribeQueues — batch-by-transport-session
+
+`subscribeQueues` is the main entry point for subscribing to receive queues:
+
+1. `checkQueues` filters out queues with active GET locks (prevents concurrent GET + SUB on the same queue)
+2. `batchQueues` groups queues by transport session
+3. `addPendingSubs` marks all queues as pending before the RPC
+4. `mapConcurrently` subscribes each session batch in parallel
+
+### subscribeSessQueues_ — post-hoc session validation and atomicity
+
+After the subscription RPC completes, `subscribeSessQueues_` validates `activeClientSession` — checking that the SessionVar still holds the same client that was used for the RPC. If the client was replaced during the RPC (reconnection happened), the results are discarded (errors converted to temporary `BROKER NETWORK` to ensure retry) and resubscription is triggered.
+
+The post-RPC processing runs under `uninterruptibleMask_` for atomicity. The sequence is:
+1. **Atomically**: `processSubResults` partitions results and updates subscription state; if there are client notices, takes `clientNoticesLock` TMVar
+2. **IO**: `processRcvServiceAssocs` updates service associations in the DB
+3. **IO**: `processClientNotices` updates notice state, always releases `clientNoticesLock` in `finally`
+
+The `clientNoticesLock` TMVar serializes notice processing across concurrent subscription batches.
+
+**UP events for newly-active connections only**: After processing, UP events are sent only for connections that were NOT already active before this batch — existing active subscriptions (from `SS.getActiveSubs`) are excluded to prevent duplicate notifications.
+
+**Client close on all-temporary-error**: When ALL subscription results are temporary errors, no connections were already active, and the session is still current, the SMP client session is closed. This forces a fresh connection on the next attempt rather than reusing a potentially broken one.
+
+### processSubResults — partitioning
+
+Subscription results are partitioned into five categories:
+1. **Failed with client notice** — error has an associated router-side notice (e.g., queue status change). Queue is treated as failed (removed from pending, added to `removedSubs`) AND the notice is recorded for processing.
+2. **Failed permanently** — non-temporary error without notice, queue is removed from pending and added to `removedSubs`
+3. **Failed temporarily** — error is transient, queue stays in pending unchanged for retry on reconnect
+4. **Subscribed** — moved from pending to active. Further split into: queues whose service ID matches the session service (added as service-associated) and others. If the queue had a tracked `clientNoticeId`, it is cleared (notice resolved by successful subscription).
+5. **Ignored** — queue was not in the pending map (already activated by a concurrent path), counted for statistics only
+
+### Resubscription worker
+
+`resubscribeSMPSession` spawns a worker per transport session that retries pending subscriptions with exponential backoff (`withRetryForeground`). The worker:
+
+1. Reads pending subs and pending service sub
+2. Waits for foreground and network
+3. Resubscribes service and queues
+4. Loops until no pending subs remain
+
+**Spawn guard**: Before creating a new worker, `resubscribeSMPSession` checks `SS.hasPendingSubs`. If there are no pending subs, it returns without spawning. This prevents creating idle workers.
+
+**Cleanup blocks on TMVar fill** — the `cleanup` STM action retries (`whenM (isEmptyTMVar $ sessionVar v) retry`) until the async handle is inserted. This prevents the race where cleanup runs before the worker async is stored, which would leave a terminated worker in the map.
+
+## Proxy routing — sendOrProxySMPCommand
+
+Implements SMP proxy/direct routing with fallback:
+
+1. `shouldUseProxy` checks `smpProxyMode` (Always/Unknown/Unprotected/Never) and whether the destination router is "known" (in the user's router list)
+2. If proxying: `getSMPProxyClient` creates or reuses a proxy connection, then `connectSMPProxiedRelay` establishes the relay session. On `NO_SESSION` error, re-creates the relay session through the same proxy.
+3. If proxying fails with a host error and `smpProxyFallback` allows it: falls back to direct connection
+4. `deleteRelaySession` carefully validates that the current relay session matches the one that failed before removing it (prevents removing a concurrently-created replacement session)
+
+**NO_SESSION retry limit**: On `NO_SESSION`, `sendViaProxy` is called recursively with `Just proxySrv` to reuse the same proxy router. If the recursive call also gets `NO_SESSION`, it throws `proxyError` instead of recursing again — `proxySrv_` is `Just`, so the `Nothing` branch (which recurses) is not taken. This limits retry to exactly one attempt.
+
+**Proxy selection caching** (`smpProxiedRelays`): When `getSMPProxyClient` selects a proxy for a destination, it atomically inserts the proxy→destination mapping into `smpProxiedRelays`. If a mapping already exists (another thread selected a proxy for the same destination), the existing mapping is used. On relay creation failure with non-host errors, both the relay session and proxy mapping are removed. On host errors, they are preserved to allow fallback logic.
+
+## Service credentials lifecycle
+
+`getServiceCredentials` manages per-user, per-router service certificate credentials:
+
+1. Checks `useClientServices` — if the user has services disabled, returns `Nothing`
+2. Looks up existing credentials in DB via `getClientServiceCredentials`
+3. If none exist, generates new TLS credentials on-the-fly (`genCredentials`) and stores them
+4. Extracts the private signing key from the X.509 certificate
+
+The generated credentials are Ed25519 self-signed certificates with `simplex` organization, valid for ~2740 years. The certificate chain and hash are bundled into `ServiceCredentials` for the SMP handshake.
+
+## withStore — database access bracket
+
+`withStore` wraps database access with `agentOperationBracket c AODatabase`, ensuring the operation suspension cascade is respected. SQLite errors are classified:
+- `ErrorBusy`/`ErrorLocked` → `SEDatabaseBusy` → `CRITICAL True` (prompts user restart)
+- Other SQL errors → `SEInternal`
+
+`SEAgentError` is a special wrapper that allows agent-level errors to be threaded through store operations — used when "transaction-like" access is needed but the operation involves agent logic, not just DB queries. See source comment: "network IO should NOT be used inside AgentStoreMonad."
+
+`withStoreBatch` / `withStoreBatch'` run multiple DB operations in a single transaction, catching exceptions per-operation to report individual failures. The entire batch is within one `agentOperationBracket`.
+
+## Router selection — getNextServer / withNextSrv
+
+Router selection has two-level diversity:
+1. **Operator diversity**: prefer routers from operators not already used (tracked by `usedOperators` set)
+2. **Host diversity**: prefer routers with hosts not already used (tracked by `usedHosts` set)
+
+`filterOrAll` ensures that if all routers are "used," the full list is returned rather than an empty one.
+
+`withNextSrv` is designed for retry loops — it re-reads user routers on each call (allowing configuration changes during retries) and tracks `triedHosts` across attempts. When all hosts are tried, the tried set is reset (`S.empty`), creating a round-robin effect.
+
+## Locking primitives
+
+**`withConnLock`**: Per-connection lock via `connLocks` TMap. Non-obvious: `withConnLock'` with empty `ConnId` is a no-op (identity function) — allows agent operations on entities without real connection IDs to skip locking.
+
+**`withConnLocks`**: Takes a `Set ConnId` and acquires locks for all connections. Uses `withGetLocks` which acquires all locks concurrently via `forConcurrently`. Note: concurrent acquisition of overlapping lock sets from different threads could theoretically deadlock, so callers must ensure non-overlapping lock sets or use a higher-level coordination.
+
+**`getMapLock`**: Creates a lock on first access and caches it in the TMap. Locks are never removed — the TMap grows monotonically.
+
+## Network configuration — slow/fast selection
+
+`getNetworkConfig` selects between slow and fast network configs based on `userNetworkInfo`:
+- `UNCellular` or `UNNone` → slow config (1.5× timeouts via `slowNetworkConfig`)
+- `UNWifi`, `UNEthernet`, `UNOther` → fast config
+
+Both configs are stored together in `useNetworkConfig :: TVar (NetworkConfig, NetworkConfig)`. The slow config is derived from the fast config in `newAgentClient`.
+
+## closeAgentClient — shutdown sequence
+
+1. Sets `active = False` — all protocol client getters will throw `INACTIVE`
+2. Closes all protocol server clients (SMP, NTF, XFTP) by swapping maps to empty and forking close threads
+3. Clears proxied relays
+4. Cancels resubscription workers — forks cancellation threads (fire-and-forget, `closeAgentClient` may return before all workers are cancelled)
+5. Clears delivery and async command workers (delivery workers are also cancelled via `cancelWorker`)
+6. Clears subscription state
+
+The cancellation of resubscription workers reads the TMVar first (to get the Async handle), then calls `uninterruptibleCancel`. This is wrapped in a forked thread to avoid blocking the shutdown sequence.
+
+**`closeClient_` edge case**: When closing individual clients, `closeClient_` handles `BlockedIndefinitelyOnSTM` — which occurs if the SessionVar TMVar was never filled (connection attempt in progress when shutdown started). The exception is caught and treated as a no-op.
+
+**`reconnectServerClients` vs `closeProtocolServerClients`**: `closeProtocolServerClients` swaps the map to empty and closes all clients — no new connections can be made to those sessions. `reconnectServerClients` reads the map without clearing it and closes current clients — the disconnect callbacks trigger reconnection, effectively forcing fresh connections while keeping the session entries.
+
+## Transport session modes
+
+`TransportSessionMode` (`TSMEntity` vs other) determines whether the transport session key includes the entity ID (connection/queue ID). When `TSMEntity`, each queue gets its own TLS connection to the router. When not, queues to the same router share a connection. This is controlled by `sessionMode` in the network config.
+
+`mkSMPTSession` and related functions compute the transport session key based on the current mode. This affects connection multiplexing — entity-mode sessions provide better privacy (router can't correlate queues) at the cost of more connections.
+
+## getMsgLocks — GET exclusion
+
+`getQueueMessage` creates a TMVar lock keyed by `(server, rcvId)` and takes it before sending GET. This prevents concurrent GET and SUB on the same queue (SUB is checked via `hasGetLock` in `checkQueues`). The lock is released by `releaseGetLock` after ACK or on error.
+
+The lock creation uses `TM.alterF` to atomically create-or-reuse: if no lock exists, creates a new `TMVar ()` and immediately takes it; if one exists, takes it. This avoids a race between two concurrent GET attempts on the same queue.
+
+## Error classification — temporaryAgentError
+
+Classifies errors as temporary (retryable) or permanent. Notable non-obvious classifications:
+- `TEHandshake BAD_SERVICE` is temporary — it indicates a DB error on the router, not a permanent rejection
+- `CRITICAL True` is temporary — `True` means the error shows a restart button, implying the user should retry. `CRITICAL False` is permanent.
+- `INACTIVE` is temporary — the agent may be reactivated
+- `SMP.PROXY NO_SESSION` via proxy is temporary — session can be re-established
+- `SMP.STORE _` is temporary — router-side store error, not a client issue
+
+`temporaryOrHostError` extends `temporaryAgentError` to also include host-related errors (`HOST`, `TRANSPORT TEVersion`). Used in subscription management where host errors should trigger resubscription rather than permanent failure.

spec/modules/Simplex/Messaging/Agent/Env/SQLite.md

@@ -0,0 +1,9 @@
+# Simplex.Messaging.Agent.Env.SQLite
+
+> Agent environment configuration, default values, and worker/supervisor record types.
+
+**Source**: [`Agent/Env/SQLite.hs`](../../../../../../src/Simplex/Messaging/Agent/Env/SQLite.hs)
+
+## mkUserServers — silent fallback on all-disabled
+
+See comment on `mkUserServers`. If filtering routers by `enabled && role` yields an empty list, `fromMaybe srvs` falls back to *all* routers regardless of enabled/role status. This prevents a configuration where all routers are disabled from leaving the user with no routers — but means disabled routers can still be used if every router in a role is disabled.

spec/modules/Simplex/Messaging/Agent/Lock.md

@@ -0,0 +1,7 @@
+# Simplex.Messaging.Agent.Lock
+
+> TMVar-based named mutex with concurrent multi-lock acquisition.
+
+**Source**: [`Agent/Lock.hs`](../../../../../src/Simplex/Messaging/Agent/Lock.hs)
+
+No non-obvious behavior. See source. See comment on `getPutLock` for the atomicity argument.