simplex-chat/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-10-12 02:22:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: add missing encodings and other protocol corrections

Browse Source
This commit is contained in:
Evgeny @ SimpleX Chat
2026-03-10 16:36:24 +00:00
parent 01785d5de8
commit 98351cf097
6 changed files with 78 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
protocol/agent-protocol.md
+49 -11
View File
@@ -24,12 +24,11 @@ Version 7, 2025-01-24
- [Full connection link syntax](#full-connection-link-syntax)
- [Short connection link syntax](#short-connection-link-syntax)
- [Short links](#short-links)
- [Short link structure](#short-link-structure)
- [Link key derivation](#link-key-derivation)
- [Link data encryption](#link-data-encryption)
- [Short link resolution](#short-link-resolution)
- [Link data management](#link-data-management)
- [Appendix A: SMP agent API](#smp-agent-api)
- [Appendix A: SMP agent API](#appendix-a-smp-agent-api)
- [API functions](#api-functions)
- [API events](#api-events)
@@ -177,7 +176,7 @@ These messages are encrypted with per-queue shared secret using NaCL crypto_box
decryptedSMPClientMessage = agentConfirmation / agentMsgEnvelope / agentInvitation / agentRatchetKey
agentConfirmation = agentVersion %s"C" ("0" / "1" sndE2EEncryptionParams) encConnInfo
agentVersion = 2*2 OCTET
sndE2EEncryptionParams = <sender E2E ratchet parameters, see crypto-ratchet.md>
sndE2EEncryptionParams = <sender E2E ratchet parameters, see pqdr.md>
encConnInfo = doubleRatchetEncryptedMessage
agentMsgEnvelope = agentVersion %s"M" encAgentMessage
@@ -185,11 +184,14 @@ encAgentMessage = doubleRatchetEncryptedMessage
agentInvitation = agentVersion %s"I" connReqLength connReq connInfo
connReqLength = 2*2 OCTET ; Word16
connReq = *OCTET ; URI text encoding of connection link, length given by connReqLength
connInfo = *OCTET ; opaque connection information (remaining bytes)
agentRatchetKey = agentVersion %s"R" rcvE2EEncryptionParams agentRatchetInfo
rcvE2EEncryptionParams = <receiver E2E ratchet parameters, see crypto-ratchet.md>
agentRatchetKey = agentVersion %s"R" rcvE2EEncryptionParams ratchetKeyInfo
rcvE2EEncryptionParams = <receiver E2E ratchet parameters, see pqdr.md>
ratchetKeyInfo = *OCTET ; additional ratchet renegotiation info (remaining bytes)
doubleRatchetEncryptedMessage = <double ratchet encrypted message, see crypto-ratchet.md>
doubleRatchetEncryptedMessage = <double ratchet encrypted message, see pqdr.md>
```
The maximum size of the encrypted connection info and agent message depend on whether post-quantum key exchange is used:
@@ -217,7 +219,8 @@ Decrypted SMP message client body can be one of 4 types:
- to confirm that the new double ratchet encryption is agreed (`EREADY`).
- to notify another party that it can continue sending messages after queue capacity was exceeded (`A_QCONT`).
- to manage SMP queue rotation (`QADD`, `QKEY`, `QUSE`, `QTEST`).
- `msgPadding` - an optional message padding to make all SMP messages have constant size, to prevent routers from observing the actual message size. The only case the message padding can be absent is when the message has exactly the maximum size, in all other cases the message MUST be padded to a fixed size.
The encoded `agentMessage` is padded to a fixed size by the double ratchet encryption layer (see [ratchet message wire format](./pqdr.md#ratchet-message-wire-format)) to make all SMP messages have constant size, preventing routers from observing the actual message size.
### Messages between SMP agents
@@ -232,7 +235,7 @@ smpQueues = length 1*newQueueInfo ; NonEmpty list of reply queues
agentRatchetInfo = %s"R" ratchetInfo
ratchetInfo = *OCTET
agentMessage = %s"M" agentMsgHeader aMessage msgPadding
agentMessage = %s"M" agentMsgHeader aMessage
agentMsgHeader = agentMsgId prevMsgHash
agentMsgId = 8*8 OCTET ; Int64
prevMsgHash = shortString
@@ -243,12 +246,13 @@ aMessage = HELLO / A_MSG / A_RCVD / EREADY / A_QCONT /
HELLO = %s"H"
A_MSG = %s"M" userMsgBody
userMsgBody = *OCTET
userMsgBody = *OCTET ; remaining bytes
A_RCVD = %s"V" msgReceipts
msgReceipts = length 1*msgReceipt ; NonEmpty list
msgReceipt = agentMsgId msgHash rcptLength rcptInfo
msgHash = shortString
rcptInfo = *OCTET ; opaque receipt info, length given by rcptLength (Word16)
EREADY = %s"E" agentMsgId
@@ -284,7 +288,6 @@ senderId = shortString
clientVRange = version version
version = 2*2 OCTET
msgPadding = *OCTET
rcptLength = 2*2 OCTET
shortString = length *OCTET
length = 1*1 OCTET
@@ -474,12 +477,15 @@ fixedData = agentVersionRange rootKey linkConnReq [linkEntityId]
agentVersionRange = version version ; min and max agent protocol version
version = 2*2 OCTET
rootKey = length x509encoded ; Ed25519 public key
linkConnReq = connectionRequestUri ; see full connection link syntax above
linkConnReq = invitationConnReq / contactConnReq ; binary encoding of connection request
invitationConnReq = %s"I" connReqData e2eRatchetParams
contactConnReq = %s"C" connReqData
linkEntityId = shortString
userData = invitationLinkData / contactLinkData
invitationLinkData = %s"I" agentVersionRange userLinkData
contactLinkData = %s"C" agentVersionRange userContactData
userLinkData = shortString / (%xFF largeString) ; opaque application data (e.g., user profile)
; shortString length byte 0x00-0xFE (max 254 bytes); 0xFF is reserved as largeString sentinel
userContactData = direct ownersList relaysList userLinkData
direct = %s"T" / %s"F" ; whether direct connection via connReq is allowed
ownersList = length *ownerAuth
@@ -488,6 +494,38 @@ ownerId = shortString ; application-specific owner ID (e.g., MemberId)
ownerKey = length x509encoded ; Ed25519 public key
authOwnerSig = length 64*64 OCTET ; Ed25519 signature of (ownerId || ownerKey) by previous owner
relaysList = length *connShortLink ; alternative relay short links
; Binary encoding of connection request (used in linkConnReq)
connReqData = agentVersionRange smpQueueUris clientData
smpQueueUris = length 1*smpQueueUri
clientData = %s"0" / (%s"1" largeString) ; Maybe (Large ByteString)
smpQueueUri = smpClientVersionRange smpServer senderId smpDhPublicKey [queueMode]
smpClientVersionRange = version version ; min and max SMP client versions
smpServer = hosts port serverKeyHash
hosts = length 1*host
host = shortString ; text-encoded hostname or IP address
port = shortString ; text-encoded port number
serverKeyHash = shortString ; CA certificate fingerprint
senderId = shortString ; queue sender ID
smpDhPublicKey = length x509encoded ; X25519 DH public key
queueMode = %s"M" / %s"C" ; messaging or contact (version-dependent trailing field)
e2eRatchetParams = e2eVersionRange e2eDhKey e2eDhKey kemParams
e2eVersionRange = version version ; min and max e2e encryption versions
e2eDhKey = length x509encoded ; X448 DH public key
kemParams = %s"0" / (%s"1" ratchetKEMParams)
ratchetKEMParams = %s"P" kemPublicKey / %s"A" kemCiphertext kemPublicKey
kemPublicKey = largeString ; sntrup761 public key
kemCiphertext = largeString ; sntrup761 ciphertext
; Binary encoding of short link (used in relaysList)
connShortLink = invShortLink / contactShortLink
invShortLink = %s"I" smpServer linkId linkKey
contactShortLink = %s"C" contactConnType smpServer linkKey
contactConnType = %s"A" / %s"C" / %s"G" / %s"R" ; contact / channel / group / relay
linkId = shortString
linkKey = shortString
x509encoded = *OCTET ; DER-encoded X.509 SubjectPublicKeyInfo
largeString = 2*2 OCTET *OCTET ; Word16 length prefix
length = 1*1 OCTET
shortString = length *OCTET
protocol/push-notifications.md
+3 -2
View File
@@ -100,8 +100,9 @@ The client and router use ALPN extension with `ntf/1` protocol name to agree han
Protocol commands have this syntax:
```
ntfRouterTransmission =
```abnf
ntfRouterTransmission = authorization corrId entityId ntfRouterCmd
; same transmission structure as SMP, see simplex-messaging.md
ntfRouterCmd = newTokenCmd / verifyTokenCmd / checkTokenCmd /
replaceTokenCmd / deleteTokenCmd / cronCmd /
newSubCmd / checkSubCmd / deleteSubCmd / pingCmd
protocol/security.md
+1 -1
View File
@@ -172,7 +172,7 @@ This document describes the cryptographic primitives and threat model for the Si
- undetectably communicate at the same time as Alice with her contacts. Doing so would result in the contact getting different messages with repeated IDs.
- undetectably monitor message queues in realtime without alerting the user they are doing so, as a second subscription request unsubscribes the first and notifies the second.
- undetectably monitor message queues in realtime without alerting the user they are doing so, as a second subscription request unsubscribes the first and notifies the first.
### A user's contact
protocol/simplex-messaging.md
+23 -22
View File
@@ -466,9 +466,9 @@ To send service commands, services should authenticate themselves to SMP routers
Service certificates are included in the client handshake and verified by the router. The service receives a service ID in the handshake response, which is then used as entity ID in service transmissions.
```abnf
clientHandshakeService = serviceRole serviceCertKey
clientService = serviceRole serviceCertKey
serviceRole = %s"M" / %s"N" / %s"P" ; Messaging / Notifier / Proxy
serviceCertKey = certChainPubKey
serviceCertKey = certChain signedServiceKey
```
### Service subscriptions
@@ -537,14 +537,14 @@ Commands syntax below is provided using [ABNF][8] with [case-sensitive strings e
smpCommand = ping / recipientCmd / senderCommand /
proxyCommand / notifierCommand / linkCommand / routerMsg
recipientCmd = create / subscribe / subscribeMultiple / rcvSecure / recipientKeys /
enableNotifications / disableNotifications / getMessage
enableNotifications / disableNotifications / getMessage /
acknowledge / suspend / delete / getQueueInfo / setShortLink / deleteShortLink
senderCommand = send / sndSecure
linkCommand = setLinkKey / getLinkData
proxyCommand = proxySession / proxyForward / relayForward
notifierCommand = subscribeNotifications / subscribeNotificationsMultiple
routerMsg = queueIds / linkResponse / serviceOk / serviceOkMultiple /
message / allReceived / notifierIdResp / messageNotification /
message / allReceived / notifierIdResponse / messageNotification /
proxySessionKey / proxyResponse / relayResponse /
unsubscribed / serviceUnsubscribed / deleted /
queueInfo / ok / error / pong
@@ -663,7 +663,7 @@ This command is used by recipient services to subscribe to multiple queues at on
```abnf
subscribeMultiple = %s"SUBS " count idsHash
count = 8*8 OCTET ; Int64, network byte order (big-endian)
idsHash = 16*16 OCTET ; XOR of MD5 hashes of all queue IDs
idsHash = length 16*16 OCTET ; XOR of MD5 hashes of all queue IDs
```
The count and idsHash allow the router to detect subscription drift. The router responds with `serviceOkMultiple` (`SOKS`) response.
@@ -883,19 +883,19 @@ This command is sent to the router by the sender both to confirm the queue after
```abnf
send = %s"SEND " msgFlags SP smpEncMessage
msgFlags = notificationFlag reserved
msgFlags = notificationFlag
notificationFlag = %s"T" / %s"F"
smpEncMessage = smpEncClientMessage / smpEncConfirmation ; message up to 16048 bytes (v11+)
smpEncClientMessage = smpPubHeaderNoKey msgNonce sentClientMsgBody ; message up to maxMessageLength bytes
smpPubHeaderNoKey = smpClientVersion "0"
sentClientMsgBody = 16000*16000 OCTET ; = maxMessageLength(v11+) - 48 = 16048 - 48
sentClientMsgBody = 16016*16016 OCTET ; = e2eEncMessageLength(16000) + authTagSize(16)
smpEncConfirmation = smpPubHeaderWithKey msgNonce sentConfirmationBody
smpPubHeaderWithKey = smpClientVersion "1" senderPublicDhKey
; sender's Curve25519 public key to agree DH secret for E2E encryption in this queue
; it is only sent in confirmation message
sentConfirmationBody = 15904*15904 OCTET ; E2E-encrypted smpClientMessage padded to e2eEncMessageLength before encryption
sentConfirmationBody = 15920*15920 OCTET ; E2E-encrypted smpConfirmation padded to e2eEncConfirmationLength(15904), + authTagSize(16)
senderPublicDhKey = length x509encoded
smpClientVersion = word16
@@ -1123,7 +1123,7 @@ Transmission sent to proxy router should use session ID as entity ID and use a r
Encrypted transmission should use the received session ID from the connection between proxy router and destination router in the authorized body.
```abnf
proxyCommand = %s"PFWD" SP smpVersion commandKey <encrypted padded(transmission, 16226)>
proxyForward = %s"PFWD" SP smpVersion commandKey <encrypted padded(transmission, 16226)>
smpVersion = 2*2 OCTET
commandKey = length x509encoded
```
@@ -1134,6 +1134,7 @@ Having received the `RRES` response from the destination router, proxy router wi
```abnf
proxyResponse = %s"PRES" SP <encrypted padded(forwardedResponse, 16226)>
forwardedResponse = *OCTET ; client-encrypted SMP response, decrypted by client using per-command DH secret
```
#### Forward command to destination router
@@ -1143,7 +1144,7 @@ Having received `PFWD` command from the client, the router should additionally e
Transmission forwarded to destination router uses empty entity ID and its unique random correlation ID is used as a nonce to encrypt forwarded transmission. Correlation ID increased by 1 is used by the destination router as a nonce to encrypt responses.
```abnf
relayCommand = %s"RFWD" SP <encrypted(forwardedTransmission)>
relayForward = %s"RFWD" SP <encrypted(forwardedTransmission)>
forwardedTransmission = fwdCorrId fwdSmpVersion fwdCommandKey transmission
fwdCorrId = length 24*24 OCTET
; `fwdCorrId` - correlation ID used in `PFWD` command transmission - it is used as a nonce for client encryption,
@@ -1160,6 +1161,8 @@ The shared secret for encrypting transmission bodies between proxy router and de
```abnf
relayResponse = %s"RRES" SP <encrypted(responseTransmission)>
responseTransmission = fwdCorrId forwardedResponse
; fwdCorrId and forwardedResponse defined above in RFWD section
```
### Short link commands
@@ -1212,7 +1215,7 @@ This command is used by notifier services to subscribe to multiple queues at onc
```abnf
subscribeNotificationsMultiple = %s"NSUBS " count idsHash
count = 8*8 OCTET ; Int64, network byte order (big-endian)
idsHash = 16*16 OCTET ; XOR of MD5 hashes of all queue IDs
idsHash = length 16*16 OCTET ; XOR of MD5 hashes of all queue IDs
```
The router responds with `serviceOkMultiple` (`SOKS`) response.
@@ -1253,7 +1256,7 @@ Sent in response to `SUBS` or `NSUBS` commands:
```abnf
serviceOkMultiple = %s"SOKS " count idsHash
count = 8*8 OCTET ; Int64, network byte order (big-endian)
idsHash = 16*16 OCTET ; XOR of MD5 hashes of all subscribed queue IDs
idsHash = length 16*16 OCTET ; XOR of MD5 hashes of all subscribed queue IDs
```
#### All service messages received
@@ -1297,15 +1300,13 @@ The router must deliver message notifications to all simplex queues that were su
```abnf
messageNotification = %s"NMSG " nmsgNonce encryptedNMsgMeta
encryptedNMsgMeta = <encrypted message metadata passed in notification>
; metadata E2E encrypted between router and recipient containing router's message ID and timestamp (allows extension),
; to be passed to the recipient by the notifier for them to decrypt
; with key negotiated in NKEY and NID commands using nmsgNonce
nmsgNonce = <nonce used in NaCl crypto_box encryption scheme>
; nonce used by the router for encryption of message metadata, to be passed to the recipient by the notifier
; for them to use in decryption of E2E encrypted metadata
nmsgNonce = 24*24 OCTET ; 192-bit NaCl crypto_box nonce
encryptedNMsgMeta = shortString
; NaCl crypto_box encrypted padded(nmsgMeta, 128): 128 + 16 (auth tag) = 144 bytes
; metadata E2E encrypted between router and recipient,
; to be passed to the recipient by the notifier for them to decrypt
; with key negotiated in NKEY and NID commands using nmsgNonce
nmsgMeta = msgId timestamp ; message ID and timestamp, allows future extension
```
Message notification does not contain any message data or non E2E encrypted metadata.
@@ -1327,7 +1328,7 @@ Sent when service subscription is terminated (can be sent when service re-connec
```abnf
serviceUnsubscribed = %s"ENDS " count idsHash
count = 8*8 OCTET ; Int64, network byte order (big-endian)
idsHash = 16*16 OCTET ; XOR of MD5 hashes of terminated queue IDs
idsHash = length 16*16 OCTET ; XOR of MD5 hashes of terminated queue IDs
```
#### Queue deleted notification
protocol/xftp.md
+1 -1
View File
@@ -421,7 +421,7 @@ pong = %s"PONG"
### File sender commands
Sending any of the commands in this section (other than `register`, that is sent without data packet ID) is only allowed with sender's ID.
Sending any of the commands in this section (other than `register`, that is sent without data packet ID) is only allowed with sender's ID. The `register` command must be signed (using `sndKey` included in `fileInfo` for verification) but must NOT include a data packet ID.
#### Register new data packet
protocol/xrcp.md
+1 -1
View File
@@ -105,7 +105,7 @@ Multicast session announcement is a binary encoded packet with this syntax:
sessionAddressPacket = dhPubKey nonce encrypted(unpaddedSize sessionAddress packetPad)
dhPubKey = length x509encoded ; same as announced
nonce = 24*24 OCTET ; NaCl 192-bit nonce, no length prefix
sessionAddress = largeLength sessionAddressUri ; as above
sessionAddress = sessionAddressUri ; length given by unpaddedSize
length = 1*1 OCTET ; for binary data up to 255 bytes
largeLength = 2*2 OCTET ; for binary data up to 65535 bytes
packetPad = <pad invitation content to 900 bytes before encryption>