Compare commits

..
Author SHA1 Message Date
Evgeny Poberezkin 1f7df5e404 trying to use semaphores 2022-06-17 13:56:06 +01:00
80 changed files with 2849 additions and 6271 deletions
-2
View File
@@ -4,5 +4,3 @@
*.session.sql
tests/tmp
dist-newstyle/
src/tor
-41
View File
@@ -1,44 +1,3 @@
# 3.2.0
SMP agent:
- Support multiple server hostnames (including onion hostnames) in server addresses.
- Network configuration options.
- Options to define rules to choose server hostname.
# 3.1.0
SMP server and agent:
- SMP protocol v4: batching multiple server commands/responses in a transport block.
# 3.0.0
SMP server:
- restore undeliverd messages when the server is restarted.
- SMP protocol v3 to support push notification:
- updated SEND and MSG to add message flags (for notification flag that contros whether the notification is sent and for any future extensions) and to move message meta-data sent to the recipient into the encrypted envelope.
- update NKEY and NID to add e2e encryption keys (for the notification meta-data encryption between SMP server and the client), and update NMSG to include this meta-data.
- update ACK command to include message ID (to avoid acknowledging unprocessed message).
- add NDEL commands to remove notification subscription credentials from SMP queue.
- add GET command to receive messages without subscription - to be used in iOS notification service extension to receive messages without terminating app subscriptions.
SMP agent:
- new protocol for duplex connection handshake reducing traffic and connection time.
- support for SMP notifications server and managing device token.
- remove redundant FQDN validation from TLS handshake to prepare for access via Tor.
- support for fully stopping agent and for termporary suspending agent operations.
- improve management of duplicate message delivery.
SMP notifications server v1.0:
- SMP notifications protocol with version negotiation during handshake.
- device token registration and verification (via background notification).
- SMP notification subscriptions and push notifications via APNS.
- restoring notification subscriptions when the server is restarted.
# 2.3.0
SMP server:
+12 -20
View File
@@ -11,7 +11,7 @@ If you have a server deployed please deploy a new server to a new host and retir
## Message broker for unidirectional (simplex) queues
SimpleXMQ is a message broker for managing message queues and sending messages over public network. It consists of SMP server, SMP client library and SMP agent that implement [SMP protocol](./protocol/simplex-messaging.md) for client-server communication and [SMP agent protocol](./protocol/agent-protocol.md) to manage duplex connections via simplex queues on multiple SMP servers.
SimpleXMQ is a message broker for managing message queues and sending messages over public network. It consists of SMP server, SMP client library and SMP agent that implement [SMP protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md) for client-server communication and [SMP agent protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md) to manage duplex connections via simplex queues on multiple SMP servers.
SMP protocol is inspired by [Redis serialization protocol](https://redis.io/topics/protocol), but it is much simpler - it currently has only 10 client commands and 8 server responses.
@@ -27,7 +27,7 @@ SimpleXMQ is implemented in Haskell - it benefits from robust software transacti
### SMP server
[SMP server](./apps/smp-server/Main.hs) can be run on any Linux distribution, including low power/low memory devices. OpenSSL library is required for initialization.
[SMP server](https://github.com/simplex-chat/simplexmq/blob/master/apps/smp-server/Main.hs) can be run on any Linux distribution, including low power/low memory devices. OpenSSL library is required for initialization.
To initialize the server use `smp-server init -n <fqdn>` (or `smp-server init --ip <ip>` for IP based address) command - it will generate keys and certificates for TLS transport. The fingerprint of offline certificate is used as part of the server address to protect client/server connection against man-in-the-middle attacks: `smp://<fingerprint>@<hostname>[:5223]`.
@@ -39,7 +39,7 @@ Starting from version 2.3.0, when store log is enabled, the server would also en
> **Please note:** On initialization SMP server creates a chain of two certificates: a self-signed CA certificate ("offline") and a server certificate used for TLS handshake ("online"). **You should store CA certificate private key securely and delete it from the server. If server TLS credential is compromised this key can be used to sign a new one, keeping the same server identity and established connections.** CA private key location by default is `/etc/opt/simplex/ca.key`.
SMP server implements [SMP protocol](./protocol/simplex-messaging.md).
SMP server implements [SMP protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md).
#### Running SMP server on MacOS
@@ -62,7 +62,7 @@ Now `openssl version` should be saying "OpenSSL". You can now run `smp-server in
### SMP client library
[SMP client](./src/Simplex/Messaging/Client.hs) is a Haskell library to connect to SMP servers that allows to:
[SMP client](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Client.hs) is a Haskell library to connect to SMP servers that allows to:
- execute commands with a functional API.
- receive messages and other notifications via STM queue.
@@ -70,13 +70,13 @@ Now `openssl version` should be saying "OpenSSL". You can now run `smp-server in
### SMP agent
[SMP agent library](./src/Simplex/Messaging/Agent.hs) can be used to run SMP agent as part of another application and to communicate with the agent via STM queues, without serializing and parsing commands and responses.
[SMP agent library](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent.hs) can be used to run SMP agent as part of another application and to communicate with the agent via STM queues, without serializing and parsing commands and responses.
Haskell type [ACommand](./src/Simplex/Messaging/Agent/Protocol.hs) represents SMP agent protocol to communicate via STM queues.
Haskell type [ACommand](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent/Protocol.hs) represents SMP agent protocol to communicate via STM queues.
See [simplex-chat](https://github.com/simplex-chat/simplex-chat) terminal UI for the example of integrating SMP agent into another application.
[SMP agent executable](./apps/smp-agent/Main.hs) can be used to run a standalone SMP agent process that implements plaintext [SMP agent protocol](./protocol/agent-protocol.md) via TCP port 5224, so it can be used via telnet. It can be deployed in private networks to share access to the connections between multiple applications and services.
[SMP agent executable](https://github.com/simplex-chat/simplexmq/blob/master/apps/smp-agent/Main.hs) can be used to run a standalone SMP agent process that implements plaintext [SMP agent protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md) via TCP port 5224, so it can be used via telnet. It can be deployed in private networks to share access to the connections between multiple applications and services.
## Using SMP server and SMP agent
@@ -110,15 +110,7 @@ You can run your SMP server as a Linux process, optionally using a service manag
See [this section](#smp-server) for more information. Run `smp-server -h` and `smp-server init -h` for explanation of commands and options.
<img alt="Docker" src="./img/docker.svg" align="right" width="200">
## Deploy SMP server with Docker
SMP server could also be deployed using `Docker`.
See: [`scripts/docker`](./scripts/docker/)
[<img alt="Linode" src="./img/linode.svg" align="right" width="200">](https://cloud.linode.com/stackscripts/748014)
[<img alt="Linode" src="https://raw.githubusercontent.com/simplex-chat/simplexmq/master/img/linode.svg" align="right" width="200">](https://cloud.linode.com/stackscripts/748014)
## Deploy SMP server on Linode
@@ -142,7 +134,7 @@ Deployment on Linode is performed via StackScripts, which serve as recipes for L
Please submit an [issue](https://github.com/simplex-chat/simplexmq/issues) if any problems occur.
[<img alt="DigitalOcean" src="/img/digitalocean.png" align="right" width="300">](https://marketplace.digitalocean.com/apps/simplex-server)
[<img alt="DigitalOcean" src="https://raw.githubusercontent.com/simplex-chat/simplexmq/master/img/digitalocean.png" align="right" width="300">](https://marketplace.digitalocean.com/apps/simplex-server)
## Deploy SMP server on DigitalOcean
@@ -170,12 +162,12 @@ smp-server init [-l] -n <fqdn>
## SMP server design
![SMP server design](./design/server.svg)
![SMP server design](https://raw.githubusercontent.com/simplex-chat/simplexmq/master/design/server.svg)
## SMP agent design
![SMP agent design](./design/agent2.svg)
![SMP agent design](https://raw.githubusercontent.com/simplex-chat/simplexmq/master/design/agent2.svg)
## License
[AGPL v3](./LICENSE)
[AGPL v3](https://github.com/simplex-chat/simplexmq/blob/master/LICENSE)
+19 -33
View File
@@ -1,12 +1,8 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Logger.Simple
import Data.Functor (($>))
import Data.Ini (lookupValue)
import Simplex.Messaging.Client.Agent (defaultSMPClientAgentConfig)
import Simplex.Messaging.Notifications.Server (runNtfServer)
import Simplex.Messaging.Notifications.Server.Env (NtfServerConfig (..))
@@ -45,7 +41,7 @@ ntfServerCLIConfig =
fingerprintFile = combine cfgPath "fingerprint",
defaultServerPort = "443",
executableName = "ntf-server",
serverVersion = "SMP notifications server v1.2.0",
serverVersion = "SMP notifications server v0.1.0",
mkIniFile = \enableStoreLog defaultServerPort ->
"[STORE_LOG]\n\
\# The server uses STM memory for persistence,\n\
@@ -53,36 +49,26 @@ ntfServerCLIConfig =
\# This option enables saving memory to append only log,\n\
\# and restoring it when the server is started.\n\
\# Log is compacted on start (deleted objects are removed).\n\
\enable: "
<> (if enableStoreLog then "on" else "off")
<> "\n\
\log_stats: off\n\n\
\[TRANSPORT]\n\
\# The messages are not logged.\n"
<> ("enable: " <> (if enableStoreLog then "on" else "off # on") <> "\n\n")
<> "[TRANSPORT]\n\
\port: "
<> defaultServerPort
<> "\n\
\websockets: off\n",
mkServerConfig = \storeLogFile transports ini ->
let settingIsOn section name = if lookupValue section name ini == Right "on" then Just () else Nothing
logStats = settingIsOn "STORE_LOG" "log_stats"
in NtfServerConfig
{ transports,
subIdBytes = 24,
regCodeBytes = 32,
clientQSize = 16,
subQSize = 64,
pushQSize = 128,
smpAgentCfg = defaultSMPClientAgentConfig,
apnsConfig = defaultAPNSPushClientConfig,
inactiveClientExpiration = Nothing,
storeLogFile,
resubscribeDelay = 50000, -- 50ms
caCertificateFile = caCrtFile,
privateKeyFile = serverKeyFile,
certificateFile = serverCrtFile,
logStatsInterval = logStats $> 86400, -- seconds
logStatsStartTime = 0, -- seconds from 00:00 UTC
serverStatsLogFile = combine logPath "ntf-server-stats.daily.log",
serverStatsBackupFile = logStats $> combine logPath "ntf-server-stats.log"
}
mkServerConfig = \_storeLogFile transports _ ->
NtfServerConfig
{ transports,
subIdBytes = 24,
regCodeBytes = 32,
clientQSize = 16,
subQSize = 64,
pushQSize = 128,
smpAgentCfg = defaultSMPClientAgentConfig,
apnsConfig = defaultAPNSPushClientConfig,
inactiveClientExpiration = Nothing,
caCertificateFile = caCrtFile,
privateKeyFile = serverKeyFile,
certificateFile = serverCrtFile
}
}
+1 -4
View File
@@ -1,5 +1,4 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TypeApplications #-}
@@ -9,7 +8,6 @@ import Control.Logger.Simple
import qualified Data.List.NonEmpty as L
import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Server (runSMPAgent)
import Simplex.Messaging.Client (defaultNetworkConfig)
import Simplex.Messaging.Transport (TLS, Transport (..))
cfg :: AgentConfig
@@ -19,8 +17,7 @@ servers :: InitialAgentServers
servers =
InitialAgentServers
{ smp = L.fromList ["smp://bU0K-bRg24xWW__lS0umO1Zdw_SXqpJNtm1_RrPLViE=@localhost:5223"],
ntf = [],
netCfg = defaultNetworkConfig
ntf = []
}
logCfg :: LogConfig
+36 -37
View File
@@ -2,6 +2,7 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TypeApplications #-}
module Main where
@@ -58,11 +59,10 @@ smpServerCLIConfig =
\# This option enables saving memory to append only log,\n\
\# and restoring it when the server is started.\n\
\# Log is compacted on start (deleted objects are removed).\n"
<> ("enable: " <> (if enableStoreLog then "on" else "off") <> "\n")
<> "# Undelivered messages are optionally saved and restored when the server restarts,\n\
\# they are preserved in the .bak file until the next restart.\n"
<> ("restore_messages: " <> (if enableStoreLog then "on" else "off") <> "\n")
<> ("log_stats: off\n\n")
<> ("enable: " <> (if enableStoreLog then "on" else "off # on") <> "\n")
<> "# The messages are optionally saved and restored when the server restarts,\n\
\# they are deleted after restarting.\n"
<> ("restore_messages: " <> (if enableStoreLog then "on" else "off # on") <> "\n\n")
<> "[TRANSPORT]\n"
<> ("port: " <> defaultServerPort <> "\n")
<> "websockets: off\n\n"
@@ -72,38 +72,37 @@ smpServerCLIConfig =
<> ("# ttl: " <> show (ttl defaultInactiveClientExpiration) <> "\n")
<> ("# check_interval: " <> show (checkInterval defaultInactiveClientExpiration) <> "\n"),
mkServerConfig = \storeLogFile transports ini ->
let settingIsOn section name = if lookupValue section name ini == Right "on" then Just () else Nothing
logStats = settingIsOn "STORE_LOG" "log_stats"
in ServerConfig
{ transports,
tbqSize = 16,
serverTbqSize = 64,
msgQueueQuota = 128,
queueIdBytes = 24,
msgIdBytes = 24, -- must be at least 24 bytes, it is used as 192-bit nonce for XSalsa20
caCertificateFile = caCrtFile,
privateKeyFile = serverKeyFile,
certificateFile = serverCrtFile,
storeLogFile,
storeMsgsFile =
let messagesPath = combine logPath "smp-server-messages.log"
in case lookupValue "STORE_LOG" "restore_messages" ini of
Right "on" -> Just messagesPath
Right _ -> Nothing
-- if the setting is not set, it is enabled when store log is enabled
_ -> storeLogFile $> messagesPath,
allowNewQueues = True,
messageExpiration = Just defaultMessageExpiration,
inactiveClientExpiration =
settingIsOn "INACTIVE_CLIENTS" "disconnect"
$> ExpirationConfig
ServerConfig
{ transports,
tbqSize = 16,
serverTbqSize = 64,
msgQueueQuota = 128,
queueIdBytes = 24,
msgIdBytes = 24, -- must be at least 24 bytes, it is used as 192-bit nonce for XSalsa20
caCertificateFile = caCrtFile,
privateKeyFile = serverKeyFile,
certificateFile = serverCrtFile,
storeLogFile,
storeMsgsFile =
let messagesPath = combine logPath "smp-server-messages.log"
in case lookupValue "STORE_LOG" "restore_messages" ini of
Right "on" -> Just messagesPath
Right _ -> Nothing
-- if the setting is not set, it is enabled when store log is enabled
_ -> storeLogFile $> messagesPath,
allowNewQueues = True,
messageExpiration = Just defaultMessageExpiration,
inactiveClientExpiration =
if lookupValue "INACTIVE_CLIENTS" "disconnect" ini == Right "on"
then
Just
ExpirationConfig
{ ttl = readStrictIni "INACTIVE_CLIENTS" "ttl" ini,
checkInterval = readStrictIni "INACTIVE_CLIENTS" "check_interval" ini
},
logStatsInterval = logStats $> 86400, -- seconds
logStatsStartTime = 0, -- seconds from 00:00 UTC
serverStatsLogFile = combine logPath "smp-server-stats.daily.log",
serverStatsBackupFile = logStats $> combine logPath "smp-server-stats.log",
smpServerVRange = supportedSMPServerVRange
}
}
else Nothing,
logStatsInterval = Just 86400, -- seconds
logStatsStartTime = 0, -- seconds from 00:00 UTC
smpServerVRange = supportedSMPServerVRange
}
}
-1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 8.1 KiB

+1 -1
View File
@@ -1 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 506.94 131.5"><path d="m109.65 58.21-18.21-10.07-15.37 9.38-.19 9.59-7.47-4.92-10.16 6.19-.44-10.48-10.48-7 10-5.17c-.08 0 0 1-1.48-34.34l-23.65-11.39-32.2 10 7.19 34.63 10.81 8.37-8.22 3.9 5.37 26.1 7.55 7.09-5.4 3.29 4.19 20.18 16.94 17.94c.08-.1 1.78-1.41 21.79-17.32l-.58-13.71 8.58 7.28c.12-.12 1.84-1.39 17.56-13.89l.61-10.14 6.48 4.5c.13-.12 1.58-1.22 14.26-11.33z" fill="#231f20"/><path d="m94.54 69 15.11-10.79-18.21-10.07-15.37 9.38z"/><path d="m92.87 88.2 1.67-19.2-18.47-11.48-.38 18.75z" fill="#004b16"/><path d="m68.22 107.73v-19.89l-18.66-14.15 1.47 19.54z" fill="#004b16"/><path d="m68.22 87.84 18.74-13.43-18.55-12.22-18.85 11.5z"/><path d="m38.43 131.48-2.98-20.32-18.15-17.8 4.19 20.18z" fill="#004b16"/><path d="m35.45 111.16 23.91-17.06-18.53-15.09-23.53 14.35z"/><path d="m33.9 100.6-3.94-26.88-20.22-16.81 5.41 26.07z" fill="#004b16"/><path d="m29.96 73.72 27.85-15.82-20.8-13.94-27.27 12.95z"/><path d="m28.07 60.88-5.4-36.81-22.67-14.06 7.19 34.62z" fill="#004b16"/><path d="m22.67 24.07 33.14-12.7-23.61-11.37-32.2 10.01z"/><g fill="#1cb35c"><path d="m107.13 76.87c-14.02 11.13-14.33 11.33-14.26 11.33 1.74-20.1 1.59-19.2 1.67-19.2 16-11.45 15-10.79 15.11-10.79z"/><path d="m85.78 93.84c-17.35 13.8-17.63 13.89-17.56 13.89-.17-20.82-.07-19.89 0-19.89 20-14.3 18.67-13.43 18.74-13.43z"/><path d="m60.22 114.16c-21.66 17.22-21.86 17.32-21.79 17.32-3.07-20.94-3-20.32-3-20.32 25.47-18.16 23.86-17.06 23.93-17.06z"/><path d="m55.81 11.37c1.52 35.37 1.4 34.34 1.48 34.34-28.66 14.89-29.29 15.17-29.22 15.17-5.52-37.63-5.47-36.81-5.4-36.81z"/><path d="m57.81 57.9c1.15 26.81 1 25.88 1.11 25.88-24.81 16.67-25.09 16.82-25 16.82-4-27.58-4-26.88-3.94-26.88z"/></g><path xmlns="http://www.w3.org/2000/svg" d="m151.61 14.24 16.58-4v79.88q0 13.13 7.83 15.65-3.84 7.31-13.13 7.3-11.28 0-11.28-15.66z"/><path d="m186.66 111.72v-57.42h-9.08v-13.6h25.86v71zm8.56-98.54a9.62 9.62 0 1 1 -9.62 9.62 9.63 9.63 0 0 1 9.62-9.62z"/><path d="m262.94 111.74v-41.06c0-6.05-1.16-10.48-3.48-13.26s-6.11-4.18-11.37-4.18a17.74 17.74 0 0 0 -7.8 2.06 18 18 0 0 0 -6.46 5.1v51.34h-16.59v-71h11.94l3 6.64q6.76-8 20-8 12.66 0 20 7.59t7.33 21.19v43.58z"/><path d="m288.42 76.06q0-16.26 9.38-26.47t24.77-10.21q16.19 0 25.14 9.82t9 26.86q0 17-9.12 27t-25 10q-16.18 0-25.17-10.12t-9-26.88zm17.24 0q0 23.47 16.91 23.48a14.54 14.54 0 0 0 12.31-6.11q4.55-6.09 4.54-17.37 0-23.14-16.85-23.15a14.61 14.61 0 0 0 -12.33 6.09q-4.58 6.11-4.58 17.06z"/><path d="m412.37 111.74v-4.31c-1.37 1.5-3.71 2.82-7 3.95a31.33 31.33 0 0 1 -10.15 1.69q-14.87 0-23.38-9.42t-8.52-26.27q0-16.84 9.78-27.42a32 32 0 0 1 24.51-10.58 32.37 32.37 0 0 1 14.72 3.32v-28.46l16.58-4v101.5zm0-54.06a17.6 17.6 0 0 0 -11.07-4.24q-10 0-15.33 6.07t-5.37 17.41q0 22.15 21.36 22.15a16.11 16.11 0 0 0 5.87-1.42c2.32-1 3.83-1.92 4.54-2.89z"/><path d="m505.55 81.3h-50.74q.46 8.49 5.83 13.2t14.46 4.7q11.34 0 17.25-5.9l6.43 12.7q-8.75 7.09-26.13 7.1-16.25 0-25.7-9.52t-9.45-26.58q0-16.78 10.38-27.2a33.91 33.91 0 0 1 24.9-10.41q15.45 0 24.81 9.22t9.35 23.48a46.28 46.28 0 0 1 -1.39 9.21zm-50.15-12.47h34.89q-1.73-15.58-17.24-15.59-14.2 0-17.65 15.59z"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 506.94 131.5"><path d="m109.65 58.21-18.21-10.07-15.37 9.38-.19 9.59-7.47-4.92-10.16 6.19-.44-10.48-10.48-7 10-5.17c-.08 0 0 1-1.48-34.34l-23.65-11.39-32.2 10 7.19 34.63 10.81 8.37-8.22 3.9 5.37 26.1 7.55 7.09-5.4 3.29 4.19 20.18 16.94 17.94c.08-.1 1.78-1.41 21.79-17.32l-.58-13.71 8.58 7.28c.12-.12 1.84-1.39 17.56-13.89l.61-10.14 6.48 4.5c.13-.12 1.58-1.22 14.26-11.33z" fill="#231f20"/><path d="m94.54 69 15.11-10.79-18.21-10.07-15.37 9.38z"/><path d="m92.87 88.2 1.67-19.2-18.47-11.48-.38 18.75z" fill="#004b16"/><path d="m68.22 107.73v-19.89l-18.66-14.15 1.47 19.54z" fill="#004b16"/><path d="m68.22 87.84 18.74-13.43-18.55-12.22-18.85 11.5z"/><path d="m38.43 131.48-2.98-20.32-18.15-17.8 4.19 20.18z" fill="#004b16"/><path d="m35.45 111.16 23.91-17.06-18.53-15.09-23.53 14.35z"/><path d="m33.9 100.6-3.94-26.88-20.22-16.81 5.41 26.07z" fill="#004b16"/><path d="m29.96 73.72 27.85-15.82-20.8-13.94-27.27 12.95z"/><path d="m28.07 60.88-5.4-36.81-22.67-14.06 7.19 34.62z" fill="#004b16"/><path d="m22.67 24.07 33.14-12.7-23.61-11.37-32.2 10.01z"/><g fill="#1cb35c"><path d="m107.13 76.87c-14.02 11.13-14.33 11.33-14.26 11.33 1.74-20.1 1.59-19.2 1.67-19.2 16-11.45 15-10.79 15.11-10.79z"/><path d="m85.78 93.84c-17.35 13.8-17.63 13.89-17.56 13.89-.17-20.82-.07-19.89 0-19.89 20-14.3 18.67-13.43 18.74-13.43z"/><path d="m60.22 114.16c-21.66 17.22-21.86 17.32-21.79 17.32-3.07-20.94-3-20.32-3-20.32 25.47-18.16 23.86-17.06 23.93-17.06z"/><path d="m55.81 11.37c1.52 35.37 1.4 34.34 1.48 34.34-28.66 14.89-29.29 15.17-29.22 15.17-5.52-37.63-5.47-36.81-5.4-36.81z"/><path d="m57.81 57.9c1.15 26.81 1 25.88 1.11 25.88-24.81 16.67-25.09 16.82-25 16.82-4-27.58-4-26.88-3.94-26.88z"/></g><path xmlns="http://www.w3.org/2000/svg" d="m151.61 14.24 16.58-4v79.88q0 13.13 7.83 15.65-3.84 7.31-13.13 7.3-11.28 0-11.28-15.66z"/><path d="m186.66 111.72v-57.42h-9.08v-13.6h25.86v71zm8.56-98.54a9.62 9.62 0 1 1 -9.62 9.62 9.63 9.63 0 0 1 9.62-9.62z"/><path d="m262.94 111.74v-41.06c0-6.05-1.16-10.48-3.48-13.26s-6.11-4.18-11.37-4.18a17.74 17.74 0 0 0 -7.8 2.06 18 18 0 0 0 -6.46 5.1v51.34h-16.59v-71h11.94l3 6.64q6.76-8 20-8 12.66 0 20 7.59t7.33 21.19v43.58z"/><path d="m288.42 76.06q0-16.26 9.38-26.47t24.77-10.21q16.19 0 25.14 9.82t9 26.86q0 17-9.12 27t-25 10q-16.18 0-25.17-10.12t-9-26.88zm17.24 0q0 23.47 16.91 23.48a14.54 14.54 0 0 0 12.31-6.11q4.55-6.09 4.54-17.37 0-23.14-16.85-23.15a14.61 14.61 0 0 0 -12.33 6.09q-4.58 6.11-4.58 17.06z"/><path d="m412.37 111.74v-4.31c-1.37 1.5-3.71 2.82-7 3.95a31.33 31.33 0 0 1 -10.15 1.69q-14.87 0-23.38-9.42t-8.52-26.27q0-16.84 9.78-27.42a32 32 0 0 1 24.51-10.58 32.37 32.37 0 0 1 14.72 3.32v-28.46l16.58-4v101.5zm0-54.06a17.6 17.6 0 0 0 -11.07-4.24q-10 0-15.33 6.07t-5.37 17.41q0 22.15 21.36 22.15a16.11 16.11 0 0 0 5.87-1.42c2.32-1 3.83-1.92 4.54-2.89z"/><path d="m505.55 81.3h-50.74q.46 8.49 5.83 13.2t14.46 4.7q11.34 0 17.25-5.9l6.43 12.7q-8.75 7.09-26.13 7.1-16.25 0-25.7-9.52t-9.45-26.58q0-16.78 10.38-27.2a33.91 33.91 0 0 1 24.9-10.41q15.45 0 24.81 9.22t9.35 23.48a46.28 46.28 0 0 1 -1.39 9.21zm-50.15-12.47h34.89q-1.73-15.58-17.24-15.59-14.2 0-17.65 15.59z"/></svg>

Before

Width:  |  Height:  |  Size: 3.1 KiB

After

Width:  |  Height:  |  Size: 3.1 KiB

+3 -3
View File
@@ -1,5 +1,5 @@
name: simplexmq
version: 3.2.0
version: 2.3.1
synopsis: SimpleXMQ message broker
description: |
This package includes <./docs/Simplex-Messaging-Server.html server>,
@@ -28,7 +28,7 @@ dependencies:
- asn1-types == 0.3.*
- async == 2.2.*
- attoparsec == 0.14.*
- base >= 4.14 && < 5
- base >= 4.7 && < 5
- base64-bytestring >= 1.0 && < 1.3
- bytestring == 0.10.*
- case-insensitive == 1.2.*
@@ -55,7 +55,6 @@ dependencies:
- process == 1.6.*
- random >= 1.1 && < 1.3
- simple-logger == 0.1.*
- socks == 0.6.*
- sqlite-simple == 0.4.*
- stm == 2.5.*
- template-haskell == 2.16.*
@@ -65,6 +64,7 @@ dependencies:
- time-manager == 0.0.*
- tls >= 1.6.0 && < 1.7
- transformers == 0.5.*
- unix == 2.7.*
- unliftio == 0.2.*
- unliftio-core == 0.2.*
- websockets == 0.12.*
+42 -95
View File
@@ -24,7 +24,6 @@
- [Subscribe to queue](#subscribe-to-queue)
- [Secure queue command](#secure-queue-command)
- [Enable notifications command](#enable-notifications-command)
- [Disable notifications command](#disable-notifications-command)
- [Acknowledge message delivery](#acknowledge-message-delivery)
- [Suspend queue](#suspend-queue)
- [Delete queue](#delete-queue)
@@ -51,8 +50,6 @@ It's designed with the focus on communication security and integrity, under the
It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system.
This document describes SMP protocol versions 3 and 4, the previous versions are discontinued.
## Introduction
The objective of Simplex Messaging Protocol (SMP) is to facilitate the secure and private unidirectional transfer of messages from senders to recipients via persistent simplex queues managed by the message broker (server).
@@ -149,61 +146,61 @@ To create and start using a simplex queue Alice and Bob follow these steps:
1. Alice creates a simplex queue on the server:
1. Decides which SMP server to use (can be the same or different server that Alice uses for other queues) and opens secure encrypted transport connection to the chosen SMP server (see [Appendix A](#appendix-a)).
1. Decides which SMP server to use (can be the same or different server that Alice uses for other queues) and opens secure encrypted transport connection to the chosen SMP server (see [Appendix A](#appendix-a)).
2. Generates a new random public/private key pair (encryption key - `EK`) that she did not use before for Bob to encrypt the messages.
2. Generates a new random public/private key pair (encryption key - `EK`) that she did not use before for Bob to encrypt the messages.
3. Generates another new random public/private key pair (recipient key - `RK`) that she did not use before for her to sign commands and to decrypt the transmissions received from the server.
3. Generates another new random public/private key pair (recipient key - `RK`) that she did not use before for her to sign commands and to decrypt the transmissions received from the server.
4. Generates one more random key pair (recipient DH key - `RDHK`) to negotiate symmetric key that will be used by the server to encrypt message bodies delivered to Alice (to avoid shared cipher-text inside transport connection).
4. Generates one more random key pair (recipient DH key - `RDHK`) to negotiate symmetric key that will be used by the server to encrypt message bodies delivered to Alice (to avoid shared cipher-text inside transport connection).
5. Sends `"NEW"` command to the server to create a simplex queue (see `create` in [Create queue command](#create-queue-command)). This command contains previously generated unique "public" keys `RK` and `RDHK`. `RK` will be used to verify the following commands related to the same queue signed by its private counterpart, for example to subscribe to the messages received to this queue or to update the queue, e.g. by setting the key required to send the messages (initially Alice creates the queue that accepts unsigned messages, so anybody could send the message via this queue if they knew the queue sender's ID and server address).
5. Sends `"NEW"` command to the server to create a simplex queue (see `create` in [Create queue command](#create-queue-command)). This command contains previously generated unique "public" keys `RK` and `RDHK`. `RK` will be used to verify the following commands related to the same queue signed by its private counterpart, for example to subscribe to the messages received to this queue or to update the queue, e.g. by setting the key required to send the messages (initially Alice creates the queue that accepts unsigned messages, so anybody could send the message via this queue if they knew the queue sender's ID and server address).
6. The server sends `"IDS"` response with queue IDs (`queueIds`):
6. The server sends `"IDS"` response with queue IDs (`queueIds`):
- Recipient ID `RID` for Alice to manage the queue and to receive the messages.
- Recipient ID `RID` for Alice to manage the queue and to receive the messages.
- Sender ID `SID` for Bob to send messages to the queue.
- Sender ID `SID` for Bob to send messages to the queue.
- Server public DH key (`SDHK`) to negotiate a shared secret for message body encryption, that Alice uses to derive a shared secret with the server `SS`.
- Server public DH key (`SDHK`) to negotiate a shared secret for message body encryption, that Alice uses to derive a shared secret with the server `SS`.
2. Alice sends an out-of-band message to Bob via the alternative channel that both Alice and Bob trust (see [protocol abstract](#simplex-messaging-protocol-abstract)). The message must include:
- Unique "public" key (`EK`) that Bob must use for E2E key agreement.
- Unique "public" key (`EK`) that Bob must use for E2E key agreement.
- SMP server hostname and information to open secure encrypted transport connection (see [Appendix A](#appendix-a)).
- SMP server hostname and information to open secure encrypted transport connection (see [Appendix A](#appendix-a)).
- Sender queue ID `SID` for Bob to use.
- Sender queue ID `SID` for Bob to use.
3. Bob, having received the out-of-band message from Alice, connects to the queue:
1. Generates a new random public/private key pair (sender key - `SK`) that he did not use before for him to sign messages sent to Alice's server.
1. Generates a new random public/private key pair (sender key - `SK`) that he did not use before for him to sign messages sent to Alice's server.
2. Prepares the confirmation message for Alice to secure the queue. This message includes:
2. Prepares the confirmation message for Alice to secure the queue. This message includes:
- Previously generated "public" key `SK` that will be used by Alice's server to authenticate Bob's messages, once the queue is secured.
- Previously generated "public" key `SK` that will be used by Alice's server to authenticate Bob's messages, once the queue is secured.
- Optionally, any additional information (application specific, e.g. Bob's profile name and details).
- Optionally, any additional information (application specific, e.g. Bob's profile name and details).
3. Encrypts the confirmation body with the "public" key `EK` (that Alice provided via the out-of-band message).
3. Encrypts the confirmation body with the "public" key `EK` (that Alice provided via the out-of-band message).
4. Sends the encrypted message to the server with queue ID `SID` (see `send` in [Send message](#send-message)). This initial message to the queue must not be signed - signed messages will be rejected until Alice secures the queue (below).
4. Sends the encrypted message to the server with queue ID `SID` (see `send` in [Send message](#send-message)). This initial message to the queue must not be signed - signed messages will be rejected until Alice secures the queue (below).
4. Alice receives Bob's message from the server using recipient queue ID `RID` (possibly, via the same transport connection she already has opened - see `message` in [Deliver queue message](#deliver-queue-message)):
1. She decrypts received message body using the secret `SS`.
1. She decrypts received message body using the secret `SS`.
2. She decrypts received message with [key agreed with sender using] "private" key `EK`.
2. She decrypts received message with [key agreed with sender using] "private" key `EK`.
3. Even though anybody could have sent the message to the queue with ID `SID` before it is secured (e.g. if communication is compromised), Alice would ignore all messages until the decryption succeeds (i.e. the result contains the expected message format). Optionally, in the client application, she also may identify Bob using the information provided, but it is out of scope of SMP protocol.
3. Even though anybody could have sent the message to the queue with ID `SID` before it is secured (e.g. if communication is compromised), Alice would ignore all messages until the decryption succeeds (i.e. the result contains the expected message format). Optionally, in the client application, she also may identify Bob using the information provided, but it is out of scope of SMP protocol.
5. Alice secures the queue `RID` with `"KEY"` command so only Bob can send messages to it (see [Secure queue command](#secure-queue-command)):
1. She sends the `KEY` command with `RID` signed with "private" key `RK` to update the queue to only accept requests signed by "private" key `SK` provided by Bob. This command contains unique "public" key `SK` previously generated by Bob.
1. She sends the `KEY` command with `RID` signed with "private" key `RK` to update the queue to only accept requests signed by "private" key `SK` provided by Bob. This command contains unique "public" key `SK` previously generated by Bob.
2. From this moment the server will accept only signed commands to `SID`, so only Bob will be able to send messages to the queue `SID` (corresponding to `RID` that Alice has).
2. From this moment the server will accept only signed commands to `SID`, so only Bob will be able to send messages to the queue `SID` (corresponding to `RID` that Alice has).
3. Once queue is secured, Alice deletes `SID` and `SK` - even if Alice's client is compromised in the future, the attacker would not be able to send messages pretending to be Bob.
3. Once queue is secured, Alice deletes `SID` and `SK` - even if Alice's client is compromised in the future, the attacker would not be able to send messages pretending to be Bob.
6. The simplex queue `RID` is now ready to be used.
@@ -217,21 +214,21 @@ Bob now can securely send messages to Alice:
1. Bob sends the message:
1. He encrypts the message to Alice with "public" key `EK` (provided by Alice, only known to Bob, used only for one simplex queue).
1. He encrypts the message to Alice with "public" key `EK` (provided by Alice, only known to Bob, used only for one simplex queue).
2. He signs `"SEND"` command to the server queue `SID` using the "private" key `SK` (that only he knows, used only for this queue).
2. He signs `"SEND"` command to the server queue `SID` using the "private" key `SK` (that only he knows, used only for this queue).
3. He sends the command to the server (see `send` in [Send message](#send-message)), that the server will authenticate using the "public" key `SK` (that Alice earlier received from Bob and provided to the server via `"KEY"` command).
3. He sends the command to the server (see `send` in [Send message](#send-message)), that the server will authenticate using the "public" key `SK` (that Alice earlier received from Bob and provided to the server via `"KEY"` command).
2. Alice receives the message(s):
1. She signs `"SUB"` command to the server to subscribe to the queue `RID` with the "private" key `RK` (see `subscribe` in [Subscribe to queue](#subscribe-to-queue)).
1. She signs `"SUB"` command to the server to subscribe to the queue `RID` with the "private" key `RK` (see `subscribe` in [Subscribe to queue](#subscribe-to-queue)).
2. The server, having authenticated Alice's command with the "public" key `RK` that she provided, delivers Bob's message(s) (see `message` in [Deliver queue message](#deliver-queue-message)).
2. The server, having authenticated Alice's command with the "public" key `RK` that she provided, delivers Bob's message(s) (see `message` in [Deliver queue message](#deliver-queue-message)).
3. She decrypts Bob's message(s) with the "private" key `EK` (that only she has).
3. She decrypts Bob's message(s) with the "private" key `EK` (that only she has).
4. She acknowledges the message reception to the server with `"ACK"` so that the server can delete the message and deliver the next messages.
4. She acknowledges the message reception to the server with `"ACK"` so that the server can delete the message and deliver the next messages.
This flow is show on sequence diagram below.
@@ -358,22 +355,18 @@ To protect the privacy of the recipients, there are several commands in SMP prot
The clients can optionally instruct a dedicated push notification server to subscribe to notifications and deliver push notifications to the device, which can then retrieve the messages in the background and send local notifications to the user - this is out of scope of SMP protocol. The commands that SMP protocol provides to allow it:
- `enableNotifications` (`"NKEY"`) with `notifierId` (`"NID"`) response - see [Enable notifications command](#enable-notifications-command).
- `disableNotifications` (`"NDEL"`) - see [Disable notifications command](#disable-notifications-command).
- `subscribeNotifications` (`"NSUB"`) - see [Subscribe to queue notifications](#subscribe-to-queue-notifications).
- `messageNotification` (`"NMSG"`) - see [Deliver message notification](#deliver-message-notification).
[`SEND` command](#send-message) includes the notification flag to instruct SMP server whether to send the notification - this flag is forwarded to the recepient inside encrypted envelope, together with the timestamp and the message body, so even if TLS is compromised this flag cannot be used for traffic correlation.
## SMP Transmission andtransport block structure
## SMP Transmission structure
Each transport block (SMP transmission) has a fixed size of 16384 bytes for traffic uniformity.
From SMP version 4 each block can contain multiple transmissions, version 3 blocks have 1 transmission.
Some parts of SMP transmission are padded to a fixed size; this padding is uniformly added as a word16 encoded in network byte order - see `paddedString` syntax.
In places where some part of the transmission should be padded, the syntax for `paddedNotation` is used:
```abnf
```
paddedString = originalLength string pad
originalLength = 2*2 OCTET
pad = N*N"#" ; where N = paddedLength - originalLength - 2
@@ -383,9 +376,9 @@ paddedNotation = <padded(string, paddedLength)>
; paddedLength - required length after padding, including 2 bytes for originalLength
```
Each transmission/block for SMP v3 between the client and the server must have this format/syntax:
Each transmission between the client and the server must have this format/syntax:
```abnf
```
paddedTransmission = <padded(transmission), 16384>
transmission = [signature] SP signed
signed = sessionIdentifier SP [corrId] SP [queueId] SP smpCommand
@@ -402,23 +395,13 @@ encoded = <base64 encoded binary>
`base64` encoding should be used with padding, as defined in section 4 of [RFC 4648][9]
Transport block for SMP v4 has this syntax:
```abnf
paddedTransportBlock = <padded(transportBlock), 16384>
transportBlock = transmissionCount transmissions
transmissionCount = 1*1 OCTET ; equal or greater than 1
transmissions = transmissionLength transmission [transmissions]
transmissionLength = 2*2 OCTET ; word16 encoded in network byte order
```
## SMP commands
Commands syntax below is provided using [ABNF][8] with [case-sensitive strings extension][8a].
```abnf
smpCommand = ping / recipientCmd / send / subscribeNotifications / serverMsg
recipientCmd = create / subscribe / secure / enableNotifications / disableNotifications /
recipientCmd = create / subscribe / secure / enableNotifications /
acknowledge / suspend / delete
serverMsg = queueIds / message / notifierId / messageNotification /
unsubscribed / ok / error
@@ -519,42 +502,22 @@ Once the queue is secured only signed messages can be sent to it.
This command is sent by the recipient to the server to add notifier's key to the queue, to allow push notifications server to receive notifications when the message arrives, via a separate queue ID, without receiving message content.
```abnf
enableNotifications = %s"NKEY " notifierKey recipientNotificationDhPublicKey
enableNotifications = %s"NKEY " notifierKey
notifierKey = length x509encoded
; the notifier's Ed25519 or Ed448 public key public key to verify NSUB command for this queue
recipientNotificationDhPublicKey = length x509encoded
; the recipient's Curve25519 key for DH exchange to derive the secret
; that the server will use to encrypt notification metadata (encryptedNMsgMeta in NMSG)
; using [NaCl crypto_box][16] encryption scheme (curve25519xsalsa20poly1305).
```
The server will respond with `notifierId` response if notifications were enabled and the notifier's key was successfully added to the queue:
```abnf
notifierId = %s"NID " notifierId srvNotificationDhPublicKey
notifierId = %s"NID " notifierId
notifierId = length *OCTET ; 16-24 bytes
srvNotificationDhPublicKey = length x509encoded
; the server's Curve25519 key for DH exchange to derive the secret
; that the server will use to encrypt notification metadata to the recipient (encryptedNMsgMeta in NMSG)
```
This response is sent with the recipient's queue ID (the third part of the transmission).
To receive the message notifications, `subscribeNotifications` command ("NSUB") must be sent signed with the notifier's key.
#### Disable notifications command
This command is sent by the recipient to the server to remove notifier's credentials from the queue:
```abnf
disableNotifications = %s"NDEL"
```
The server must respond `ok` to this command if it was successful.
Once notifier's credentials are removed server will no longer send "NMSG" for this queue to notifier.
#### Acknowledge message delivery
The recipient should send the acknowledgement of message delivery once the message was stored in the client, to notify the server that the message should be deleted:
@@ -602,9 +565,7 @@ Currently SMP defines only one command that can be used by senders - `send` mess
This command is sent to the server by the sender both to confirm the queue after the sender received out-of-band message from the recipient and to send messages after the queue is secured:
```abnf
send = %s"SEND " msgFlags SP smpEncMessage
msgFlags = notificationFlag reserved
notificationFlag = %s"T" / %s"F"
send = %s"SEND " smpEncMessage
smpEncMessage = smpPubHeader sentMsgBody ; message up to 16088 bytes
smpPubHeader = smpClientVersion ("1" senderPublicDhKey / "0")
smpClientVersion = word16
@@ -746,11 +707,9 @@ See its syntax in [Create queue command](#create-queue-command)
The server must deliver messages to all subscribed simplex queues on the currently open transport connection. The syntax for the message delivery is:
```abnf
message = %s"MSG " msgId encryptedRcvMsgBody
message = %s"MSG " msgId SP timestamp SP encryptedMsgBody
encryptedMsgBody = <encrypt paddedSentMsgBody> ; server-encrypted padded sent msgBody
paddedSentMsgBody = <padded(sentMsgBody, maxMessageLength + 2)> ; maxMessageLength = 16088
encryptedRcvMsgBody = <encrypt rcvMsgBody> ; server-encrypted meta-data and padded sent msgBody
rcvMsgBody = timestamp msgFlags SP paddedSentMsgBody
msgId = length 24*24OCTET
timestamp = 8*8OCTET
```
@@ -776,19 +735,10 @@ See its syntax in [Enable notifications command](#enable-notifications-command)
The server must deliver message notifications to all simplex queues that were subscribed with `subscribeNotifications` command ("NSUB") on the currently open transport connection. The syntax for the message notification delivery is:
```abnf
messageNotification = %s"NMSG " nmsgNonce encryptedNMsgMeta
encryptedNMsgMeta = <encrypted message metadata passed in notification>
; metadata E2E encrypted between server and recipient containing server's message ID and timestamp (allows extension),
; to be passed to the recipient by the notifier for them to decrypt
; with key negotiated in NKEY and NID commands using nmsgNonce
nmsgNonce = <nonce used in NaCl crypto_box encryption scheme>
; nonce used by the server for encryption of message metadata, to be passed to the recipient by the notifier
; for them to use in decryption of E2E encrypted metadata
messageNotification = %s"NMSG"
```
Message notification does not contain any message data or non E2E encrypted metadata.
Message notification does not contain any message data or meta-data, it only notifies that the message is available.
#### Subscription END notification
@@ -842,7 +792,6 @@ ok = %s"OK"
Both the recipient and the sender can use TCP or some other, possibly higher level, transport protocol to communicate with the server. The default TCP port for SMP server is 5223.
For scenarios when meta-data privacy is critical, it is recommended that clients:
- communicating over Tor network,
- establish a separate connection for each SMP queue,
- send noise traffic (using PING command).
@@ -850,14 +799,12 @@ For scenarios when meta-data privacy is critical, it is recommended that clients
In addition to that, the servers can be deployed as Tor onion services.
The transport protocol should provide the following:
- server authentication (by matching server certificate hash with `serverIdentity`),
- forward secrecy (by encrypting the traffic using ephemeral keys agreed during transport handshake),
- integrity (preventing data modification by the attacker without detection),
- unique channel binding (`sessionIdentifier`) to include in the signed part of SMP transmissions.
By default, the client and server communicate using [TLS 1.3 protocol][13] restricted to:
- TLS_CHACHA20_POLY1305_SHA256 cipher suite (for better performance on mobile devices),
- ed25519 and ed448 EdDSA algorithms for signatures,
- x25519 and x448 ECDHE groups for key exchange.
-26
View File
@@ -1,26 +0,0 @@
# Accessing SMP servers via Tor
## Problem
While SMP protocol is focussed on minimizing application-level meta-data by using pair-wise identifiers instead of user profile identifiers, it is important for many users to protect their IP addresses.
Further, even if IP addresses are hidden by onion routing, clients should be able to choose to use a separate TCP connection to subscribe to each queue, even though it increases traffic and battery consumption, as otherwise the servers can observe multiple queues accessed by the same client.
## Solution and requirements
While some users may want to access SMP servers via tor, some other users (even their contacts) may want the opposite - e.g., if they use the network when accessing Tor would be suspicious (or blocked).
Therefore we need to support the connections when one of the user accesses the same server via Tor (and, possibly, via onion address), while another user accesses this server without Tor.
At the same time the user accessing the server via Tor may not want that their contacts access this server without Tor, and it also may be possible that the server is not available under a normal (not .onion) address.
The proposed options for connecting via Tor are:
1. Access servers via Socks proxy: no/yes (specify port?)
2. Use .onion addresses: no/when available/warn/always
3. Require senders to use .onion addresses: yes/no
4. Use separate TCP connection for each queue
While it should be possible for SMP servers to have two addresses (with and without Tor), the queues should only use one server address - if the queue started being accessed via .onion address it should not be possible to access it via a normal address. Queue addresses in connection invitations should support dual server addresses (when senders are not required ot use .onion address).
At the same time, the queue with the normal addresses can be accessed with and without Tor, depending on the current device settings.
-25
View File
@@ -1,25 +0,0 @@
# smp-server docker container
0. Install `docker` to your host.
1. Build your `smp-server` image:
- **Option 1** - Compile `smp-server` from source (stable branch):
```sh
DOCKER_BUILDKIT=1 docker build -t smp-server -f smp-server-build.Dockerfile .
```
- **Option 2** - Download latest `smp-server` from [latest Github release](https://github.com/simplex-chat/simplexmq/releases/latest):
```sh
DOCKER_BUILDKIT=1 docker build -t smp-server -f smp-server-download.Dockerfile .
```
2. Run new docker container:
```sh
docker run -d \
--name smp-server \
-e addr="your_ip_or_domain" \
-p 5223:5223 \
-v ${PWD}/config:/etc/opt/simplex \
-v ${PWD}/logs:/var/opt/simplex \
smp-server
```
Configuration files and logs will be written to [`config`](./config) and [`logs`](./logs) folders respectively.
-1
View File
@@ -1 +0,0 @@
smp-server configuration, certificate and fingerprint will be stored here
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env sh
confd="/etc/opt/simplex"
logd="/var/opt/simplex/"
# Check if server has been initialized
if [ ! -f "$confd/smp-server.ini" ]; then
# If not, determine ip or domain
case $addr in
'') printf "Please specify \$addr environment variable.\n"; exit 1 ;;
*[a-zA-Z]*) smp-server init -l -n "$addr" ;;
*) smp-server init -l --ip "$addr" ;;
esac
fi
# backup store log
[ -f "$logd/smp-server-store.log" ] && cp "$logd"/smp-server-store.log "$logd"/smp-server-store.log.bak
# rotate server log
[ -f "$logd/smp-server.log" ] && mv "$logd"/smp-server.log "$logd"/smp-server-"$(date +'%FT%T')".log
# Finally, run smp-sever. Notice that "exec" here is important:
# smp-server replaces our helper script, so that it can catch INT signal
exec smp-server start > "$logd"/smp-server.log 2>&1
-1
View File
@@ -1 +0,0 @@
smp-server general logs, stored messages and statistics (if enabled) will be stored here
@@ -1,53 +0,0 @@
FROM ubuntu:focal AS final
FROM ubuntu:focal AS build
### Build stage
# Install curl and git and smp-related dependencies
RUN apt-get update && apt-get install -y curl git build-essential libgmp3-dev zlib1g-dev
# Install ghcup
RUN curl https://downloads.haskell.org/~ghcup/x86_64-linux-ghcup -o /usr/bin/ghcup && \
chmod +x /usr/bin/ghcup
# Install ghc
RUN ghcup install ghc
# Install cabal
RUN ghcup install cabal
# Set both as default
RUN ghcup set ghc && \
ghcup set cabal
# Clone simplexmq repository
RUN git clone https://github.com/simplex-chat/simplexmq
# and cd to it
WORKDIR ./simplexmq
# Adjust PATH
ENV PATH="/root/.cabal/bin:/root/.ghcup/bin:$PATH"
# Compile smp-server
RUN cabal update
RUN cabal install
### Final stage
FROM final
# Install OpenSSL dependency
RUN apt-get update && apt-get install -y openssl
# Copy compiled smp-server from build stage
COPY --from=build /root/.cabal/bin/smp-server /usr/bin/smp-server
# Copy our helper script
COPY ./entrypoint /usr/bin/entrypoint
# Open smp-server listening port
EXPOSE 5223
# SimpleX requires using SIGINT to correctly preserve undelivered messages and restore them on restart
STOPSIGNAL SIGINT
# Finally, execute helper script
ENTRYPOINT [ "/usr/bin/entrypoint" ]
@@ -1,20 +0,0 @@
FROM ubuntu:focal
# Install curl
RUN apt-get update && apt-get install -y curl
# Download latest smp-server release and assign executable permission
RUN curl -L https://github.com/simplex-chat/simplexmq/releases/latest/download/smp-server-ubuntu-20_04-x86-64 -o /usr/bin/smp-server && \
chmod +x /usr/bin/smp-server
# Copy our helper script
COPY ./entrypoint /usr/bin/entrypoint
# Open smp-server listening port
EXPOSE 5223
# SimpleX requires using SIGINT to correctly preserve undelivered messages and restore them on restart
STOPSIGNAL SIGINT
# Finally, execute helper script
ENTRYPOINT [ "/usr/bin/entrypoint" ]
-178
View File
@@ -1,178 +0,0 @@
#!/bin/bash
# <UDF name="enable_store_log" label="Store log - persist notification subscriptions to append only log and restore them upon server restart." default="on" oneof="on, off" />
# <UDF name="api_token" label="Linode API token - enable Linode to create tags with server address, fingerprint and version. Note: minimal permissions token should have are read/write access to `linodes` (to create tags) and `domains` (to add A record for the third level domain if FQDN is provided)." default="" />
# <UDF name="fqdn" label="FQDN (Fully Qualified Domain Name) - provide third level domain name (e.g. ntf.example.com). If provided use `ntf://fingerprint@FQDN` as server address in the client. If FQDN is not provided use `ntf://fingerprint@IP` instead." default="" />
# <UDF name="apns_key_id" label="APNS key ID." default="" />
# Log all stdout output to stackscript.log
exec &> >(tee -i /var/log/stackscript.log)
# Uncomment next line to enable debugging features
# set -xeo pipefail
cd $HOME
# https://superuser.com/questions/1638779/automatic-yess-to-linux-update-upgrade
# https://superuser.com/questions/1412054/non-interactive-apt-upgrade
sudo DEBIAN_FRONTEND=noninteractive \
apt-get \
-o Dpkg::Options::=--force-confold \
-o Dpkg::Options::=--force-confdef \
-y --allow-downgrades --allow-remove-essential --allow-change-held-packages \
update
sudo DEBIAN_FRONTEND=noninteractive \
apt-get \
-o Dpkg::Options::=--force-confold \
-o Dpkg::Options::=--force-confdef \
-y --allow-downgrades --allow-remove-essential --allow-change-held-packages \
dist-upgrade
# TODO install unattended-upgrades
sudo DEBIAN_FRONTEND=noninteractive \
apt-get \
-o Dpkg::Options::=--force-confold \
-o Dpkg::Options::=--force-confdef \
-y --allow-downgrades --allow-remove-essential --allow-change-held-packages \
install jq
# Add firewall
echo "y" | ufw enable
# Open ports
ufw allow ssh
ufw allow https
ufw allow 5223
# Increase file descriptors limit
echo 'fs.file-max = 1000000' >> /etc/sysctl.conf
echo 'fs.inode-max = 1000000' >> /etc/sysctl.conf
echo 'root soft nofile unlimited' >> /etc/security/limits.conf
echo 'root hard nofile unlimited' >> /etc/security/limits.conf
# Download latest release
bin_dir="/opt/simplex-notifications/bin"
binary="$bin_dir/ntf-server"
mkdir -p $bin_dir
curl -L -o $binary https://github.com/simplex-chat/simplexmq/releases/latest/download/ntf-server-ubuntu-20_04-x86-64
chmod +x $binary
# / Add to PATH
cat > /etc/profile.d/simplex.sh << EOF
#!/bin/bash
export PATH="$PATH:$bin_dir"
EOF
# Add to PATH /
# Source and test PATH
source /etc/profile.d/simplex.sh
ntf-server --version
# Initialize server
init_opts=()
[[ $ENABLE_STORE_LOG == "on" ]] && init_opts+=(-l)
ip_address=$(curl ifconfig.me)
init_opts+=(--ip $ip_address)
[[ -n "$FQDN" ]] && init_opts+=(-n $FQDN)
ntf-server init "${init_opts[@]}"
# Server fingerprint
fingerprint=$(cat /etc/opt/simplex-notifications/fingerprint)
# Determine server address to specify in welcome script and Linode tag
if [[ -n "$FQDN" ]]; then
server_address=$FQDN
else
server_address=$ip_address
fi
# Set up welcome script
on_login_script="/opt/simplex-notifications/on_login.sh"
# / Welcome script
cat > $on_login_script << EOF
#!/bin/bash
fingerprint=\$1
server_address=\$2
cat << EOF2
********************************************************************************
SimpleX notifications server address: ntf://\$fingerprint@\$server_address
Check server status with: systemctl status ntf-server
To keep this server secure, the UFW firewall is enabled.
All ports are BLOCKED except 22 (SSH), 443 (HTTPS), 5223 (notifications server).
********************************************************************************
To stop seeing this message delete line - bash /opt/simplex-notifications/on_login.sh - from /root/.bashrc
EOF2
EOF
# Welcome script /
chmod +x $on_login_script
echo "bash $on_login_script $fingerprint $server_address" >> /root/.bashrc
# Create A record and update Linode's tags
if [[ -n "$API_TOKEN" ]]; then
if [[ -n "$FQDN" ]]; then
domain_address=$(echo $FQDN | rev | cut -d "." -f 1,2 | rev)
domain_id=$(curl -H "Authorization: Bearer $API_TOKEN" https://api.linode.com/v4/domains \
| jq --arg da "$domain_address" '.data[] | select( .domain == $da ) | .id')
if [[ -n $domain_id ]]; then
curl \
-s -H "Content-Type: application/json" \
-H "Authorization: Bearer $API_TOKEN" \
-X POST -d "{\"type\":\"A\",\"name\":\"$FQDN\",\"target\":\"$ip_address\"}" \
https://api.linode.com/v4/domains/${domain_id}/records
fi
fi
version=$(ntf-server --version | cut -d ' ' -f 3-)
curl \
-s -H "Content-Type: application/json" \
-H "Authorization: Bearer $API_TOKEN" \
-X PUT -d "{\"tags\":[\"$server_address\",\"$fingerprint\",\"$version\"]}" \
https://api.linode.com/v4/linode/instances/$LINODE_ID
fi
# / Create systemd service
cat > /etc/systemd/system/ntf-server.service << EOF
[Unit]
Description=SimpleX notifications server
[Service]
Environment="APNS_KEY_FILE=/etc/opt/simplex-notifications/AuthKey.p8"
Environment="APNS_KEY_ID=$APNS_KEY_ID"
Type=simple
ExecStart=/bin/sh -c "exec $binary start >> /var/opt/simplex-notifications/ntf-server.log 2>&1"
KillSignal=SIGINT
Restart=always
RestartSec=10
LimitNOFILE=1000000
LimitNOFILESoft=1000000
[Install]
WantedBy=multi-user.target
EOF
# Create systemd service /
# Start systemd service
chmod 644 /etc/systemd/system/ntf-server.service
sudo systemctl enable ntf-server
# ! APNS key file and certificate have to be copied manually
# sudo systemctl start ntf-server
# Reboot Linode to apply upgrades
# sudo reboot
+4 -1
View File
@@ -86,6 +86,10 @@ smp-server init "${init_opts[@]}"
fingerprint=$(cat /etc/opt/simplex/fingerprint)
# Determine server address to specify in welcome script and Linode tag
# ! If FQDN was provided and used as part of server initialization, server's certificate will not pass validation at client
# ! if client tries to connect by server's IP address, so we have to specify FQDN as server address in Linode tag and
# ! in welcome script regardless of creation of A record in Linode
# ! https://hackage.haskell.org/package/x509-validation-1.6.10/docs/src/Data-X509-Validation.html#validateCertificateName
if [[ -n "$FQDN" ]]; then
server_address=$FQDN
else
@@ -154,7 +158,6 @@ Description=SMP server
Type=simple
ExecStart=/bin/sh -c "exec $binary start >> /var/opt/simplex/smp-server.log 2>&1"
KillSignal=SIGINT
TimeoutStopSec=infinity
Restart=always
RestartSec=10
LimitNOFILE=1000000
+11 -17
View File
@@ -5,7 +5,7 @@ cabal-version: 1.12
-- see: https://github.com/sol/hpack
name: simplexmq
version: 3.2.0
version: 2.3.1
synopsis: SimpleXMQ message broker
description: This package includes <./docs/Simplex-Messaging-Server.html server>,
<./docs/Simplex-Messaging-Client.html client> and
@@ -49,8 +49,6 @@ library
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220301_snd_queue_keys
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220322_notifications
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220608_v2
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220625_v2_ntf_mode
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220811_onion_hosts
Simplex.Messaging.Client
Simplex.Messaging.Client.Agent
Simplex.Messaging.Crypto
@@ -62,11 +60,8 @@ library
Simplex.Messaging.Notifications.Server
Simplex.Messaging.Notifications.Server.Env
Simplex.Messaging.Notifications.Server.Push.APNS
Simplex.Messaging.Notifications.Server.Stats
Simplex.Messaging.Notifications.Server.Store
Simplex.Messaging.Notifications.Server.StoreLog
Simplex.Messaging.Notifications.Transport
Simplex.Messaging.Notifications.Types
Simplex.Messaging.Parsers
Simplex.Messaging.Protocol
Simplex.Messaging.Server
@@ -77,7 +72,6 @@ library
Simplex.Messaging.Server.MsgStore.STM
Simplex.Messaging.Server.QueueStore
Simplex.Messaging.Server.QueueStore.STM
Simplex.Messaging.Server.Stats
Simplex.Messaging.Server.StoreLog
Simplex.Messaging.TMap
Simplex.Messaging.Transport
@@ -103,7 +97,7 @@ library
, asn1-types ==0.3.*
, async ==2.2.*
, attoparsec ==0.14.*
, base >=4.14 && <5
, base >=4.7 && <5
, base64-bytestring >=1.0 && <1.3
, bytestring ==0.10.*
, case-insensitive ==1.2.*
@@ -129,7 +123,6 @@ library
, process ==1.6.*
, random >=1.1 && <1.3
, simple-logger ==0.1.*
, socks ==0.6.*
, sqlite-simple ==0.4.*
, stm ==2.5.*
, template-haskell ==2.16.*
@@ -139,6 +132,7 @@ library
, time-manager ==0.0.*
, tls >=1.6.0 && <1.7
, transformers ==0.5.*
, unix ==2.7.*
, unliftio ==0.2.*
, unliftio-core ==0.2.*
, websockets ==0.12.*
@@ -164,7 +158,7 @@ executable ntf-server
, asn1-types ==0.3.*
, async ==2.2.*
, attoparsec ==0.14.*
, base >=4.14 && <5
, base >=4.7 && <5
, base64-bytestring >=1.0 && <1.3
, bytestring ==0.10.*
, case-insensitive ==1.2.*
@@ -191,7 +185,6 @@ executable ntf-server
, random >=1.1 && <1.3
, simple-logger ==0.1.*
, simplexmq
, socks ==0.6.*
, sqlite-simple ==0.4.*
, stm ==2.5.*
, template-haskell ==2.16.*
@@ -201,6 +194,7 @@ executable ntf-server
, time-manager ==0.0.*
, tls >=1.6.0 && <1.7
, transformers ==0.5.*
, unix ==2.7.*
, unliftio ==0.2.*
, unliftio-core ==0.2.*
, websockets ==0.12.*
@@ -226,7 +220,7 @@ executable smp-agent
, asn1-types ==0.3.*
, async ==2.2.*
, attoparsec ==0.14.*
, base >=4.14 && <5
, base >=4.7 && <5
, base64-bytestring >=1.0 && <1.3
, bytestring ==0.10.*
, case-insensitive ==1.2.*
@@ -253,7 +247,6 @@ executable smp-agent
, random >=1.1 && <1.3
, simple-logger ==0.1.*
, simplexmq
, socks ==0.6.*
, sqlite-simple ==0.4.*
, stm ==2.5.*
, template-haskell ==2.16.*
@@ -263,6 +256,7 @@ executable smp-agent
, time-manager ==0.0.*
, tls >=1.6.0 && <1.7
, transformers ==0.5.*
, unix ==2.7.*
, unliftio ==0.2.*
, unliftio-core ==0.2.*
, websockets ==0.12.*
@@ -288,7 +282,7 @@ executable smp-server
, asn1-types ==0.3.*
, async ==2.2.*
, attoparsec ==0.14.*
, base >=4.14 && <5
, base >=4.7 && <5
, base64-bytestring >=1.0 && <1.3
, bytestring ==0.10.*
, case-insensitive ==1.2.*
@@ -315,7 +309,6 @@ executable smp-server
, random >=1.1 && <1.3
, simple-logger ==0.1.*
, simplexmq
, socks ==0.6.*
, sqlite-simple ==0.4.*
, stm ==2.5.*
, template-haskell ==2.16.*
@@ -325,6 +318,7 @@ executable smp-server
, time-manager ==0.0.*
, tls >=1.6.0 && <1.7
, transformers ==0.5.*
, unix ==2.7.*
, unliftio ==0.2.*
, unliftio-core ==0.2.*
, websockets ==0.12.*
@@ -367,7 +361,7 @@ test-suite smp-server-test
, asn1-types ==0.3.*
, async ==2.2.*
, attoparsec ==0.14.*
, base >=4.14 && <5
, base >=4.7 && <5
, base64-bytestring >=1.0 && <1.3
, bytestring ==0.10.*
, case-insensitive ==1.2.*
@@ -396,7 +390,6 @@ test-suite smp-server-test
, random >=1.1 && <1.3
, simple-logger ==0.1.*
, simplexmq
, socks ==0.6.*
, sqlite-simple ==0.4.*
, stm ==2.5.*
, template-haskell ==2.16.*
@@ -407,6 +400,7 @@ test-suite smp-server-test
, timeit ==2.0.*
, tls >=1.6.0 && <1.7
, transformers ==0.5.*
, unix ==2.7.*
, unliftio ==0.2.*
, unliftio-core ==0.2.*
, websockets ==0.12.*
File diff suppressed because it is too large Load Diff
+192 -366
View File
@@ -11,7 +11,6 @@
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TupleSections #-}
{-# LANGUAGE TypeApplications #-}
module Simplex.Messaging.Agent.Client
@@ -19,30 +18,24 @@ module Simplex.Messaging.Agent.Client
newAgentClient,
withAgentLock,
closeAgentClient,
closeProtocolServerClients,
newRcvQueue,
subscribeQueue,
subscribeQueues,
getQueueMessage,
decryptSMPMessage,
addSubscription,
getSubscriptions,
sendConfirmation,
sendInvitation,
temporaryAgentError,
RetryInterval (..),
secureQueue,
enableQueueNotifications,
disableQueueNotifications,
sendAgentMessage,
agentNtfRegisterToken,
agentNtfVerifyToken,
agentNtfCheckToken,
agentNtfReplaceToken,
agentNtfDeleteToken,
agentNtfEnableCron,
agentNtfCreateSubscription,
agentNtfCheckSubscription,
agentNtfDeleteSubscription,
agentCbEncrypt,
agentCbDecrypt,
cryptoError,
@@ -53,20 +46,10 @@ module Simplex.Messaging.Agent.Client
removeSubscription,
hasActiveSubscription,
agentDbPath,
AgentOperation (..),
AgentOpState (..),
AgentState (..),
agentOperations,
agentOperationBracket,
beginAgentOperation,
endAgentOperation,
suspendSendingAndDatabase,
suspendOperation,
notifySuspended,
whenSuspending,
notifyAgentPhaseChanged,
withStore,
withStore',
storeError,
)
where
@@ -77,64 +60,38 @@ import Control.Logger.Simple
import Control.Monad.Except
import Control.Monad.IO.Unlift
import Control.Monad.Reader
import Data.Bifunctor (bimap, first, second)
import Data.Bifunctor (first)
import Data.ByteString.Base64
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Either (partitionEithers)
import Data.List.NonEmpty (NonEmpty)
import qualified Data.List.NonEmpty as L
import Data.Map.Strict (Map)
import qualified Data.Map.Strict as M
import Data.Maybe (listToMaybe)
import Data.Maybe (catMaybes)
import Data.Set (Set)
import Data.Text.Encoding
import Data.Tuple (swap)
import Data.Word (Word16)
import qualified Database.SQLite.Simple as DB
import Database.SQLite.Simple (SQLError)
import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Agent.RetryInterval
import Simplex.Messaging.Agent.Store
import Simplex.Messaging.Agent.Store.SQLite (SQLiteStore (..), withTransaction)
import Simplex.Messaging.Agent.Store.SQLite (AgentStoreMonad, SQLiteStore (..))
import Simplex.Messaging.Client
import Simplex.Messaging.Client.Agent ()
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Client
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Types
import Simplex.Messaging.Parsers (parse)
import Simplex.Messaging.Protocol
( AProtocolType (..),
BrokerMsg,
ErrorType,
MsgFlags (..),
MsgId,
NotifierId,
NtfPrivateSignKey,
NtfPublicVerifyKey,
NtfServer,
ProtoServer,
Protocol (..),
ProtocolServer (..),
ProtocolTypeI (..),
QueueId,
QueueIdsKeys (..),
RcvMessage (..),
RcvNtfPublicDhKey,
SMPMsgMeta (..),
SndPublicVerifyKey,
)
import Simplex.Messaging.Protocol (BrokerMsg, ErrorType, MsgFlags (..), MsgId, NotifierId, NtfPrivateSignKey, NtfPublicVerifyKey, ProtocolServer (..), QueueId, QueueIdsKeys (..), SndPublicVerifyKey)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport.Client (TransportHost)
import Simplex.Messaging.Util
import Simplex.Messaging.Util (bshow, catchAll_, ifM, liftEitherError, liftError, tryError, unlessM, whenM)
import Simplex.Messaging.Version
import System.Posix.Semaphore (semPost, semThreadWait)
import System.Timeout (timeout)
import UnliftIO (async)
import UnliftIO (async, pooledForConcurrentlyN)
import qualified UnliftIO.Exception as E
import UnliftIO.STM
@@ -153,20 +110,13 @@ data AgentClient = AgentClient
smpClients :: TMap SMPServer SMPClientVar,
ntfServers :: TVar [NtfServer],
ntfClients :: TMap NtfServer NtfClientVar,
useNetworkConfig :: TVar NetworkConfig,
subscrSrvrs :: TMap SMPServer (TMap ConnId RcvQueue),
pendingSubscrSrvrs :: TMap SMPServer (TMap ConnId RcvQueue),
subscrConns :: TMap ConnId SMPServer,
connMsgsQueued :: TMap ConnId Bool,
smpQueueMsgQueues :: TMap (ConnId, SMPServer, SMP.SenderId) (TQueue InternalId),
smpQueueMsgDeliveries :: TMap (ConnId, SMPServer, SMP.SenderId) (Async ()),
ntfNetworkOp :: TVar AgentOpState,
rcvNetworkOp :: TVar AgentOpState,
msgDeliveryOp :: TVar AgentOpState,
sndNetworkOp :: TVar AgentOpState,
databaseOp :: TVar AgentOpState,
agentState :: TVar AgentState,
getMsgLocks :: TMap (SMPServer, SMP.RecipientId) (TMVar ()),
getMsgLocks :: TMap (ConnId, SMPServer, SMP.RecipientId) (TMVar ()),
reconnections :: TVar [Async ()],
asyncClients :: TVar [Async ()],
clientId :: Int,
@@ -174,27 +124,8 @@ data AgentClient = AgentClient
lock :: TMVar ()
}
data AgentOperation = AONtfNetwork | AORcvNetwork | AOMsgDelivery | AOSndNetwork | AODatabase
deriving (Eq, Show)
agentOpSel :: AgentOperation -> (AgentClient -> TVar AgentOpState)
agentOpSel = \case
AONtfNetwork -> ntfNetworkOp
AORcvNetwork -> rcvNetworkOp
AOMsgDelivery -> msgDeliveryOp
AOSndNetwork -> sndNetworkOp
AODatabase -> databaseOp
agentOperations :: [AgentClient -> TVar AgentOpState]
agentOperations = [ntfNetworkOp, rcvNetworkOp, msgDeliveryOp, sndNetworkOp, databaseOp]
data AgentOpState = AgentOpState {opSuspended :: Bool, opsInProgress :: Int}
data AgentState = ASActive | ASSuspending | ASSuspended
deriving (Eq, Show)
newAgentClient :: InitialAgentServers -> Env -> STM AgentClient
newAgentClient InitialAgentServers {smp, ntf, netCfg} agentEnv = do
newAgentClient InitialAgentServers {smp, ntf} agentEnv = do
let qSize = tbqSize $ config agentEnv
active <- newTVar True
rcvQ <- newTBQueue qSize
@@ -204,31 +135,24 @@ newAgentClient InitialAgentServers {smp, ntf, netCfg} agentEnv = do
smpClients <- TM.empty
ntfServers <- newTVar ntf
ntfClients <- TM.empty
useNetworkConfig <- newTVar netCfg
subscrSrvrs <- TM.empty
pendingSubscrSrvrs <- TM.empty
subscrConns <- TM.empty
connMsgsQueued <- TM.empty
smpQueueMsgQueues <- TM.empty
smpQueueMsgDeliveries <- TM.empty
ntfNetworkOp <- newTVar $ AgentOpState False 0
rcvNetworkOp <- newTVar $ AgentOpState False 0
msgDeliveryOp <- newTVar $ AgentOpState False 0
sndNetworkOp <- newTVar $ AgentOpState False 0
databaseOp <- newTVar $ AgentOpState False 0
agentState <- newTVar ASActive
getMsgLocks <- TM.empty
reconnections <- newTVar []
asyncClients <- newTVar []
clientId <- stateTVar (clientCounter agentEnv) $ \i -> let i' = i + 1 in (i', i')
clientId <- stateTVar (clientCounter agentEnv) $ \i -> (i + 1, i + 1)
lock <- newTMVar ()
return AgentClient {active, rcvQ, subQ, msgQ, smpServers, smpClients, ntfServers, ntfClients, useNetworkConfig, subscrSrvrs, pendingSubscrSrvrs, subscrConns, connMsgsQueued, smpQueueMsgQueues, smpQueueMsgDeliveries, ntfNetworkOp, rcvNetworkOp, msgDeliveryOp, sndNetworkOp, databaseOp, agentState, getMsgLocks, reconnections, asyncClients, clientId, agentEnv, lock}
return AgentClient {active, rcvQ, subQ, msgQ, smpServers, smpClients, ntfServers, ntfClients, subscrSrvrs, pendingSubscrSrvrs, subscrConns, connMsgsQueued, smpQueueMsgQueues, smpQueueMsgDeliveries, getMsgLocks, reconnections, asyncClients, clientId, agentEnv, lock}
agentDbPath :: AgentClient -> FilePath
agentDbPath AgentClient {agentEnv = Env {store = SQLiteStore {dbFilePath}}} = dbFilePath
class ProtocolServerClient msg where
getProtocolServerClient :: AgentMonad m => AgentClient -> ProtoServer msg -> m (ProtocolClient msg)
getProtocolServerClient :: AgentMonad m => AgentClient -> ProtocolServer -> m (ProtocolClient msg)
clientProtocolError :: ErrorType -> AgentErrorType
instance ProtocolServerClient BrokerMsg where
@@ -245,72 +169,83 @@ getSMPServerClient c@AgentClient {active, smpClients, msgQ} srv = do
atomically (getClientVar srv smpClients)
>>= either
(newProtocolClient c srv smpClients connectClient reconnectClient)
(waitForProtocolClient c)
(waitForProtocolClient smpCfg)
where
connectClient :: m SMPClient
connectClient = do
cfg <- atomically . updateClientConfig c =<< asks (smpCfg . config)
cfg <- asks $ smpCfg . config
u <- askUnliftIO
liftEitherError (protocolClientError SMP) (getProtocolClient srv cfg (Just msgQ) $ clientDisconnected u)
clientDisconnected :: UnliftIO m -> SMPClient -> IO ()
clientDisconnected u client = do
removeClientAndSubs >>= (`forM_` serverDown)
clientDisconnected :: UnliftIO m -> IO ()
clientDisconnected u = do
removeClientAndSubs >>= (`forM_` serverDown u)
logInfo . decodeUtf8 $ "Agent disconnected from " <> showServer srv
removeClientAndSubs :: IO (Maybe (Map ConnId RcvQueue))
removeClientAndSubs = atomically $ do
TM.delete srv smpClients
TM.lookupDelete srv (subscrSrvrs c) >>= mapM updateSubs
where
removeClientAndSubs :: IO (Maybe (Map ConnId RcvQueue))
removeClientAndSubs = atomically $ do
TM.delete srv smpClients
TM.lookupDelete srv (subscrSrvrs c) >>= mapM updateSubs
where
updateSubs cVar = do
cs <- readTVar cVar
modifyTVar' (subscrConns c) (`M.withoutKeys` M.keysSet cs)
addPendingSubs cVar cs
pure cs
updateSubs cVar = do
cs <- readTVar cVar
modifyTVar' (subscrConns c) (`M.withoutKeys` M.keysSet cs)
addPendingSubs cVar cs
pure cs
addPendingSubs cVar cs = do
let ps = pendingSubscrSrvrs c
TM.lookup srv ps >>= \case
Just v -> TM.union cs v
_ -> TM.insert srv cVar ps
addPendingSubs cVar cs = do
let ps = pendingSubscrSrvrs c
TM.lookup srv ps >>= \case
Just v -> TM.union cs v
_ -> TM.insert srv cVar ps
serverDown :: Map ConnId RcvQueue -> IO ()
serverDown cs = unless (M.null cs) $
whenM (readTVarIO active) $ do
let conns = M.keys cs
notifySub "" $ hostEvent DISCONNECT client
unless (null conns) . notifySub "" $ DOWN srv conns
atomically $ mapM_ (releaseGetLock c) cs
unliftIO u reconnectServer
serverDown :: UnliftIO m -> Map ConnId RcvQueue -> IO ()
serverDown u cs = unless (M.null cs) $
whenM (readTVarIO active) $ do
let conns = M.keys cs
unless (null conns) . notifySub "" $ DOWN srv conns
unliftIO u reconnectServer
reconnectServer :: m ()
reconnectServer = do
a <- async tryReconnectClient
atomically $ modifyTVar' (reconnections c) (a :)
reconnectServer :: m ()
reconnectServer = do
a <- async tryReconnectClient
atomically $ modifyTVar' (reconnections c) (a :)
tryReconnectClient :: m ()
tryReconnectClient = do
ri <- asks $ reconnectInterval . config
withRetryInterval ri $ \loop ->
reconnectClient `catchError` const loop
tryReconnectClient :: m ()
tryReconnectClient = do
ri <- asks $ reconnectInterval . config
withRetryInterval ri $ \loop ->
reconnectClient `catchError` const loop
reconnectClient :: m ()
reconnectClient =
withAgentLock c $
atomically (TM.lookup srv (pendingSubscrSrvrs c) >>= mapM readTVar)
>>= mapM_ resubscribe
reconnectClient = do
n <- asks $ resubscriptionConcurrency . config
withAgentLock c . withClient c srv $ \smp -> do
cs <- atomically $ mapM readTVar =<< TM.lookup srv (pendingSubscrSrvrs c)
-- TODO if any of the subscriptions fails here (e.g. because of timeout), it terminates the whole process for all subscriptions
-- instead it should only report successful subscriptions and schedule the next call to reconnectClient to subscribe for the remaining subscriptions
-- this way, for each DOWN event there can be several UP events
conns <- pooledForConcurrentlyN n (maybe [] M.toList cs) $ \sub@(connId, _) ->
ifM
(atomically $ hasActiveSubscription c connId)
(pure $ Just connId)
(subscribe_ smp sub `catchError` handleError connId)
liftIO . unless (null conns) . notifySub "" . UP srv $ catMaybes conns
where
resubscribe :: Map ConnId RcvQueue -> m ()
resubscribe qs = do
(client_, (errs, oks)) <- second (M.mapEither id) <$> subscribeQueues c srv qs
liftIO $ do
mapM_ (notifySub "" . hostEvent CONNECT) client_
unless (M.null oks) $ do
notifySub "" . UP srv $ M.keys oks
let (tempErrs, finalErrs) = M.partition temporaryAgentError errs
liftIO . mapM_ (\(connId, e) -> notifySub connId $ ERR e) $ M.assocs finalErrs
mapM_ throwError . listToMaybe $ M.elems tempErrs
subscribe_ :: SMPClient -> (ConnId, RcvQueue) -> ExceptT ProtocolClientError IO (Maybe ConnId)
subscribe_ smp (connId, rq@RcvQueue {rcvPrivateKey, rcvId}) = do
subscribeSMPQueue smp rcvPrivateKey rcvId
addSubscription c rq connId
pure $ Just connId
handleError :: ConnId -> ProtocolClientError -> ExceptT ProtocolClientError IO (Maybe ConnId)
handleError connId = \case
e@PCEResponseTimeout -> throwError e
e@PCENetworkError -> throwError e
e -> do
liftIO . notifySub connId . ERR $ protocolClientError SMP e
atomically $ removePendingSubscription c srv connId
pure Nothing
notifySub :: ConnId -> ACommand 'Agent -> IO ()
notifySub connId cmd = atomically $ writeTBQueue (subQ c) ("", connId, cmd)
@@ -321,20 +256,19 @@ getNtfServerClient c@AgentClient {active, ntfClients} srv = do
atomically (getClientVar srv ntfClients)
>>= either
(newProtocolClient c srv ntfClients connectClient $ pure ())
(waitForProtocolClient c)
(waitForProtocolClient ntfCfg)
where
connectClient :: m NtfClient
connectClient = do
cfg <- atomically . updateClientConfig c =<< asks (ntfCfg . config)
cfg <- asks $ ntfCfg . config
liftEitherError (protocolClientError NTF) (getProtocolClient srv cfg Nothing clientDisconnected)
clientDisconnected :: NtfClient -> IO ()
clientDisconnected client = do
clientDisconnected :: IO ()
clientDisconnected = do
atomically $ TM.delete srv ntfClients
atomically $ writeTBQueue (subQ c) ("", "", hostEvent DISCONNECT client)
logInfo . decodeUtf8 $ "Agent disconnected from " <> showServer srv
getClientVar :: forall a s. ProtocolServer s -> TMap (ProtocolServer s) (TMVar a) -> STM (Either (TMVar a) (TMVar a))
getClientVar :: forall a. ProtocolServer -> TMap ProtocolServer (TMVar a) -> STM (Either (TMVar a) (TMVar a))
getClientVar srv clients = maybe (Left <$> newClientVar) (pure . Right) =<< TM.lookup srv clients
where
newClientVar :: STM (TMVar a)
@@ -343,10 +277,10 @@ getClientVar srv clients = maybe (Left <$> newClientVar) (pure . Right) =<< TM.l
TM.insert srv var clients
pure var
waitForProtocolClient :: AgentMonad m => AgentClient -> ClientVar msg -> m (ProtocolClient msg)
waitForProtocolClient c clientVar = do
NetworkConfig {tcpConnectTimeout} <- readTVarIO $ useNetworkConfig c
client_ <- liftIO $ tcpConnectTimeout `timeout` atomically (readTMVar clientVar)
waitForProtocolClient :: AgentMonad m => (AgentConfig -> ProtocolClientConfig) -> ClientVar msg -> m (ProtocolClient msg)
waitForProtocolClient clientConfig clientVar = do
ProtocolClientConfig {tcpTimeout} <- asks $ clientConfig . config
client_ <- liftIO $ tcpTimeout `timeout` atomically (readTMVar clientVar)
liftEither $ case client_ of
Just (Right smpClient) -> Right smpClient
Just (Left e) -> Left e
@@ -354,10 +288,10 @@ waitForProtocolClient c clientVar = do
newProtocolClient ::
forall msg m.
(AgentMonad m, ProtocolTypeI (ProtoType msg)) =>
AgentMonad m =>
AgentClient ->
ProtoServer msg ->
TMap (ProtoServer msg) (ClientVar msg) ->
ProtocolServer ->
TMap ProtocolServer (ClientVar msg) ->
m (ProtocolClient msg) ->
m () ->
ClientVar msg ->
@@ -370,10 +304,9 @@ newProtocolClient c srv clients connectClient reconnectClient clientVar = tryCon
Right client -> do
logInfo . decodeUtf8 $ "Agent connected to " <> showServer srv
atomically $ putTMVar clientVar r
atomically $ writeTBQueue (subQ c) ("", "", hostEvent CONNECT client)
successAction client
Left e -> do
if temporaryAgentError e
if e == BROKER NETWORK || e == BROKER TIMEOUT
then retryAction
else atomically $ do
putTMVar clientVar (Left e)
@@ -388,19 +321,11 @@ newProtocolClient c srv clients connectClient reconnectClient clientVar = tryCon
ri <- asks $ reconnectInterval . config
withRetryInterval ri $ \loop -> void $ tryConnectClient (const reconnectClient) loop
hostEvent :: forall msg. ProtocolTypeI (ProtoType msg) => (AProtocolType -> TransportHost -> ACommand 'Agent) -> ProtocolClient msg -> ACommand 'Agent
hostEvent event client = event (AProtocolType $ protocolTypeI @(ProtoType msg)) $ transportHost client
updateClientConfig :: AgentClient -> ProtocolClientConfig -> STM ProtocolClientConfig
updateClientConfig AgentClient {useNetworkConfig} cfg = do
networkConfig <- readTVar useNetworkConfig
pure cfg {networkConfig}
closeAgentClient :: MonadIO m => AgentClient -> m ()
closeAgentClient c = liftIO $ do
atomically $ writeTVar (active c) False
closeProtocolServerClients c smpClients
closeProtocolServerClients c ntfClients
closeProtocolServerClients (clientTimeout smpCfg) $ smpClients c
closeProtocolServerClients (clientTimeout ntfCfg) $ ntfClients c
cancelActions $ reconnections c
cancelActions $ asyncClients c
cancelActions $ smpQueueMsgDeliveries c
@@ -409,19 +334,16 @@ closeAgentClient c = liftIO $ do
clear subscrConns
clear connMsgsQueued
clear smpQueueMsgQueues
clear getMsgLocks
where
clientTimeout sel = tcpTimeout . sel . config $ agentEnv c
clear :: (AgentClient -> TMap k a) -> IO ()
clear sel = atomically $ writeTVar (sel c) M.empty
closeProtocolServerClients :: AgentClient -> (AgentClient -> TMap (ProtoServer msg) (ClientVar msg)) -> IO ()
closeProtocolServerClients c clientsSel =
readTVarIO cs >>= mapM_ (forkIO . closeClient) >> atomically (writeTVar cs M.empty)
closeProtocolServerClients :: Int -> TMap ProtocolServer (ClientVar msg) -> IO ()
closeProtocolServerClients tcpTimeout cs = readTVarIO cs >>= mapM_ (forkIO . closeClient) >> atomically (writeTVar cs M.empty)
where
cs = clientsSel c
closeClient cVar = do
NetworkConfig {tcpConnectTimeout} <- readTVarIO $ useNetworkConfig c
tcpConnectTimeout `timeout` atomically (readTMVar cVar) >>= \case
closeClient cVar =
tcpTimeout `timeout` atomically (readTMVar cVar) >>= \case
Just (Right client) -> closeProtocolClient client `catchAll_` pure ()
_ -> pure ()
@@ -434,7 +356,7 @@ withAgentLock AgentClient {lock} =
(void . atomically $ takeTMVar lock)
(atomically $ putTMVar lock ())
withClient_ :: forall a m msg. (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtoServer msg -> (ProtocolClient msg -> m a) -> m a
withClient_ :: forall a m msg. (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtocolServer -> (ProtocolClient msg -> m a) -> m a
withClient_ c srv action = (getProtocolServerClient c srv >>= action) `catchError` logServerError
where
logServerError :: AgentErrorType -> m a
@@ -442,17 +364,17 @@ withClient_ c srv action = (getProtocolServerClient c srv >>= action) `catchErro
logServer "<--" c srv "" $ bshow e
throwError e
withLogClient_ :: (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtoServer msg -> QueueId -> ByteString -> (ProtocolClient msg -> m a) -> m a
withLogClient_ :: (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtocolServer -> QueueId -> ByteString -> (ProtocolClient msg -> m a) -> m a
withLogClient_ c srv qId cmdStr action = do
logServer "-->" c srv qId cmdStr
res <- withClient_ c srv action
logServer "<--" c srv qId "OK"
return res
withClient :: forall m msg a. (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtoServer msg -> (ProtocolClient msg -> ExceptT ProtocolClientError IO a) -> m a
withClient :: forall m msg a. (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtocolServer -> (ProtocolClient msg -> ExceptT ProtocolClientError IO a) -> m a
withClient c srv action = withClient_ c srv $ liftClient (clientProtocolError @msg) . action
withLogClient :: forall m msg a. (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtoServer msg -> QueueId -> ByteString -> (ProtocolClient msg -> ExceptT ProtocolClientError IO a) -> m a
withLogClient :: forall m msg a. (AgentMonad m, ProtocolServerClient msg) => AgentClient -> ProtocolServer -> QueueId -> ByteString -> (ProtocolClient msg -> ExceptT ProtocolClientError IO a) -> m a
withLogClient c srv qId cmdStr action = withLogClient_ c srv qId cmdStr $ liftClient (clientProtocolError @msg) . action
liftClient :: AgentMonad m => (ErrorType -> AgentErrorType) -> ExceptT ProtocolClientError IO a -> m a
@@ -462,27 +384,25 @@ protocolClientError :: (ErrorType -> AgentErrorType) -> ProtocolClientError -> A
protocolClientError protocolError_ = \case
PCEProtocolError e -> protocolError_ e
PCEResponseError e -> BROKER $ RESPONSE e
PCEUnexpectedResponse _ -> BROKER UNEXPECTED
PCEUnexpectedResponse -> BROKER UNEXPECTED
PCEResponseTimeout -> BROKER TIMEOUT
PCENetworkError -> BROKER NETWORK
PCEIncompatibleHost -> BROKER HOST
PCETransportError e -> BROKER $ TRANSPORT e
e@PCESignatureError {} -> INTERNAL $ show e
e@PCEIOError {} -> INTERNAL $ show e
newRcvQueue :: AgentMonad m => AgentClient -> SMPServer -> VersionRange -> m (RcvQueue, SMPQueueUri)
newRcvQueue c srv vRange =
newRcvQueue :: AgentMonad m => AgentClient -> SMPServer -> m (RcvQueue, SMPQueueUri)
newRcvQueue c srv =
asks (cmdSignAlg . config) >>= \case
C.SignAlg a -> newRcvQueue_ a c srv vRange
C.SignAlg a -> newRcvQueue_ a c srv
newRcvQueue_ ::
(C.SignatureAlgorithm a, C.AlgorithmI a, AgentMonad m) =>
C.SAlgorithm a ->
AgentClient ->
SMPServer ->
VersionRange ->
m (RcvQueue, SMPQueueUri)
newRcvQueue_ a c srv vRange = do
newRcvQueue_ a c srv = do
(recipientKey, rcvPrivateKey) <- liftIO $ C.generateSignatureKeyPair a
(dhKey, privDhKey) <- liftIO C.generateKeyPair'
(e2eDhKey, e2ePrivKey) <- liftIO C.generateKeyPair'
@@ -500,63 +420,24 @@ newRcvQueue_ a c srv vRange = do
e2eDhSecret = Nothing,
sndId = Just sndId,
status = New,
smpClientVersion = maxVersion vRange,
clientNtfCreds = Nothing
ntfPublicKey = Nothing,
ntfPrivateKey = Nothing,
notifierId = Nothing
}
pure (rq, SMPQueueUri vRange $ SMPQueueAddress srv sndId e2eDhKey)
pure (rq, SMPQueueUri srv sndId SMP.smpClientVRange e2eDhKey)
subscribeQueue :: AgentMonad m => AgentClient -> RcvQueue -> ConnId -> m ()
subscribeQueue c rq@RcvQueue {server, rcvPrivateKey, rcvId} connId = do
whenM (atomically . TM.member (server, rcvId) $ getMsgLocks c) . throwError $ CMD PROHIBITED
whenM (atomically . TM.member (connId, server, rcvId) $ getMsgLocks c) . throwError $ CMD PROHIBITED
atomically $ addPendingSubscription c rq connId
withLogClient c server rcvId "SUB" $ \smp ->
liftIO (runExceptT (subscribeSMPQueue smp rcvPrivateKey rcvId) >>= processSubResult c rq connId)
>>= either throwError pure
processSubResult :: AgentClient -> RcvQueue -> ConnId -> Either ProtocolClientError () -> IO (Either ProtocolClientError ())
processSubResult c rq@RcvQueue {server} connId r = do
case r of
Left e ->
atomically . unless (temporaryClientError e) $
removePendingSubscription c server connId
_ -> addSubscription c rq connId
pure r
temporaryClientError :: ProtocolClientError -> Bool
temporaryClientError = \case
PCENetworkError -> True
PCEResponseTimeout -> True
_ -> False
temporaryAgentError :: AgentErrorType -> Bool
temporaryAgentError = \case
BROKER NETWORK -> True
BROKER TIMEOUT -> True
_ -> False
-- | subscribe multiple queues - all passed queues should be on the same server
subscribeQueues :: AgentMonad m => AgentClient -> SMPServer -> Map ConnId RcvQueue -> m (Maybe SMPClient, Map ConnId (Either AgentErrorType ()))
subscribeQueues c srv qs = do
(errs, qs_) <- partitionEithers <$> mapM checkQueue (M.assocs qs)
forM_ qs_ $ atomically . uncurry (addPendingSubscription c) . swap
case L.nonEmpty qs_ of
Just qs' -> do
smp_ <- tryError (getSMPServerClient c srv)
(eitherToMaybe smp_,) . M.fromList . (errs <>) <$> case smp_ of
Left e -> pure $ map (second . const $ Left e) qs_
Right smp -> do
logServer "-->" c srv (bshow (length qs_) <> " queues") "SUB"
let qs2 = L.map (queueCreds . snd) qs'
rs' :: [((ConnId, RcvQueue), Either ProtocolClientError ())] <-
liftIO $ zip qs_ . L.toList <$> subscribeSMPQueues smp qs2
forM_ rs' $ \((connId, rq), r) -> liftIO $ processSubResult c rq connId r
pure $ map (bimap fst (first $ protocolClientError SMP)) rs'
_ -> pure $ (Nothing, M.fromList errs)
where
checkQueue rq@(connId, RcvQueue {rcvId, server}) = do
prohibited <- atomically . TM.member (server, rcvId) $ getMsgLocks c
pure $ if prohibited || srv /= server then Left (connId, Left $ CMD PROHIBITED) else Right rq
queueCreds RcvQueue {rcvPrivateKey, rcvId} = (rcvPrivateKey, rcvId)
withLogClient c server rcvId "SUB" $ \smp -> do
liftIO (runExceptT $ subscribeSMPQueue smp rcvPrivateKey rcvId) >>= \case
Left e -> do
atomically . when (e /= PCENetworkError && e /= PCEResponseTimeout) $
removePendingSubscription c server connId
throwError e
Right _ -> do
addSubscription c rq connId
addSubscription :: MonadIO m => AgentClient -> RcvQueue -> ConnId -> m ()
addSubscription c rq@RcvQueue {server} connId = atomically $ do
@@ -593,13 +474,13 @@ getSubscriptions AgentClient {subscrConns} = do
m <- readTVar subscrConns
pure $ M.keysSet m
logServer :: MonadIO m => ByteString -> AgentClient -> ProtocolServer s -> QueueId -> ByteString -> m ()
logServer :: MonadIO m => ByteString -> AgentClient -> SMPServer -> QueueId -> ByteString -> m ()
logServer dir AgentClient {clientId} srv qId cmdStr =
logInfo . decodeUtf8 $ B.unwords ["A", "(" <> bshow clientId <> ")", dir, showServer srv, ":", logSecret qId, cmdStr]
showServer :: ProtocolServer s -> ByteString
showServer :: SMPServer -> ByteString
showServer ProtocolServer {host, port} =
strEncode host <> B.pack (if null port then "" else ':' : port)
B.pack $ host <> if null port then "" else ':' : port
logSecret :: ByteString -> ByteString
logSecret bs = encode $ B.take 3 bs
@@ -612,8 +493,8 @@ sendConfirmation c sq@SndQueue {server, sndId, sndPublicKey = Just sndPublicKey,
liftClient SMP $ sendSMPMessage smp Nothing sndId (SMP.MsgFlags {notification = True}) msg
sendConfirmation _ _ _ = throwError $ INTERNAL "sendConfirmation called without snd_queue public key(s) in the database"
sendInvitation :: forall m. AgentMonad m => AgentClient -> Compatible SMPQueueInfo -> Compatible Version -> ConnectionRequestUri 'CMInvitation -> ConnInfo -> m ()
sendInvitation c (Compatible (SMPQueueInfo v SMPQueueAddress {smpServer, senderId, dhPublicKey})) (Compatible agentVersion) connReq connInfo =
sendInvitation :: forall m. AgentMonad m => AgentClient -> Compatible SMPQueueInfo -> ConnectionRequestUri 'CMInvitation -> ConnInfo -> m ()
sendInvitation c (Compatible SMPQueueInfo {smpServer, senderId, dhPublicKey}) connReq connInfo =
withLogClient_ c smpServer senderId "SEND <INV>" $ \smp -> do
msg <- mkInvitation
liftClient SMP $ sendSMPMessage smp Nothing senderId MsgFlags {notification = True} msg
@@ -621,56 +502,42 @@ sendInvitation c (Compatible (SMPQueueInfo v SMPQueueAddress {smpServer, senderI
mkInvitation :: m ByteString
-- this is only encrypted with per-queue E2E, not with double ratchet
mkInvitation = do
agentVersion <- asks $ smpAgentVersion . config
let agentEnvelope = AgentInvitation {agentVersion, connReq, connInfo}
agentCbEncryptOnce v dhPublicKey . smpEncode $
agentCbEncryptOnce dhPublicKey . smpEncode $
SMP.ClientMessage SMP.PHEmpty $ smpEncode agentEnvelope
getQueueMessage :: AgentMonad m => AgentClient -> RcvQueue -> m (Maybe SMPMsgMeta)
getQueueMessage c rq@RcvQueue {server, rcvId, rcvPrivateKey} = do
atomically createTakeGetLock
(v, msg_) <- withLogClient c server rcvId "GET" $ \smp ->
(thVersion smp,) <$> getSMPMessage smp rcvPrivateKey rcvId
mapM (decryptMeta v) msg_
getQueueMessage :: AgentMonad m => AgentClient -> RcvQueue -> ConnId -> m (Maybe (MsgId, MsgFlags))
getQueueMessage c@AgentClient {getMsgLocks} RcvQueue {server, rcvId, rcvPrivateKey} connId =
E.bracket (atomically createTakeLock) (atomically . (`putTMVar` ())) $ \_ ->
withLogClient c server rcvId "GET" $ \smp ->
getSMPMessage smp rcvPrivateKey rcvId
where
decryptMeta v msg@SMP.RcvMessage {msgId} = SMP.rcvMessageMeta msgId <$> decryptSMPMessage v rq msg
createTakeGetLock = TM.alterF takeLock (server, rcvId) $ getMsgLocks c
k = (connId, server, rcvId)
createTakeLock = do
l <- TM.lookup k getMsgLocks >>= maybe newLock pure
takeTMVar l
pure l
where
takeLock l_ = do
l <- maybe (newTMVar ()) pure l_
takeTMVar l
pure $ Just l
decryptSMPMessage :: AgentMonad m => Version -> RcvQueue -> SMP.RcvMessage -> m SMP.ClientRcvMsgBody
decryptSMPMessage v rq SMP.RcvMessage {msgId, msgTs, msgFlags, msgBody = SMP.EncRcvMsgBody body}
| v == 1 || v == 2 = SMP.ClientRcvMsgBody msgTs msgFlags <$> decrypt body
| otherwise = liftEither . parse SMP.clientRcvMsgBodyP (AGENT A_MESSAGE) =<< decrypt body
where
decrypt = agentCbDecrypt (rcvDhSecret rq) (C.cbNonce msgId)
newLock = do
l <- newTMVar ()
TM.insert k l getMsgLocks
pure l
secureQueue :: AgentMonad m => AgentClient -> RcvQueue -> SndPublicVerifyKey -> m ()
secureQueue c RcvQueue {server, rcvId, rcvPrivateKey} senderKey =
withLogClient c server rcvId "KEY <key>" $ \smp ->
secureSMPQueue smp rcvPrivateKey rcvId senderKey
enableQueueNotifications :: AgentMonad m => AgentClient -> RcvQueue -> NtfPublicVerifyKey -> RcvNtfPublicDhKey -> m (NotifierId, RcvNtfPublicDhKey)
enableQueueNotifications c RcvQueue {server, rcvId, rcvPrivateKey} notifierKey rcvNtfPublicDhKey =
enableQueueNotifications :: AgentMonad m => AgentClient -> RcvQueue -> NtfPublicVerifyKey -> m NotifierId
enableQueueNotifications c RcvQueue {server, rcvId, rcvPrivateKey} notifierKey =
withLogClient c server rcvId "NKEY <nkey>" $ \smp ->
enableSMPQueueNotifications smp rcvPrivateKey rcvId notifierKey rcvNtfPublicDhKey
disableQueueNotifications :: AgentMonad m => AgentClient -> RcvQueue -> m ()
disableQueueNotifications c RcvQueue {server, rcvId, rcvPrivateKey} =
withLogClient c server rcvId "NDEL" $ \smp ->
disableSMPQueueNotifications smp rcvPrivateKey rcvId
enableSMPQueueNotifications smp rcvPrivateKey rcvId notifierKey
sendAck :: AgentMonad m => AgentClient -> RcvQueue -> MsgId -> m ()
sendAck c rq@RcvQueue {server, rcvId, rcvPrivateKey} msgId = do
sendAck c RcvQueue {server, rcvId, rcvPrivateKey} msgId =
withLogClient c server rcvId "ACK" $ \smp ->
ackSMPMessage smp rcvPrivateKey rcvId msgId
atomically $ releaseGetLock c rq
releaseGetLock :: AgentClient -> RcvQueue -> STM ()
releaseGetLock c RcvQueue {server, rcvId} =
TM.lookup (server, rcvId) (getMsgLocks c) >>= mapM_ (`tryPutTMVar` ())
suspendQueue :: AgentMonad m => AgentClient -> RcvQueue -> m ()
suspendQueue c RcvQueue {server, rcvId, rcvPrivateKey} =
@@ -701,10 +568,6 @@ agentNtfCheckToken :: AgentMonad m => AgentClient -> NtfTokenId -> NtfToken -> m
agentNtfCheckToken c tknId NtfToken {ntfServer, ntfPrivKey} =
withLogClient c ntfServer tknId "TCHK" $ \ntf -> ntfCheckToken ntf ntfPrivKey tknId
agentNtfReplaceToken :: AgentMonad m => AgentClient -> NtfTokenId -> NtfToken -> DeviceToken -> m ()
agentNtfReplaceToken c tknId NtfToken {ntfServer, ntfPrivKey} token =
withLogClient c ntfServer tknId "TRPL" $ \ntf -> ntfReplaceToken ntf ntfPrivKey tknId token
agentNtfDeleteToken :: AgentMonad m => AgentClient -> NtfTokenId -> NtfToken -> m ()
agentNtfDeleteToken c tknId NtfToken {ntfServer, ntfPrivKey} =
withLogClient c ntfServer tknId "TDEL" $ \ntf -> ntfDeleteToken ntf ntfPrivKey tknId
@@ -721,30 +584,28 @@ agentNtfCheckSubscription :: AgentMonad m => AgentClient -> NtfSubscriptionId ->
agentNtfCheckSubscription c subId NtfToken {ntfServer, ntfPrivKey} =
withLogClient c ntfServer subId "SCHK" $ \ntf -> ntfCheckSubscription ntf ntfPrivKey subId
agentNtfDeleteSubscription :: AgentMonad m => AgentClient -> NtfSubscriptionId -> NtfToken -> m ()
agentNtfDeleteSubscription c subId NtfToken {ntfServer, ntfPrivKey} =
withLogClient c ntfServer subId "SDEL" $ \ntf -> ntfDeleteSubscription ntf ntfPrivKey subId
agentCbEncrypt :: AgentMonad m => SndQueue -> Maybe C.PublicKeyX25519 -> ByteString -> m ByteString
agentCbEncrypt SndQueue {e2eDhSecret, smpClientVersion} e2ePubKey msg = do
agentCbEncrypt SndQueue {e2eDhSecret} e2ePubKey msg = do
cmNonce <- liftIO C.randomCbNonce
let paddedLen = maybe SMP.e2eEncMessageLength (const SMP.e2eEncConfirmationLength) e2ePubKey
cmEncBody <-
liftEither . first cryptoError $
C.cbEncrypt e2eDhSecret cmNonce msg paddedLen
let cmHeader = SMP.PubHeader smpClientVersion e2ePubKey
-- TODO per-queue client version
let cmHeader = SMP.PubHeader (maxVersion SMP.smpClientVRange) e2ePubKey
pure $ smpEncode SMP.ClientMsgEnvelope {cmHeader, cmNonce, cmEncBody}
-- add encoding as AgentInvitation'?
agentCbEncryptOnce :: AgentMonad m => Version -> C.PublicKeyX25519 -> ByteString -> m ByteString
agentCbEncryptOnce clientVersion dhRcvPubKey msg = do
agentCbEncryptOnce :: AgentMonad m => C.PublicKeyX25519 -> ByteString -> m ByteString
agentCbEncryptOnce dhRcvPubKey msg = do
(dhSndPubKey, dhSndPrivKey) <- liftIO C.generateKeyPair'
let e2eDhSecret = C.dh' dhRcvPubKey dhSndPrivKey
cmNonce <- liftIO C.randomCbNonce
cmEncBody <-
liftEither . first cryptoError $
C.cbEncrypt e2eDhSecret cmNonce msg SMP.e2eEncConfirmationLength
let cmHeader = SMP.PubHeader clientVersion (Just dhSndPubKey)
-- TODO per-queue client version
let cmHeader = SMP.PubHeader (maxVersion SMP.smpClientVRange) (Just dhSndPubKey)
pure $ smpEncode SMP.ClientMsgEnvelope {cmHeader, cmNonce, cmEncBody}
-- | NaCl crypto-box decrypt - both for messages received from the server
@@ -764,85 +625,50 @@ cryptoError = \case
e -> INTERNAL $ show e
endAgentOperation :: AgentClient -> AgentOperation -> STM ()
endAgentOperation c op = endOperation c op $ case op of
AONtfNetwork -> pure ()
AORcvNetwork ->
suspendOperation c AOMsgDelivery $
suspendSendingAndDatabase c
AOMsgDelivery ->
suspendSendingAndDatabase c
AOSndNetwork ->
suspendOperation c AODatabase $
notifySuspended c
AODatabase ->
notifySuspended c
suspendSendingAndDatabase :: AgentClient -> STM ()
suspendSendingAndDatabase c =
suspendOperation c AOSndNetwork $
suspendOperation c AODatabase $
notifySuspended c
suspendOperation :: AgentClient -> AgentOperation -> STM () -> STM ()
suspendOperation c op endedAction = do
n <- stateTVar (agentOpSel op c) $ \s -> (opsInProgress s, s {opSuspended = True})
-- unsafeIOToSTM $ putStrLn $ "suspendOperation_ " <> show op <> " " <> show n
when (n == 0) $ whenSuspending c endedAction
notifySuspended :: AgentClient -> STM ()
notifySuspended c = do
-- unsafeIOToSTM $ putStrLn "notifySuspended"
writeTBQueue (subQ c) ("", "", SUSPENDED)
writeTVar (agentState c) ASSuspended
endOperation :: AgentClient -> AgentOperation -> STM () -> STM ()
endOperation c op endedAction = do
(suspended, n) <- stateTVar (agentOpSel op c) $ \s ->
let n = max 0 (opsInProgress s - 1)
in ((opSuspended s, n), s {opsInProgress = n})
-- unsafeIOToSTM $ putStrLn $ "endOperation: " <> show op <> " " <> show suspended <> " " <> show n
when (suspended && n == 0) $ whenSuspending c endedAction
whenSuspending :: AgentClient -> STM () -> STM ()
whenSuspending c = whenM ((== ASSuspending) <$> readTVar (agentState c))
endAgentOperation c@AgentClient {agentEnv = Env {agentOperations}} op = do
TM.alter (Just . maybe 0 (\n -> min 0 $ n - 1)) op agentOperations
notifyAgentPhaseChanged c
beginAgentOperation :: AgentClient -> AgentOperation -> STM ()
beginAgentOperation c op = do
let opVar = agentOpSel op c
s <- readTVar opVar
-- unsafeIOToSTM $ putStrLn $ "beginOperation? " <> show op <> " " <> show (opsInProgress s)
when (opSuspended s) retry
-- unsafeIOToSTM $ putStrLn $ "beginOperation! " <> show op <> " " <> show (opsInProgress s + 1)
writeTVar opVar $! s {opsInProgress = opsInProgress s + 1}
beginAgentOperation AgentClient {agentEnv = Env {agentPhase, agentOperations}} op = do
(p, _) <- readTVar agentPhase
when (op `elem` disallowedOperations p) retry
TM.alter (Just . maybe 1 (+ 1)) op agentOperations
agentOperationBracket :: MonadUnliftIO m => AgentClient -> AgentOperation -> m a -> m a
agentOperationBracket c op action =
E.bracket
(atomically $ beginAgentOperation c op)
(\_ -> atomically $ endAgentOperation c op)
(const action)
notifyAgentPhaseChanged :: AgentClient -> STM ()
notifyAgentPhaseChanged AgentClient {subQ, agentEnv = Env {agentPhase, agentOperations}} = do
(p, notified) <- readTVar agentPhase
unless notified $ do
ops <- readTVar agentOperations
let opsPaused = all (maybe True (== 0) . (`M.lookup` ops)) $ disallowedOperations p
when opsPaused $ do
writeTBQueue subQ ("", "", PHASE p)
writeTVar agentPhase (p, True)
withStore' :: AgentMonad m => AgentClient -> (DB.Connection -> IO a) -> m a
withStore' c action = withStore c $ fmap Right . action
withStore :: AgentMonad m => AgentClient -> (DB.Connection -> IO (Either StoreError a)) -> m a
withStore :: AgentMonad m => AgentClient -> (forall m'. AgentStoreMonad m' => SQLiteStore -> m' a) -> m a
withStore c action = do
st <- asks store
liftEitherError storeError . agentOperationBracket c AODatabase $
withTransaction st action `E.catch` handleInternal
st@SQLiteStore {dbSemaphore} <- asks store
E.bracket_ (liftIO $ mapM_ semThreadWait dbSemaphore) (liftIO $ mapM_ semPost dbSemaphore) $ do
atomically $ beginAgentOperation c AODatabase
r <- runExceptT (action st `E.catch` handleInternal)
atomically $ endAgentOperation c AODatabase
case r of
Right res -> pure res
Left e -> throwError $ storeError e
where
handleInternal :: E.SomeException -> IO (Either StoreError a)
handleInternal = pure . Left . SEInternal . bshow
storeError :: StoreError -> AgentErrorType
storeError = \case
SEConnNotFound -> CONN NOT_FOUND
SEConnDuplicate -> CONN DUPLICATE
SEBadConnType CRcv -> CONN SIMPLEX
SEBadConnType CSnd -> CONN SIMPLEX
SEInvitationNotFound -> CMD PROHIBITED
-- this error is never reported as store error,
-- it is used to wrap agent operations when "transaction-like" store access is needed
-- NOTE: network IO should NOT be used inside AgentStoreMonad
SEAgentError e -> e
e -> INTERNAL $ show e
-- TODO when parsing exception happens in store, the agent hangs;
-- changing SQLError to SomeException does not help
handleInternal :: (MonadError StoreError m') => SQLError -> m' a
handleInternal e = throwError . SEInternal $ bshow e
storeError :: StoreError -> AgentErrorType
storeError = \case
SEConnNotFound -> CONN NOT_FOUND
SEConnDuplicate -> CONN DUPLICATE
SEBadConnType CRcv -> CONN SIMPLEX
SEBadConnType CSnd -> CONN SIMPLEX
SEInvitationNotFound -> CMD PROHIBITED
-- this error is never reported as store error,
-- it is used to wrap agent operations when "transaction-like" store access is needed
-- NOTE: network IO should NOT be used inside AgentStoreMonad
SEAgentError e -> e
e -> INTERNAL $ show e
+38 -40
View File
@@ -3,6 +3,7 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE RankNTypes #-}
@@ -13,10 +14,11 @@ module Simplex.Messaging.Agent.Env.SQLite
( AgentMonad,
AgentConfig (..),
InitialAgentServers (..),
NetworkConfig (..),
defaultAgentConfig,
defaultReconnectInterval,
Env (..),
AgentOperation (..),
disallowedOperations,
newSMPAgentEnv,
NtfSupervisor (..),
NtfSupervisorCommand (..),
@@ -29,7 +31,6 @@ import Control.Monad.Reader
import Crypto.Random
import Data.List.NonEmpty (NonEmpty)
import Data.Time.Clock (NominalDiffTime, nominalDay)
import Data.Word (Word16)
import Network.Socket
import Numeric.Natural
import Simplex.Messaging.Agent.Protocol
@@ -39,12 +40,10 @@ import qualified Simplex.Messaging.Agent.Store.SQLite.Migrations as Migrations
import Simplex.Messaging.Client
import Simplex.Messaging.Client.Agent ()
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Notifications.Types
import Simplex.Messaging.Protocol (NtfServer, supportedSMPClientVRange)
import Simplex.Messaging.Notifications.Client (NtfServer, NtfToken)
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport (TLS, Transport (..))
import Simplex.Messaging.Transport.Client (defaultSMPPort)
import Simplex.Messaging.Version
import System.Random (StdGen, newStdGen)
import UnliftIO (Async)
@@ -55,8 +54,7 @@ type AgentMonad m = (MonadUnliftIO m, MonadReader Env m, MonadError AgentErrorTy
data InitialAgentServers = InitialAgentServers
{ smp :: NonEmpty SMPServer,
ntf :: [NtfServer],
netCfg :: NetworkConfig
ntf :: [NtfServer]
}
data AgentConfig = AgentConfig
@@ -65,40 +63,31 @@ data AgentConfig = AgentConfig
connIdBytes :: Int,
tbqSize :: Natural,
dbFile :: FilePath,
dbSemName :: Maybe String,
yesToMigrations :: Bool,
smpCfg :: ProtocolClientConfig,
ntfCfg :: ProtocolClientConfig,
reconnectInterval :: RetryInterval,
messageRetryInterval :: RetryInterval,
messageTimeout :: NominalDiffTime,
helloTimeout :: NominalDiffTime,
ntfCron :: Word16,
ntfWorkerDelay :: Int,
ntfSMPWorkerDelay :: Int,
resubscriptionConcurrency :: Int,
ntfWorkerThrottle :: Int,
ntfSubCheckInterval :: NominalDiffTime,
ntfMaxMessages :: Int,
caCertificateFile :: FilePath,
privateKeyFile :: FilePath,
certificateFile :: FilePath,
smpAgentVRange :: VersionRange,
smpClientVRange :: VersionRange
smpAgentVersion :: Version,
smpAgentVRange :: VersionRange
}
defaultReconnectInterval :: RetryInterval
defaultReconnectInterval =
RetryInterval
{ initialInterval = 2_000000,
increaseAfter = 10_000000,
maxInterval = 180_000000
}
defaultMessageRetryInterval :: RetryInterval
defaultMessageRetryInterval =
RetryInterval
{ initialInterval = 1_000000,
increaseAfter = 10_000000,
maxInterval = 60_000000
{ initialInterval = second,
increaseAfter = 10 * second,
maxInterval = 10 * second
}
where
second = 1_000_000
defaultAgentConfig :: AgentConfig
defaultAgentConfig =
@@ -108,25 +97,22 @@ defaultAgentConfig =
connIdBytes = 12,
tbqSize = 64,
dbFile = "smp-agent.db",
dbSemName = Nothing,
yesToMigrations = False,
smpCfg = defaultClientConfig {defaultTransport = (show defaultSMPPort, transport @TLS)},
smpCfg = defaultClientConfig {defaultTransport = ("5223", transport @TLS)},
ntfCfg = defaultClientConfig {defaultTransport = ("443", transport @TLS)},
reconnectInterval = defaultReconnectInterval,
messageRetryInterval = defaultMessageRetryInterval,
messageTimeout = 2 * nominalDay,
helloTimeout = 2 * nominalDay,
ntfCron = 20, -- minutes
ntfWorkerDelay = 100000, -- microseconds
ntfSMPWorkerDelay = 500000, -- microseconds
resubscriptionConcurrency = 16,
ntfWorkerThrottle = 1000000, -- microseconds
ntfSubCheckInterval = nominalDay,
ntfMaxMessages = 4,
-- CA certificate private key is not needed for initialization
-- ! we do not generate these
caCertificateFile = "/etc/opt/simplex-agent/ca.crt",
privateKeyFile = "/etc/opt/simplex-agent/agent.key",
certificateFile = "/etc/opt/simplex-agent/agent.crt",
smpAgentVRange = supportedSMPAgentVRange,
smpClientVRange = supportedSMPClientVRange
smpAgentVersion = currentSMPAgentVersion,
smpAgentVRange = supportedSMPAgentVRange
}
data Env = Env
@@ -135,17 +121,30 @@ data Env = Env
idsDrg :: TVar ChaChaDRG,
clientCounter :: TVar Int,
randomServer :: TVar StdGen,
agentPhase :: TVar (AgentPhase, Bool),
agentOperations :: TMap AgentOperation Int,
ntfSupervisor :: NtfSupervisor
}
data AgentOperation = AONetwork | AODatabase
deriving (Eq, Ord)
disallowedOperations :: AgentPhase -> [AgentOperation]
disallowedOperations = \case
APActive -> []
APPaused -> [AONetwork]
APSuspended -> [AONetwork, AODatabase]
newSMPAgentEnv :: (MonadUnliftIO m, MonadRandom m) => AgentConfig -> m Env
newSMPAgentEnv config@AgentConfig {dbFile, yesToMigrations} = do
newSMPAgentEnv config@AgentConfig {dbFile, dbSemName, yesToMigrations} = do
idsDrg <- newTVarIO =<< drgNew
store <- liftIO $ createSQLiteStore dbFile Migrations.app yesToMigrations
store <- liftIO $ createSQLiteStore dbFile dbSemName Migrations.app yesToMigrations
clientCounter <- newTVarIO 0
randomServer <- newTVarIO =<< liftIO newStdGen
agentPhase <- newTVarIO (APActive, True)
agentOperations <- atomically TM.empty
ntfSupervisor <- atomically . newNtfSubSupervisor $ tbqSize config
return Env {config, store, idsDrg, clientCounter, randomServer, ntfSupervisor}
return Env {config, store, idsDrg, clientCounter, randomServer, agentPhase, agentOperations, ntfSupervisor}
data NtfSupervisor = NtfSupervisor
{ ntfTkn :: TVar (Maybe NtfToken),
@@ -154,8 +153,7 @@ data NtfSupervisor = NtfSupervisor
ntfSMPWorkers :: TMap SMPServer (TMVar (), Async ())
}
data NtfSupervisorCommand = NSCCreate | NSCDelete | NSCSmpDelete | NSCNtfWorker NtfServer | NSCNtfSMPWorker SMPServer
deriving (Show)
data NtfSupervisorCommand = NSCCreate | NSCDelete | NSCNtfWorker NtfServer
newNtfSubSupervisor :: Natural -> STM NtfSupervisor
newNtfSubSupervisor qSize = do
+135 -242
View File
@@ -4,14 +4,12 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TupleSections #-}
module Simplex.Messaging.Agent.NtfSubSupervisor
( runNtfSupervisor,
nsUpdateToken,
nsRemoveNtfToken,
sendNtfSubCommand,
instantNotifications,
closeNtfSupervisor,
getNtfServer,
)
@@ -19,130 +17,89 @@ where
import Control.Concurrent.Async (Async, uninterruptibleCancel)
import Control.Concurrent.STM (stateTVar)
import Control.Logger.Simple (logError, logInfo)
import Control.Monad
import Control.Monad.Except
import Control.Monad.Except (runExceptT)
import Control.Monad.IO.Unlift (MonadUnliftIO)
import Control.Monad.Reader
import Data.Bifunctor (first)
import Data.Fixed (Fixed (MkFixed), Pico)
import qualified Data.Map.Strict as M
import Data.Text (Text)
import Data.Time (UTCTime, addUTCTime, diffUTCTime, getCurrentTime, nominalDiffTimeToSeconds)
import Simplex.Messaging.Agent.Client
import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Protocol (AgentErrorType (..), BrokerErrorType (..), ConnId, NotificationsMode (..))
import Simplex.Messaging.Agent.Protocol (ConnId)
import qualified Simplex.Messaging.Agent.Protocol as AP
import Simplex.Messaging.Agent.RetryInterval
import Simplex.Messaging.Agent.Store
import Simplex.Messaging.Agent.Store.SQLite
import Simplex.Messaging.Client.Agent ()
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Notifications.Client
import Simplex.Messaging.Notifications.Protocol (NtfSubStatus (..), NtfTknStatus (..), SMPQueueNtf (..))
import Simplex.Messaging.Notifications.Types
import Simplex.Messaging.Protocol
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Util (tshow, unlessM)
import Simplex.Messaging.Util (unlessM)
import System.Random (randomR)
import UnliftIO (async)
import UnliftIO.Concurrent (forkIO, threadDelay)
import qualified UnliftIO.Exception as E
import UnliftIO.STM
runNtfSupervisor :: forall m. (MonadUnliftIO m, MonadReader Env m) => AgentClient -> m ()
runNtfSupervisor c = do
runNtfSupervisor :: (MonadUnliftIO m, MonadReader Env m) => AgentClient -> m ()
runNtfSupervisor c = forever $ do
ns <- asks ntfSupervisor
forever $ do
cmd@(connId, _) <- atomically . readTBQueue $ ntfSubQ ns
handleError connId . agentOperationBracket c AONtfNetwork $
runExceptT (processNtfSub c cmd) >>= \case
Left e -> notifyErr connId e
Right _ -> return ()
where
handleError :: ConnId -> m () -> m ()
handleError connId = E.handle $ \(e :: E.SomeException) -> do
logError $ "runNtfSupervisor error " <> tshow e
notifyErr connId e
notifyErr connId e = notifyInternalError c connId $ "runNtfSupervisor error " <> show e
cmd <- atomically . readTBQueue $ ntfSubQ ns
runExceptT (processNtfSub c cmd) >>= \case
Left e -> liftIO $ print e
Right _ -> return ()
processNtfSub :: forall m. AgentMonad m => AgentClient -> (ConnId, NtfSupervisorCommand) -> m ()
processNtfSub c (connId, cmd) = do
logInfo $ "processNtfSub - connId = " <> tshow connId <> " - cmd = " <> tshow cmd
ntfServer_ <- getNtfServer c
case cmd of
NSCCreate -> do
(a, RcvQueue {server = smpServer, clientNtfCreds}) <- withStore c $ \db -> runExceptT $ do
a <- liftIO $ getNtfSubscription db connId
q <- ExceptT $ getRcvQueue db connId
pure (a, q)
logInfo $ "processNtfSub, NSCCreate - a = " <> tshow a
case a of
Nothing -> do
withNtfServer c $ \ntfServer -> do
case clientNtfCreds of
Just ClientNtfCreds {notifierId} -> do
let newSub = newNtfSubscription connId smpServer (Just notifierId) ntfServer NASKey
ts <- liftIO getCurrentTime
withStore' c $ \db -> createNtfSubscription db newSub (NtfSubNTFAction NSACreate) ts
addNtfNTFWorker ntfServer
Nothing -> do
let newSub = newNtfSubscription connId smpServer Nothing ntfServer NASNew
ts <- liftIO getCurrentTime
withStore' c $ \db -> createNtfSubscription db newSub (NtfSubSMPAction NSASmpKey) ts
addNtfSMPWorker smpServer
(Just (sub@NtfSubscription {ntfSubStatus, ntfServer = subNtfServer}, action_)) -> do
case action_ of
-- action was set to NULL after worker internal error
Nothing -> resetSubscription
Just (action, _)
-- subscription was marked for deletion / is being deleted
| isDeleteNtfSubAction action -> do
if ntfSubStatus == NASNew || ntfSubStatus == NASOff || ntfSubStatus == NASDeleted
then resetSubscription
else withNtfServer c $ \ntfServer -> do
ts <- liftIO getCurrentTime
withStore' c $ \db ->
supervisorUpdateNtfSubscription db sub {ntfServer} (NtfSubNTFAction NSACreate) ts
addNtfNTFWorker ntfServer
| otherwise -> case action of
NtfSubNTFAction _ -> addNtfNTFWorker subNtfServer
NtfSubSMPAction _ -> addNtfSMPWorker smpServer
where
resetSubscription :: m ()
resetSubscription =
withNtfServer c $ \ntfServer -> do
ts <- liftIO getCurrentTime
withStore' c $ \db ->
supervisorUpdateNtfSubscription db sub {ntfQueueId = Nothing, ntfServer, ntfSubId = Nothing, ntfSubStatus = NASNew} (NtfSubSMPAction NSASmpKey) ts
addNtfSMPWorker smpServer
NSCDelete -> do
sub_ <- withStore' c $ \db -> do
ts <- liftIO getCurrentTime
supervisorUpdateNtfSubAction db connId (NtfSubNTFAction NSADelete) ts
getNtfSubscription db connId
logInfo $ "processNtfSub, NSCDelete - sub_ = " <> tshow sub_
case sub_ of
(Just (NtfSubscription {ntfServer}, _)) -> addNtfNTFWorker ntfServer
_ -> pure () -- err "NSCDelete - no subscription"
NSCSmpDelete -> do
withStore' c (`getRcvQueue` connId) >>= \case
Right rq@RcvQueue {server = smpServer} -> do
logInfo $ "processNtfSub, NSCSmpDelete - rq = " <> tshow rq
ts <- liftIO getCurrentTime
withStore' c $ \db -> supervisorUpdateNtfSubAction db connId (NtfSubSMPAction NSASmpDelete) ts
-- TODO merge getNtfSubscription and getRcvQueue into one method to read both in same transaction?
sub_ <- withStore c $ \st -> getNtfSubscription st connId
RcvQueue {notifierId, server = smpServer} <- withStore c $ \st -> getRcvQueue st connId
case (sub_, ntfServer_) of
(Nothing, Just ntfServer) -> do
currentTime <- liftIO getCurrentTime
case notifierId of
(Just nId) -> do
let newSub = newNtfSubscription connId smpServer (Just nId) ntfServer NASKey currentTime
withStore c $ \st -> createNtfSubscription st newSub (NtfSubAction NSACreate)
_ -> do
let newSub = newNtfSubscription connId smpServer Nothing ntfServer NASNew currentTime
withStore c $ \st -> createNtfSubscription st newSub (NtfSubSMPAction NSAKey)
-- TODO optimize?
-- TODO - read action in getNtfSubscription and decide which worker to create
-- TODO - SMP worker can create Ntf worker on NKEY completion
addNtfSMPWorker smpServer
_ -> notifyInternalError c connId "NSCSmpDelete - no rcv queue"
addNtfWorker ntfServer
(Just _, Just ntfServer) -> do
-- TODO subscription may have to be updated depending on current state:
-- TODO - e.g., if it was previously marked for deletion action has to be updated
-- TODO - should action depend on subscription status or always be NSAKey (NSACreate if notifierId exists)
-- TODO in case worker is currently deleting it? When deleting worker should check for updated_by_supervisor
-- TODO and if it is set perform update instead of delete. If worker was not deleting it yet it should
-- TODO idempotently replay commands.
addNtfSMPWorker smpServer
addNtfWorker ntfServer
_ -> pure () -- error - notification server not configured
NSCDelete -> do
-- TODO delete notifier ID and Key from SMP server (SDEL, then NDEL)
withStore c $ \st -> markNtfSubscriptionForDeletion st connId
case ntfServer_ of
(Just ntfServer) -> addNtfWorker ntfServer
_ -> pure ()
NSCNtfWorker ntfServer ->
addNtfNTFWorker ntfServer
NSCNtfSMPWorker smpServer ->
addNtfSMPWorker smpServer
addNtfWorker ntfServer
where
addNtfNTFWorker = addWorker ntfWorkers runNtfWorker
addNtfWorker = addWorker ntfWorkers runNtfWorker
addNtfSMPWorker = addWorker ntfSMPWorkers runNtfSMPWorker
addWorker ::
(NtfSupervisor -> TMap (ProtocolServer s) (TMVar (), Async ())) ->
(AgentClient -> ProtocolServer s -> TMVar () -> m ()) ->
ProtocolServer s ->
(NtfSupervisor -> TMap ProtocolServer (TMVar (), Async ())) ->
(AgentClient -> ProtocolServer -> TMVar () -> m ()) ->
ProtocolServer ->
m ()
addWorker wsSel runWorker srv = do
ws <- asks $ wsSel . ntfSupervisor
@@ -154,138 +111,95 @@ processNtfSub c (connId, cmd) = do
Just (doWork, _) ->
void . atomically $ tryPutTMVar doWork ()
withNtfServer :: AgentMonad m => AgentClient -> (NtfServer -> m ()) -> m ()
withNtfServer c action = getNtfServer c >>= mapM_ action
runNtfWorker :: forall m. AgentMonad m => AgentClient -> NtfServer -> TMVar () -> m ()
runNtfWorker c srv doWork = do
delay <- asks $ ntfWorkerDelay . config
forever $ do
void . atomically $ readTMVar doWork
agentOperationBracket c AONtfNetwork runNtfOperation
threadDelay delay
where
runNtfOperation :: m ()
runNtfOperation = do
nextSub_ <- withStore' c (`getNextNtfSubNTFAction` srv)
logInfo $ "runNtfWorker, nextSub_ " <> tshow nextSub_
runNtfWorker :: AgentMonad m => AgentClient -> NtfServer -> TMVar () -> m ()
runNtfWorker c srv doWork = forever $ do
void . atomically $ readTMVar doWork
getNtfToken >>= \case
Just tkn@NtfToken {ntfTokenId = Just tknId, ntfTknStatus} -> do
nextSub_ <- withStore c (`getNextNtfSubAction` srv)
ts <- liftIO getCurrentTime
case nextSub_ of
Nothing -> noWorkToDo
Just a@(NtfSubscription {connId}, _, _) -> do
ri <- asks $ reconnectInterval . config
withRetryInterval ri $ \loop ->
processAction a
`catchError` retryOnError c "NtfWorker" loop (workerInternalError c connId . show)
Just (ntfSub@NtfSubscription {connId, smpServer, ntfSubId}, ntfSubAction, RcvQueue {ntfPrivateKey, notifierId}) ->
unlessM (rescheduleAction doWork ts ntfSub) $
case ntfSubAction of
NSACreate -> case (ntfPrivateKey, notifierId) of
(Just ntfPrivKey, Just nId)
| ntfTknStatus == NTActive -> do
nSubId <- agentNtfCreateSubscription c tknId tkn (SMPQueueNtf smpServer nId) ntfPrivKey
let actionTs = addUTCTime 30 ts
withStore c $ \st ->
updateNtfSubscription st connId ntfSub {ntfSubId = Just nSubId, ntfSubStatus = NASCreated NSNew, ntfSubActionTs = actionTs} (NtfSubAction NSACheck)
| otherwise -> ntfInternalError c connId "NSACreate - token not active"
_ -> ntfInternalError c connId "NSACreate - no notifier key or ID"
NSACheck -> case ntfSubId of
Just nSubId ->
agentNtfCheckSubscription c nSubId tkn >>= \case
NSNew -> updateSubNextCheck NSNew
NSPending -> updateSubNextCheck NSPending
NSActive -> updateSubNextCheck NSActive
NSEnd -> updateSubNextCheck NSEnd
NSSMPAuth -> updateSub (NASCreated NSSMPAuth) (NtfSubAction NSADelete) ts
Nothing -> ntfInternalError c connId "NSACheck - no subscription ID"
NSADelete -> pure ()
where
updateSubNextCheck toStatus = do
checkInterval <- asks $ ntfSubCheckInterval . config
let nextCheckTs = addUTCTime checkInterval ts
updateSub (NASCreated toStatus) (NtfSubAction NSACheck) nextCheckTs
updateSub toStatus toAction actionTs =
withStore c $ \st ->
updateNtfSubscription st connId ntfSub {ntfSubStatus = toStatus, ntfSubActionTs = actionTs} toAction
_ -> noWorkToDo
delay <- asks $ ntfWorkerThrottle . config
liftIO $ threadDelay delay
where
noWorkToDo = void . atomically $ tryTakeTMVar doWork
processAction :: (NtfSubscription, NtfSubNTFAction, NtfActionTs) -> m ()
processAction (sub@NtfSubscription {connId, smpServer, ntfSubId}, action, actionTs) = do
ts <- liftIO getCurrentTime
unlessM (rescheduleAction doWork ts actionTs) $
case action of
NSACreate ->
getNtfToken >>= \case
Just tkn@NtfToken {ntfTokenId = Just tknId, ntfTknStatus = NTActive, ntfMode = NMInstant} -> do
RcvQueue {clientNtfCreds} <- withStore c (`getRcvQueue` connId)
case clientNtfCreds of
Just ClientNtfCreds {ntfPrivateKey, notifierId} -> do
nSubId <- agentNtfCreateSubscription c tknId tkn (SMPQueueNtf smpServer notifierId) ntfPrivateKey
-- TODO smaller retry until Active, less frequently (daily?) once Active
let actionTs' = addUTCTime 30 ts
withStore' c $ \db ->
updateNtfSubscription db sub {ntfSubId = Just nSubId, ntfSubStatus = NASCreated NSNew} (NtfSubNTFAction NSACheck) actionTs'
_ -> workerInternalError c connId "NSACreate - no notifier queue credentials"
_ -> workerInternalError c connId "NSACreate - no active token"
NSACheck ->
getNtfToken >>= \case
Just tkn ->
case ntfSubId of
Just nSubId ->
agentNtfCheckSubscription c nSubId tkn >>= \case
NSAuth -> do
getNtfServer c >>= \case
Just ntfServer -> do
withStore' c $ \db ->
updateNtfSubscription db sub {ntfServer, ntfQueueId = Nothing, ntfSubId = Nothing, ntfSubStatus = NASNew} (NtfSubSMPAction NSASmpKey) ts
ns <- asks ntfSupervisor
atomically $ writeTBQueue (ntfSubQ ns) (connId, NSCNtfSMPWorker smpServer)
_ -> workerInternalError c connId "NSACheck - failed to reset subscription, notification server not configured"
status -> updateSubNextCheck ts status
Nothing -> workerInternalError c connId "NSACheck - no subscription ID"
_ -> workerInternalError c connId "NSACheck - no active token"
NSADelete -> case ntfSubId of
Just nSubId ->
(getNtfToken >>= \tkn -> forM_ tkn $ agentNtfDeleteSubscription c nSubId)
`E.finally` carryOnWithDeletion
Nothing -> carryOnWithDeletion
where
carryOnWithDeletion :: m ()
carryOnWithDeletion = do
withStore' c $ \db ->
updateNtfSubscription db sub {ntfSubId = Nothing, ntfSubStatus = NASOff} (NtfSubSMPAction NSASmpDelete) ts
ns <- asks ntfSupervisor
atomically $ writeTBQueue (ntfSubQ ns) (connId, NSCNtfSMPWorker smpServer)
where
updateSubNextCheck ts toStatus = do
checkInterval <- asks $ ntfSubCheckInterval . config
let nextCheckTs = addUTCTime checkInterval ts
updateSub (NASCreated toStatus) (NtfSubNTFAction NSACheck) nextCheckTs
updateSub toStatus toAction actionTs' =
withStore' c $ \db ->
updateNtfSubscription db sub {ntfSubStatus = toStatus} toAction actionTs'
runNtfSMPWorker :: forall m. AgentMonad m => AgentClient -> SMPServer -> TMVar () -> m ()
runNtfSMPWorker c srv doWork = do
delay <- asks $ ntfSMPWorkerDelay . config
forever $ do
void . atomically $ readTMVar doWork
agentOperationBracket c AONtfNetwork runNtfSMPOperation
threadDelay delay
where
runNtfSMPOperation = do
nextSub_ <- withStore' c (`getNextNtfSubSMPAction` srv)
logInfo $ "runNtfSMPWorker, nextSub_ " <> tshow nextSub_
runNtfSMPWorker c srv doWork = forever $ do
void . atomically $ readTMVar doWork
getNtfToken >>= \case
Just NtfToken {ntfTknStatus} -> do
nextSub_ <- withStore c (`getNextNtfSubSMPAction` srv)
ts <- liftIO getCurrentTime
case nextSub_ of
Nothing -> noWorkToDo
Just a@(NtfSubscription {connId}, _, _) -> do
ri <- asks $ reconnectInterval . config
withRetryInterval ri $ \loop ->
processAction a
`catchError` retryOnError c "NtfSMPWorker" loop (workerInternalError c connId . show)
Just (ntfSub@NtfSubscription {connId, ntfServer}, ntfSubAction, rq@RcvQueue {ntfPublicKey}) ->
unlessM (rescheduleAction doWork ts ntfSub) $
case ntfSubAction of
NSAKey
| ntfTknStatus == NTActive ->
case ntfPublicKey of
Just ntfPubKey ->
enableNotificationsWithNKey ntfPubKey
_ -> do
C.SignAlg a <- asks (cmdSignAlg . config)
(ntfPubKey, ntfPrivKey) <- liftIO $ C.generateSignatureKeyPair a
withStore c $ \st -> setRcvQueueNotifierKey st connId ntfPubKey ntfPrivKey
enableNotificationsWithNKey ntfPubKey
| otherwise -> ntfInternalError c connId "NSAKey - token not active"
where
enableNotificationsWithNKey ntfPubKey = do
nId <- enableQueueNotifications c rq ntfPubKey
withStore c $ \st -> do
setRcvQueueNotifierId st connId nId
updateNtfSubscription st connId ntfSub {ntfQueueId = Just nId, ntfSubStatus = NASKey, ntfSubActionTs = ts} (NtfSubAction NSACreate)
ns <- asks ntfSupervisor
atomically $ sendNtfSubCommand ns (connId, NSCNtfWorker ntfServer)
_ -> noWorkToDo
delay <- asks $ ntfWorkerThrottle . config
liftIO $ threadDelay delay
where
noWorkToDo = void . atomically $ tryTakeTMVar doWork
processAction :: (NtfSubscription, NtfSubSMPAction, NtfActionTs) -> m ()
processAction (sub@NtfSubscription {connId, ntfServer}, smpAction, actionTs) = do
ts <- liftIO getCurrentTime
unlessM (rescheduleAction doWork ts actionTs) $
case smpAction of
NSASmpKey ->
getNtfToken >>= \case
Just NtfToken {ntfTknStatus = NTActive, ntfMode = NMInstant} -> do
rq <- withStore c (`getRcvQueue` connId)
C.SignAlg a <- asks (cmdSignAlg . config)
(ntfPublicKey, ntfPrivateKey) <- liftIO $ C.generateSignatureKeyPair a
(rcvNtfPubDhKey, rcvNtfPrivDhKey) <- liftIO C.generateKeyPair'
(notifierId, rcvNtfSrvPubDhKey) <- enableQueueNotifications c rq ntfPublicKey rcvNtfPubDhKey
let rcvNtfDhSecret = C.dh' rcvNtfSrvPubDhKey rcvNtfPrivDhKey
withStore' c $ \db -> do
setRcvQueueNtfCreds db connId $ Just ClientNtfCreds {ntfPublicKey, ntfPrivateKey, notifierId, rcvNtfDhSecret}
updateNtfSubscription db sub {ntfQueueId = Just notifierId, ntfSubStatus = NASKey} (NtfSubNTFAction NSACreate) ts
ns <- asks ntfSupervisor
atomically $ sendNtfSubCommand ns (connId, NSCNtfWorker ntfServer)
_ -> workerInternalError c connId "NSASmpKey - no active token"
NSASmpDelete -> do
rq_ <- withStore' c $ \db -> do
setRcvQueueNtfCreds db connId Nothing
getRcvQueue db connId
forM_ rq_ $ \rq -> disableQueueNotifications c rq
withStore' c $ \db -> deleteNtfSubscription db connId
rescheduleAction :: AgentMonad m => TMVar () -> UTCTime -> UTCTime -> m Bool
rescheduleAction doWork ts actionTs
| actionTs <= ts = pure False
rescheduleAction :: AgentMonad m => TMVar () -> UTCTime -> NtfSubscription -> m Bool
rescheduleAction doWork ts NtfSubscription {ntfSubActionTs}
| ntfSubActionTs <= ts = pure False
| otherwise = do
void . atomically $ tryTakeTMVar doWork
void . forkIO $ do
threadDelay $ diffInMicros actionTs ts
threadDelay $ diffInMicros ntfSubActionTs ts
void . atomically $ tryPutTMVar doWork ()
pure True
@@ -295,26 +209,10 @@ fromPico (MkFixed i) = i
diffInMicros :: UTCTime -> UTCTime -> Int
diffInMicros a b = (`div` 1000000) . fromInteger . fromPico . nominalDiffTimeToSeconds $ diffUTCTime a b
retryOnError :: AgentMonad m => AgentClient -> Text -> m () -> (AgentErrorType -> m ()) -> AgentErrorType -> m ()
retryOnError c name loop done e = do
logError $ name <> " error: " <> tshow e
case e of
BROKER NETWORK -> retryLoop
BROKER TIMEOUT -> retryLoop
_ -> done e
where
retryLoop = do
atomically $ endAgentOperation c AONtfNetwork
atomically $ beginAgentOperation c AONtfNetwork
loop
workerInternalError :: AgentMonad m => AgentClient -> ConnId -> String -> m ()
workerInternalError c connId internalErrStr = do
withStore' c $ \db -> setNullNtfSubscriptionAction db connId
notifyInternalError c connId internalErrStr
notifyInternalError :: (MonadUnliftIO m) => AgentClient -> ConnId -> String -> m ()
notifyInternalError AgentClient {subQ} connId internalErrStr = atomically $ writeTBQueue subQ ("", connId, AP.ERR $ AP.INTERNAL internalErrStr)
ntfInternalError :: AgentMonad m => AgentClient -> ConnId -> String -> m ()
ntfInternalError c@AgentClient {subQ} connId internalErrStr = do
withStore c $ \st -> setNullNtfSubscriptionAction st connId
atomically $ writeTBQueue subQ ("", connId, AP.ERR $ AP.INTERNAL internalErrStr)
getNtfToken :: AgentMonad m => m (Maybe NtfToken)
getNtfToken = do
@@ -328,23 +226,18 @@ nsRemoveNtfToken :: NtfSupervisor -> STM ()
nsRemoveNtfToken ns = writeTVar (ntfTkn ns) Nothing
sendNtfSubCommand :: NtfSupervisor -> (ConnId, NtfSupervisorCommand) -> STM ()
sendNtfSubCommand ns cmd = do
tkn <- readTVar (ntfTkn ns)
when (instantNotifications tkn) $ writeTBQueue (ntfSubQ ns) cmd
instantNotifications :: Maybe NtfToken -> Bool
instantNotifications = \case
Just NtfToken {ntfTknStatus = NTActive, ntfMode = NMInstant} -> True
_ -> False
sendNtfSubCommand ns cmd =
readTVar (ntfTkn ns)
>>= mapM_ (\NtfToken {ntfTknStatus} -> when (ntfTknStatus == NTActive) $ writeTBQueue (ntfSubQ ns) cmd)
closeNtfSupervisor :: NtfSupervisor -> IO ()
closeNtfSupervisor ns = do
cancelNtfWorkers_ $ ntfWorkers ns
cancelNtfWorkers_ $ ntfSMPWorkers ns
cancelNtfWorkers_ :: TMap (ProtocolServer s) (TMVar (), Async ()) -> IO ()
cancelNtfWorkers_ :: TMap ProtocolServer (TMVar (), Async ()) -> IO ()
cancelNtfWorkers_ wsVar = do
ws <- atomically $ stateTVar wsVar (,M.empty)
ws <- atomically $ stateTVar wsVar $ \ws -> (ws, M.empty)
forM_ ws $ uninterruptibleCancel . snd
getNtfServer :: AgentMonad m => AgentClient -> m (Maybe NtfServer)
+60 -146
View File
@@ -6,7 +6,6 @@
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE PolyKinds #-}
@@ -33,6 +32,7 @@
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md
module Simplex.Messaging.Agent.Protocol
( -- * Protocol parameters
currentSMPAgentVersion,
supportedSMPAgentVRange,
e2eEncConnInfoLength,
e2eEncUserMsgLength,
@@ -40,12 +40,10 @@ module Simplex.Messaging.Agent.Protocol
-- * SMP agent protocol types
ConnInfo,
ACommand (..),
ACmd (..),
AParty (..),
SAParty (..),
MsgHash,
MsgMeta (..),
ConnectionStats (..),
SMPConfirmation (..),
AgentMsgEnvelope (..),
AgentMessage (..),
@@ -57,7 +55,6 @@ module Simplex.Messaging.Agent.Protocol
SrvLoc (..),
SMPQueueUri (..),
SMPQueueInfo (..),
SMPQueueAddress (..),
ConnectionMode (..),
SConnectionMode (..),
AConnectionMode (..),
@@ -85,8 +82,7 @@ module Simplex.Messaging.Agent.Protocol
QueueStatus (..),
ACorrId,
AgentMsgId,
NotificationsMode (..),
NotificationInfo (..),
AgentPhase (..),
-- * Encode/decode
serializeCommand,
@@ -97,8 +93,6 @@ module Simplex.Messaging.Agent.Protocol
serializeQueueStatus,
queueStatusT,
agentMessageType,
extraSMPServerHosts,
updateSMPServerHosts,
-- * TCP transport functions
tPut,
@@ -121,19 +115,13 @@ import Data.Composition ((.:), (.:.))
import Data.Functor (($>))
import Data.Int (Int64)
import Data.Kind (Type)
import Data.List.NonEmpty (NonEmpty (..))
import qualified Data.List.NonEmpty as L
import Data.Map (Map)
import qualified Data.Map as M
import Data.Maybe (isJust)
import Data.Text (Text)
import Data.Time.Clock (UTCTime)
import Data.Time.Clock.System (SystemTime)
import Data.Time.ISO8601
import Data.Type.Equality
import Data.Typeable ()
import Database.SQLite.Simple.FromField
import Database.SQLite.Simple.ToField
import GHC.Generics (Generic)
import Generic.Random (genericArbitraryU)
import Simplex.Messaging.Agent.QueryString
@@ -143,24 +131,17 @@ import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers
import Simplex.Messaging.Protocol
( AProtocolType,
ErrorType,
( ErrorType,
MsgBody,
MsgFlags,
MsgId,
NMsgMeta,
ProtocolServer (..),
SMPServer,
SndPublicVerifyKey,
SrvLoc (..),
legacyEncodeServer,
legacyServerP,
legacyStrEncodeServer,
pattern SMPServer,
)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Transport (Transport (..), TransportError, serializeTransportError, transportErrorP)
import Simplex.Messaging.Transport.Client (TransportHost, TransportHosts_ (..))
import Simplex.Messaging.Util
import Simplex.Messaging.Version
import Test.QuickCheck (Arbitrary (..))
@@ -222,17 +203,15 @@ data ACommand (p :: AParty) where
NEW :: AConnectionMode -> ACommand Client -- response INV
INV :: AConnectionRequestUri -> ACommand Agent
JOIN :: AConnectionRequestUri -> ConnInfo -> ACommand Client -- response OK
CONF :: ConfirmationId -> [SMPServer] -> ConnInfo -> ACommand Agent -- ConnInfo is from sender, [SMPServer] will be empty only in v1 handshake
CONF :: ConfirmationId -> ConnInfo -> ACommand Agent -- ConnInfo is from sender
LET :: ConfirmationId -> ConnInfo -> ACommand Client -- ConnInfo is from client
REQ :: InvitationId -> L.NonEmpty SMPServer -> ConnInfo -> ACommand Agent -- ConnInfo is from sender
REQ :: InvitationId -> ConnInfo -> ACommand Agent -- ConnInfo is from sender
ACPT :: InvitationId -> ConnInfo -> ACommand Client -- ConnInfo is from client
RJCT :: InvitationId -> ACommand Client
INFO :: ConnInfo -> ACommand Agent
CON :: ACommand Agent -- notification that connection is established
SUB :: ACommand Client
END :: ACommand Agent
CONNECT :: AProtocolType -> TransportHost -> ACommand Agent
DISCONNECT :: AProtocolType -> TransportHost -> ACommand Agent
DOWN :: SMPServer -> [ConnId] -> ACommand Agent
UP :: SMPServer -> [ConnId] -> ACommand Agent
SEND :: MsgFlags -> MsgBody -> ACommand Client
@@ -243,59 +222,35 @@ data ACommand (p :: AParty) where
ACK :: AgentMsgId -> ACommand Client
OFF :: ACommand Client
DEL :: ACommand Client
CHK :: ACommand Client
STAT :: ConnectionStats -> ACommand Agent
OK :: ACommand Agent
ERR :: AgentErrorType -> ACommand Agent
SUSPENDED :: ACommand Agent
PHASE :: AgentPhase -> ACommand Agent
deriving instance Eq (ACommand p)
deriving instance Show (ACommand p)
data ConnectionStats = ConnectionStats
{ rcvServers :: [SMPServer],
sndServers :: [SMPServer]
}
deriving (Eq, Show, Generic)
instance StrEncoding ConnectionStats where
strEncode ConnectionStats {rcvServers, sndServers} =
"rcv=" <> strEncodeList rcvServers <> " snd=" <> strEncodeList sndServers
strP = do
rcvServers <- "rcv=" *> strListP
sndServers <- " snd=" *> strListP
pure ConnectionStats {rcvServers, sndServers}
instance ToJSON ConnectionStats where toEncoding = J.genericToEncoding J.defaultOptions
data NotificationsMode = NMPeriodic | NMInstant
-- | Agent phase allows to have two agent processes concurrently working with the same database
data AgentPhase
= -- | agent is operating normally
APActive
| -- | agent is paused - no new send/receive operations will be started - they will STM-retry
APPaused
| -- | agent is suspended - no new send/receive/database operations will be started - they will STM-retry
APSuspended
deriving (Eq, Show)
instance StrEncoding NotificationsMode where
instance StrEncoding AgentPhase where
strEncode = \case
NMPeriodic -> "PERIODIC"
NMInstant -> "INSTANT"
APActive -> "ACTIVE"
APPaused -> "PAUSED"
APSuspended -> "SUSPENDED"
strP =
A.takeTill (== ' ') >>= \case
"PERIODIC" -> pure NMPeriodic
"INSTANT" -> pure NMInstant
_ -> fail "bad NotificationsMode"
instance ToJSON NotificationsMode where
toEncoding = strToJEncoding
toJSON = strToJSON
instance ToField NotificationsMode where toField = toField . strEncode
instance FromField NotificationsMode where fromField = blobFieldDecoder $ parseAll strP
data NotificationInfo = NotificationInfo
{ ntfConnId :: ConnId,
ntfTs :: SystemTime,
ntfMsgMeta :: Maybe NMsgMeta
}
deriving (Show)
"ACTIVE" -> pure APActive
"PAUSED" -> pure APPaused
"SUSPENDED" -> pure APSuspended
_ -> fail "bad AgentPhase"
data ConnectionMode = CMInvitation | CMContact
deriving (Eq, Show)
@@ -359,9 +314,7 @@ data SMPConfirmation = SMPConfirmation
-- | sender's information to be associated with the connection, e.g. sender's profile information
connInfo :: ConnInfo,
-- | optional reply queues included in confirmation (added in agent protocol v2)
smpReplyQueues :: [SMPQueueInfo],
-- | SMP client version
smpClientVersion :: Version
smpReplyQueues :: [SMPQueueInfo]
}
deriving (Show)
@@ -589,36 +542,20 @@ type ConfirmationId = ByteString
type InvitationId = ByteString
extraSMPServerHosts :: Map TransportHost TransportHost
extraSMPServerHosts =
M.fromList
[ ("smp4.simplex.im", "o5vmywmrnaxalvz6wi3zicyftgio6psuvyniis6gco6bp6ekl4cqj4id.onion"),
("smp5.simplex.im", "jjbyvoemxysm7qxap7m5d5m35jzv5qq6gnlv7s4rsn7tdwwmuqciwpid.onion"),
("smp6.simplex.im", "bylepyau3ty4czmn77q4fglvperknl4bi2eb2fdy2bh4jxtf32kf73yd.onion"),
("smp8.simplex.im", "beccx4yfxxbvyhqypaavemqurytl6hozr47wfc7uuecacjqdvwpw2xid.onion"),
("smp9.simplex.im", "jssqzccmrcws6bhmn77vgmhfjmhwlyr3u7puw4erkyoosywgl67slqqd.onion"),
("smp10.simplex.im", "rb2pbttocvnbrngnwziclp2f4ckjq65kebafws6g4hy22cdaiv5dwjqd.onion")
]
updateSMPServerHosts :: SMPServer -> SMPServer
updateSMPServerHosts srv@ProtocolServer {host} = case host of
h :| [] -> case M.lookup h extraSMPServerHosts of
Just h' -> srv {host = [h, h']}
_ -> srv
_ -> srv
data SMPQueueInfo = SMPQueueInfo {clientVersion :: Version, queueAddress :: SMPQueueAddress}
data SMPQueueInfo = SMPQueueInfo
{ clientVersion :: Version,
smpServer :: SMPServer,
senderId :: SMP.SenderId,
dhPublicKey :: C.PublicKeyX25519
}
deriving (Eq, Show)
instance Encoding SMPQueueInfo where
smpEncode (SMPQueueInfo clientVersion SMPQueueAddress {smpServer, senderId, dhPublicKey})
| clientVersion > 1 = smpEncode (clientVersion, smpServer, senderId, dhPublicKey)
| otherwise = smpEncode clientVersion <> legacyEncodeServer smpServer <> smpEncode (senderId, dhPublicKey)
smpEncode SMPQueueInfo {clientVersion, smpServer, senderId, dhPublicKey} =
smpEncode (clientVersion, smpServer, senderId, dhPublicKey)
smpP = do
clientVersion <- smpP
smpServer <- if clientVersion > 1 then smpP else updateSMPServerHosts <$> legacyServerP
(senderId, dhPublicKey) <- smpP
pure $ SMPQueueInfo clientVersion SMPQueueAddress {smpServer, senderId, dhPublicKey}
(clientVersion, smpServer, senderId, dhPublicKey) <- smpP
pure SMPQueueInfo {clientVersion, smpServer, senderId, dhPublicKey}
-- This instance seems contrived and there was a temptation to split a common part of both types.
-- But this is created to allow backward and forward compatibility where SMPQueueUri
@@ -627,51 +564,43 @@ instance Encoding SMPQueueInfo where
instance VersionI SMPQueueInfo where
type VersionRangeT SMPQueueInfo = SMPQueueUri
version = clientVersion
toVersionRangeT (SMPQueueInfo _v addr) vr = SMPQueueUri vr addr
toVersionRangeT SMPQueueInfo {smpServer, senderId, dhPublicKey} vr =
SMPQueueUri {clientVRange = vr, smpServer, senderId, dhPublicKey}
instance VersionRangeI SMPQueueUri where
type VersionT SMPQueueUri = SMPQueueInfo
versionRange = clientVRange
toVersionT (SMPQueueUri _vr addr) v = SMPQueueInfo v addr
toVersionT SMPQueueUri {smpServer, senderId, dhPublicKey} v =
SMPQueueInfo {clientVersion = v, smpServer, senderId, dhPublicKey}
-- | SMP queue information sent out-of-band.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#out-of-band-messages
data SMPQueueUri = SMPQueueUri {clientVRange :: VersionRange, queueAddress :: SMPQueueAddress}
deriving (Eq, Show)
data SMPQueueAddress = SMPQueueAddress
data SMPQueueUri = SMPQueueUri
{ smpServer :: SMPServer,
senderId :: SMP.SenderId,
clientVRange :: VersionRange,
dhPublicKey :: C.PublicKeyX25519
}
deriving (Eq, Show)
instance StrEncoding SMPQueueUri where
strEncode (SMPQueueUri vr SMPQueueAddress {smpServer = srv, senderId = qId, dhPublicKey})
| minVersion vr > 1 = strEncode srv <> "/" <> strEncode qId <> "#/?" <> query queryParams
| otherwise = legacyStrEncodeServer srv <> "/" <> strEncode qId <> "#/?" <> query (queryParams <> srvParam)
where
query = strEncode . QSP QEscape
queryParams = [("v", strEncode vr), ("dh", strEncode dhPublicKey)]
srvParam = [("srv", strEncode $ TransportHosts_ hs) | length hs > 0]
hs = L.tail $ host srv
-- v1 uses short SMP queue URI format
strEncode SMPQueueUri {smpServer = srv, senderId = qId, clientVRange = _vr, dhPublicKey = k} =
strEncode srv <> "/" <> strEncode qId <> "#" <> strEncode k
strP = do
srv@ProtocolServer {host = h :| host} <- strP <* A.char '/'
smpServer <- strP <* A.char '/'
senderId <- strP <* optional (A.char '/') <* A.char '#'
(vr, hs, dhPublicKey) <- unversioned <|> versioned
let srv' = srv {host = h :| host <> hs}
smpServer = if maxVersion vr == 1 then updateSMPServerHosts srv' else srv'
pure $ SMPQueueUri vr SMPQueueAddress {smpServer, senderId, dhPublicKey}
(vr, dhPublicKey) <- unversioned <|> versioned
pure SMPQueueUri {smpServer, senderId, clientVRange = vr, dhPublicKey}
where
unversioned = (versionToRange 1,[],) <$> strP <* A.endOfInput
unversioned = (SMP.smpClientVRange,) <$> strP <* A.endOfInput
versioned = do
dhKey_ <- optional strP
query <- optional (A.char '/') *> A.char '?' *> strP
vr <- queryParam "v" query
dhKey <- maybe (queryParam "dh" query) pure dhKey_
hs_ <- queryParam_ "srv" query
pure (vr, maybe [] thList_ hs_, dhKey)
pure (vr, dhKey)
data ConnectionRequestUri (m :: ConnectionMode) where
CRInvitationUri :: ConnReqUriData -> E2ERatchetParamsUri 'C.X448 -> ConnectionRequestUri CMInvitation
@@ -807,7 +736,7 @@ data AgentErrorType
AGENT {agentErr :: SMPAgentError}
| -- | agent implementation or dependency errors
INTERNAL {internalErr :: String}
deriving (Eq, Generic, Show, Exception)
deriving (Eq, Generic, Read, Show, Exception)
instance ToJSON AgentErrorType where
toJSON = J.genericToJSON $ sumTypeJSON id
@@ -857,8 +786,6 @@ data BrokerErrorType
UNEXPECTED
| -- | network error
NETWORK
| -- | no compatible server host (e.g. onion when public is required, or vice versa)
HOST
| -- | handshake or other transport error
TRANSPORT {transportErr :: TransportError}
| -- | command response timeout
@@ -934,10 +861,8 @@ commandP =
<|> "INFO " *> infoCmd
<|> "SUB" $> ACmd SClient SUB
<|> "END" $> ACmd SAgent END
<|> "CONNECT " *> connectResp
<|> "DISCONNECT " *> disconnectResp
<|> "DOWN " *> downResp
<|> "UP " *> upResp
<|> "DOWN " *> downsResp
<|> "UP " *> upsResp
<|> "SEND " *> sendCmd
<|> "MID " *> msgIdResp
<|> "SENT " *> sentResp
@@ -946,8 +871,6 @@ commandP =
<|> "ACK " *> ackCmd
<|> "OFF" $> ACmd SClient OFF
<|> "DEL" $> ACmd SClient DEL
<|> "CHK" $> ACmd SClient CHK
<|> "STAT " *> statResp
<|> "ERR " *> agentError
<|> "CON" $> ACmd SAgent CON
<|> "OK" $> ACmd SAgent OK
@@ -955,24 +878,21 @@ commandP =
newCmd = ACmd SClient . NEW <$> strP
invResp = ACmd SAgent . INV <$> strP
joinCmd = ACmd SClient .: JOIN <$> strP_ <*> A.takeByteString
confMsg = ACmd SAgent .:. CONF <$> A.takeTill (== ' ') <* A.space <*> strListP <* A.space <*> A.takeByteString
confMsg = ACmd SAgent .: CONF <$> A.takeTill (== ' ') <* A.space <*> A.takeByteString
letCmd = ACmd SClient .: LET <$> A.takeTill (== ' ') <* A.space <*> A.takeByteString
reqMsg = ACmd SAgent .:. REQ <$> A.takeTill (== ' ') <* A.space <*> strP_ <*> A.takeByteString
reqMsg = ACmd SAgent .: REQ <$> A.takeTill (== ' ') <* A.space <*> A.takeByteString
acptCmd = ACmd SClient .: ACPT <$> A.takeTill (== ' ') <* A.space <*> A.takeByteString
rjctCmd = ACmd SClient . RJCT <$> A.takeByteString
infoCmd = ACmd SAgent . INFO <$> A.takeByteString
connectResp = ACmd SAgent .: CONNECT <$> strP_ <*> strP
disconnectResp = ACmd SAgent .: DISCONNECT <$> strP_ <*> strP
downResp = ACmd SAgent .: DOWN <$> strP_ <*> connections
upResp = ACmd SAgent .: UP <$> strP_ <*> connections
downsResp = ACmd SAgent .: DOWN <$> strP_ <*> connections
upsResp = ACmd SAgent .: UP <$> strP_ <*> connections
sendCmd = ACmd SClient .: SEND <$> smpP <* A.space <*> A.takeByteString
msgIdResp = ACmd SAgent . MID <$> A.decimal
sentResp = ACmd SAgent . SENT <$> A.decimal
msgErrResp = ACmd SAgent .: MERR <$> A.decimal <* A.space <*> strP
message = ACmd SAgent .:. MSG <$> msgMetaP <* A.space <*> smpP <* A.space <*> A.takeByteString
ackCmd = ACmd SClient . ACK <$> A.decimal
statResp = ACmd SAgent . STAT <$> strP
connections = strP `A.sepBy'` A.char ','
connections = strP `A.sepBy'` (A.char ',')
msgMetaP = do
integrity <- strP
recipient <- " R=" *> partyMeta A.decimal
@@ -991,16 +911,14 @@ serializeCommand = \case
NEW cMode -> "NEW " <> strEncode cMode
INV cReq -> "INV " <> strEncode cReq
JOIN cReq cInfo -> B.unwords ["JOIN", strEncode cReq, serializeBinary cInfo]
CONF confId srvs cInfo -> B.unwords ["CONF", confId, strEncodeList srvs, serializeBinary cInfo]
CONF confId cInfo -> B.unwords ["CONF", confId, serializeBinary cInfo]
LET confId cInfo -> B.unwords ["LET", confId, serializeBinary cInfo]
REQ invId srvs cInfo -> B.unwords ["REQ", invId, strEncode srvs, serializeBinary cInfo]
REQ invId cInfo -> B.unwords ["REQ", invId, serializeBinary cInfo]
ACPT invId cInfo -> B.unwords ["ACPT", invId, serializeBinary cInfo]
RJCT invId -> "RJCT " <> invId
INFO cInfo -> "INFO " <> serializeBinary cInfo
SUB -> "SUB"
END -> "END"
CONNECT p h -> B.unwords ["CONNECT", strEncode p, strEncode h]
DISCONNECT p h -> B.unwords ["DISCONNECT", strEncode p, strEncode h]
DOWN srv conns -> B.unwords ["DOWN", strEncode srv, connections conns]
UP srv conns -> B.unwords ["UP", strEncode srv, connections conns]
SEND msgFlags msgBody -> "SEND " <> smpEncode msgFlags <> " " <> serializeBinary msgBody
@@ -1011,12 +929,10 @@ serializeCommand = \case
ACK mId -> "ACK " <> bshow mId
OFF -> "OFF"
DEL -> "DEL"
CHK -> "CHK"
STAT srvs -> "STAT " <> strEncode srvs
CON -> "CON"
ERR e -> "ERR " <> strEncode e
OK -> "OK"
SUSPENDED -> "SUSPENDED"
PHASE p -> "PHASE " <> strEncode p
where
showTs :: UTCTime -> ByteString
showTs = B.pack . formatISO8601Millis
@@ -1073,8 +989,6 @@ tGet party h = liftIO (tGetRaw h) >>= tParseLoadBody
ACPT {} -> Right cmd
-- ERROR response does not always have connId
ERR _ -> Right cmd
CONNECT {} -> Right cmd
DISCONNECT {} -> Right cmd
DOWN {} -> Right cmd
UP {} -> Right cmd
-- other responses must have connId
@@ -1087,9 +1001,9 @@ tGet party h = liftIO (tGetRaw h) >>= tParseLoadBody
SEND msgFlags body -> SEND msgFlags <$$> getBody body
MSG msgMeta msgFlags body -> MSG msgMeta msgFlags <$$> getBody body
JOIN qUri cInfo -> JOIN qUri <$$> getBody cInfo
CONF confId srvs cInfo -> CONF confId srvs <$$> getBody cInfo
CONF confId cInfo -> CONF confId <$$> getBody cInfo
LET confId cInfo -> LET confId <$$> getBody cInfo
REQ invId srvs cInfo -> REQ invId srvs <$$> getBody cInfo
REQ invId cInfo -> REQ invId <$$> getBody cInfo
ACPT invId cInfo -> ACPT invId <$$> getBody cInfo
INFO cInfo -> INFO <$$> getBody cInfo
cmd -> pure $ Right cmd
@@ -28,9 +28,3 @@ queryParam name (QSP _ q) =
case find ((== name) . fst) q of
Just (_, p) -> either fail pure $ parseAll strP p
_ -> fail $ "no qs param " <> B.unpack name
queryParam_ :: StrEncoding a => ByteString -> QueryStringParams -> Parser (Maybe a)
queryParam_ name (QSP _ q) =
case find ((== name) . fst) q of
Just (_, p) -> either fail pure $ parseAll strP p
_ -> pure Nothing
+80 -19
View File
@@ -9,7 +9,9 @@
module Simplex.Messaging.Agent.Store where
import Control.Concurrent.STM (TVar)
import Control.Exception (Exception)
import Crypto.Random (ChaChaDRG)
import Data.ByteString.Char8 (ByteString)
import Data.Int (Int64)
import Data.Kind (Type)
@@ -17,7 +19,9 @@ import Data.Time (UTCTime)
import Data.Type.Equality
import Simplex.Messaging.Agent.Protocol
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Crypto.Ratchet (RatchetX448)
import Simplex.Messaging.Crypto.Ratchet (RatchetX448, SkippedMsgDiff, SkippedMsgKeys)
import Simplex.Messaging.Notifications.Client
import Simplex.Messaging.Notifications.Protocol (DeviceToken, NtfTknStatus, NtfTokenId)
import Simplex.Messaging.Protocol
( MsgBody,
MsgFlags,
@@ -26,13 +30,82 @@ import Simplex.Messaging.Protocol
NtfPrivateSignKey,
NtfPublicVerifyKey,
RcvDhSecret,
RcvNtfDhSecret,
RcvPrivateSignKey,
SndPrivateSignKey,
)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Version
-- * Store management
-- | Store class type. Defines store access methods for implementations.
class Monad m => MonadAgentStore s m where
-- Queue and Connection management
createRcvConn :: s -> TVar ChaChaDRG -> ConnData -> RcvQueue -> SConnectionMode c -> m ConnId
createSndConn :: s -> TVar ChaChaDRG -> ConnData -> SndQueue -> m ConnId
getConn :: s -> ConnId -> m SomeConn
getRcvConn :: s -> SMPServer -> SMP.RecipientId -> m SomeConn
deleteConn :: s -> ConnId -> m ()
upgradeRcvConnToDuplex :: s -> ConnId -> SndQueue -> m ()
upgradeSndConnToDuplex :: s -> ConnId -> RcvQueue -> m ()
setRcvQueueStatus :: s -> RcvQueue -> QueueStatus -> m ()
setRcvQueueConfirmedE2E :: s -> RcvQueue -> C.DhSecretX25519 -> m ()
setSndQueueStatus :: s -> SndQueue -> QueueStatus -> m ()
getRcvQueue :: s -> ConnId -> m RcvQueue
-- RcvQueue notifier key and ID
setRcvQueueNotifierKey :: s -> ConnId -> NtfPublicVerifyKey -> NtfPrivateSignKey -> m ()
setRcvQueueNotifierId :: s -> ConnId -> NotifierId -> m ()
-- Confirmations
createConfirmation :: s -> TVar ChaChaDRG -> NewConfirmation -> m ConfirmationId
acceptConfirmation :: s -> ConfirmationId -> ConnInfo -> m AcceptedConfirmation
getAcceptedConfirmation :: s -> ConnId -> m AcceptedConfirmation
removeConfirmations :: s -> ConnId -> m ()
setHandshakeVersion :: s -> ConnId -> Version -> Bool -> m ()
-- Invitations - sent via Contact connections
createInvitation :: s -> TVar ChaChaDRG -> NewInvitation -> m InvitationId
getInvitation :: s -> InvitationId -> m Invitation
acceptInvitation :: s -> InvitationId -> ConnInfo -> m ()
deleteInvitation :: s -> ConnId -> InvitationId -> m ()
-- Msg management
updateRcvIds :: s -> ConnId -> m (InternalId, InternalRcvId, PrevExternalSndId, PrevRcvMsgHash)
createRcvMsg :: s -> ConnId -> RcvMsgData -> m ()
updateSndIds :: s -> ConnId -> m (InternalId, InternalSndId, PrevSndMsgHash)
createSndMsg :: s -> ConnId -> SndMsgData -> m ()
getPendingMsgData :: s -> ConnId -> InternalId -> m (Maybe RcvQueue, PendingMsgData)
getPendingMsgs :: s -> ConnId -> m [InternalId]
setMsgUserAck :: s -> ConnId -> InternalId -> m MsgId
getLastMsg :: s -> ConnId -> SMP.MsgId -> m (Maybe RcvMsg)
deleteMsg :: s -> ConnId -> InternalId -> m ()
-- Double ratchet persistence
createRatchetX3dhKeys :: s -> ConnId -> C.PrivateKeyX448 -> C.PrivateKeyX448 -> m ()
getRatchetX3dhKeys :: s -> ConnId -> m (C.PrivateKeyX448, C.PrivateKeyX448)
createRatchet :: s -> ConnId -> RatchetX448 -> m ()
getRatchet :: s -> ConnId -> m RatchetX448
getSkippedMsgKeys :: s -> ConnId -> m SkippedMsgKeys
updateRatchet :: s -> ConnId -> RatchetX448 -> SkippedMsgDiff -> m ()
-- Notification device token persistence
createNtfToken :: s -> NtfToken -> m ()
getDeviceNtfToken :: s -> DeviceToken -> m (Maybe NtfToken, [NtfToken])
updateNtfTokenRegistration :: s -> NtfToken -> NtfTokenId -> C.DhSecretX25519 -> m ()
updateNtfToken :: s -> NtfToken -> NtfTknStatus -> Maybe NtfTknAction -> m ()
removeNtfToken :: s -> NtfToken -> m ()
-- Notification subscription persistence
getNtfSubscription :: s -> ConnId -> m (Maybe NtfSubscription)
createNtfSubscription :: s -> NtfSubscription -> NtfSubOrSMPAction -> m ()
markNtfSubscriptionForDeletion :: s -> ConnId -> m ()
updateNtfSubscription :: s -> ConnId -> NtfSubscription -> NtfSubOrSMPAction -> m ()
setNullNtfSubscriptionAction :: s -> ConnId -> m ()
deleteNtfSubscription :: s -> ConnId -> m ()
getNextNtfSubAction :: s -> NtfServer -> m (Maybe (NtfSubscription, NtfSubAction, RcvQueue))
getNextNtfSubSMPAction :: s -> SMPServer -> m (Maybe (NtfSubscription, NtfSubSMPAction, RcvQueue))
-- * Queue types
-- | A receive queue. SMP queue through which the agent receives messages from a sender.
@@ -52,21 +125,11 @@ data RcvQueue = RcvQueue
sndId :: Maybe SMP.SenderId,
-- | queue status
status :: QueueStatus,
-- | SMP client version
smpClientVersion :: Version,
-- | credentials used in context of notifications
clientNtfCreds :: Maybe ClientNtfCreds
}
deriving (Eq, Show)
data ClientNtfCreds = ClientNtfCreds
{ -- | key pair to be used by the notification server to sign transmissions
ntfPublicKey :: NtfPublicVerifyKey,
ntfPrivateKey :: NtfPrivateSignKey,
-- | key pair to be used by the notification server to sign transmissions
ntfPublicKey :: Maybe NtfPublicVerifyKey,
ntfPrivateKey :: Maybe NtfPrivateSignKey,
-- | queue ID to be used by the notification server for NSUB command
notifierId :: NotifierId,
-- | shared DH secret used to encrypt/decrypt notification metadata (NMsgMeta) from server to recipient
rcvNtfDhSecret :: RcvNtfDhSecret
notifierId :: Maybe NotifierId
}
deriving (Eq, Show)
@@ -83,9 +146,7 @@ data SndQueue = SndQueue
-- | shared DH secret agreed for simple per-queue e2e encryption
e2eDhSecret :: C.DhSecretX25519,
-- | queue status
status :: QueueStatus,
-- | SMP client version
smpClientVersion :: Version
status :: QueueStatus
}
deriving (Eq, Show)
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,5 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes #-}
{-# LANGUAGE ScopedTypeVariables #-}
@@ -17,26 +16,18 @@ module Simplex.Messaging.Agent.Store.SQLite.Migrations
where
import Control.Monad (forM_)
import Data.Function (on)
import Data.List (intercalate, sortBy)
import Data.List.NonEmpty (NonEmpty)
import qualified Data.Map as M
import Data.Ord (comparing)
import Data.Text (Text)
import Data.Text.Encoding (decodeLatin1)
import Data.Time.Clock (getCurrentTime)
import Database.SQLite.Simple (Connection, Only (..), Query (..))
import qualified Database.SQLite.Simple as DB
import Database.SQLite.Simple.QQ (sql)
import qualified Database.SQLite3 as SQLite3
import Simplex.Messaging.Agent.Protocol (extraSMPServerHosts)
import Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220101_initial
import Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220301_snd_queue_keys
import Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220322_notifications
import Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220608_v2
import Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220625_v2_ntf_mode
import Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220811_onion_hosts
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Transport.Client (TransportHost)
data Migration = Migration {name :: String, up :: Text}
deriving (Show)
@@ -46,14 +37,12 @@ schemaMigrations =
[ ("20220101_initial", m20220101_initial),
("20220301_snd_queue_keys", m20220301_snd_queue_keys),
("20220322_notifications", m20220322_notifications),
("20220607_v2", m20220608_v2),
("m20220625_v2_ntf_mode", m20220625_v2_ntf_mode),
("m20220811_onion_hosts", m20220811_onion_hosts)
("20220607_v2", m20220608_v2)
]
-- | The list of migrations in ascending order by date
app :: [Migration]
app = sortBy (comparing name) $ map migration schemaMigrations
app = sortBy (compare `on` name) $ map migration schemaMigrations
where
migration (name, query) = Migration {name = name, up = fromQuery query}
@@ -64,15 +53,10 @@ get conn migrations =
run :: Connection -> [Migration] -> IO ()
run conn ms = DB.withImmediateTransaction conn . forM_ ms $
\Migration {name, up} -> insert name >> execSQL up >> updateServers name
\Migration {name, up} -> insert name >> execSQL up
where
insert name = DB.execute conn "INSERT INTO migrations (name, ts) VALUES (?, ?);" . (name,) =<< getCurrentTime
execSQL = SQLite3.exec $ DB.connectionHandle conn
updateServers = \case
"m20220811_onion_hosts" -> forM_ (M.assocs extraSMPServerHosts) $ \(h, h') ->
let hs = decodeLatin1 . strEncode $ ([h, h'] :: NonEmpty TransportHost)
in DB.execute conn "UPDATE servers SET host = ? WHERE host = ?" (hs, decodeLatin1 $ strEncode h)
_ -> pure ()
initialize :: Connection -> IO ()
initialize conn =
@@ -18,7 +18,7 @@ CREATE TABLE ntf_servers (
) WITHOUT ROWID;
CREATE TABLE ntf_tokens (
provider TEXT NOT NULL, -- apns
provider TEXT NOT NULL, -- apn
device_token TEXT NOT NULL, -- ! this field is mislabeled and is actually saved as binary
ntf_host TEXT NOT NULL,
ntf_port TEXT NOT NULL,
@@ -22,12 +22,10 @@ ALTER TABLE rcv_queues ADD COLUMN ntf_private_key BLOB;
ALTER TABLE rcv_queues ADD COLUMN ntf_id BLOB;
ALTER TABLE rcv_queues ADD COLUMN rcv_ntf_dh_secret BLOB;
CREATE UNIQUE INDEX idx_rcv_queues_ntf ON rcv_queues (host, port, ntf_id);
CREATE TABLE ntf_subscriptions (
conn_id BLOB NOT NULL,
conn_id BLOB NOT NULL, -- ? make nullable for when connection is deleted but we still need to delete subscription
smp_host TEXT NULL,
smp_port TEXT NULL,
smp_ntf_id BLOB,
@@ -35,13 +33,15 @@ CREATE TABLE ntf_subscriptions (
ntf_port TEXT NOT NULL,
ntf_sub_id BLOB,
ntf_sub_status TEXT NOT NULL, -- see NtfAgentSubStatus
ntf_sub_action TEXT, -- if there is an action required on this subscription: NtfSubNTFAction
ntf_sub_action TEXT, -- if there is an action required on this subscription: NtfSubAction
ntf_sub_smp_action TEXT, -- action with SMP server: NtfSubSMPAction; only one of this and ntf_sub_action can (should) be not null in same record
ntf_sub_action_ts TEXT, -- the earliest time for the action, e.g. checks can be scheduled every X hours
updated_by_supervisor INTEGER NOT NULL DEFAULT 0, -- to be checked on updates by workers to not overwrite supervisor command (state still should be updated)
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (conn_id),
-- FOREIGN KEY (conn_id) REFERENCES connections
-- ON DELETE SET NULL ON UPDATE CASCADE,
FOREIGN KEY (smp_host, smp_port) REFERENCES servers (host, port)
ON DELETE SET NULL ON UPDATE CASCADE,
FOREIGN KEY (ntf_host, ntf_port) REFERENCES ntf_servers
@@ -1,14 +0,0 @@
{-# LANGUAGE QuasiQuotes #-}
module Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220625_v2_ntf_mode where
import Database.SQLite.Simple (Query)
import Database.SQLite.Simple.QQ (sql)
m20220625_v2_ntf_mode :: Query
m20220625_v2_ntf_mode =
[sql|
ALTER TABLE ntf_tokens ADD COLUMN ntf_mode TEXT NULL;
DELETE FROM ntf_tokens;
|]
@@ -1,16 +0,0 @@
{-# LANGUAGE QuasiQuotes #-}
module Simplex.Messaging.Agent.Store.SQLite.Migrations.M20220811_onion_hosts where
import Database.SQLite.Simple (Query)
import Database.SQLite.Simple.QQ (sql)
m20220811_onion_hosts :: Query
m20220811_onion_hosts =
[sql|
ALTER TABLE conn_confirmations ADD COLUMN smp_client_version INTEGER;
UPDATE ntf_servers
SET ntf_host = 'ntf2.simplex.im,ntg7jdjy2i3qbib3sykiho3enekwiaqg3icctliqhtqcg6jmoh6cxiad.onion'
WHERE ntf_host = 'ntf2.simplex.im';
|]
@@ -39,7 +39,6 @@ CREATE TABLE rcv_queues(
ntf_public_key BLOB,
ntf_private_key BLOB,
ntf_id BLOB,
rcv_ntf_dh_secret BLOB,
PRIMARY KEY(host, port, rcv_id),
FOREIGN KEY(host, port) REFERENCES servers
ON DELETE RESTRICT ON UPDATE CASCADE,
@@ -113,8 +112,7 @@ CREATE TABLE conn_confirmations(
own_conn_info BLOB,
created_at TEXT NOT NULL DEFAULT(datetime('now'))
,
smp_reply_queues BLOB NULL,
smp_client_version INTEGER
smp_reply_queues BLOB NULL
) WITHOUT ROWID;
CREATE TABLE conn_invitations(
invitation_id BLOB NOT NULL PRIMARY KEY,
@@ -152,7 +150,7 @@ CREATE TABLE ntf_servers(
PRIMARY KEY(ntf_host, ntf_port)
) WITHOUT ROWID;
CREATE TABLE ntf_tokens(
provider TEXT NOT NULL, -- apns
provider TEXT NOT NULL, -- apn
device_token TEXT NOT NULL, -- ! this field is mislabeled and is actually saved as binary
ntf_host TEXT NOT NULL,
ntf_port TEXT NOT NULL,
@@ -165,15 +163,14 @@ tkn_dh_secret BLOB, -- DH secret for e2e encryption of notifications
tkn_status TEXT NOT NULL,
tkn_action BLOB,
created_at TEXT NOT NULL DEFAULT(datetime('now')),
updated_at TEXT NOT NULL DEFAULT(datetime('now')),
ntf_mode TEXT NULL, -- this is to check token status periodically to know when it was last checked
updated_at TEXT NOT NULL DEFAULT(datetime('now')), -- this is to check token status periodically to know when it was last checked
PRIMARY KEY(provider, device_token, ntf_host, ntf_port),
FOREIGN KEY(ntf_host, ntf_port) REFERENCES ntf_servers
ON DELETE RESTRICT ON UPDATE CASCADE
) WITHOUT ROWID;
CREATE UNIQUE INDEX idx_rcv_queues_ntf ON rcv_queues(host, port, ntf_id);
CREATE TABLE ntf_subscriptions(
conn_id BLOB NOT NULL,
conn_id BLOB NOT NULL, -- ? make nullable for when connection is deleted but we still need to delete subscription
smp_host TEXT NULL,
smp_port TEXT NULL,
smp_ntf_id BLOB,
@@ -181,13 +178,15 @@ CREATE TABLE ntf_subscriptions(
ntf_port TEXT NOT NULL,
ntf_sub_id BLOB,
ntf_sub_status TEXT NOT NULL, -- see NtfAgentSubStatus
ntf_sub_action TEXT, -- if there is an action required on this subscription: NtfSubNTFAction
ntf_sub_action TEXT, -- if there is an action required on this subscription: NtfSubAction
ntf_sub_smp_action TEXT, -- action with SMP server: NtfSubSMPAction; only one of this and ntf_sub_action can(should) be not null in same record
ntf_sub_action_ts TEXT, -- the earliest time for the action, e.g. checks can be scheduled every X hours
updated_by_supervisor INTEGER NOT NULL DEFAULT 0, -- to be checked on updates by workers to not overwrite supervisor command(state still should be updated)
created_at TEXT NOT NULL DEFAULT(datetime('now')),
updated_at TEXT NOT NULL DEFAULT(datetime('now')),
PRIMARY KEY(conn_id),
-- FOREIGN KEY(conn_id) REFERENCES connections
-- ON DELETE SET NULL ON UPDATE CASCADE,
FOREIGN KEY(smp_host, smp_port) REFERENCES servers(host, port)
ON DELETE SET NULL ON UPDATE CASCADE,
FOREIGN KEY(ntf_host, ntf_port) REFERENCES ntf_servers
+79 -220
View File
@@ -1,13 +1,11 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
@@ -26,7 +24,7 @@
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
module Simplex.Messaging.Client
( -- * Connect (disconnect) client to (from) SMP server
ProtocolClient (thVersion, sessionId, transportHost),
ProtocolClient (sessionId),
SMPClient,
getProtocolClient,
closeProtocolClient,
@@ -34,14 +32,10 @@ module Simplex.Messaging.Client
-- * SMP protocol command functions
createSMPQueue,
subscribeSMPQueue,
subscribeSMPQueues,
getSMPMessage,
subscribeSMPQueueNotifications,
secureSMPQueue,
enableSMPQueueNotifications,
disableSMPQueueNotifications,
enableSMPQueuesNtfs,
disableSMPQueuesNtfs,
sendSMPMessage,
ackSMPMessage,
suspendSMPQueue,
@@ -51,10 +45,7 @@ module Simplex.Messaging.Client
-- * Supporting types and client configuration
ProtocolClientError (..),
ProtocolClientConfig (..),
NetworkConfig (..),
defaultClientConfig,
defaultNetworkConfig,
chooseTransportHost,
ServerTransmission,
)
where
@@ -64,28 +55,19 @@ import Control.Concurrent.Async
import Control.Concurrent.STM
import Control.Exception
import Control.Monad
import Control.Monad.IO.Class (liftIO)
import Control.Monad.Trans.Class
import Control.Monad.Trans.Except
import Data.Aeson (FromJSON (..), ToJSON (..))
import qualified Data.Aeson as J
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Either (rights)
import Data.Functor (($>))
import Data.List (find)
import Data.List.NonEmpty (NonEmpty)
import qualified Data.List.NonEmpty as L
import Data.Maybe (fromMaybe)
import GHC.Generics (Generic)
import Network.Socket (ServiceName)
import Numeric.Natural
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Parsers (dropPrefix, enumJSON)
import Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.Client (SocksProxy, TransportHost (..), runTransportClient)
import Simplex.Messaging.Transport.Client (runTransportClient)
import Simplex.Messaging.Transport.KeepAlive
import Simplex.Messaging.Transport.WebSockets (WS)
import Simplex.Messaging.Util (bshow, liftError, raceAny_)
@@ -100,74 +82,19 @@ data ProtocolClient msg = ProtocolClient
connected :: TVar Bool,
sessionId :: SessionId,
thVersion :: Version,
protocolServer :: ProtoServer msg,
transportHost :: TransportHost,
protocolServer :: ProtocolServer,
tcpTimeout :: Int,
clientCorrId :: TVar Natural,
sentCommands :: TMap CorrId (Request msg),
sndQ :: TBQueue (NonEmpty (SentRawTransmission)),
rcvQ :: TBQueue (NonEmpty (SignedTransmission msg)),
sndQ :: TBQueue SentRawTransmission,
rcvQ :: TBQueue (SignedTransmission msg),
msgQ :: Maybe (TBQueue (ServerTransmission msg))
}
type SMPClient = ProtocolClient SMP.BrokerMsg
-- | Type for client command data
type ClientCommand msg = (Maybe C.APrivateSignKey, QueueId, ProtoCommand msg)
-- | Type synonym for transmission from some SPM server queue.
type ServerTransmission msg = (ProtoServer msg, Version, SessionId, QueueId, msg)
data HostMode
= -- | prefer (or require) onion hosts when connecting via SOCKS proxy
HMOnionViaSocks
| -- | prefer (or require) onion hosts
HMOnion
| -- | prefer (or require) public hosts
HMPublic
deriving (Eq, Show, Generic)
instance FromJSON HostMode where
parseJSON = J.genericParseJSON . enumJSON $ dropPrefix "HM"
instance ToJSON HostMode where
toJSON = J.genericToJSON . enumJSON $ dropPrefix "HM"
toEncoding = J.genericToEncoding . enumJSON $ dropPrefix "HM"
-- | network configuration for the client
data NetworkConfig = NetworkConfig
{ -- | use SOCKS5 proxy
socksProxy :: Maybe SocksProxy,
-- | determines critera which host is chosen from the list
hostMode :: HostMode,
-- | if above criteria is not met, if the below setting is True return error, otherwise use the first host
requiredHostMode :: Bool,
-- | timeout for the initial client TCP/TLS connection (microseconds)
tcpConnectTimeout :: Int,
-- | timeout of protocol commands (microseconds)
tcpTimeout :: Int,
-- | TCP keep-alive options, Nothing to skip enabling keep-alive
tcpKeepAlive :: Maybe KeepAliveOpts,
-- | period for SMP ping commands (microseconds)
smpPingInterval :: Int
}
deriving (Eq, Show, Generic, FromJSON)
instance ToJSON NetworkConfig where
toJSON = J.genericToJSON J.defaultOptions {J.omitNothingFields = True}
toEncoding = J.genericToEncoding J.defaultOptions {J.omitNothingFields = True}
defaultNetworkConfig :: NetworkConfig
defaultNetworkConfig =
NetworkConfig
{ socksProxy = Nothing,
hostMode = HMOnionViaSocks,
requiredHostMode = False,
tcpConnectTimeout = 7_500_000,
tcpTimeout = 5_000_000,
tcpKeepAlive = Just defaultKeepAliveOpts,
smpPingInterval = 600_000_000 -- 10min
}
type ServerTransmission msg = (ProtocolServer, SessionId, QueueId, msg)
-- | protocol client configuration.
data ProtocolClientConfig = ProtocolClientConfig
@@ -175,8 +102,12 @@ data ProtocolClientConfig = ProtocolClientConfig
qSize :: Natural,
-- | default server port if port is not specified in ProtocolServer
defaultTransport :: (ServiceName, ATransport),
-- | network configuration
networkConfig :: NetworkConfig,
-- | timeout of TCP commands (microseconds)
tcpTimeout :: Int,
-- | TCP keep-alive options, Nothing to skip enabling keep-alive
tcpKeepAlive :: Maybe KeepAliveOpts,
-- | period for SMP ping commands (microseconds)
smpPing :: Int,
-- | SMP client-server protocol version range
smpServerVRange :: VersionRange
}
@@ -187,7 +118,9 @@ defaultClientConfig =
ProtocolClientConfig
{ qSize = 64,
defaultTransport = ("443", transport @TLS),
networkConfig = defaultNetworkConfig,
tcpTimeout = 5_000_000,
tcpKeepAlive = Just defaultKeepAliveOpts,
smpPing = 600_000_000, -- 10min
smpServerVRange = supportedSMPServerVRange
}
@@ -198,36 +131,18 @@ data Request msg = Request
type Response msg = Either ProtocolClientError msg
chooseTransportHost :: NetworkConfig -> NonEmpty TransportHost -> Either ProtocolClientError TransportHost
chooseTransportHost NetworkConfig {socksProxy, hostMode, requiredHostMode} hosts =
firstOrError $ case hostMode of
HMOnionViaSocks -> maybe publicHost (const onionHost) socksProxy
HMOnion -> onionHost
HMPublic -> publicHost
where
firstOrError
| requiredHostMode = maybe (Left PCEIncompatibleHost) Right
| otherwise = Right . fromMaybe (L.head hosts)
isOnionHost = \case THOnionHost _ -> True; _ -> False
onionHost = find isOnionHost hosts
publicHost = find (not . isOnionHost) hosts
-- | Connects to 'ProtocolServer' using passed client configuration
-- and queue for messages and notifications.
--
-- A single queue can be used for multiple 'SMPClient' instances,
-- as 'SMPServerTransmission' includes server information.
getProtocolClient :: forall msg. Protocol msg => ProtoServer msg -> ProtocolClientConfig -> Maybe (TBQueue (ServerTransmission msg)) -> (ProtocolClient msg -> IO ()) -> IO (Either ProtocolClientError (ProtocolClient msg))
getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, networkConfig, smpServerVRange} msgQ disconnected = do
case chooseTransportHost networkConfig (host protocolServer) of
Right useHost ->
(atomically (mkProtocolClient useHost) >>= runClient useTransport useHost)
`catch` \(e :: IOException) -> pure . Left $ PCEIOError e
Left e -> pure $ Left e
getProtocolClient :: forall msg. Protocol msg => ProtocolServer -> ProtocolClientConfig -> Maybe (TBQueue (ServerTransmission msg)) -> IO () -> IO (Either ProtocolClientError (ProtocolClient msg))
getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, tcpTimeout, tcpKeepAlive, smpPing, smpServerVRange} msgQ disconnected =
(atomically mkProtocolClient >>= runClient useTransport)
`catch` \(e :: IOException) -> pure . Left $ PCEIOError e
where
NetworkConfig {tcpConnectTimeout, tcpTimeout, tcpKeepAlive, socksProxy, smpPingInterval} = networkConfig
mkProtocolClient :: TransportHost -> STM (ProtocolClient msg)
mkProtocolClient transportHost = do
mkProtocolClient :: STM (ProtocolClient msg)
mkProtocolClient = do
connected <- newTVar False
clientCorrId <- newTVar 0
sentCommands <- TM.empty
@@ -240,7 +155,6 @@ getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, networkConfig,
thVersion = undefined,
connected,
protocolServer,
transportHost,
tcpTimeout,
clientCorrId,
sentCommands,
@@ -249,14 +163,14 @@ getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, networkConfig,
msgQ
}
runClient :: (ServiceName, ATransport) -> TransportHost -> ProtocolClient msg -> IO (Either ProtocolClientError (ProtocolClient msg))
runClient (port', ATransport t) useHost c = do
runClient :: (ServiceName, ATransport) -> ProtocolClient msg -> IO (Either ProtocolClientError (ProtocolClient msg))
runClient (port', ATransport t) c = do
thVar <- newEmptyTMVarIO
action <-
async $
runTransportClient socksProxy useHost port' (Just $ keyHash protocolServer) tcpKeepAlive (client t c thVar)
runTransportClient (host protocolServer) port' (Just $ keyHash protocolServer) tcpKeepAlive (client t c thVar)
`finally` atomically (putTMVar thVar $ Left PCENetworkError)
th_ <- tcpConnectTimeout `timeout` atomically (takeTMVar thVar)
th_ <- tcpTimeout `timeout` atomically (takeTMVar thVar)
pure $ case th_ of
Just (Right THandle {sessionId, thVersion}) -> Right c {action, sessionId, thVersion}
Just (Left e) -> Left e
@@ -279,7 +193,7 @@ getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, networkConfig,
let c' = c {sessionId, thVersion} :: ProtocolClient msg
-- TODO remove ping if 0 is passed (or Nothing?)
raceAny_ [send c' th, process c', receive c' th, ping c']
`finally` disconnected c'
`finally` disconnected
send :: Transport c => ProtocolClient msg -> THandle c -> IO ()
send ProtocolClient {sndQ} h = forever $ atomically (readTBQueue sndQ) >>= tPut h
@@ -289,19 +203,17 @@ getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, networkConfig,
ping :: ProtocolClient msg -> IO ()
ping c = forever $ do
threadDelay smpPingInterval
threadDelay smpPing
runExceptT $ sendProtocolCommand c Nothing "" protocolPing
process :: ProtocolClient msg -> IO ()
process c = forever $ atomically (readTBQueue $ rcvQ c) >>= mapM_ (processMsg c)
processMsg :: ProtocolClient msg -> SignedTransmission msg -> IO ()
processMsg c@ProtocolClient {sentCommands} (_, _, (corrId, qId, respOrErr)) =
process ProtocolClient {sessionId, rcvQ, sentCommands} = forever $ do
(_, _, (corrId, qId, respOrErr)) <- atomically $ readTBQueue rcvQ
if B.null $ bs corrId
then sendMsg respOrErr
then sendMsg qId respOrErr
else do
atomically (TM.lookup corrId sentCommands) >>= \case
Nothing -> sendMsg respOrErr
Nothing -> sendMsg qId respOrErr
Just Request {queueId, responseVar} -> atomically $ do
TM.delete corrId sentCommands
putTMVar responseVar $
@@ -311,11 +223,11 @@ getProtocolClient protocolServer cfg@ProtocolClientConfig {qSize, networkConfig,
Right r -> case protocolError r of
Just e -> Left $ PCEProtocolError e
_ -> Right r
else Left . PCEUnexpectedResponse $ bshow respOrErr
else Left PCEUnexpectedResponse
where
sendMsg :: Either ErrorType msg -> IO ()
sendMsg = \case
Right msg -> atomically $ mapM_ (`writeTBQueue` serverTransmission c qId msg) msgQ
sendMsg :: QueueId -> Either ErrorType msg -> IO ()
sendMsg qId = \case
Right cmd -> atomically $ mapM_ (`writeTBQueue` (protocolServer, sessionId, qId, cmd)) msgQ
-- TODO send everything else to errQ and log in agent
_ -> return ()
@@ -335,15 +247,13 @@ data ProtocolClientError
-- e.g. server should respond `IDS` or `ERR` to `NEW` command,
-- other responses would result in this error.
-- Forwarded to the agent client as `ERR BROKER UNEXPECTED`.
PCEUnexpectedResponse ByteString
PCEUnexpectedResponse
| -- | Used for TCP connection and command response timeouts.
-- Forwarded to the agent client as `ERR BROKER TIMEOUT`.
PCEResponseTimeout
| -- | Failure to establish TCP connection.
-- Forwarded to the agent client as `ERR BROKER NETWORK`.
PCENetworkError
| -- | No host compatible with network configuration
PCEIncompatibleHost
| -- | TCP transport handshake or some other transport error.
-- Forwarded to the agent client as `ERR BROKER TRANSPORT e`.
PCETransportError TransportError
@@ -365,45 +275,30 @@ createSMPQueue ::
createSMPQueue c rpKey rKey dhKey =
sendSMPCommand c (Just rpKey) "" (NEW rKey dhKey) >>= \case
IDS qik -> pure qik
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
-- | Subscribe to the SMP queue.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#subscribe-to-queue
subscribeSMPQueue :: SMPClient -> RcvPrivateSignKey -> RecipientId -> ExceptT ProtocolClientError IO ()
subscribeSMPQueue c rpKey rId =
subscribeSMPQueue c@ProtocolClient {protocolServer, sessionId, msgQ} rpKey rId =
sendSMPCommand c (Just rpKey) rId SUB >>= \case
OK -> return ()
cmd@MSG {} -> liftIO $ writeSMPMessage c rId cmd
r -> throwE . PCEUnexpectedResponse $ bshow r
-- | Subscribe to multiple SMP queues batching commands if supported.
subscribeSMPQueues :: SMPClient -> NonEmpty (RcvPrivateSignKey, RecipientId) -> IO (NonEmpty (Either ProtocolClientError ()))
subscribeSMPQueues c qs = sendProtocolCommands c cs >>= mapM response . L.zip qs
where
cs = L.map (\(rpKey, rId) -> (Just rpKey, rId, Cmd SRecipient SUB)) qs
response ((_, rId), r) = case r of
Right OK -> pure $ Right ()
Right cmd@MSG {} -> writeSMPMessage c rId cmd $> Right ()
Right r' -> pure . Left . PCEUnexpectedResponse $ bshow r'
Left e -> pure $ Left e
writeSMPMessage :: SMPClient -> RecipientId -> BrokerMsg -> IO ()
writeSMPMessage c rId msg = atomically $ mapM_ (`writeTBQueue` serverTransmission c rId msg) (msgQ c)
serverTransmission :: ProtocolClient msg -> RecipientId -> msg -> ServerTransmission msg
serverTransmission ProtocolClient {protocolServer, thVersion, sessionId} entityId message =
(protocolServer, thVersion, sessionId, entityId, message)
cmd@MSG {} ->
lift . atomically $ mapM_ (`writeTBQueue` (protocolServer, sessionId, rId, cmd)) msgQ
_ -> throwE PCEUnexpectedResponse
-- | Get message from SMP queue. The server returns ERR PROHIBITED if a client uses SUB and GET via the same transport connection for the same queue
--
-- https://github.covm/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#receive-a-message-from-the-queue
getSMPMessage :: SMPClient -> RcvPrivateSignKey -> RecipientId -> ExceptT ProtocolClientError IO (Maybe RcvMessage)
getSMPMessage c rpKey rId =
getSMPMessage :: SMPClient -> RcvPrivateSignKey -> RecipientId -> ExceptT ProtocolClientError IO (Maybe (MsgId, MsgFlags))
getSMPMessage c@ProtocolClient {protocolServer, sessionId, msgQ} rpKey rId =
sendSMPCommand c (Just rpKey) rId GET >>= \case
OK -> pure Nothing
cmd@(MSG msg) -> liftIO (writeSMPMessage c rId cmd) $> Just msg
r -> throwE . PCEUnexpectedResponse $ bshow r
cmd@(MSG msgId _ msgFlags _) -> do
lift . atomically $ mapM_ (`writeTBQueue` (protocolServer, sessionId, rId, cmd)) msgQ
pure $ Just (msgId, msgFlags)
_ -> throwE PCEUnexpectedResponse
-- | Subscribe to the SMP queue notifications.
--
@@ -420,37 +315,11 @@ secureSMPQueue c rpKey rId senderKey = okSMPCommand (KEY senderKey) c rpKey rId
-- | Enable notifications for the queue for push notifications server.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#enable-notifications-command
enableSMPQueueNotifications :: SMPClient -> RcvPrivateSignKey -> RecipientId -> NtfPublicVerifyKey -> RcvNtfPublicDhKey -> ExceptT ProtocolClientError IO (NotifierId, RcvNtfPublicDhKey)
enableSMPQueueNotifications c rpKey rId notifierKey rcvNtfPublicDhKey =
sendSMPCommand c (Just rpKey) rId (NKEY notifierKey rcvNtfPublicDhKey) >>= \case
NID nId rcvNtfSrvPublicDhKey -> pure (nId, rcvNtfSrvPublicDhKey)
r -> throwE . PCEUnexpectedResponse $ bshow r
-- | Enable notifications for the multiple queues for push notifications server.
enableSMPQueuesNtfs :: SMPClient -> NonEmpty (RcvPrivateSignKey, RecipientId, NtfPublicVerifyKey, RcvNtfPublicDhKey) -> IO (NonEmpty (Either ProtocolClientError (NotifierId, RcvNtfPublicDhKey)))
enableSMPQueuesNtfs c qs = L.map response <$> sendProtocolCommands c cs
where
cs = L.map (\(rpKey, rId, notifierKey, rcvNtfPublicDhKey) -> (Just rpKey, rId, Cmd SRecipient $ NKEY notifierKey rcvNtfPublicDhKey)) qs
response = \case
Right (NID nId rcvNtfSrvPublicDhKey) -> Right (nId, rcvNtfSrvPublicDhKey)
Right r -> Left . PCEUnexpectedResponse $ bshow r
Left e -> Left e
-- | Disable notifications for the queue for push notifications server.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#disable-notifications-command
disableSMPQueueNotifications :: SMPClient -> RcvPrivateSignKey -> RecipientId -> ExceptT ProtocolClientError IO ()
disableSMPQueueNotifications = okSMPCommand NDEL
-- | Disable notifications for multiple queues for push notifications server.
disableSMPQueuesNtfs :: SMPClient -> NonEmpty (RcvPrivateSignKey, RecipientId) -> IO (NonEmpty (Either ProtocolClientError ()))
disableSMPQueuesNtfs c qs = L.map response <$> sendProtocolCommands c cs
where
cs = L.map (\(rpKey, rId) -> (Just rpKey, rId, Cmd SRecipient NDEL)) qs
response = \case
Right OK -> Right ()
Right r -> Left . PCEUnexpectedResponse $ bshow r
Left e -> Left e
enableSMPQueueNotifications :: SMPClient -> RcvPrivateSignKey -> RecipientId -> NtfPublicVerifyKey -> ExceptT ProtocolClientError IO NotifierId
enableSMPQueueNotifications c rpKey rId notifierKey =
sendSMPCommand c (Just rpKey) rId (NKEY notifierKey) >>= \case
NID nId -> pure nId
_ -> throwE PCEUnexpectedResponse
-- | Send SMP message.
--
@@ -459,17 +328,18 @@ sendSMPMessage :: SMPClient -> Maybe SndPrivateSignKey -> SenderId -> MsgFlags -
sendSMPMessage c spKey sId flags msg =
sendSMPCommand c spKey sId (SEND flags msg) >>= \case
OK -> pure ()
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
-- | Acknowledge message delivery (server deletes the message).
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#acknowledge-message-delivery
ackSMPMessage :: SMPClient -> RcvPrivateSignKey -> QueueId -> MsgId -> ExceptT ProtocolClientError IO ()
ackSMPMessage c rpKey rId msgId =
ackSMPMessage c@ProtocolClient {protocolServer, sessionId, msgQ} rpKey rId msgId =
sendSMPCommand c (Just rpKey) rId (ACK msgId) >>= \case
OK -> return ()
cmd@MSG {} -> liftIO $ writeSMPMessage c rId cmd
r -> throwE . PCEUnexpectedResponse $ bshow r
cmd@MSG {} ->
lift . atomically $ mapM_ (`writeTBQueue` (protocolServer, sessionId, rId, cmd)) msgQ
_ -> throwE PCEUnexpectedResponse
-- | Irreversibly suspend SMP queue.
-- The existing messages from the queue will still be delivered.
@@ -488,54 +358,43 @@ okSMPCommand :: PartyI p => Command p -> SMPClient -> C.APrivateSignKey -> Queue
okSMPCommand cmd c pKey qId =
sendSMPCommand c (Just pKey) qId cmd >>= \case
OK -> return ()
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
-- | Send SMP command
sendSMPCommand :: PartyI p => SMPClient -> Maybe C.APrivateSignKey -> QueueId -> Command p -> ExceptT ProtocolClientError IO BrokerMsg
sendSMPCommand c pKey qId cmd = sendProtocolCommand c pKey qId (Cmd sParty cmd)
-- | Send multiple commands with batching and collect responses
sendProtocolCommands :: forall msg. ProtocolEncoding (ProtoCommand msg) => ProtocolClient msg -> NonEmpty (ClientCommand msg) -> IO (NonEmpty (Either ProtocolClientError msg))
sendProtocolCommands c@ProtocolClient {sndQ, tcpTimeout} cs = do
ts <- mapM (runExceptT . mkTransmission c) cs
mapM_ (atomically . writeTBQueue sndQ . L.map fst) . L.nonEmpty . rights $ L.toList ts
forConcurrently ts $ \case
Right (_, r) -> withTimeout . atomically $ takeTMVar r
Left e -> pure $ Left e
where
withTimeout a = fromMaybe (Left PCEResponseTimeout) <$> timeout tcpTimeout a
-- | Send Protocol command
sendProtocolCommand :: forall msg. ProtocolEncoding (ProtoCommand msg) => ProtocolClient msg -> Maybe C.APrivateSignKey -> QueueId -> ProtoCommand msg -> ExceptT ProtocolClientError IO msg
sendProtocolCommand c@ProtocolClient {sndQ, tcpTimeout} pKey qId cmd = do
(t, r) <- mkTransmission c (pKey, qId, cmd)
ExceptT $ sendRecv t r
where
-- two separate "atomically" needed to avoid blocking
sendRecv :: SentRawTransmission -> TMVar (Response msg) -> IO (Response msg)
sendRecv t r = atomically (writeTBQueue sndQ [t]) >> withTimeout (atomically $ takeTMVar r)
where
withTimeout a = fromMaybe (Left PCEResponseTimeout) <$> timeout tcpTimeout a
mkTransmission :: forall msg. ProtocolEncoding (ProtoCommand msg) => ProtocolClient msg -> ClientCommand msg -> ExceptT ProtocolClientError IO (SentRawTransmission, TMVar (Response msg))
mkTransmission ProtocolClient {clientCorrId, sessionId, thVersion, sentCommands} (pKey, qId, cmd) = do
corrId <- liftIO $ atomically getNextCorrId
sendProtocolCommand :: forall msg. ProtocolEncoding (ProtocolCommand msg) => ProtocolClient msg -> Maybe C.APrivateSignKey -> QueueId -> ProtocolCommand msg -> ExceptT ProtocolClientError IO msg
sendProtocolCommand ProtocolClient {sndQ, sentCommands, clientCorrId, sessionId, thVersion, tcpTimeout} pKey qId cmd = do
corrId <- lift_ getNextCorrId
t <- signTransmission $ encodeTransmission thVersion sessionId (corrId, qId, cmd)
r <- liftIO . atomically $ mkRequest corrId
pure (t, r)
ExceptT $ sendRecv corrId t
where
lift_ :: STM a -> ExceptT ProtocolClientError IO a
lift_ action = ExceptT $ Right <$> atomically action
getNextCorrId :: STM CorrId
getNextCorrId = do
i <- stateTVar clientCorrId $ \i -> (i, i + 1)
pure . CorrId $ bshow i
signTransmission :: ByteString -> ExceptT ProtocolClientError IO SentRawTransmission
signTransmission t = case pKey of
Nothing -> pure (Nothing, t)
Nothing -> return (Nothing, t)
Just pk -> do
sig <- liftError PCESignatureError $ C.sign pk t
return (Just sig, t)
mkRequest :: CorrId -> STM (TMVar (Response msg))
mkRequest corrId = do
-- two separate "atomically" needed to avoid blocking
sendRecv :: CorrId -> SentRawTransmission -> IO (Response msg)
sendRecv corrId t = atomically (send corrId t) >>= withTimeout . atomically . takeTMVar
where
withTimeout a = fromMaybe (Left PCEResponseTimeout) <$> timeout tcpTimeout a
send :: CorrId -> SentRawTransmission -> STM (TMVar (Response msg))
send corrId t = do
r <- newEmptyTMVar
TM.insert corrId (Request qId r) sentCommands
pure r
writeTBQueue sndQ t
return r
+9 -12
View File
@@ -5,7 +5,6 @@
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
{-# OPTIONS_GHC -fno-warn-orphans #-}
module Simplex.Messaging.Client.Agent where
@@ -26,12 +25,10 @@ import Numeric.Natural
import Simplex.Messaging.Agent.RetryInterval
import Simplex.Messaging.Client
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol (BrokerMsg, ProtocolServer (..), QueueId, SMPServer)
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport
import Simplex.Messaging.Util (catchAll_, tryE, unlessM, ($>>=))
import Simplex.Messaging.Util (catchAll_, tryE, whenM, ($>>=))
import System.Timeout (timeout)
import UnliftIO (async, forConcurrently_)
import UnliftIO.Exception (Exception)
@@ -48,7 +45,7 @@ data SMPClientAgentEvent
| CASubError SMPServer SMPSub ProtocolClientError
data SMPSubParty = SPRecipient | SPNotifier
deriving (Eq, Ord, Show)
deriving (Eq, Ord)
type SMPSub = (SMPSubParty, QueueId)
@@ -64,7 +61,7 @@ data SMPClientAgentConfig = SMPClientAgentConfig
defaultSMPClientAgentConfig :: SMPClientAgentConfig
defaultSMPClientAgentConfig =
SMPClientAgentConfig
{ smpCfg = defaultClientConfig {defaultTransport = ("5223", transport @TLS)},
{ smpCfg = defaultClientConfig,
reconnectInterval =
RetryInterval
{ initialInterval = second,
@@ -126,8 +123,8 @@ getSMPServerClient' ca@SMPClientAgent {agentCfg, smpClients, msgQ} srv =
waitForSMPClient :: SMPClientVar -> ExceptT ProtocolClientError IO SMPClient
waitForSMPClient smpVar = do
let ProtocolClientConfig {networkConfig = NetworkConfig {tcpConnectTimeout}} = smpCfg agentCfg
smpClient_ <- liftIO $ tcpConnectTimeout `timeout` atomically (readTMVar smpVar)
let ProtocolClientConfig {tcpTimeout} = smpCfg agentCfg
smpClient_ <- liftIO $ tcpTimeout `timeout` atomically (readTMVar smpVar)
liftEither $ case smpClient_ of
Just (Right smpClient) -> Right smpClient
Just (Left e) -> Left e
@@ -162,8 +159,8 @@ getSMPServerClient' ca@SMPClientAgent {agentCfg, smpClients, msgQ} srv =
connectClient :: ExceptT ProtocolClientError IO SMPClient
connectClient = ExceptT $ getProtocolClient srv (smpCfg agentCfg) (Just msgQ) clientDisconnected
clientDisconnected :: SMPClient -> IO ()
clientDisconnected _ = do
clientDisconnected :: IO ()
clientDisconnected = do
removeClientAndSubs >>= (`forM_` serverDown)
logInfo . decodeUtf8 $ "Agent disconnected from " <> showServer srv
@@ -204,7 +201,7 @@ getSMPServerClient' ca@SMPClientAgent {agentCfg, smpClients, msgQ} srv =
notify $ CAReconnected srv
cs <- atomically $ mapM readTVar =<< TM.lookup srv (pendingSrvSubs ca)
forConcurrently_ (maybe [] M.assocs cs) $ \sub@(s, _) ->
unlessM (atomically $ hasSub (srvSubs ca) srv s) $
whenM (atomically $ hasSub (srvSubs ca) srv s) $
subscribe_ smp sub `catchE` handleError s
where
subscribe_ :: SMPClient -> (SMPSub, C.APrivateSignKey) -> ExceptT ProtocolClientError IO ()
@@ -265,7 +262,7 @@ subscribeQueue ca srv sub = do
showServer :: SMPServer -> ByteString
showServer ProtocolServer {host, port} =
strEncode host <> (B.pack $ if null port then "" else ':' : port)
B.pack $ host <> if null port then "" else ':' : port
smpSubscribe :: SMPClient -> (SMPSub, C.APrivateSignKey) -> ExceptT ProtocolClientError IO ()
smpSubscribe smp ((party, queueId), privKey) = subscribe_ smp privKey queueId
+8 -69
View File
@@ -8,7 +8,6 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StandaloneDeriving #-}
@@ -102,7 +101,6 @@ module Simplex.Messaging.Crypto
-- * NaCl crypto_box
CbNonce (unCbNonce),
cbEncrypt,
cbEncryptMaxLenBS,
cbDecrypt,
cbNonce,
randomCbNonce,
@@ -120,13 +118,6 @@ module Simplex.Messaging.Crypto
-- * Cryptography error type
CryptoError (..),
-- * Limited size ByteStrings
MaxLenBS,
pattern MaxLenBS,
maxLenBS,
unsafeMaxLenBS,
appendMaxLenBS,
)
where
@@ -162,11 +153,11 @@ import Data.Constraint (Dict (..))
import Data.Kind (Constraint, Type)
import Data.String
import Data.Type.Equality
import Data.Typeable (Proxy (Proxy), Typeable)
import Data.Typeable (Typeable)
import Data.X509
import Database.SQLite.Simple.FromField (FromField (..))
import Database.SQLite.Simple.ToField (ToField (..))
import GHC.TypeLits (ErrorMessage (..), KnownNat, Nat, TypeError, natVal, type (+))
import GHC.TypeLits (ErrorMessage (..), TypeError)
import Network.Transport.Internal (decodeWord16, encodeWord16)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
@@ -259,12 +250,6 @@ deriving instance Eq (PrivateKey a)
deriving instance Show (PrivateKey a)
instance StrEncoding (PrivateKey X25519) where
strEncode = strEncode . encodePrivKey
{-# INLINE strEncode #-}
strDecode = decodePrivKey
{-# INLINE strDecode #-}
data APrivateKey
= forall a.
AlgorithmI a =>
@@ -311,12 +296,6 @@ instance Encoding APrivateSignKey where
smpDecode = decodePrivKey
{-# INLINE smpDecode #-}
instance StrEncoding APrivateSignKey where
strEncode = strEncode . encodePrivKey
{-# INLINE strEncode #-}
strDecode = decodePrivKey
{-# INLINE strDecode #-}
data APublicVerifyKey
= forall a.
(AlgorithmI a, SignatureAlgorithm a) =>
@@ -818,41 +797,6 @@ unPad padded
(lenWrd, rest) = B.splitAt 2 padded
len = fromIntegral $ decodeWord16 lenWrd
newtype MaxLenBS (i :: Nat) = MLBS {unMaxLenBS :: ByteString}
pattern MaxLenBS :: ByteString -> MaxLenBS i
pattern MaxLenBS s <- MLBS s
{-# COMPLETE MaxLenBS #-}
instance KnownNat i => Encoding (MaxLenBS i) where
smpEncode (MLBS s) = smpEncode s
smpP = first show . maxLenBS <$?> smpP
instance KnownNat i => StrEncoding (MaxLenBS i) where
strEncode (MLBS s) = strEncode s
strP = first show . maxLenBS <$?> strP
maxLenBS :: forall i. KnownNat i => ByteString -> Either CryptoError (MaxLenBS i)
maxLenBS s
| B.length s > maxLength @i = Left CryptoLargeMsgError
| otherwise = Right $ MLBS s
unsafeMaxLenBS :: forall i. KnownNat i => ByteString -> MaxLenBS i
unsafeMaxLenBS = MLBS
padMaxLenBS :: forall i. KnownNat i => MaxLenBS i -> MaxLenBS (i + 2)
padMaxLenBS (MLBS msg) = MLBS $ encodeWord16 (fromIntegral len) <> msg <> B.replicate padLen '#'
where
len = B.length msg
padLen = maxLength @i - len
appendMaxLenBS :: (KnownNat i, KnownNat j) => MaxLenBS i -> MaxLenBS j -> MaxLenBS (i + j)
appendMaxLenBS (MLBS s1) (MLBS s2) = MLBS $ s1 <> s2
maxLength :: forall i. KnownNat i => Int
maxLength = fromIntegral (natVal $ Proxy @i)
initAEAD :: forall c. AES.BlockCipher c => Key -> IV -> ExceptT CryptoError IO (AES.AEAD c)
initAEAD (Key aesKey) (IV ivBytes) = do
iv <- makeIV @c ivBytes
@@ -908,17 +852,12 @@ dh' (PublicKeyX448 k) (PrivateKeyX448 pk _) = DhSecretX448 $ X448.dh k pk
-- | NaCl @crypto_box@ encrypt with a shared DH secret and 192-bit nonce.
cbEncrypt :: DhSecret X25519 -> CbNonce -> ByteString -> Int -> Either CryptoError ByteString
cbEncrypt secret (CbNonce nonce) msg paddedLen = cryptoBox secret nonce <$> pad msg paddedLen
-- | NaCl @crypto_box@ encrypt with a shared DH secret and 192-bit nonce.
cbEncryptMaxLenBS :: KnownNat i => DhSecret X25519 -> CbNonce -> MaxLenBS i -> ByteString
cbEncryptMaxLenBS secret (CbNonce nonce) = cryptoBox secret nonce . unMaxLenBS . padMaxLenBS
cryptoBox :: DhSecret 'X25519 -> ByteString -> ByteString -> ByteString
cryptoBox secret nonce s = BA.convert tag <> c
cbEncrypt secret (CbNonce nonce) msg paddedLen = cryptoBox <$> pad msg paddedLen
where
(rs, c) = xSalsa20 secret nonce s
tag = Poly1305.auth rs c
cryptoBox s = BA.convert tag <> c
where
(rs, c) = xSalsa20 secret nonce s
tag = Poly1305.auth rs c
-- | NaCl @crypto_box@ decrypt with a shared DH secret and 192-bit nonce.
cbDecrypt :: DhSecret X25519 -> CbNonce -> ByteString -> Either CryptoError ByteString
@@ -932,7 +871,7 @@ cbDecrypt secret (CbNonce nonce) packet
tag = Poly1305.auth rs c
newtype CbNonce = CbNonce {unCbNonce :: ByteString}
deriving (Eq, Show)
deriving (Show)
instance StrEncoding CbNonce where
strEncode (CbNonce s) = strEncode s
-7
View File
@@ -13,7 +13,6 @@ module Simplex.Messaging.Encoding
Large (..),
smpEncodeList,
smpListP,
lenEncode,
)
where
@@ -198,9 +197,3 @@ instance (Encoding a, Encoding b, Encoding c, Encoding d, Encoding e, Encoding f
{-# INLINE smpEncode #-}
smpP = (,,,,,,) <$> smpP <*> smpP <*> smpP <*> smpP <*> smpP <*> smpP <*> smpP
{-# INLINE smpP #-}
instance (Encoding a, Encoding b, Encoding c, Encoding d, Encoding e, Encoding f, Encoding g, Encoding h) => Encoding (a, b, c, d, e, f, g, h) where
smpEncode (a, b, c, d, e, f, g, h) = smpEncode a <> smpEncode b <> smpEncode c <> smpEncode d <> smpEncode e <> smpEncode f <> smpEncode g <> smpEncode h
{-# INLINE smpEncode #-}
smpP = (,,,,,,,) <$> smpP <*> smpP <*> smpP <*> smpP <*> smpP <*> smpP <*> smpP <*> smpP
{-# INLINE smpP #-}
+1 -21
View File
@@ -28,13 +28,9 @@ import qualified Data.ByteString.Char8 as B
import Data.Char (isAlphaNum)
import Data.Int (Int64)
import qualified Data.List.NonEmpty as L
import Data.Set (Set)
import qualified Data.Set as S
import Data.Text (Text)
import Data.Text.Encoding (decodeLatin1, encodeUtf8)
import Data.Time.Clock (UTCTime)
import Data.Time.Clock.System (SystemTime (..))
import Data.Time.Format.ISO8601
import Data.Word (Word16)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Parsers (parseAll)
@@ -105,26 +101,14 @@ instance StrEncoding Bool where
strP = smpP
{-# INLINE strP #-}
instance StrEncoding Int where
strEncode = B.pack . show
{-# INLINE strEncode #-}
strP = A.decimal
{-# INLINE strP #-}
instance StrEncoding Int64 where
strEncode = B.pack . show
{-# INLINE strEncode #-}
strP = A.decimal
{-# INLINE strP #-}
instance StrEncoding SystemTime where
strEncode = strEncode . systemSeconds
strP = MkSystemTime <$> strP <*> pure 0
instance StrEncoding UTCTime where
strEncode = B.pack . iso8601Show
strP = maybe (Left "bad UTCTime") Right . iso8601ParseM . B.unpack <$?> A.takeTill (\c -> c == ' ' || c == '\n')
-- lists encode/parse as comma-separated strings
strEncodeList :: StrEncoding a => [a] -> ByteString
strEncodeList = B.intercalate "," . map strEncode
@@ -137,12 +121,8 @@ instance StrEncoding a => StrEncoding (L.NonEmpty a) where
strEncode = strEncodeList . L.toList
strP = L.fromList <$> listItem `A.sepBy1'` A.char ','
instance (StrEncoding a, Ord a) => StrEncoding (Set a) where
strEncode = strEncodeList . S.toList
strP = S.fromList <$> listItem `A.sepBy'` A.char ','
listItem :: StrEncoding a => Parser a
listItem = parseAll strP <$?> A.takeTill (\c -> c == ',' || c == ' ' || c == '\n')
listItem = parseAll strP <$?> A.takeTill (== ',')
instance (StrEncoding a, StrEncoding b) => StrEncoding (a, b) where
strEncode (a, b) = B.unwords [strEncode a, strEncode b]
+174 -9
View File
@@ -1,16 +1,29 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Simplex.Messaging.Notifications.Client where
import Control.Monad.Except
import Control.Monad.Trans.Except
import qualified Data.Attoparsec.ByteString.Char8 as A
import Data.Text.Encoding (decodeLatin1, encodeUtf8)
import Data.Time (UTCTime)
import Data.Word (Word16)
import Database.SQLite.Simple.FromField (FromField (..))
import Database.SQLite.Simple.ToField (ToField (..))
import Simplex.Messaging.Agent.Protocol (ConnId)
import Simplex.Messaging.Client
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Util (bshow)
import Simplex.Messaging.Parsers (blobFieldDecoder, fromTextField_)
import Simplex.Messaging.Protocol (NotifierId, ProtocolServer, SMPServer)
type NtfServer = ProtocolServer
type NtfClient = ProtocolClient NtfResponse
@@ -18,7 +31,7 @@ ntfRegisterToken :: NtfClient -> C.APrivateSignKey -> NewNtfEntity 'Token -> Exc
ntfRegisterToken c pKey newTkn =
sendNtfCommand c (Just pKey) "" (TNEW newTkn) >>= \case
NRTknId tknId dhKey -> pure (tknId, dhKey)
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
ntfVerifyToken :: NtfClient -> C.APrivateSignKey -> NtfTokenId -> NtfRegCode -> ExceptT ProtocolClientError IO ()
ntfVerifyToken c pKey tknId code = okNtfCommand (TVFY code) c pKey tknId
@@ -27,10 +40,7 @@ ntfCheckToken :: NtfClient -> C.APrivateSignKey -> NtfTokenId -> ExceptT Protoco
ntfCheckToken c pKey tknId =
sendNtfCommand c (Just pKey) tknId TCHK >>= \case
NRTkn stat -> pure stat
r -> throwE . PCEUnexpectedResponse $ bshow r
ntfReplaceToken :: NtfClient -> C.APrivateSignKey -> NtfTokenId -> DeviceToken -> ExceptT ProtocolClientError IO ()
ntfReplaceToken c pKey tknId token = okNtfCommand (TRPL token) c pKey tknId
_ -> throwE PCEUnexpectedResponse
ntfDeleteToken :: NtfClient -> C.APrivateSignKey -> NtfTokenId -> ExceptT ProtocolClientError IO ()
ntfDeleteToken = okNtfCommand TDEL
@@ -42,13 +52,13 @@ ntfCreateSubscription :: NtfClient -> C.APrivateSignKey -> NewNtfEntity 'Subscri
ntfCreateSubscription c pKey newSub =
sendNtfCommand c (Just pKey) "" (SNEW newSub) >>= \case
NRSubId subId -> pure subId
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
ntfCheckSubscription :: NtfClient -> C.APrivateSignKey -> NtfSubscriptionId -> ExceptT ProtocolClientError IO NtfSubStatus
ntfCheckSubscription c pKey subId =
sendNtfCommand c (Just pKey) subId SCHK >>= \case
NRSub stat -> pure stat
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
ntfDeleteSubscription :: NtfClient -> C.APrivateSignKey -> NtfSubscriptionId -> ExceptT ProtocolClientError IO ()
ntfDeleteSubscription = okNtfCommand SDEL
@@ -61,4 +71,159 @@ okNtfCommand :: NtfEntityI e => NtfCommand e -> NtfClient -> C.APrivateSignKey -
okNtfCommand cmd c pKey entId =
sendNtfCommand c (Just pKey) entId cmd >>= \case
NROk -> return ()
r -> throwE . PCEUnexpectedResponse $ bshow r
_ -> throwE PCEUnexpectedResponse
data NtfTknAction
= NTARegister
| NTAVerify NtfRegCode -- code to verify token
| NTACheck
| NTACron Word16
| NTADelete
deriving (Show)
instance Encoding NtfTknAction where
smpEncode = \case
NTARegister -> "R"
NTAVerify code -> smpEncode ('V', code)
NTACheck -> "C"
NTACron interval -> smpEncode ('I', interval)
NTADelete -> "D"
smpP =
A.anyChar >>= \case
'R' -> pure NTARegister
'V' -> NTAVerify <$> smpP
'C' -> pure NTACheck
'I' -> NTACron <$> smpP
'D' -> pure NTADelete
_ -> fail "bad NtfTknAction"
instance FromField NtfTknAction where fromField = blobFieldDecoder smpDecode
instance ToField NtfTknAction where toField = toField . smpEncode
data NtfToken = NtfToken
{ deviceToken :: DeviceToken,
ntfServer :: NtfServer,
ntfTokenId :: Maybe NtfTokenId,
-- | key used by the ntf server to verify transmissions
ntfPubKey :: C.APublicVerifyKey,
-- | key used by the ntf client to sign transmissions
ntfPrivKey :: C.APrivateSignKey,
-- | client's DH keys (to repeat registration if necessary)
ntfDhKeys :: C.KeyPair 'C.X25519,
-- | shared DH secret used to encrypt/decrypt notifications e2e
ntfDhSecret :: Maybe C.DhSecretX25519,
-- | token status
ntfTknStatus :: NtfTknStatus,
-- | pending token action and the earliest time
ntfTknAction :: Maybe NtfTknAction
}
deriving (Show)
newNtfToken :: DeviceToken -> NtfServer -> C.ASignatureKeyPair -> C.KeyPair 'C.X25519 -> NtfToken
newNtfToken deviceToken ntfServer (ntfPubKey, ntfPrivKey) ntfDhKeys =
NtfToken
{ deviceToken,
ntfServer,
ntfTokenId = Nothing,
ntfPubKey,
ntfPrivKey,
ntfDhKeys,
ntfDhSecret = Nothing,
ntfTknStatus = NTNew,
ntfTknAction = Just NTARegister
}
data NtfSubOrSMPAction = NtfSubAction NtfSubAction | NtfSubSMPAction NtfSubSMPAction
data NtfSubAction
= NSACreate
| NSACheck
| NSADelete
deriving (Show)
instance Encoding NtfSubAction where
smpEncode = \case
NSACreate -> "N"
NSACheck -> "C"
NSADelete -> "D"
smpP =
A.anyChar >>= \case
'N' -> pure NSACreate
'C' -> pure NSACheck
'D' -> pure NSADelete
_ -> fail "bad NtfSubAction"
instance FromField NtfSubAction where fromField = blobFieldDecoder smpDecode
instance ToField NtfSubAction where toField = toField . smpEncode
data NtfSubSMPAction
= NSAKey
deriving (Show)
instance Encoding NtfSubSMPAction where
smpEncode = \case
NSAKey -> "K"
smpP =
A.anyChar >>= \case
'K' -> pure NSAKey
_ -> fail "bad NtfSubSMPAction"
instance FromField NtfSubSMPAction where fromField = blobFieldDecoder smpDecode
instance ToField NtfSubSMPAction where toField = toField . smpEncode
data NtfAgentSubStatus
= -- | subscription started
NASNew
| -- | state after NKEY - notifier ID is assigned to queue on SMP server
NASKey
| -- | state after SNEW - subscription created on notification server
NASCreated NtfSubStatus
| -- | state after SDEL (subscription is deleted on notification server)
NASDeleted
deriving (Eq, Show)
instance Encoding NtfAgentSubStatus where
smpEncode = \case
NASNew -> "NEW"
NASKey -> "KEY"
NASCreated status -> "CREATED " <> smpEncode status
NASDeleted -> "DELETED"
smpP =
A.takeTill (== ' ') >>= \case
"NEW" -> pure NASNew
"KEY" -> pure NASKey
"CREATED" -> do
_ <- A.space
NASCreated <$> smpP
"DELETED" -> pure NASDeleted
_ -> fail "bad NtfAgentSubStatus"
instance FromField NtfAgentSubStatus where fromField = fromTextField_ $ either (const Nothing) Just . smpDecode . encodeUtf8
instance ToField NtfAgentSubStatus where toField = toField . decodeLatin1 . smpEncode
data NtfSubscription = NtfSubscription
{ connId :: ConnId,
smpServer :: SMPServer,
ntfQueueId :: Maybe NotifierId,
ntfServer :: NtfServer,
ntfSubId :: Maybe NtfSubscriptionId,
ntfSubStatus :: NtfAgentSubStatus,
ntfSubActionTs :: UTCTime
}
deriving (Show)
newNtfSubscription :: ConnId -> SMPServer -> Maybe NotifierId -> NtfServer -> NtfAgentSubStatus -> UTCTime -> NtfSubscription
newNtfSubscription connId smpServer ntfQueueId ntfServer ntfSubStatus ntfSubActionTs =
NtfSubscription
{ connId,
smpServer,
ntfQueueId,
ntfServer,
ntfSubId = Nothing,
ntfSubStatus,
ntfSubActionTs
}
+23 -87
View File
@@ -10,7 +10,7 @@
module Simplex.Messaging.Notifications.Protocol where
import Data.Aeson (FromJSON (..), ToJSON (..), (.=))
import Data.Aeson (FromJSON (..), ToJSON (..))
import qualified Data.Aeson as J
import qualified Data.Aeson.Encoding as JE
import qualified Data.Attoparsec.ByteString.Char8 as A
@@ -23,14 +23,13 @@ import Data.Type.Equality
import Data.Word (Word16)
import Database.SQLite.Simple.FromField (FromField (..))
import Database.SQLite.Simple.ToField (ToField (..))
import Simplex.Messaging.Agent.Protocol (updateSMPServerHosts)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Transport (ntfClientHandshake)
import Simplex.Messaging.Parsers (fromTextField_)
import Simplex.Messaging.Protocol hiding (Command (..), CommandTag (..))
import Simplex.Messaging.Util (eitherToMaybe, (<$?>))
import Simplex.Messaging.Util ((<$?>))
data NtfEntity = Token | Subscription
deriving (Show)
@@ -56,7 +55,6 @@ data NtfCommandTag (e :: NtfEntity) where
TNEW_ :: NtfCommandTag 'Token
TVFY_ :: NtfCommandTag 'Token
TCHK_ :: NtfCommandTag 'Token
TRPL_ :: NtfCommandTag 'Token
TDEL_ :: NtfCommandTag 'Token
TCRN_ :: NtfCommandTag 'Token
SNEW_ :: NtfCommandTag 'Subscription
@@ -73,7 +71,6 @@ instance NtfEntityI e => Encoding (NtfCommandTag e) where
TNEW_ -> "TNEW"
TVFY_ -> "TVFY"
TCHK_ -> "TCHK"
TRPL_ -> "TRPL"
TDEL_ -> "TDEL"
TCRN_ -> "TCRN"
SNEW_ -> "SNEW"
@@ -91,7 +88,6 @@ instance ProtocolMsgTag NtfCmdTag where
"TNEW" -> Just $ NCT SToken TNEW_
"TVFY" -> Just $ NCT SToken TVFY_
"TCHK" -> Just $ NCT SToken TCHK_
"TRPL" -> Just $ NCT SToken TRPL_
"TDEL" -> Just $ NCT SToken TDEL_
"TCRN" -> Just $ NCT SToken TCRN_
"SNEW" -> Just $ NCT SSubscription SNEW_
@@ -147,8 +143,7 @@ instance Encoding ANewNtfEntity where
_ -> fail "bad ANewNtfEntity"
instance Protocol NtfResponse where
type ProtoCommand NtfResponse = NtfCmd
type ProtoType NtfResponse = 'PNTF
type ProtocolCommand NtfResponse = NtfCmd
protocolClientHandshake = ntfClientHandshake
protocolPing = NtfCmd SSubscription PING
protocolError = \case
@@ -162,8 +157,6 @@ data NtfCommand (e :: NtfEntity) where
TVFY :: NtfRegCode -> NtfCommand 'Token
-- | check token status
TCHK :: NtfCommand 'Token
-- | replace device token (while keeping all existing subscriptions)
TRPL :: DeviceToken -> NtfCommand 'Token
-- | delete token - all subscriptions will be removed and no more notifications will be sent
TDEL :: NtfCommand 'Token
-- | enable periodic background notification to fetch the new messages - interval is in minutes, minimum is 20, 0 to disable
@@ -189,7 +182,6 @@ instance NtfEntityI e => ProtocolEncoding (NtfCommand e) where
TNEW newTkn -> e (TNEW_, ' ', newTkn)
TVFY code -> e (TVFY_, ' ', code)
TCHK -> e TCHK_
TRPL tkn -> e (TRPL_, ' ', tkn)
TDEL -> e TDEL_
TCRN int -> e (TCRN_, ' ', int)
SNEW newSub -> e (SNEW_, ' ', newSub)
@@ -229,7 +221,6 @@ instance ProtocolEncoding NtfCmd where
TNEW_ -> TNEW <$> _smpP
TVFY_ -> TVFY <$> _smpP
TCHK_ -> pure TCHK
TRPL_ -> TRPL <$> _smpP
TDEL_ -> pure TDEL
TCRN_ -> TCRN <$> _smpP
NCT SSubscription tag ->
@@ -253,7 +244,7 @@ data NtfResponseTag
instance Encoding NtfResponseTag where
smpEncode = \case
NRTknId_ -> "IDTKN" -- it should be "TID", "SID"
NRTknId_ -> "IDTKN"
NRSubId_ -> "IDSUB"
NROk_ -> "OK"
NRErr_ -> "ERR"
@@ -325,7 +316,7 @@ instance ProtocolEncoding NtfResponse where
| otherwise = Left $ CMD HAS_AUTH
data SMPQueueNtf = SMPQueueNtf
{ smpServer :: SMPServer,
{ smpServer :: ProtocolServer,
notifierId :: NotifierId
}
deriving (Eq, Ord, Show)
@@ -333,47 +324,30 @@ data SMPQueueNtf = SMPQueueNtf
instance Encoding SMPQueueNtf where
smpEncode SMPQueueNtf {smpServer, notifierId} = smpEncode (smpServer, notifierId)
smpP = do
smpServer <- updateSMPServerHosts <$> smpP
notifierId <- smpP
pure SMPQueueNtf {smpServer, notifierId}
(smpServer, notifierId) <- smpP
pure $ SMPQueueNtf smpServer notifierId
instance StrEncoding SMPQueueNtf where
strEncode SMPQueueNtf {smpServer, notifierId} = strEncode smpServer <> "/" <> strEncode notifierId
strP = do
smpServer <- updateSMPServerHosts <$> strP
notifierId <- A.char '/' *> strP
pure SMPQueueNtf {smpServer, notifierId}
data PushProvider = PPApnsDev | PPApnsProd | PPApnsTest
data PushProvider = PPApns
deriving (Eq, Ord, Show)
instance Encoding PushProvider where
smpEncode = \case
PPApnsDev -> "AD"
PPApnsProd -> "AP"
PPApnsTest -> "AT"
PPApns -> "A"
smpP =
A.take 2 >>= \case
"AD" -> pure PPApnsDev
"AP" -> pure PPApnsProd
"AT" -> pure PPApnsTest
A.anyChar >>= \case
'A' -> pure PPApns
_ -> fail "bad PushProvider"
instance StrEncoding PushProvider where
strEncode = \case
PPApnsDev -> "apns_dev"
PPApnsProd -> "apns_prod"
PPApnsTest -> "apns_test"
strP =
A.takeTill (== ' ') >>= \case
"apns_dev" -> pure PPApnsDev
"apns_prod" -> pure PPApnsProd
"apns_test" -> pure PPApnsTest
_ -> fail "bad PushProvider"
instance TextEncoding PushProvider where
textEncode = \case
PPApns -> "apple"
textDecode = \case
"apple" -> Just PPApns
_ -> Nothing
instance FromField PushProvider where fromField = fromTextField_ $ eitherToMaybe . strDecode . encodeUtf8
instance FromField PushProvider where fromField = fromTextField_ textDecode
instance ToField PushProvider where toField = toField . decodeLatin1 . strEncode
instance ToField PushProvider where toField = toField . textEncode
data DeviceToken = DeviceToken PushProvider ByteString
deriving (Eq, Ord, Show)
@@ -382,18 +356,6 @@ instance Encoding DeviceToken where
smpEncode (DeviceToken p t) = smpEncode (p, t)
smpP = DeviceToken <$> smpP <*> smpP
instance StrEncoding DeviceToken where
strEncode (DeviceToken p t) = strEncode p <> " " <> t
strP = DeviceToken <$> strP <* A.space <*> hexStringP
where
hexStringP =
A.takeWhile (\c -> (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) >>= \s ->
if even (B.length s) then pure s else fail "odd number of hex characters"
instance ToJSON DeviceToken where
toEncoding (DeviceToken pp t) = J.pairs $ "pushProvider" .= decodeLatin1 (strEncode pp) <> "token" .= decodeLatin1 t
toJSON (DeviceToken pp t) = J.object ["pushProvider" .= decodeLatin1 (strEncode pp), "token" .= decodeLatin1 t]
type NtfEntityId = ByteString
type NtfSubscriptionId = NtfEntityId
@@ -407,50 +369,28 @@ data NtfSubStatus
NSPending
| -- | connected and subscribed to SMP server
NSActive
| -- | disconnected/unsubscribed from SMP server
NSInactive
| -- | END received
| -- | NEND received (we currently do not support it)
NSEnd
| -- | SMP AUTH error
NSAuth
| -- | SMP error other than AUTH
NSErr ByteString
NSSMPAuth
deriving (Eq, Show)
ntfShouldSubscribe :: NtfSubStatus -> Bool
ntfShouldSubscribe = \case
NSNew -> True
NSPending -> True
NSActive -> True
NSInactive -> True
NSEnd -> False
NSAuth -> False
NSErr _ -> False
instance Encoding NtfSubStatus where
smpEncode = \case
NSNew -> "NEW"
NSPending -> "PENDING" -- e.g. after SMP server disconnect/timeout while ntf server is retrying to connect
NSActive -> "ACTIVE"
NSInactive -> "INACTIVE"
NSEnd -> "END"
NSAuth -> "AUTH"
NSErr err -> "ERR " <> err
NSSMPAuth -> "SMP_AUTH"
smpP =
A.takeTill (== ' ') >>= \case
"NEW" -> pure NSNew
"PENDING" -> pure NSPending
"ACTIVE" -> pure NSActive
"INACTIVE" -> pure NSInactive
"END" -> pure NSEnd
"AUTH" -> pure NSAuth
"ERR" -> NSErr <$> (A.space *> A.takeByteString)
"SMP_AUTH" -> pure NSSMPAuth
_ -> fail "bad NtfSubStatus"
instance StrEncoding NtfSubStatus where
strEncode = smpEncode
strP = smpP
data NtfTknStatus
= -- | Token created in DB
NTNew
@@ -484,10 +424,6 @@ instance Encoding NtfTknStatus where
"EXPIRED" -> pure NTExpired
_ -> fail "bad NtfTknStatus"
instance StrEncoding NtfTknStatus where
strEncode = smpEncode
strP = smpP
instance FromField NtfTknStatus where fromField = fromTextField_ $ either (const Nothing) Just . smpDecode . encodeUtf8
instance ToField NtfTknStatus where toField = toField . decodeLatin1 . smpEncode
+121 -297
View File
@@ -4,57 +4,38 @@
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TupleSections #-}
module Simplex.Messaging.Notifications.Server where
import Control.Concurrent.STM (stateTVar)
import Control.Logger.Simple
import Control.Monad.Except
import Control.Monad.IO.Unlift (MonadUnliftIO)
import Control.Monad.Reader
import Crypto.Random (MonadRandom)
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Functor (($>))
import Data.List (intercalate)
import Data.Map.Strict (Map)
import qualified Data.Text as T
import Data.Text.Encoding (decodeLatin1)
import Data.Time.Clock (UTCTime (..), diffTimeToPicoseconds, getCurrentTime)
import Data.Time.Clock.System (getSystemTime)
import Data.Time.Format.ISO8601 (iso8601Show)
import Network.Socket (ServiceName)
import Simplex.Messaging.Client (ProtocolClientError (..))
import Simplex.Messaging.Client.Agent
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Env
import Simplex.Messaging.Notifications.Server.Push.APNS (PNMessageData (..), PushNotification (..), PushProviderError (..))
import Simplex.Messaging.Notifications.Server.Stats
import Simplex.Messaging.Notifications.Server.Push.APNS
import Simplex.Messaging.Notifications.Server.Store
import Simplex.Messaging.Notifications.Server.StoreLog
import Simplex.Messaging.Notifications.Transport
import Simplex.Messaging.Protocol (ErrorType (..), ProtocolServer (host), SMPServer, SignedTransmission, Transmission, encodeTransmission, tGet, tPut)
import Simplex.Messaging.Protocol (ErrorType (..), SMPServer, SignedTransmission, Transmission, encodeTransmission, tGet, tPut)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Server
import Simplex.Messaging.Server.Stats
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport (ATransport (..), THandle (..), TProxy, Transport (..))
import Simplex.Messaging.Transport.Server (runTransportServer)
import Simplex.Messaging.Util
import System.Exit (exitFailure)
import System.IO (BufferMode (..), hPutStrLn, hSetBuffering)
import System.Mem.Weak (deRefWeak)
import UnliftIO (IOMode (..), async, uninterruptibleCancel, withFile)
import UnliftIO.Concurrent (forkIO, killThread, mkWeakThreadId, threadDelay)
import UnliftIO.Directory (doesFileExist, renameFile)
import UnliftIO (async, uninterruptibleCancel)
import UnliftIO.Concurrent (forkFinally, threadDelay)
import UnliftIO.Exception
import UnliftIO.STM
@@ -67,13 +48,10 @@ runNtfServerBlocking :: (MonadRandom m, MonadUnliftIO m) => TMVar Bool -> NtfSer
runNtfServerBlocking started cfg = runReaderT (ntfServer cfg started) =<< newNtfServerEnv cfg
ntfServer :: forall m. (MonadUnliftIO m, MonadReader NtfEnv m) => NtfServerConfig -> TMVar Bool -> m ()
ntfServer cfg@NtfServerConfig {transports} started = do
restoreServerStats
ntfServer NtfServerConfig {transports} started = do
s <- asks subscriber
ps <- asks pushServer
subs <- readTVarIO =<< asks (subscriptions . store)
void . forkIO $ resubscribe s subs
raceAny_ (ntfSubscriber s : ntfPush ps : map runServer transports <> serverStatsThread_ cfg) `finally` stopServer
raceAny_ (ntfSubscriber s : ntfPush ps : map runServer transports)
where
runServer :: (ServiceName, ATransport) -> m ()
runServer (tcpPort, ATransport t) = do
@@ -87,67 +65,6 @@ ntfServer cfg@NtfServerConfig {transports} started = do
Right th -> runNtfClientTransport th
Left _ -> pure ()
stopServer :: m ()
stopServer = do
withNtfLog closeStoreLog
saveServerStats
asks (smpSubscribers . subscriber) >>= readTVarIO >>= mapM_ (\SMPSubscriber {subThreadId} -> readTVarIO subThreadId >>= mapM_ (liftIO . deRefWeak >=> mapM_ killThread))
serverStatsThread_ :: NtfServerConfig -> [m ()]
serverStatsThread_ NtfServerConfig {logStatsInterval = Just interval, logStatsStartTime, serverStatsLogFile} =
[logServerStats logStatsStartTime interval serverStatsLogFile]
serverStatsThread_ _ = []
logServerStats :: Int -> Int -> FilePath -> m ()
logServerStats startAt logInterval statsFilePath = do
initialDelay <- (startAt -) . fromIntegral . (`div` 1000000_000000) . diffTimeToPicoseconds . utctDayTime <$> liftIO getCurrentTime
liftIO $ putStrLn $ "server stats log enabled: " <> statsFilePath
threadDelay $ 1000000 * (initialDelay + if initialDelay < 0 then 86400 else 0)
NtfServerStats {fromTime, tknCreated, tknVerified, tknDeleted, subCreated, subDeleted, ntfReceived, ntfDelivered, activeTokens, activeSubs} <- asks serverStats
let interval = 1000000 * logInterval
withFile statsFilePath AppendMode $ \h -> liftIO $ do
hSetBuffering h LineBuffering
forever $ do
ts <- getCurrentTime
fromTime' <- atomically $ swapTVar fromTime ts
tknCreated' <- atomically $ swapTVar tknCreated 0
tknVerified' <- atomically $ swapTVar tknVerified 0
tknDeleted' <- atomically $ swapTVar tknDeleted 0
subCreated' <- atomically $ swapTVar subCreated 0
subDeleted' <- atomically $ swapTVar subDeleted 0
ntfReceived' <- atomically $ swapTVar ntfReceived 0
ntfDelivered' <- atomically $ swapTVar ntfDelivered 0
tkn <- atomically $ periodStatCounts activeTokens ts
sub <- atomically $ periodStatCounts activeSubs ts
hPutStrLn h $
intercalate
","
[ iso8601Show $ utctDay fromTime',
show tknCreated',
show tknVerified',
show tknDeleted',
show subCreated',
show subDeleted',
show ntfReceived',
show ntfDelivered',
dayCount tkn,
weekCount tkn,
monthCount tkn,
dayCount sub,
weekCount sub,
monthCount sub
]
threadDelay interval
resubscribe :: (MonadUnliftIO m, MonadReader NtfEnv m) => NtfSubscriber -> Map NtfSubscriptionId NtfSubData -> m ()
resubscribe NtfSubscriber {newSubQ} subs = do
d <- asks $ resubscribeDelay . config
forM_ subs $ \sub@NtfSubData {} ->
whenM (ntfShouldSubscribe <$> readTVarIO (subStatus sub)) $ do
atomically $ writeTBQueue newSubQ $ NtfSub sub
threadDelay d
liftIO $ logInfo "SMP connections resubscribed"
ntfSubscriber :: forall m. (MonadUnliftIO m, MonadReader NtfEnv m) => NtfSubscriber -> m ()
ntfSubscriber NtfSubscriber {smpSubscribers, newSubQ, smpAgent = ca@SMPClientAgent {msgQ, agentQ}} = do
raceAny_ [subscribe, receiveSMP, receiveAgent]
@@ -158,53 +75,44 @@ ntfSubscriber NtfSubscriber {smpSubscribers, newSubQ, smpAgent = ca@SMPClientAge
atomically (readTBQueue newSubQ) >>= \case
sub@(NtfSub NtfSubData {smpQueue = SMPQueueNtf {smpServer}}) -> do
SMPSubscriber {newSubQ = subscriberSubQ} <- getSMPSubscriber smpServer
atomically $ writeTQueue subscriberSubQ sub
atomically $ writeTBQueue subscriberSubQ sub
getSMPSubscriber :: SMPServer -> m SMPSubscriber
getSMPSubscriber smpServer =
atomically (TM.lookup smpServer smpSubscribers) >>= maybe createSMPSubscriber pure
where
createSMPSubscriber = do
sub@SMPSubscriber {subThreadId} <- atomically newSMPSubscriber
atomically $ TM.insert smpServer sub smpSubscribers
tId <- mkWeakThreadId =<< forkIO (runSMPSubscriber sub)
atomically . writeTVar subThreadId $ Just tId
pure sub
qSize <- asks $ subQSize . config
newSubscriber <- atomically $ newSMPSubscriber qSize
atomically $ TM.insert smpServer newSubscriber smpSubscribers
_ <- runSMPSubscriber newSubscriber `forkFinally` \_ -> atomically (TM.delete smpServer smpSubscribers >> failSubscriptions newSubscriber)
pure newSubscriber
-- TODO mark subscriptions as failed
failSubscriptions _ = pure ()
runSMPSubscriber :: SMPSubscriber -> m ()
runSMPSubscriber SMPSubscriber {newSubQ = subscriberSubQ} =
forever $
atomically (peekTQueue subscriberSubQ)
>>= \(NtfSub NtfSubData {smpQueue, notifierKey}) -> do
updateSubStatus smpQueue NSPending
atomically (peekTBQueue subscriberSubQ) >>= \case
NtfSub NtfSubData {smpQueue, notifierKey} -> do
let SMPQueueNtf {smpServer, notifierId} = smpQueue
liftIO (runExceptT $ subscribeQueue ca smpServer ((SPNotifier, notifierId), notifierKey)) >>= \case
Right _ -> do
updateSubStatus smpQueue NSActive
void . atomically $ readTQueue subscriberSubQ
Left err -> do
handleSubError smpQueue err
case err of
PCEResponseTimeout -> pure ()
PCENetworkError -> pure ()
_ -> void . atomically $ readTQueue subscriberSubQ
-- update subscription status
_ <- atomically $ readTBQueue subscriberSubQ
pure ()
Left _e -> pure ()
receiveSMP :: m ()
receiveSMP = forever $ do
(srv, _, _, ntfId, msg) <- atomically $ readTBQueue msgQ
let smpQueue = SMPQueueNtf srv ntfId
(srv, _sessId, ntfId, msg) <- atomically $ readTBQueue msgQ
case msg of
SMP.NMSG nmsgNonce encNMsgMeta -> do
ntfTs <- liftIO getSystemTime
st <- asks store
SMP.NMSG -> do
NtfPushServer {pushQ} <- asks pushServer
stats <- asks serverStats
atomically $ updatePeriodStats (activeSubs stats) ntfId
st <- asks store
atomically $
findNtfSubscriptionToken st smpQueue
>>= mapM_ (\tkn -> writeTBQueue pushQ (tkn, PNMessage PNMessageData {smpQueue, ntfTs, nmsgNonce, encNMsgMeta}))
incNtfStat ntfReceived
SMP.END -> updateSubStatus smpQueue NSEnd
findNtfSubscriptionToken st (SMPQueueNtf srv ntfId)
>>= mapM_ (\tkn -> writeTBQueue pushQ (tkn, PNMessage srv ntfId))
_ -> pure ()
pure ()
@@ -212,94 +120,39 @@ ntfSubscriber NtfSubscriber {smpSubscribers, newSubQ, smpAgent = ca@SMPClientAge
forever $
atomically (readTBQueue agentQ) >>= \case
CAConnected _ -> pure ()
CADisconnected srv subs -> do
logInfo $ "SMP server disconnected " <> showServer' srv <> " (" <> tshow (length subs) <> ") subscriptions"
forM_ subs $ \(_, ntfId) -> do
let smpQueue = SMPQueueNtf srv ntfId
updateSubStatus smpQueue NSInactive
CAReconnected srv ->
logInfo $ "SMP server reconnected " <> showServer' srv
CAResubscribed srv sub -> do
let ntfId = snd sub
smpQueue = SMPQueueNtf srv ntfId
updateSubStatus smpQueue NSActive
CASubError srv (_, ntfId) err -> do
logError $ "SMP subscription error on server " <> showServer' srv <> ": " <> tshow err
handleSubError (SMPQueueNtf srv ntfId) err
where
showServer' = decodeLatin1 . strEncode . host
CADisconnected _srv _subs -> do
-- update subscription statuses
pure ()
CAReconnected _ -> pure ()
CAResubscribed _srv _sub -> do
-- update subscription status
pure ()
CASubError _srv _sub _err -> do
-- update subscription status
pure ()
handleSubError :: SMPQueueNtf -> ProtocolClientError -> m ()
handleSubError smpQueue = \case
PCEProtocolError AUTH -> updateSubStatus smpQueue NSAuth
PCEProtocolError e -> updateErr "SMP error " e
PCEIOError e -> updateErr "IOError " e
PCEResponseError e -> updateErr "ResponseError " e
PCEUnexpectedResponse r -> updateErr "UnexpectedResponse " r
PCETransportError e -> updateErr "TransportError " e
PCESignatureError e -> updateErr "SignatureError " e
PCEIncompatibleHost -> updateSubStatus smpQueue $ NSErr "IncompatibleHost"
PCEResponseTimeout -> pure ()
PCENetworkError -> pure ()
where
updateErr :: Show e => ByteString -> e -> m ()
updateErr errType e = updateSubStatus smpQueue . NSErr $ errType <> bshow e
updateSubStatus smpQueue status = do
st <- asks store
atomically (findNtfSubscription st smpQueue)
>>= mapM_
( \NtfSubData {ntfSubId, subStatus} -> do
atomically $ writeTVar subStatus status
withNtfLog $ \sl -> logSubscriptionStatus sl ntfSubId status
)
ntfPush :: forall m. (MonadUnliftIO m, MonadReader NtfEnv m) => NtfPushServer -> m ()
ntfPush s@NtfPushServer {pushQ} = forever $ do
(tkn@NtfTknData {ntfTknId, token = DeviceToken pp _, tknStatus}, ntf) <- atomically (readTBQueue pushQ)
liftIO $ logDebug $ "sending push notification to " <> T.pack (show pp)
ntfPush :: MonadUnliftIO m => NtfPushServer -> m ()
ntfPush s@NtfPushServer {pushQ} = liftIO . forever . runExceptT $ do
(tkn@NtfTknData {token = DeviceToken pp _, tknStatus}, ntf) <- atomically (readTBQueue pushQ)
logDebug $ "sending push notification to " <> T.pack (show pp)
status <- readTVarIO tknStatus
case (status, ntf) of
(_, PNVerification _) ->
(_, PNVerification _) -> do
-- TODO check token status
deliverNotification pp tkn ntf >>= \case
Right _ -> do
status_ <- atomically $ stateTVar tknStatus $ \status' -> if status' == NTActive then (Nothing, NTActive) else (Just NTConfirmed, NTConfirmed)
forM_ status_ $ \status' -> withNtfLog $ \sl -> logTokenStatus sl ntfTknId status'
_ -> pure ()
(NTActive, PNCheckMessages) ->
void $ deliverNotification pp tkn ntf
(NTActive, PNMessage {}) -> do
stats <- asks serverStats
atomically $ updatePeriodStats (activeTokens stats) ntfTknId
void $ deliverNotification pp tkn ntf
incNtfStat ntfDelivered
_ ->
liftIO $ logError "bad notification token status"
deliverNotification pp tkn ntf
atomically $ modifyTVar tknStatus $ \status' -> if status' == NTActive then NTActive else NTConfirmed
(NTActive, PNCheckMessages) -> do
deliverNotification pp tkn ntf
(NTActive, PNMessage _ _) -> do
deliverNotification pp tkn ntf
_ -> do
logError "bad notification token status"
where
deliverNotification :: PushProvider -> NtfTknData -> PushNotification -> m (Either PushProviderError ())
deliverNotification pp tkn@NtfTknData {ntfTknId, tknStatus} ntf = do
deliverNotification :: PushProvider -> PushProviderClient
deliverNotification pp tkn ntf = do
deliver <- liftIO $ getPushClient s pp
liftIO (runExceptT $ deliver tkn ntf) >>= \case
Right _ -> pure $ Right ()
Left e -> case e of
PPConnection _ -> retryDeliver
PPRetryLater -> retryDeliver
-- TODO alert
PPCryptoError _ -> err e
PPResponseError _ _ -> err e
PPTokenInvalid -> updateTknStatus NTInvalid >> err e
PPPermanentError -> err e
where
retryDeliver :: m (Either PushProviderError ())
retryDeliver = do
deliver <- liftIO $ newPushClient s pp
liftIO (runExceptT $ deliver tkn ntf) >>= either err (pure . Right)
updateTknStatus :: NtfTknStatus -> m ()
updateTknStatus status = do
atomically $ writeTVar tknStatus status
withNtfLog $ \sl -> logTokenStatus sl ntfTknId status
err e = logError (T.pack $ "Push provider error (" <> show pp <> "): " <> show e) $> Left e
-- TODO retry later based on the error
deliver tkn ntf `catchError` \e -> logError (T.pack $ "Push provider error (" <> show pp <> "): " <> show e) >> throwError e
runNtfClientTransport :: (Transport c, MonadUnliftIO m, MonadReader NtfEnv m) => THandle c -> m ()
runNtfClientTransport th@THandle {sessionId} = do
@@ -319,23 +172,22 @@ clientDisconnected NtfServerClient {connected} = atomically $ writeTVar connecte
receive :: (Transport c, MonadUnliftIO m, MonadReader NtfEnv m) => THandle c -> NtfServerClient -> m ()
receive th NtfServerClient {rcvQ, sndQ, activeAt} = forever $ do
ts <- tGet th
forM_ ts $ \t@(_, _, (corrId, entId, cmdOrError)) -> do
atomically . writeTVar activeAt =<< liftIO getSystemTime
logDebug "received transmission"
case cmdOrError of
Left e -> write sndQ (corrId, entId, NRErr e)
Right cmd ->
verifyNtfTransmission t cmd >>= \case
VRVerified req -> write rcvQ req
VRFailed -> write sndQ (corrId, entId, NRErr AUTH)
t@(_, _, (corrId, entId, cmdOrError)) <- tGet th
atomically . writeTVar activeAt =<< liftIO getSystemTime
logDebug "received transmission"
case cmdOrError of
Left e -> write sndQ (corrId, entId, NRErr e)
Right cmd ->
verifyNtfTransmission t cmd >>= \case
VRVerified req -> write rcvQ req
VRFailed -> write sndQ (corrId, entId, NRErr AUTH)
where
write q t = atomically $ writeTBQueue q t
send :: (Transport c, MonadUnliftIO m) => THandle c -> NtfServerClient -> m ()
send h@THandle {thVersion = v} NtfServerClient {sndQ, sessionId, activeAt} = forever $ do
t <- atomically $ readTBQueue sndQ
void . liftIO $ tPut h [(Nothing, encodeTransmission v sessionId t)]
void . liftIO $ tPut h (Nothing, encodeTransmission v sessionId t)
atomically . writeTVar activeAt =<< liftIO getSystemTime
-- instance Show a => Show (TVar a) where
@@ -398,7 +250,7 @@ verifyNtfTransmission (sig_, signed, (corrId, entId, _)) cmd = do
verifyToken' t_ = verifyToken t_ . const
client :: forall m. (MonadUnliftIO m, MonadReader NtfEnv m) => NtfServerClient -> NtfSubscriber -> NtfPushServer -> m ()
client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ, smpAgent = ca} NtfPushServer {pushQ, intervalNotifiers} =
client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ} NtfPushServer {pushQ, intervalNotifiers} =
forever $
atomically (readTBQueue rcvQ)
>>= processCommand
@@ -413,13 +265,12 @@ client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ, smpAgent = ca} NtfPu
let dhSecret = C.dh' dhPubKey srvDhPrivKey
tknId <- getId
regCode <- getRegCode
tkn <- atomically $ mkNtfTknData tknId newTkn ks dhSecret regCode
atomically $ addNtfToken st tknId tkn
atomically $ writeTBQueue pushQ (tkn, PNVerification regCode)
withNtfLog (`logCreateToken` tkn)
incNtfStat tknCreated
atomically $ do
tkn <- mkNtfTknData tknId newTkn ks dhSecret regCode
addNtfToken st tknId tkn
writeTBQueue pushQ (tkn, PNVerification regCode)
pure (corrId, "", NRTknId tknId srvDhPubKey)
NtfReqCmd SToken (NtfTkn tkn@NtfTknData {ntfTknId, tknStatus, tknRegCode, tknDhSecret, tknDhKeys = (srvDhPubKey, srvDhPrivKey), tknCronInterval}) (corrId, tknId, cmd) -> do
NtfReqCmd SToken (NtfTkn tkn@NtfTknData {ntfTknId, tknStatus, tknRegCode, tknDhSecret, tknDhKeys = (srvDhPubKey, srvDhPrivKey)}) (corrId, tknId, cmd) -> do
status <- readTVarIO tknStatus
(corrId,tknId,) <$> case cmd of
TNEW (NewNtfTkn _ _ dhPubKey) -> do
@@ -438,57 +289,31 @@ client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ, smpAgent = ca} NtfPu
atomically $ writeTVar tknStatus NTActive
tIds <- atomically $ removeInactiveTokenRegistrations st tkn
forM_ tIds cancelInvervalNotifications
withNtfLog $ \s -> logTokenStatus s tknId NTActive
incNtfStat tknVerified
pure NROk
| otherwise -> do
logDebug "TVFY - incorrect code or token status"
pure $ NRErr AUTH
TCHK -> do
logDebug "TCHK"
pure $ NRTkn status
TRPL token' -> do
logDebug "TRPL - replace token"
st <- asks store
regCode <- getRegCode
atomically $ do
removeTokenRegistration st tkn
writeTVar tknStatus NTRegistered
let tkn' = tkn {token = token', tknRegCode = regCode}
addNtfToken st tknId tkn'
writeTBQueue pushQ (tkn', PNVerification regCode)
withNtfLog $ \s -> logUpdateToken s tknId token' regCode
incNtfStat tknDeleted
incNtfStat tknCreated
pure NROk
TCHK -> pure $ NRTkn status
TDEL -> do
logDebug "TDEL"
st <- asks store
qs <- atomically $ deleteNtfToken st tknId
forM_ qs $ \SMPQueueNtf {smpServer, notifierId} ->
atomically $ removeSubscription ca smpServer (SPNotifier, notifierId)
atomically $ deleteNtfToken st tknId
cancelInvervalNotifications tknId
withNtfLog (`logDeleteToken` tknId)
incNtfStat tknDeleted
pure NROk
TCRN 0 -> do
logDebug "TCRN 0"
atomically $ writeTVar tknCronInterval 0
cancelInvervalNotifications tknId
withNtfLog $ \s -> logTokenCron s tknId 0
pure NROk
TCRN int
| int < 20 -> pure $ NRErr QUOTA
| otherwise -> do
logDebug "TCRN"
atomically $ writeTVar tknCronInterval int
atomically (TM.lookup tknId intervalNotifiers) >>= \case
Nothing -> runIntervalNotifier int
Just IntervalNotifier {interval, action} ->
unless (interval == int) $ do
uninterruptibleCancel action
runIntervalNotifier int
withNtfLog $ \s -> logTokenCron s tknId int
pure NROk
where
runIntervalNotifier interval = do
@@ -503,15 +328,14 @@ client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ, smpAgent = ca} NtfPu
logDebug "SNEW - new subscription"
st <- asks store
subId <- getId
sub <- atomically $ mkNtfSubData subId newSub
resp <-
atomically (addNtfSubscription st subId sub) >>= \case
Just _ -> atomically (writeTBQueue newSubQ $ NtfSub sub) $> NRSubId subId
_ -> pure $ NRErr AUTH
withNtfLog (`logCreateSubscription` sub)
incNtfStat subCreated
pure (corrId, "", resp)
NtfReqCmd SSubscription (NtfSub NtfSubData {smpQueue = SMPQueueNtf {smpServer, notifierId}, notifierKey = registeredNKey, subStatus}) (corrId, subId, cmd) -> do
atomically $ do
sub <- mkNtfSubData newSub
(corrId,"",)
<$> ( addNtfSubscription st subId sub >>= \case
Just _ -> writeTBQueue newSubQ (NtfSub sub) $> NRSubId subId
_ -> pure $ NRErr AUTH
)
NtfReqCmd SSubscription (NtfSub NtfSubData {notifierKey = registeredNKey, subStatus}) (corrId, subId, cmd) -> do
status <- readTVarIO subStatus
(corrId,subId,) <$> case cmd of
SNEW (NewNtfSub _ _ notifierKey) -> do
@@ -521,17 +345,8 @@ client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ, smpAgent = ca} NtfPu
if notifierKey == registeredNKey
then NRSubId subId
else NRErr AUTH
SCHK -> do
logDebug "SCHK"
pure $ NRSub status
SDEL -> do
logDebug "SDEL"
st <- asks store
atomically $ deleteNtfSubscription st subId
atomically $ removeSubscription ca smpServer (SPNotifier, notifierId)
withNtfLog (`logDeleteSubscription` subId)
incNtfStat subDeleted
pure NROk
SCHK -> pure $ NRSub status
SDEL -> pure NROk
PING -> pure NRPong
getId :: m NtfEntityId
getId = getRandomBytes =<< asks (subIdBytes . config)
@@ -546,35 +361,44 @@ client NtfServerClient {rcvQ, sndQ} NtfSubscriber {newSubQ, smpAgent = ca} NtfPu
atomically (TM.lookupDelete tknId intervalNotifiers)
>>= mapM_ (uninterruptibleCancel . action)
withNtfLog :: (MonadUnliftIO m, MonadReader NtfEnv m) => (StoreLog 'WriteMode -> IO a) -> m ()
withNtfLog action = liftIO . mapM_ action =<< asks storeLog
incNtfStat :: (MonadUnliftIO m, MonadReader NtfEnv m) => (NtfServerStats -> TVar Int) -> m ()
incNtfStat statSel = do
stats <- asks serverStats
atomically $ modifyTVar (statSel stats) (+ 1)
saveServerStats :: (MonadUnliftIO m, MonadReader NtfEnv m) => m ()
saveServerStats =
asks (serverStatsBackupFile . config)
>>= mapM_ (\f -> asks serverStats >>= atomically . getNtfServerStatsData >>= liftIO . saveStats f)
where
saveStats f stats = do
logInfo $ "saving server stats to file " <> T.pack f
B.writeFile f $ strEncode stats
logInfo "server stats saved"
restoreServerStats :: (MonadUnliftIO m, MonadReader NtfEnv m) => m ()
restoreServerStats = asks (serverStatsBackupFile . config) >>= mapM_ restoreStats
where
restoreStats f = whenM (doesFileExist f) $ do
logInfo $ "restoring server stats from file " <> T.pack f
liftIO (strDecode <$> B.readFile f) >>= \case
Right d -> do
s <- asks serverStats
atomically $ setNtfServerStats s d
renameFile f $ f <> ".bak"
logInfo "server stats restored"
Left e -> do
logInfo $ "error restoring server stats: " <> T.pack e
liftIO exitFailure
-- NReqCreate corrId tokenId smpQueue -> pure (corrId, "", NROk)
-- do
-- st <- asks store
-- (pubDhKey, privDhKey) <- liftIO C.generateKeyPair'
-- let dhSecret = C.dh' dhPubKey privDhKey
-- sub <- atomically $ mkNtfSubsciption smpQueue token verifyKey dhSecret
-- addSubRetry 3 st sub >>= \case
-- Nothing -> pure (corrId, "", NRErr INTERNAL)
-- Just sId -> do
-- atomically $ writeTBQueue subQ sub
-- pure (corrId, sId, NRSubId pubDhKey)
-- where
-- addSubRetry :: Int -> NtfSubscriptionsStore -> NtfSubsciption -> m (Maybe NtfSubsciptionId)
-- addSubRetry 0 _ _ = pure Nothing
-- addSubRetry n st sub = do
-- sId <- getId
-- -- create QueueRec record with these ids and keys
-- atomically (addNtfSub st sId sub) >>= \case
-- Nothing -> addSubRetry (n - 1) st sub
-- _ -> pure $ Just sId
-- getId :: m NtfSubsciptionId
-- getId = do
-- n <- asks $ subIdBytes . config
-- gVar <- asks idsDrg
-- atomically (randomBytes n gVar)
-- NReqCommand sub@NtfSubsciption {tokenId, subStatus} (corrId, subId, cmd) ->
-- (corrId,subId,) <$> case cmd of
-- NCSubCreate tokenId smpQueue -> pure NROk
-- do
-- st <- asks store
-- (pubDhKey, privDhKey) <- liftIO C.generateKeyPair'
-- let dhSecret = C.dh' (dhPubKey newSub) privDhKey
-- atomically (updateNtfSub st sub newSub dhSecret) >>= \case
-- Nothing -> pure $ NRErr INTERNAL
-- _ -> atomically $ do
-- whenM ((== NSEnd) <$> readTVar status) $ writeTBQueue subQ sub
-- pure $ NRSubId pubDhKey
-- NCSubCheck -> NRStat <$> readTVarIO subStatus
-- NCSubDelete -> do
-- st <- asks store
-- atomically (deleteNtfSub st subId) $> NROk
@@ -2,16 +2,16 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
module Simplex.Messaging.Notifications.Server.Env where
import Control.Concurrent (ThreadId)
import Control.Concurrent.Async (Async)
import Control.Monad (void)
import Control.Monad.IO.Unlift
import Crypto.Random
import Data.ByteString.Char8 (ByteString)
import Data.Time.Clock (getCurrentTime)
import Data.Time.Clock.System (SystemTime)
import Data.Word (Word16)
import Data.X509.Validation (Fingerprint (..))
@@ -22,17 +22,13 @@ import Simplex.Messaging.Client.Agent
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Push.APNS
import Simplex.Messaging.Notifications.Server.Stats
import Simplex.Messaging.Notifications.Server.Store
import Simplex.Messaging.Notifications.Server.StoreLog
import Simplex.Messaging.Protocol (CorrId, SMPServer, Transmission)
import Simplex.Messaging.Server.Expiration
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport (ATransport)
import Simplex.Messaging.Transport.Server (loadFingerprint, loadTLSServerParams)
import System.IO (IOMode (..))
import System.Mem.Weak (Weak)
import UnliftIO.STM
data NtfServerConfig = NtfServerConfig
@@ -45,17 +41,10 @@ data NtfServerConfig = NtfServerConfig
smpAgentCfg :: SMPClientAgentConfig,
apnsConfig :: APNSPushClientConfig,
inactiveClientExpiration :: Maybe ExpirationConfig,
storeLogFile :: Maybe FilePath,
resubscribeDelay :: Int, -- microseconds
-- CA certificate private key is not needed for initialization
caCertificateFile :: FilePath,
privateKeyFile :: FilePath,
certificateFile :: FilePath,
-- stats config - see SMP server config
logStatsInterval :: Maybe Int,
logStatsStartTime :: Int,
serverStatsLogFile :: FilePath,
serverStatsBackupFile :: Maybe FilePath
certificateFile :: FilePath
}
defaultInactiveClientExpiration :: ExpirationConfig
@@ -70,25 +59,23 @@ data NtfEnv = NtfEnv
subscriber :: NtfSubscriber,
pushServer :: NtfPushServer,
store :: NtfStore,
storeLog :: Maybe (StoreLog 'WriteMode),
idsDrg :: TVar ChaChaDRG,
serverIdentity :: C.KeyHash,
tlsServerParams :: T.ServerParams,
serverIdentity :: C.KeyHash,
serverStats :: NtfServerStats
serverIdentity :: C.KeyHash
}
newNtfServerEnv :: (MonadUnliftIO m, MonadRandom m) => NtfServerConfig -> m NtfEnv
newNtfServerEnv config@NtfServerConfig {subQSize, pushQSize, smpAgentCfg, apnsConfig, storeLogFile, caCertificateFile, certificateFile, privateKeyFile} = do
newNtfServerEnv config@NtfServerConfig {subQSize, pushQSize, smpAgentCfg, apnsConfig, caCertificateFile, certificateFile, privateKeyFile} = do
idsDrg <- newTVarIO =<< drgNew
store <- atomically newNtfStore
storeLog <- liftIO $ mapM (`readWriteNtfStore` store) storeLogFile
subscriber <- atomically $ newNtfSubscriber subQSize smpAgentCfg
pushServer <- atomically $ newNtfPushServer pushQSize apnsConfig
-- TODO not creating APNS client on start to pass CI test, has to be replaced with mock APNS server
void . liftIO $ newPushClient pushServer PPApns
tlsServerParams <- liftIO $ loadTLSServerParams caCertificateFile certificateFile privateKeyFile
Fingerprint fp <- liftIO $ loadFingerprint caCertificateFile
serverStats <- atomically . newNtfServerStats =<< liftIO getCurrentTime
pure NtfEnv {config, subscriber, pushServer, store, storeLog, idsDrg, tlsServerParams, serverIdentity = C.KeyHash fp, serverStats}
pure NtfEnv {config, subscriber, pushServer, store, idsDrg, tlsServerParams, serverIdentity = C.KeyHash fp}
data NtfSubscriber = NtfSubscriber
{ smpSubscribers :: TMap SMPServer SMPSubscriber,
@@ -103,16 +90,14 @@ newNtfSubscriber qSize smpAgentCfg = do
smpAgent <- newSMPClientAgent smpAgentCfg
pure NtfSubscriber {smpSubscribers, newSubQ, smpAgent}
data SMPSubscriber = SMPSubscriber
{ newSubQ :: TQueue (NtfEntityRec 'Subscription),
subThreadId :: TVar (Maybe (Weak ThreadId))
newtype SMPSubscriber = SMPSubscriber
{ newSubQ :: TBQueue (NtfEntityRec 'Subscription)
}
newSMPSubscriber :: STM SMPSubscriber
newSMPSubscriber = do
newSubQ <- newTQueue
subThreadId <- newTVar Nothing
pure SMPSubscriber {newSubQ, subThreadId}
newSMPSubscriber :: Natural -> STM SMPSubscriber
newSMPSubscriber qSize = do
newSubQ <- newTBQueue qSize
pure SMPSubscriber {newSubQ}
data NtfPushServer = NtfPushServer
{ pushQ :: TBQueue (NtfTknData, PushNotification),
@@ -135,10 +120,11 @@ newNtfPushServer qSize apnsConfig = do
pure NtfPushServer {pushQ, pushClients, intervalNotifiers, apnsConfig}
newPushClient :: NtfPushServer -> PushProvider -> IO PushProviderClient
newPushClient NtfPushServer {apnsConfig, pushClients} pp = do
c <- apnsPushProviderClient <$> createAPNSPushClient (apnsProviderHost pp) apnsConfig
atomically $ TM.insert pp c pushClients
pure c
newPushClient NtfPushServer {apnsConfig, pushClients} = \case
PPApns -> do
c <- apnsPushProviderClient <$> createAPNSPushClient apnsConfig
atomically $ TM.insert PPApns c pushClients
pure c
getPushClient :: NtfPushServer -> PushProvider -> IO PushProviderClient
getPushClient s@NtfPushServer {pushClients} pp =
@@ -10,7 +10,6 @@
module Simplex.Messaging.Notifications.Server.Push.APNS where
import Control.Exception (Exception)
import Control.Logger.Simple
import Control.Monad.Except
import Crypto.Hash.Algorithms (SHA256 (..))
@@ -47,7 +46,7 @@ import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Store (NtfTknData (..))
import Simplex.Messaging.Protocol (EncNMsgMeta)
import Simplex.Messaging.Protocol (NotifierId, SMPServer)
import Simplex.Messaging.Transport.HTTP2.Client
import System.Environment (getEnv)
import UnliftIO.STM
@@ -96,25 +95,9 @@ readECPrivateKey f = do
data PushNotification
= PNVerification NtfRegCode
| PNMessage PNMessageData
| PNMessage SMPServer NotifierId
| PNAlert Text
| PNCheckMessages
deriving (Show)
data PNMessageData = PNMessageData
{ smpQueue :: SMPQueueNtf,
ntfTs :: SystemTime,
nmsgNonce :: C.CbNonce,
encNMsgMeta :: EncNMsgMeta
}
deriving (Show)
instance StrEncoding PNMessageData where
strEncode PNMessageData {smpQueue, ntfTs, nmsgNonce, encNMsgMeta} =
strEncode (smpQueue, ntfTs, nmsgNonce, encNMsgMeta)
strP = do
(smpQueue, ntfTs, nmsgNonce, encNMsgMeta) <- strP
pure PNMessageData {smpQueue, ntfTs, nmsgNonce, encNMsgMeta}
data APNSNotification = APNSNotification {aps :: APNSNotificationBody, notificationData :: Maybe J.Value}
deriving (Show, Generic)
@@ -124,9 +107,9 @@ instance ToJSON APNSNotification where
toEncoding = J.genericToEncoding J.defaultOptions {J.omitNothingFields = True}
data APNSNotificationBody
= APNSBackground {contentAvailable :: Int}
= APNSAlert {alert :: APNSAlertBody, badge :: Maybe Int, sound :: Maybe Text, category :: Maybe Text}
| APNSBackground {contentAvailable :: Int}
| APNSMutableContent {mutableContent :: Int, alert :: APNSAlertBody, category :: Maybe Text}
| APNSAlert {alert :: APNSAlertBody, badge :: Maybe Int, sound :: Maybe Text, category :: Maybe Text}
deriving (Show, Generic)
apnsJSONOptions :: J.Options
@@ -193,27 +176,23 @@ data APNSPushClientConfig = APNSPushClientConfig
paddedNtfLength :: Int,
appName :: ByteString,
appTeamId :: Text,
apnsHost :: HostName,
apnsPort :: ServiceName,
http2cfg :: HTTP2ClientConfig
}
deriving (Show)
apnsProviderHost :: PushProvider -> HostName
apnsProviderHost = \case
PPApnsTest -> "localhost"
PPApnsDev -> "api.sandbox.push.apple.com"
PPApnsProd -> "api.push.apple.com"
defaultAPNSPushClientConfig :: APNSPushClientConfig
defaultAPNSPushClientConfig =
APNSPushClientConfig
{ tokenTTL = 1800, -- 30 minutes
{ tokenTTL = 1200, -- 20 minutes
authKeyFileEnv = "APNS_KEY_FILE", -- the environment variables APNS_KEY_FILE and APNS_KEY_ID must be set, or the server would fail to start
authKeyAlg = "ES256",
authKeyIdEnv = "APNS_KEY_ID",
paddedNtfLength = 512,
paddedNtfLength = 256,
appName = "chat.simplex.app",
appTeamId = "5NN7GUYB6T",
apnsHost = "api.sandbox.push.apple.com",
apnsPort = "443",
http2cfg = defaultHTTP2ClientConfig
}
@@ -224,20 +203,19 @@ data APNSPushClient = APNSPushClient
jwtHeader :: JWTHeader,
jwtToken :: TVar (JWTToken, SignedJWTToken),
nonceDrg :: TVar ChaChaDRG,
apnsHost :: HostName,
apnsCfg :: APNSPushClientConfig
}
createAPNSPushClient :: HostName -> APNSPushClientConfig -> IO APNSPushClient
createAPNSPushClient apnsHost apnsCfg@APNSPushClientConfig {authKeyFileEnv, authKeyAlg, authKeyIdEnv, appTeamId} = do
createAPNSPushClient :: APNSPushClientConfig -> IO APNSPushClient
createAPNSPushClient apnsCfg@APNSPushClientConfig {authKeyFileEnv, authKeyAlg, authKeyIdEnv, appTeamId} = do
https2Client <- newTVarIO Nothing
void $ connectHTTPS2 apnsHost apnsCfg https2Client
void $ connectHTTPS2 apnsCfg https2Client
privateKey <- readECPrivateKey =<< getEnv authKeyFileEnv
authKeyId <- T.pack <$> getEnv authKeyIdEnv
let jwtHeader = JWTHeader {alg = authKeyAlg, kid = authKeyId}
jwtToken <- newTVarIO =<< mkApnsJWTToken appTeamId jwtHeader privateKey
nonceDrg <- drgNew >>= newTVarIO
pure APNSPushClient {https2Client, privateKey, jwtHeader, jwtToken, nonceDrg, apnsHost, apnsCfg}
pure APNSPushClient {https2Client, privateKey, jwtHeader, jwtToken, nonceDrg, apnsCfg}
getApnsJWTToken :: APNSPushClient -> IO SignedJWTToken
getApnsJWTToken APNSPushClient {apnsCfg = APNSPushClientConfig {appTeamId, tokenTTL}, privateKey, jwtHeader, jwtToken} = do
@@ -258,8 +236,8 @@ mkApnsJWTToken appTeamId jwtHeader privateKey = do
signedJWT <- signedJWTToken privateKey jwt
pure (jwt, signedJWT)
connectHTTPS2 :: HostName -> APNSPushClientConfig -> TVar (Maybe HTTP2Client) -> IO (Either HTTP2ClientError HTTP2Client)
connectHTTPS2 apnsHost APNSPushClientConfig {apnsPort, http2cfg} https2Client = do
connectHTTPS2 :: APNSPushClientConfig -> TVar (Maybe HTTP2Client) -> IO (Either HTTP2ClientError HTTP2Client)
connectHTTPS2 APNSPushClientConfig {apnsHost, apnsPort, http2cfg} https2Client = do
r <- getHTTP2Client apnsHost apnsPort http2cfg disconnected
case r of
Right client -> atomically . writeTVar https2Client $ Just client
@@ -269,31 +247,28 @@ connectHTTPS2 apnsHost APNSPushClientConfig {apnsPort, http2cfg} https2Client =
disconnected = atomically $ writeTVar https2Client Nothing
getApnsHTTP2Client :: APNSPushClient -> IO (Either HTTP2ClientError HTTP2Client)
getApnsHTTP2Client APNSPushClient {https2Client, apnsHost, apnsCfg} =
readTVarIO https2Client >>= maybe (connectHTTPS2 apnsHost apnsCfg https2Client) (pure . Right)
getApnsHTTP2Client APNSPushClient {https2Client, apnsCfg} =
readTVarIO https2Client >>= maybe (connectHTTPS2 apnsCfg https2Client) (pure . Right)
disconnectApnsHTTP2Client :: APNSPushClient -> IO ()
disconnectApnsHTTP2Client APNSPushClient {https2Client} =
readTVarIO https2Client >>= mapM_ closeHTTP2Client >> atomically (writeTVar https2Client Nothing)
ntfCategoryCheckMessage :: Text
ntfCategoryCheckMessage = "NTF_CAT_CHECK_MESSAGE"
apnsNotification :: NtfTknData -> C.CbNonce -> Int -> PushNotification -> Either C.CryptoError APNSNotification
apnsNotification NtfTknData {tknDhSecret} nonce paddedLen = \case
PNVerification (NtfRegCode code) ->
encrypt code $ \code' ->
apn APNSBackground {contentAvailable = 1} . Just $ J.object ["nonce" .= nonce, "verification" .= code']
PNMessage pnMessageData ->
encrypt (strEncode pnMessageData) $ \ntfData ->
apn apnMutableContent . Just $ J.object ["nonce" .= nonce, "message" .= ntfData]
apn APNSBackground {contentAvailable = 1} . Just $ J.object ["verification" .= code', "nonce" .= nonce]
PNMessage srv nId ->
encrypt (strEncode srv <> "/" <> strEncode nId) $ \ntfQueue ->
apn apnMutableContent . Just $ J.object ["checkMessage" .= ntfQueue, "nonce" .= nonce]
PNAlert text -> Right $ apn (apnAlert $ APNSAlertText text) Nothing
PNCheckMessages -> Right $ apn APNSBackground {contentAvailable = 1} . Just $ J.object ["checkMessages" .= True]
where
encrypt :: ByteString -> (Text -> APNSNotification) -> Either C.CryptoError APNSNotification
encrypt ntfData f = f . safeDecodeUtf8 . U.encode <$> C.cbEncrypt tknDhSecret nonce ntfData paddedLen
apn aps notificationData = APNSNotification {aps, notificationData}
apnMutableContent = APNSMutableContent {mutableContent = 1, alert = APNSAlertText "Encrypted message or another app event", category = Just ntfCategoryCheckMessage}
apnMutableContent = APNSMutableContent {mutableContent = 1, alert = APNSAlertText "Encrypted message or some other app event", category = Nothing}
apnAlert alert = APNSAlert {alert, badge = Nothing, sound = Nothing, category = Nothing}
safeDecodeUtf8 = decodeUtf8With onError where onError _ _ = Just '?'
@@ -323,7 +298,7 @@ data PushProviderError
| PPTokenInvalid
| PPRetryLater
| PPPermanentError
deriving (Show, Exception)
deriving (Show)
type PushProviderClient = NtfTknData -> PushNotification -> ExceptT PushProviderError IO ()
@@ -332,7 +307,7 @@ data APNSErrorResponse = APNSErrorResponse {reason :: Text}
deriving (Generic, FromJSON)
apnsPushProviderClient :: APNSPushClient -> PushProviderClient
apnsPushProviderClient c@APNSPushClient {nonceDrg, apnsCfg} tkn@NtfTknData {token = DeviceToken _ tknStr} pn = do
apnsPushProviderClient c@APNSPushClient {nonceDrg, apnsCfg} tkn@NtfTknData {token = DeviceToken PPApns tknStr} pn = do
http2 <- liftHTTPS2 $ getApnsHTTP2Client c
nonce <- atomically $ C.pseudoRandomCbNonce nonceDrg
apnsNtf <- liftEither $ first PPCryptoError $ apnsNotification tkn nonce (paddedNtfLength apnsCfg) pn
@@ -1,113 +0,0 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
module Simplex.Messaging.Notifications.Server.Stats where
import Control.Applicative (optional)
import qualified Data.Attoparsec.ByteString.Char8 as A
import qualified Data.ByteString.Char8 as B
import Data.Time.Clock (UTCTime)
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Protocol (NtfTokenId)
import Simplex.Messaging.Protocol (NotifierId)
import Simplex.Messaging.Server.Stats
import UnliftIO.STM
data NtfServerStats = NtfServerStats
{ fromTime :: TVar UTCTime,
tknCreated :: TVar Int,
tknVerified :: TVar Int,
tknDeleted :: TVar Int,
subCreated :: TVar Int,
subDeleted :: TVar Int,
ntfReceived :: TVar Int,
ntfDelivered :: TVar Int,
activeTokens :: PeriodStats NtfTokenId,
activeSubs :: PeriodStats NotifierId
}
data NtfServerStatsData = NtfServerStatsData
{ _fromTime :: UTCTime,
_tknCreated :: Int,
_tknVerified :: Int,
_tknDeleted :: Int,
_subCreated :: Int,
_subDeleted :: Int,
_ntfReceived :: Int,
_ntfDelivered :: Int,
_activeTokens :: PeriodStatsData NtfTokenId,
_activeSubs :: PeriodStatsData NotifierId
}
newNtfServerStats :: UTCTime -> STM NtfServerStats
newNtfServerStats ts = do
fromTime <- newTVar ts
tknCreated <- newTVar 0
tknVerified <- newTVar 0
tknDeleted <- newTVar 0
subCreated <- newTVar 0
subDeleted <- newTVar 0
ntfReceived <- newTVar 0
ntfDelivered <- newTVar 0
activeTokens <- newPeriodStats
activeSubs <- newPeriodStats
pure NtfServerStats {fromTime, tknCreated, tknVerified, tknDeleted, subCreated, subDeleted, ntfReceived, ntfDelivered, activeTokens, activeSubs}
getNtfServerStatsData :: NtfServerStats -> STM NtfServerStatsData
getNtfServerStatsData s = do
_fromTime <- readTVar $ fromTime (s :: NtfServerStats)
_tknCreated <- readTVar $ tknCreated s
_tknVerified <- readTVar $ tknVerified s
_tknDeleted <- readTVar $ tknDeleted s
_subCreated <- readTVar $ subCreated s
_subDeleted <- readTVar $ subDeleted s
_ntfReceived <- readTVar $ ntfReceived s
_ntfDelivered <- readTVar $ ntfDelivered s
_activeTokens <- getPeriodStatsData $ activeTokens s
_activeSubs <- getPeriodStatsData $ activeSubs s
pure NtfServerStatsData {_fromTime, _tknCreated, _tknVerified, _tknDeleted, _subCreated, _subDeleted, _ntfReceived, _ntfDelivered, _activeTokens, _activeSubs}
setNtfServerStats :: NtfServerStats -> NtfServerStatsData -> STM ()
setNtfServerStats s d = do
writeTVar (fromTime (s :: NtfServerStats)) (_fromTime (d :: NtfServerStatsData))
writeTVar (tknCreated s) (_tknCreated d)
writeTVar (tknVerified s) (_tknVerified d)
writeTVar (tknDeleted s) (_tknDeleted d)
writeTVar (subCreated s) (_subCreated d)
writeTVar (subDeleted s) (_subDeleted d)
writeTVar (ntfReceived s) (_ntfReceived d)
writeTVar (ntfDelivered s) (_ntfDelivered d)
setPeriodStats (activeTokens s) (_activeTokens d)
setPeriodStats (activeSubs s) (_activeSubs d)
instance StrEncoding NtfServerStatsData where
strEncode NtfServerStatsData {_fromTime, _tknCreated, _tknVerified, _tknDeleted, _subCreated, _subDeleted, _ntfReceived, _ntfDelivered, _activeTokens, _activeSubs} =
B.unlines
[ "fromTime=" <> strEncode _fromTime,
"tknCreated=" <> strEncode _tknCreated,
"tknVerified=" <> strEncode _tknVerified,
"tknDeleted=" <> strEncode _tknDeleted,
"subCreated=" <> strEncode _subCreated,
"subDeleted=" <> strEncode _subDeleted,
"ntfReceived=" <> strEncode _ntfReceived,
"ntfDelivered=" <> strEncode _ntfDelivered,
"activeTokens:",
strEncode _activeTokens,
"activeSubs:",
strEncode _activeSubs
]
strP = do
_fromTime <- "fromTime=" *> strP <* A.endOfLine
_tknCreated <- "tknCreated=" *> strP <* A.endOfLine
_tknVerified <- "tknVerified=" *> strP <* A.endOfLine
_tknDeleted <- "tknDeleted=" *> strP <* A.endOfLine
_subCreated <- "subCreated=" *> strP <* A.endOfLine
_subDeleted <- "subDeleted=" *> strP <* A.endOfLine
_ntfReceived <- "ntfReceived=" *> strP <* A.endOfLine
_ntfDelivered <- "ntfDelivered=" *> strP <* A.endOfLine
_ <- "activeTokens:" <* A.endOfLine
_activeTokens <- strP <* A.endOfLine
_ <- "activeSubs:" <* A.endOfLine
_activeSubs <- strP <* optional A.endOfLine
pure NtfServerStatsData {_fromTime, _tknCreated, _tknVerified, _tknDeleted, _subCreated, _subDeleted, _ntfReceived, _ntfDelivered, _activeTokens, _activeSubs}
@@ -12,12 +12,9 @@ module Simplex.Messaging.Notifications.Server.Store where
import Control.Concurrent.STM
import Control.Monad
import Data.ByteString.Char8 (ByteString)
import Data.Functor (($>))
import qualified Data.Map.Strict as M
import Data.Maybe (catMaybes)
import Data.Set (Set)
import qualified Data.Set as S
import Data.Word (Word16)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Protocol (NtfPrivateSignKey)
@@ -27,7 +24,6 @@ import Simplex.Messaging.Util (whenM, ($>>=))
data NtfStore = NtfStore
{ tokens :: TMap NtfTokenId NtfTknData,
-- multiple registrations exist to protect from malicious registrations if token is compromised
tokenRegistrations :: TMap DeviceToken (TMap ByteString NtfTokenId),
subscriptions :: TMap NtfSubscriptionId NtfSubData,
tokenSubscriptions :: TMap NtfTokenId (TVar (Set NtfSubscriptionId)),
@@ -50,19 +46,26 @@ data NtfTknData = NtfTknData
tknVerifyKey :: C.APublicVerifyKey,
tknDhKeys :: C.KeyPair 'C.X25519,
tknDhSecret :: C.DhSecretX25519,
tknRegCode :: NtfRegCode,
tknCronInterval :: TVar Word16
tknRegCode :: NtfRegCode
}
mkNtfTknData :: NtfTokenId -> NewNtfEntity 'Token -> C.KeyPair 'C.X25519 -> C.DhSecretX25519 -> NtfRegCode -> STM NtfTknData
mkNtfTknData ntfTknId (NewNtfTkn token tknVerifyKey _) tknDhKeys tknDhSecret tknRegCode = do
tknStatus <- newTVar NTRegistered
tknCronInterval <- newTVar 0
pure NtfTknData {ntfTknId, token, tknStatus, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval}
pure NtfTknData {ntfTknId, token, tknStatus, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode}
-- data NtfSubscriptionsStore = NtfSubscriptionsStore
-- { subscriptions :: TMap NtfSubsciptionId NtfSubsciption,
-- activeSubscriptions :: TMap (SMPServer, NotifierId) NtfSubsciptionId
-- }
-- do
-- subscriptions <- newTVar M.empty
-- activeSubscriptions <- newTVar M.empty
-- pure NtfSubscriptionsStore {subscriptions, activeSubscriptions}
data NtfSubData = NtfSubData
{ ntfSubId :: NtfSubscriptionId,
smpQueue :: SMPQueueNtf,
{ smpQueue :: SMPQueueNtf,
notifierKey :: NtfPrivateSignKey,
tokenId :: NtfTokenId,
subStatus :: TVar NtfSubStatus
@@ -108,16 +111,7 @@ removeInactiveTokenRegistrations st NtfTknData {ntfTknId = tId, token} =
TM.delete tId' $ tokens st
pure $ map snd tIds
removeTokenRegistration :: NtfStore -> NtfTknData -> STM ()
removeTokenRegistration st NtfTknData {ntfTknId = tId, token, tknVerifyKey} =
TM.lookup token (tokenRegistrations st) >>= mapM_ removeReg
where
removeReg regs =
TM.lookup k regs
>>= mapM_ (\tId' -> when (tId == tId') $ TM.delete k regs)
k = C.toPubKey C.pubKeyBytes tknVerifyKey
deleteNtfToken :: NtfStore -> NtfTokenId -> STM [SMPQueueNtf]
deleteNtfToken :: NtfStore -> NtfTokenId -> STM ()
deleteNtfToken st tknId = do
TM.lookupDelete tknId (tokens st)
>>= mapM_
@@ -129,22 +123,6 @@ deleteNtfToken st tknId = do
whenM (TM.null tIds) $ TM.delete token regs
)
)
qs <-
TM.lookupDelete tknId (tokenSubscriptions st)
>>= mapM
( readTVar
>=> mapM
( \subId -> do
TM.lookupDelete subId (subscriptions st)
>>= mapM
( \NtfSubData {smpQueue} ->
TM.delete smpQueue (subscriptionLookup st) $> smpQueue
)
)
. S.toList
)
pure $ maybe [] catMaybes qs
where
regs = tokenRegistrations st
regKey = C.toPubKey C.pubKeyBytes
@@ -169,10 +147,10 @@ getActiveNtfToken st tknId =
tStatus <- readTVar tknStatus
pure $ if tStatus == NTActive then Just tkn else Nothing
mkNtfSubData :: NtfSubscriptionId -> NewNtfEntity 'Subscription -> STM NtfSubData
mkNtfSubData ntfSubId (NewNtfSub tokenId smpQueue notifierKey) = do
mkNtfSubData :: NewNtfEntity 'Subscription -> STM NtfSubData
mkNtfSubData (NewNtfSub tokenId smpQueue notifierKey) = do
subStatus <- newTVar NSNew
pure NtfSubData {ntfSubId, smpQueue, tokenId, subStatus, notifierKey}
pure NtfSubData {smpQueue, tokenId, subStatus, notifierKey}
addNtfSubscription :: NtfStore -> NtfSubscriptionId -> NtfSubData -> STM (Maybe ())
addNtfSubscription st subId sub@NtfSubData {smpQueue, tokenId} =
@@ -189,12 +167,38 @@ addNtfSubscription st subId sub@NtfSubData {smpQueue, tokenId} =
-- return Nothing if subscription existed before
pure $ Just ()
deleteNtfSubscription :: NtfStore -> NtfSubscriptionId -> STM ()
deleteNtfSubscription st subId = do
TM.lookupDelete subId (subscriptions st)
>>= mapM_
( \NtfSubData {smpQueue, tokenId} -> do
TM.delete smpQueue $ subscriptionLookup st
ts_ <- TM.lookup tokenId (tokenSubscriptions st)
forM_ ts_ $ \ts -> modifyTVar' ts $ S.delete subId
)
-- getNtfRec :: NtfStore -> SNtfEntity e -> NtfEntityId -> STM (Maybe (NtfEntityRec e))
-- getNtfRec st ent entId = case ent of
-- SToken -> NtfTkn <$$> TM.lookup entId (tokens st)
-- SSubscription -> pure Nothing
-- getNtfVerifyKey :: NtfStore -> SNtfEntity e -> NtfEntityId -> STM (Maybe (NtfEntityRec e, C.APublicVerifyKey))
-- getNtfVerifyKey st ent entId =
-- getNtfRec st ent entId >>= \case
-- Just r@(NtfTkn NtfTknData {tknVerifyKey}) -> pure $ Just (r, tknVerifyKey)
-- Just r@(NtfSub NtfSubData {tokenId}) ->
-- getNtfRec st SToken tokenId >>= \case
-- Just (NtfTkn NtfTknData {tknVerifyKey}) -> pure $ Just (r, tknVerifyKey)
-- _ -> pure Nothing
-- _ -> pure Nothing
-- mkNtfSubsciption :: SMPQueueNtf -> NtfTokenId -> STM NtfSubsciption
-- mkNtfSubsciption smpQueue tokenId = do
-- subStatus <- newTVar NSNew
-- pure NtfSubsciption {smpQueue, tokenId, subStatus}
-- getNtfSub :: NtfSubscriptionsStore -> NtfSubsciptionId -> STM (Maybe NtfSubsciption)
-- getNtfSub st subId = pure Nothing -- maybe (pure $ Left AUTH) (fmap Right . readTVar) . M.lookup subId . subscriptions =<< readTVar st
-- getNtfSubViaSMPQueue :: NtfSubscriptionsStore -> SMPQueueNtf -> STM (Maybe NtfSubsciption)
-- getNtfSubViaSMPQueue st smpQueue = pure Nothing
-- -- replace keeping status
-- updateNtfSub :: NtfSubscriptionsStore -> NtfSubsciption -> SMPQueueNtf -> NtfTokenId -> C.DhSecretX25519 -> STM (Maybe ())
-- updateNtfSub st sub smpQueue tokenId dhSecret = pure Nothing
-- addNtfSub :: NtfSubscriptionsStore -> NtfSubsciptionId -> NtfSubsciption -> STM (Maybe ())
-- addNtfSub st subId sub = pure Nothing
-- deleteNtfSub :: NtfSubscriptionsStore -> NtfSubsciptionId -> STM ()
-- deleteNtfSub st subId = pure ()
@@ -1,227 +0,0 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
module Simplex.Messaging.Notifications.Server.StoreLog
( StoreLog,
NtfStoreLogRecord (..),
readWriteNtfStore,
logCreateToken,
logTokenStatus,
logUpdateToken,
logTokenCron,
logDeleteToken,
logCreateSubscription,
logSubscriptionStatus,
logDeleteSubscription,
closeStoreLog,
)
where
import Control.Concurrent.STM
import Control.Monad (void)
import qualified Data.Attoparsec.ByteString.Char8 as A
import qualified Data.ByteString.Char8 as B
import Data.Word (Word16)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Store
import Simplex.Messaging.Protocol (NtfPrivateSignKey)
import Simplex.Messaging.Server.StoreLog
import Simplex.Messaging.Util (whenM)
import System.Directory (doesFileExist, renameFile)
import System.IO
data NtfStoreLogRecord
= CreateToken NtfTknRec
| TokenStatus NtfTokenId NtfTknStatus
| UpdateToken NtfTokenId DeviceToken NtfRegCode
| TokenCron NtfTokenId Word16
| DeleteToken NtfTokenId
| CreateSubscription NtfSubRec
| SubscriptionStatus NtfSubscriptionId NtfSubStatus
| DeleteSubscription NtfSubscriptionId
data NtfTknRec = NtfTknRec
{ ntfTknId :: NtfTokenId,
token :: DeviceToken,
tknStatus :: NtfTknStatus,
tknVerifyKey :: C.APublicVerifyKey,
tknDhKeys :: C.KeyPair 'C.X25519,
tknDhSecret :: C.DhSecretX25519,
tknRegCode :: NtfRegCode,
tknCronInterval :: Word16
}
mkTknData :: NtfTknRec -> STM NtfTknData
mkTknData NtfTknRec {ntfTknId, token, tknStatus = status, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval = cronInt} = do
tknStatus <- newTVar status
tknCronInterval <- newTVar cronInt
pure NtfTknData {ntfTknId, token, tknStatus, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval}
mkTknRec :: NtfTknData -> STM NtfTknRec
mkTknRec NtfTknData {ntfTknId, token, tknStatus = status, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval = cronInt} = do
tknStatus <- readTVar status
tknCronInterval <- readTVar cronInt
pure NtfTknRec {ntfTknId, token, tknStatus, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval}
data NtfSubRec = NtfSubRec
{ ntfSubId :: NtfSubscriptionId,
smpQueue :: SMPQueueNtf,
notifierKey :: NtfPrivateSignKey,
tokenId :: NtfTokenId,
subStatus :: NtfSubStatus
}
mkSubData :: NtfSubRec -> STM NtfSubData
mkSubData NtfSubRec {ntfSubId, smpQueue, notifierKey, tokenId, subStatus = status} = do
subStatus <- newTVar status
pure NtfSubData {ntfSubId, smpQueue, notifierKey, tokenId, subStatus}
mkSubRec :: NtfSubData -> STM NtfSubRec
mkSubRec NtfSubData {ntfSubId, smpQueue, notifierKey, tokenId, subStatus = status} = do
subStatus <- readTVar status
pure NtfSubRec {ntfSubId, smpQueue, notifierKey, tokenId, subStatus}
instance StrEncoding NtfStoreLogRecord where
strEncode = \case
CreateToken tknRec -> strEncode (Str "TCREATE", tknRec)
TokenStatus tknId tknStatus -> strEncode (Str "TSTATUS", tknId, tknStatus)
UpdateToken tknId token regCode -> strEncode (Str "TUPDATE", tknId, token, regCode)
TokenCron tknId cronInt -> strEncode (Str "TCRON", tknId, cronInt)
DeleteToken tknId -> strEncode (Str "TDELETE", tknId)
CreateSubscription subRec -> strEncode (Str "SCREATE", subRec)
SubscriptionStatus subId subStatus -> strEncode (Str "SSTATUS", subId, subStatus)
DeleteSubscription subId -> strEncode (Str "SDELETE", subId)
strP =
A.choice
[ "TCREATE " *> (CreateToken <$> strP),
"TSTATUS " *> (TokenStatus <$> strP_ <*> strP),
"TUPDATE " *> (UpdateToken <$> strP_ <*> strP_ <*> strP),
"TCRON " *> (TokenCron <$> strP_ <*> strP),
"TDELETE " *> (DeleteToken <$> strP),
"SCREATE " *> (CreateSubscription <$> strP),
"SSTATUS " *> (SubscriptionStatus <$> strP_ <*> strP),
"SDELETE " *> (DeleteSubscription <$> strP)
]
instance StrEncoding NtfTknRec where
strEncode NtfTknRec {ntfTknId, token, tknStatus, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval} =
B.unwords
[ "tknId=" <> strEncode ntfTknId,
"token=" <> strEncode token,
"tokenStatus=" <> strEncode tknStatus,
"verifyKey=" <> strEncode tknVerifyKey,
"dhKeys=" <> strEncode tknDhKeys,
"dhSecret=" <> strEncode tknDhSecret,
"regCode=" <> strEncode tknRegCode,
"cron=" <> strEncode tknCronInterval
]
strP = do
ntfTknId <- "tknId=" *> strP_
token <- "token=" *> strP_
tknStatus <- "tokenStatus=" *> strP_
tknVerifyKey <- "verifyKey=" *> strP_
tknDhKeys <- "dhKeys=" *> strP_
tknDhSecret <- "dhSecret=" *> strP_
tknRegCode <- "regCode=" *> strP_
tknCronInterval <- "cron=" *> strP
pure NtfTknRec {ntfTknId, token, tknStatus, tknVerifyKey, tknDhKeys, tknDhSecret, tknRegCode, tknCronInterval}
instance StrEncoding NtfSubRec where
strEncode NtfSubRec {ntfSubId, smpQueue, notifierKey, tokenId, subStatus} =
B.unwords
[ "subId=" <> strEncode ntfSubId,
"smpQueue=" <> strEncode smpQueue,
"notifierKey=" <> strEncode notifierKey,
"tknId=" <> strEncode tokenId,
"subStatus=" <> strEncode subStatus
]
strP = do
ntfSubId <- "subId=" *> strP_
smpQueue <- "smpQueue=" *> strP_
notifierKey <- "notifierKey=" *> strP_
tokenId <- "tknId=" *> strP_
subStatus <- "subStatus=" *> strP
pure NtfSubRec {ntfSubId, smpQueue, notifierKey, tokenId, subStatus}
logNtfStoreRecord :: StoreLog 'WriteMode -> NtfStoreLogRecord -> IO ()
logNtfStoreRecord = writeStoreLogRecord
logCreateToken :: StoreLog 'WriteMode -> NtfTknData -> IO ()
logCreateToken s tkn = logNtfStoreRecord s . CreateToken =<< atomically (mkTknRec tkn)
logTokenStatus :: StoreLog 'WriteMode -> NtfTokenId -> NtfTknStatus -> IO ()
logTokenStatus s tknId tknStatus = logNtfStoreRecord s $ TokenStatus tknId tknStatus
logUpdateToken :: StoreLog 'WriteMode -> NtfTokenId -> DeviceToken -> NtfRegCode -> IO ()
logUpdateToken s tknId token regCode = logNtfStoreRecord s $ UpdateToken tknId token regCode
logTokenCron :: StoreLog 'WriteMode -> NtfTokenId -> Word16 -> IO ()
logTokenCron s tknId cronInt = logNtfStoreRecord s $ TokenCron tknId cronInt
logDeleteToken :: StoreLog 'WriteMode -> NtfTokenId -> IO ()
logDeleteToken s tknId = logNtfStoreRecord s $ DeleteToken tknId
logCreateSubscription :: StoreLog 'WriteMode -> NtfSubData -> IO ()
logCreateSubscription s sub = logNtfStoreRecord s . CreateSubscription =<< atomically (mkSubRec sub)
logSubscriptionStatus :: StoreLog 'WriteMode -> NtfSubscriptionId -> NtfSubStatus -> IO ()
logSubscriptionStatus s subId subStatus = logNtfStoreRecord s $ SubscriptionStatus subId subStatus
logDeleteSubscription :: StoreLog 'WriteMode -> NtfSubscriptionId -> IO ()
logDeleteSubscription s subId = logNtfStoreRecord s $ DeleteSubscription subId
readWriteNtfStore :: FilePath -> NtfStore -> IO (StoreLog 'WriteMode)
readWriteNtfStore f st = do
whenM (doesFileExist f) $ do
readNtfStore f st
renameFile f $ f <> ".bak"
s <- openWriteStoreLog f
writeNtfStore s st
pure s
readNtfStore :: FilePath -> NtfStore -> IO ()
readNtfStore f st = mapM_ addNtfLogRecord . B.lines =<< B.readFile f
where
addNtfLogRecord s = case strDecode s of
Left e -> B.putStrLn $ "Log parsing error (" <> B.pack e <> "): " <> B.take 100 s
Right lr -> atomically $ case lr of
CreateToken r@NtfTknRec {ntfTknId} -> do
tkn <- mkTknData r
addNtfToken st ntfTknId tkn
TokenStatus tknId status ->
getNtfToken st tknId
>>= mapM_ (\NtfTknData {tknStatus} -> writeTVar tknStatus status)
UpdateToken tknId token' tknRegCode ->
getNtfToken st tknId
>>= mapM_
( \tkn@NtfTknData {tknStatus} -> do
removeTokenRegistration st tkn
writeTVar tknStatus NTRegistered
addNtfToken st tknId tkn {token = token', tknRegCode}
)
TokenCron tknId cronInt ->
getNtfToken st tknId
>>= mapM_ (\NtfTknData {tknCronInterval} -> writeTVar tknCronInterval cronInt)
DeleteToken tknId ->
void $ deleteNtfToken st tknId
CreateSubscription r@NtfSubRec {ntfSubId} -> do
sub <- mkSubData r
void $ addNtfSubscription st ntfSubId sub
SubscriptionStatus subId status ->
getNtfSubscription st subId
>>= mapM_ (\NtfSubData {subStatus} -> writeTVar subStatus status)
DeleteSubscription subId ->
deleteNtfSubscription st subId
writeNtfStore :: StoreLog 'WriteMode -> NtfStore -> IO ()
writeNtfStore s NtfStore {tokens, subscriptions} = do
atomically (readTVar tokens >>= mapM mkTknRec)
>>= mapM_ (writeStoreLogRecord s . CreateToken)
atomically (readTVar subscriptions >>= mapM mkSubRec)
>>= mapM_ (writeStoreLogRecord s . CreateSubscription)
@@ -1,13 +1,7 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Simplex.Messaging.Notifications.Transport where
import Control.Monad.Except
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Transport
import Simplex.Messaging.Version
@@ -17,56 +11,13 @@ ntfBlockSize = 512
supportedNTFServerVRange :: VersionRange
supportedNTFServerVRange = mkVersionRange 1 1
data NtfServerHandshake = NtfServerHandshake
{ ntfVersionRange :: VersionRange,
sessionId :: SessionId
}
data NtfClientHandshake = NtfClientHandshake
{ -- | agreed SMP notifications server protocol version
ntfVersion :: Version,
-- | server identity - CA certificate fingerprint
keyHash :: C.KeyHash
}
instance Encoding NtfServerHandshake where
smpEncode NtfServerHandshake {ntfVersionRange, sessionId} =
smpEncode (ntfVersionRange, sessionId)
smpP = do
(ntfVersionRange, sessionId) <- smpP
pure NtfServerHandshake {ntfVersionRange, sessionId}
instance Encoding NtfClientHandshake where
smpEncode NtfClientHandshake {ntfVersion, keyHash} = smpEncode (ntfVersion, keyHash)
smpP = do
(ntfVersion, keyHash) <- smpP
pure NtfClientHandshake {ntfVersion, keyHash}
-- | Notifcations server transport handshake.
ntfServerHandshake :: forall c. Transport c => c -> C.KeyHash -> VersionRange -> ExceptT TransportError IO (THandle c)
ntfServerHandshake c kh ntfVRange = do
let th@THandle {sessionId} = ntfTHandle c
sendHandshake th $ NtfServerHandshake {sessionId, ntfVersionRange = ntfVRange}
getHandshake th >>= \case
NtfClientHandshake {ntfVersion, keyHash}
| keyHash /= kh ->
throwError $ TEHandshake IDENTITY
| ntfVersion `isCompatible` ntfVRange -> do
pure (th :: THandle c) {thVersion = ntfVersion}
| otherwise -> throwError $ TEHandshake VERSION
ntfServerHandshake :: Transport c => c -> C.KeyHash -> VersionRange -> ExceptT TransportError IO (THandle c)
ntfServerHandshake c _ _ = pure $ ntfTHandle c
-- | Notifcations server client transport handshake.
ntfClientHandshake :: forall c. Transport c => c -> C.KeyHash -> VersionRange -> ExceptT TransportError IO (THandle c)
ntfClientHandshake c keyHash ntfVRange = do
let th@THandle {sessionId} = ntfTHandle c
NtfServerHandshake {sessionId = sessId, ntfVersionRange} <- getHandshake th
if sessionId /= sessId
then throwError TEBadSession
else case ntfVersionRange `compatibleVersion` ntfVRange of
Just (Compatible ntfVersion) -> do
sendHandshake th $ NtfClientHandshake {ntfVersion, keyHash}
pure (th :: THandle c) {thVersion = ntfVersion}
Nothing -> throwError $ TEHandshake VERSION
ntfClientHandshake :: Transport c => c -> C.KeyHash -> VersionRange -> ExceptT TransportError IO (THandle c)
ntfClientHandshake c _ _ = pure $ ntfTHandle c
ntfTHandle :: Transport c => c -> THandle c
ntfTHandle c = THandle {connection = c, sessionId = tlsUnique c, blockSize = ntfBlockSize, thVersion = 0, batch = False}
ntfTHandle c = THandle {connection = c, sessionId = tlsUnique c, blockSize = ntfBlockSize, thVersion = 0}
@@ -1,194 +0,0 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
module Simplex.Messaging.Notifications.Types where
import qualified Data.Attoparsec.ByteString.Char8 as A
import Data.Text.Encoding (decodeLatin1, encodeUtf8)
import Data.Time (UTCTime)
import Database.SQLite.Simple.FromField (FromField (..))
import Database.SQLite.Simple.ToField (ToField (..))
import Simplex.Messaging.Agent.Protocol (ConnId, NotificationsMode (..))
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Parsers (blobFieldDecoder, fromTextField_)
import Simplex.Messaging.Protocol (NotifierId, NtfServer, SMPServer)
data NtfTknAction
= NTARegister
| NTAVerify NtfRegCode -- code to verify token
| NTACheck
| NTADelete
deriving (Show)
instance Encoding NtfTknAction where
smpEncode = \case
NTARegister -> "R"
NTAVerify code -> smpEncode ('V', code)
NTACheck -> "C"
NTADelete -> "D"
smpP =
A.anyChar >>= \case
'R' -> pure NTARegister
'V' -> NTAVerify <$> smpP
'C' -> pure NTACheck
'D' -> pure NTADelete
_ -> fail "bad NtfTknAction"
instance FromField NtfTknAction where fromField = blobFieldDecoder smpDecode
instance ToField NtfTknAction where toField = toField . smpEncode
data NtfToken = NtfToken
{ deviceToken :: DeviceToken,
ntfServer :: NtfServer,
ntfTokenId :: Maybe NtfTokenId,
-- | key used by the ntf server to verify transmissions
ntfPubKey :: C.APublicVerifyKey,
-- | key used by the ntf client to sign transmissions
ntfPrivKey :: C.APrivateSignKey,
-- | client's DH keys (to repeat registration if necessary)
ntfDhKeys :: C.KeyPair 'C.X25519,
-- | shared DH secret used to encrypt/decrypt notifications e2e
ntfDhSecret :: Maybe C.DhSecretX25519,
-- | token status
ntfTknStatus :: NtfTknStatus,
-- | pending token action and the earliest time
ntfTknAction :: Maybe NtfTknAction,
ntfMode :: NotificationsMode
}
deriving (Show)
newNtfToken :: DeviceToken -> NtfServer -> C.ASignatureKeyPair -> C.KeyPair 'C.X25519 -> NotificationsMode -> NtfToken
newNtfToken deviceToken ntfServer (ntfPubKey, ntfPrivKey) ntfDhKeys ntfMode =
NtfToken
{ deviceToken,
ntfServer,
ntfTokenId = Nothing,
ntfPubKey,
ntfPrivKey,
ntfDhKeys,
ntfDhSecret = Nothing,
ntfTknStatus = NTNew,
ntfTknAction = Just NTARegister,
ntfMode
}
data NtfSubAction = NtfSubNTFAction NtfSubNTFAction | NtfSubSMPAction NtfSubSMPAction
deriving (Show)
isDeleteNtfSubAction :: NtfSubAction -> Bool
isDeleteNtfSubAction = \case
NtfSubNTFAction a -> case a of
NSACreate -> False
NSACheck -> False
NSADelete -> True
NtfSubSMPAction a -> case a of
NSASmpKey -> False
NSASmpDelete -> True
type NtfActionTs = UTCTime
data NtfSubNTFAction
= NSACreate
| NSACheck
| NSADelete
deriving (Show)
instance Encoding NtfSubNTFAction where
smpEncode = \case
NSACreate -> "N"
NSACheck -> "C"
NSADelete -> "D"
smpP =
A.anyChar >>= \case
'N' -> pure NSACreate
'C' -> pure NSACheck
'D' -> pure NSADelete
_ -> fail "bad NtfSubNTFAction"
instance FromField NtfSubNTFAction where fromField = blobFieldDecoder smpDecode
instance ToField NtfSubNTFAction where toField = toField . smpEncode
data NtfSubSMPAction
= NSASmpKey
| NSASmpDelete
deriving (Show)
instance Encoding NtfSubSMPAction where
smpEncode = \case
NSASmpKey -> "K"
NSASmpDelete -> "D"
smpP =
A.anyChar >>= \case
'K' -> pure NSASmpKey
'D' -> pure NSASmpDelete
_ -> fail "bad NtfSubSMPAction"
instance FromField NtfSubSMPAction where fromField = blobFieldDecoder smpDecode
instance ToField NtfSubSMPAction where toField = toField . smpEncode
data NtfAgentSubStatus
= -- | subscription started
NASNew
| -- | state after NKEY - notifier ID is assigned to queue on SMP server
NASKey
| -- | state after SNEW - subscription created on notification server
NASCreated NtfSubStatus
| -- | state after SDEL (subscription is deleted on notification server)
NASOff
| -- | state after NDEL (notifier credentials are deleted on SMP server)
-- Can only exist transiently - if subscription record was updated by notification supervisor mid worker operation,
-- and hence got updated instead of being fully deleted in the database post operation by worker
NASDeleted
deriving (Eq, Show)
instance Encoding NtfAgentSubStatus where
smpEncode = \case
NASNew -> "NEW"
NASKey -> "KEY"
NASCreated status -> "CREATED " <> smpEncode status
NASOff -> "OFF"
NASDeleted -> "DELETED"
smpP =
A.takeTill (== ' ') >>= \case
"NEW" -> pure NASNew
"KEY" -> pure NASKey
"CREATED" -> do
_ <- A.space
NASCreated <$> smpP
"OFF" -> pure NASOff
"DELETED" -> pure NASDeleted
_ -> fail "bad NtfAgentSubStatus"
instance FromField NtfAgentSubStatus where fromField = fromTextField_ $ either (const Nothing) Just . smpDecode . encodeUtf8
instance ToField NtfAgentSubStatus where toField = toField . decodeLatin1 . smpEncode
data NtfSubscription = NtfSubscription
{ connId :: ConnId,
smpServer :: SMPServer,
ntfQueueId :: Maybe NotifierId,
ntfServer :: NtfServer,
ntfSubId :: Maybe NtfSubscriptionId,
ntfSubStatus :: NtfAgentSubStatus
}
deriving (Show)
newNtfSubscription :: ConnId -> SMPServer -> Maybe NotifierId -> NtfServer -> NtfAgentSubStatus -> NtfSubscription
newNtfSubscription connId smpServer ntfQueueId ntfServer ntfSubStatus =
NtfSubscription
{ connId,
smpServer,
ntfQueueId,
ntfServer,
ntfSubId = Nothing,
ntfSubStatus
}
+55 -348
View File
@@ -7,7 +7,6 @@
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE PolyKinds #-}
@@ -17,13 +16,9 @@
{-# LANGUAGE TupleSections #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE TypeFamilyDependencies #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE UndecidableInstances #-}
{-# OPTIONS_GHC -Wno-unrecognised-pragmas #-}
{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
{-# HLINT ignore "Use newtype instead of data" #-}
-- |
-- Module : Simplex.Messaging.ProtocolEncoding
-- Copyright : (c) simplex.chat
@@ -38,7 +33,8 @@
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
module Simplex.Messaging.Protocol
( -- * SMP protocol parameters
supportedSMPClientVRange,
smpClientVersion,
smpClientVRange,
maxMessageLength,
e2eEncConfirmationLength,
e2eEncMessageLength,
@@ -63,15 +59,9 @@ module Simplex.Messaging.Protocol
ClientMessage (..),
PrivHeader (..),
Protocol (..),
ProtocolType (..),
AProtocolType (..),
ProtocolTypeI (..),
ProtocolServer (..),
ProtoServer,
SMPServer,
pattern SMPServer,
NtfServer,
pattern NtfServer,
SrvLoc (..),
CorrId (..),
QueueId,
@@ -86,22 +76,9 @@ module Simplex.Messaging.Protocol
SndPublicVerifyKey,
NtfPrivateSignKey,
NtfPublicVerifyKey,
RcvNtfPublicDhKey,
RcvNtfDhSecret,
Message (..),
RcvMessage (..),
MsgId,
MsgBody,
MaxMessageLen,
MaxRcvMessageLen,
EncRcvMsgBody (..),
RcvMsgBody (..),
ClientRcvMsgBody (..),
EncNMsgMeta,
SMPMsgMeta (..),
NMsgMeta (..),
MsgFlags (..),
rcvMessageMeta,
noMsgFlags,
-- * Parse and serialize
@@ -110,11 +87,6 @@ module Simplex.Messaging.Protocol
encodeTransmission,
transmissionP,
_smpP,
encodeRcvMsgBody,
clientRcvMsgBodyP,
legacyEncodeServer,
legacyServerP,
legacyStrEncodeServer,
-- * TCP transport functions
tPut,
@@ -135,14 +107,11 @@ import qualified Data.Attoparsec.ByteString.Char8 as A
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Kind
import Data.List.NonEmpty (NonEmpty (..))
import qualified Data.List.NonEmpty as L
import Data.Maybe (isJust, isNothing)
import Data.Maybe (isNothing)
import Data.String
import Data.Time.Clock.System (SystemTime (..))
import Data.Time.Clock.System (SystemTime)
import Data.Type.Equality
import GHC.Generics (Generic)
import GHC.TypeLits (type (+))
import Generic.Random (genericArbitraryU)
import Network.Socket (HostName, ServiceName)
import qualified Simplex.Messaging.Crypto as C
@@ -150,25 +119,19 @@ import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.Client (TransportHost, TransportHosts (..))
import Simplex.Messaging.Util (bshow, (<$?>))
import Simplex.Messaging.Version
import Test.QuickCheck (Arbitrary (..))
currentSMPClientVersion :: Version
currentSMPClientVersion = 2
smpClientVersion :: Version
smpClientVersion = 1
supportedSMPClientVRange :: VersionRange
supportedSMPClientVRange = mkVersionRange 1 currentSMPClientVersion
smpClientVRange :: VersionRange
smpClientVRange = mkVersionRange 1 smpClientVersion
maxMessageLength :: Int
maxMessageLength = 16088
type MaxMessageLen = 16088
-- 16 extra bytes: 8 for timestamp and 8 for flags (7 flags and the space, only 1 flag is currently used)
type MaxRcvMessageLen = MaxMessageLen + 16 -- 16104, the padded size is 16106
-- it is shorter to allow per-queue e2e encryption DH key in the "public" header
e2eEncConfirmationLength :: Int
e2eEncConfirmationLength = 15936
@@ -252,8 +215,7 @@ data Command (p :: Party) where
NEW :: RcvPublicVerifyKey -> RcvPublicDhKey -> Command Recipient
SUB :: Command Recipient
KEY :: SndPublicVerifyKey -> Command Recipient
NKEY :: NtfPublicVerifyKey -> RcvNtfPublicDhKey -> Command Recipient
NDEL :: Command Recipient
NKEY :: NtfPublicVerifyKey -> Command Recipient
GET :: Command Recipient
-- ACK v1 has to be supported for encoding/decoding
-- ACK :: Command Recipient
@@ -275,124 +237,20 @@ deriving instance Eq (Command p)
data BrokerMsg where
-- SMP broker messages (responses, client messages, notifications)
IDS :: QueueIdsKeys -> BrokerMsg
-- MSG v1/2 has to be supported for encoding/decoding
-- v1: MSG :: MsgId -> SystemTime -> MsgBody -> BrokerMsg
-- v2: MsgId -> SystemTime -> MsgFlags -> MsgBody -> BrokerMsg
MSG :: RcvMessage -> BrokerMsg
NID :: NotifierId -> RcvNtfPublicDhKey -> BrokerMsg
NMSG :: C.CbNonce -> EncNMsgMeta -> BrokerMsg
-- MSG v1 has to be supported for encoding/decoding
-- MSG :: MsgId -> SystemTime -> MsgBody -> BrokerMsg
MSG :: MsgId -> SystemTime -> MsgFlags -> MsgBody -> BrokerMsg
NID :: NotifierId -> BrokerMsg
NMSG :: BrokerMsg
END :: BrokerMsg
OK :: BrokerMsg
ERR :: ErrorType -> BrokerMsg
PONG :: BrokerMsg
deriving (Eq, Show)
data RcvMessage = RcvMessage
{ msgId :: MsgId,
msgTs :: SystemTime,
msgFlags :: MsgFlags,
msgBody :: EncRcvMsgBody -- e2e encrypted, with extra encryption for recipient
}
newtype MsgFlags = MsgFlags {notification :: Bool}
deriving (Eq, Show)
-- | received message without server/recipient encryption
data Message = Message
{ msgId :: MsgId,
msgTs :: SystemTime,
msgFlags :: MsgFlags,
msgBody :: C.MaxLenBS MaxMessageLen
}
instance StrEncoding RcvMessage where
strEncode RcvMessage {msgId, msgTs, msgFlags, msgBody = EncRcvMsgBody body} =
B.unwords
[ strEncode msgId,
strEncode msgTs,
"flags=" <> strEncode msgFlags,
strEncode body
]
strP = do
msgId <- strP_
msgTs <- strP_
msgFlags <- ("flags=" *> strP_) <|> pure noMsgFlags
msgBody <- EncRcvMsgBody <$> strP
pure RcvMessage {msgId, msgTs, msgFlags, msgBody}
newtype EncRcvMsgBody = EncRcvMsgBody ByteString
deriving (Eq, Show)
data RcvMsgBody = RcvMsgBody
{ msgTs :: SystemTime,
msgFlags :: MsgFlags,
msgBody :: C.MaxLenBS MaxMessageLen
}
encodeRcvMsgBody :: RcvMsgBody -> C.MaxLenBS MaxRcvMessageLen
encodeRcvMsgBody RcvMsgBody {msgTs, msgFlags, msgBody} =
let rcvMeta :: C.MaxLenBS 16 = C.unsafeMaxLenBS $ smpEncode (msgTs, msgFlags, ' ')
in C.appendMaxLenBS rcvMeta msgBody
data ClientRcvMsgBody = ClientRcvMsgBody
{ msgTs :: SystemTime,
msgFlags :: MsgFlags,
msgBody :: ByteString
}
clientRcvMsgBodyP :: Parser ClientRcvMsgBody
clientRcvMsgBodyP = do
msgTs <- smpP
msgFlags <- smpP
Tail msgBody <- _smpP
pure ClientRcvMsgBody {msgTs, msgFlags, msgBody}
instance StrEncoding Message where
strEncode Message {msgId, msgTs, msgFlags, msgBody} =
B.unwords
[ strEncode msgId,
strEncode msgTs,
"flags=" <> strEncode msgFlags,
strEncode msgBody
]
strP = do
msgId <- strP_
msgTs <- strP_
msgFlags <- ("flags=" *> strP_) <|> pure noMsgFlags
msgBody <- strP
pure Message {msgId, msgTs, msgFlags, msgBody}
type EncNMsgMeta = ByteString
data SMPMsgMeta = SMPMsgMeta
{ msgId :: MsgId,
msgTs :: SystemTime,
msgFlags :: MsgFlags
}
deriving (Show)
rcvMessageMeta :: MsgId -> ClientRcvMsgBody -> SMPMsgMeta
rcvMessageMeta msgId ClientRcvMsgBody {msgTs, msgFlags} = SMPMsgMeta {msgId, msgTs, msgFlags}
data NMsgMeta = NMsgMeta
{ msgId :: MsgId,
msgTs :: SystemTime
}
deriving (Show)
instance Encoding NMsgMeta where
smpEncode NMsgMeta {msgId, msgTs} =
smpEncode (msgId, msgTs)
smpP = do
-- Tail here is to allow extension in the future clients/servers
(msgId, msgTs, Tail _) <- smpP
pure NMsgMeta {msgId, msgTs}
-- it must be data for correct JSON encoding
data MsgFlags = MsgFlags {notification :: Bool}
deriving (Eq, Show, Generic)
instance ToJSON MsgFlags where toEncoding = J.genericToEncoding J.defaultOptions
-- this encoding should not become bigger than 7 bytes (currently it is 1 byte)
instance Encoding MsgFlags where
smpEncode MsgFlags {notification} = smpEncode notification
smpP = do
@@ -415,7 +273,6 @@ data CommandTag (p :: Party) where
SUB_ :: CommandTag Recipient
KEY_ :: CommandTag Recipient
NKEY_ :: CommandTag Recipient
NDEL_ :: CommandTag Recipient
GET_ :: CommandTag Recipient
ACK_ :: CommandTag Recipient
OFF_ :: CommandTag Recipient
@@ -455,7 +312,6 @@ instance PartyI p => Encoding (CommandTag p) where
SUB_ -> "SUB"
KEY_ -> "KEY"
NKEY_ -> "NKEY"
NDEL_ -> "NDEL"
GET_ -> "GET"
ACK_ -> "ACK"
OFF_ -> "OFF"
@@ -471,7 +327,6 @@ instance ProtocolMsgTag CmdTag where
"SUB" -> Just $ CT SRecipient SUB_
"KEY" -> Just $ CT SRecipient KEY_
"NKEY" -> Just $ CT SRecipient NKEY_
"NDEL" -> Just $ CT SRecipient NDEL_
"GET" -> Just $ CT SRecipient GET_
"ACK" -> Just $ CT SRecipient ACK_
"OFF" -> Just $ CT SRecipient OFF_
@@ -558,143 +413,44 @@ instance Encoding ClientMessage where
smpEncode (ClientMessage h msg) = smpEncode h <> msg
smpP = ClientMessage <$> smpP <*> A.takeByteString
type SMPServer = ProtocolServer 'PSMP
type SMPServer = ProtocolServer
pattern SMPServer :: NonEmpty TransportHost -> ServiceName -> C.KeyHash -> ProtocolServer 'PSMP
pattern SMPServer host port keyHash = ProtocolServer SPSMP host port keyHash
pattern SMPServer :: HostName -> ServiceName -> C.KeyHash -> ProtocolServer
pattern SMPServer host port keyHash = ProtocolServer host port keyHash
{-# COMPLETE SMPServer #-}
type NtfServer = ProtocolServer 'PNTF
pattern NtfServer :: NonEmpty TransportHost -> ServiceName -> C.KeyHash -> ProtocolServer 'PNTF
pattern NtfServer host port keyHash = ProtocolServer SPNTF host port keyHash
{-# COMPLETE NtfServer #-}
data ProtocolType = PSMP | PNTF
deriving (Eq, Ord, Show)
instance StrEncoding ProtocolType where
strEncode = \case
PSMP -> "smp"
PNTF -> "ntf"
strP =
A.takeTill (\c -> c == ':' || c == ' ') >>= \case
"smp" -> pure PSMP
"ntf" -> pure PNTF
_ -> fail "bad ProtocolType"
data SProtocolType (p :: ProtocolType) where
SPSMP :: SProtocolType 'PSMP
SPNTF :: SProtocolType 'PNTF
deriving instance Eq (SProtocolType p)
deriving instance Ord (SProtocolType p)
deriving instance Show (SProtocolType p)
data AProtocolType = forall p. ProtocolTypeI p => AProtocolType (SProtocolType p)
deriving instance Show AProtocolType
instance Eq AProtocolType where
AProtocolType p == AProtocolType p' = isJust $ testEquality p p'
instance TestEquality SProtocolType where
testEquality SPSMP SPSMP = Just Refl
testEquality SPNTF SPNTF = Just Refl
testEquality _ _ = Nothing
protocolType :: SProtocolType p -> ProtocolType
protocolType = \case
SPSMP -> PSMP
SPNTF -> PNTF
aProtocolType :: ProtocolType -> AProtocolType
aProtocolType = \case
PSMP -> AProtocolType SPSMP
PNTF -> AProtocolType SPNTF
instance ProtocolTypeI p => StrEncoding (SProtocolType p) where
strEncode = strEncode . protocolType
strP = (\(AProtocolType p) -> checkProtocolType p) <$?> strP
instance StrEncoding AProtocolType where
strEncode (AProtocolType p) = strEncode p
strP = aProtocolType <$> strP
instance ToJSON AProtocolType where
toEncoding = strToJEncoding
toJSON = strToJSON
checkProtocolType :: forall t p p'. (ProtocolTypeI p, ProtocolTypeI p') => t p' -> Either String (t p)
checkProtocolType p = case testEquality (protocolTypeI @p) (protocolTypeI @p') of
Just Refl -> Right p
Nothing -> Left "bad ProtocolType"
class ProtocolTypeI (p :: ProtocolType) where
protocolTypeI :: SProtocolType p
instance ProtocolTypeI 'PSMP where protocolTypeI = SPSMP
instance ProtocolTypeI 'PNTF where protocolTypeI = SPNTF
-- | server location and transport key digest (hash).
data ProtocolServer p = ProtocolServer
{ scheme :: SProtocolType p,
host :: NonEmpty TransportHost,
-- | SMP server location and transport key digest (hash).
data ProtocolServer = ProtocolServer
{ host :: HostName,
port :: ServiceName,
keyHash :: C.KeyHash
}
deriving (Eq, Ord, Show)
instance ProtocolTypeI p => IsString (ProtocolServer p) where
instance IsString ProtocolServer where
fromString = parseString strDecode
instance ProtocolTypeI p => Encoding (ProtocolServer p) where
instance Encoding ProtocolServer where
smpEncode ProtocolServer {host, port, keyHash} =
smpEncode (host, port, keyHash)
smpP = do
(host, port, keyHash) <- smpP
pure ProtocolServer {scheme = protocolTypeI @p, host, port, keyHash}
pure ProtocolServer {host, port, keyHash}
instance ProtocolTypeI p => StrEncoding (ProtocolServer p) where
strEncode ProtocolServer {scheme, host, port, keyHash} =
strEncodeServer scheme (strEncode host) port keyHash
instance StrEncoding ProtocolServer where
strEncode ProtocolServer {host, port, keyHash} =
"smp://" <> strEncode keyHash <> "@" <> strEncode (SrvLoc host port)
strP = do
scheme <- strP <* "://"
_ <- "smp://"
keyHash <- strP <* A.char '@'
TransportHosts host <- strP
port <- portP <|> pure ""
pure ProtocolServer {scheme, host, port, keyHash}
where
portP = show <$> (A.char ':' *> (A.decimal :: Parser Int))
SrvLoc host port <- strP
pure ProtocolServer {host, port, keyHash}
instance ProtocolTypeI p => ToJSON (ProtocolServer p) where
instance ToJSON ProtocolServer where
toJSON = strToJSON
toEncoding = strToJEncoding
legacyEncodeServer :: ProtocolServer p -> ByteString
legacyEncodeServer ProtocolServer {host, port, keyHash} =
smpEncode (L.head host, port, keyHash)
legacyServerP :: forall p. ProtocolTypeI p => Parser (ProtocolServer p)
legacyServerP = do
(h, port, keyHash) <- smpP
pure ProtocolServer {scheme = protocolTypeI @p, host = [h], port, keyHash}
legacyStrEncodeServer :: ProtocolTypeI p => ProtocolServer p -> ByteString
legacyStrEncodeServer ProtocolServer {scheme, host, port, keyHash} =
strEncodeServer scheme (strEncode $ L.head host) port keyHash
strEncodeServer :: ProtocolTypeI p => SProtocolType p -> ByteString -> ServiceName -> C.KeyHash -> ByteString
strEncodeServer scheme host port keyHash =
strEncode scheme <> "://" <> strEncode keyHash <> "@" <> host <> portStr
where
portStr = B.pack $ if null port then "" else ':' : port
data SrvLoc = SrvLoc HostName ServiceName
deriving (Eq, Ord, Show)
@@ -703,7 +459,7 @@ instance StrEncoding SrvLoc where
strP = SrvLoc <$> host <*> (port <|> pure "")
where
host = B.unpack <$> A.takeWhile1 (A.notInClass ":#,;/ ")
port = show <$> (A.char ':' *> (A.decimal :: Parser Int))
port = B.unpack <$> (A.char ':' *> A.takeWhile1 A.isDigit)
-- | Transmission correlation ID.
newtype CorrId = CorrId {bs :: ByteString} deriving (Eq, Ord, Show)
@@ -756,12 +512,6 @@ type NtfPrivateSignKey = C.APrivateSignKey
-- | Public key used by SMP server to verify authorization of NSUB command sent by push notifications server.
type NtfPublicVerifyKey = C.APublicVerifyKey
-- | Public key used for DH exchange to encrypt notification metadata from server to recipient
type RcvNtfPublicDhKey = C.PublicKeyX25519
-- | DH Secret used to encrypt notification metadata from server to recipient
type RcvNtfDhSecret = C.DhSecretX25519
-- | SMP message server ID.
type MsgId = ByteString
@@ -782,7 +532,7 @@ data ErrorType
QUOTA
| -- | ACK command is sent without message to be acknowledged
NO_MSG
| -- | sent message is too large (> maxMessageLength = 16088 bytes)
| -- | sent message is too large (> maxMessageLength = 16078 bytes)
LARGE_MSG
| -- | internal server error
INTERNAL
@@ -838,18 +588,14 @@ transmissionP = do
command <- A.takeByteString
pure RawTransmission {signature, signed, sessId, corrId, entityId, command}
class (ProtocolEncoding msg, ProtocolEncoding (ProtoCommand msg), Show msg) => Protocol msg where
type ProtoCommand msg = cmd | cmd -> msg
type ProtoType msg = (sch :: ProtocolType) | sch -> msg
class (ProtocolEncoding msg, ProtocolEncoding (ProtocolCommand msg)) => Protocol msg where
type ProtocolCommand msg = cmd | cmd -> msg
protocolClientHandshake :: forall c. Transport c => c -> C.KeyHash -> VersionRange -> ExceptT TransportError IO (THandle c)
protocolPing :: ProtoCommand msg
protocolPing :: ProtocolCommand msg
protocolError :: msg -> Maybe ErrorType
type ProtoServer msg = ProtocolServer (ProtoType msg)
instance Protocol BrokerMsg where
type ProtoCommand BrokerMsg = Cmd
type ProtoType BrokerMsg = 'PSMP
type ProtocolCommand BrokerMsg = Cmd
protocolClientHandshake = smpClientHandshake
protocolPing = Cmd SSender PING
protocolError = \case
@@ -868,8 +614,7 @@ instance PartyI p => ProtocolEncoding (Command p) where
NEW rKey dhKey -> e (NEW_, ' ', rKey, dhKey)
SUB -> e SUB_
KEY k -> e (KEY_, ' ', k)
NKEY k dhKey -> e (NKEY_, ' ', k, dhKey)
NDEL -> e NDEL_
NKEY k -> e (NKEY_, ' ', k)
GET -> e GET_
ACK msgId
| v == 1 -> e ACK_
@@ -916,8 +661,7 @@ instance ProtocolEncoding Cmd where
NEW_ -> NEW <$> _smpP <*> smpP
SUB_ -> pure SUB
KEY_ -> KEY <$> _smpP
NKEY_ -> NKEY <$> _smpP <*> smpP
NDEL_ -> pure NDEL
NKEY_ -> NKEY <$> _smpP
GET_ -> pure GET
ACK_
| v == 1 -> pure $ ACK ""
@@ -938,12 +682,11 @@ instance ProtocolEncoding BrokerMsg where
type Tag BrokerMsg = BrokerMsgTag
encodeProtocol v = \case
IDS (QIK rcvId sndId srvDh) -> e (IDS_, ' ', rcvId, sndId, srvDh)
MSG RcvMessage {msgId, msgTs, msgFlags, msgBody = EncRcvMsgBody body}
| v == 1 -> e (MSG_, ' ', msgId, msgTs, Tail body)
| v == 2 -> e (MSG_, ' ', msgId, msgTs, msgFlags, ' ', Tail body)
| otherwise -> e (MSG_, ' ', msgId, Tail body)
NID nId srvNtfDh -> e (NID_, ' ', nId, srvNtfDh)
NMSG nmsgNonce encNMsgMeta -> e (NMSG_, ' ', nmsgNonce, encNMsgMeta)
MSG msgId ts flags msgBody
| v == 1 -> e (MSG_, ' ', msgId, ts, Tail msgBody)
| otherwise -> e (MSG_, ' ', msgId, ts, flags, ' ', Tail msgBody)
NID nId -> e (NID_, ' ', nId)
NMSG -> e NMSG_
END -> e END_
OK -> e OK_
ERR err -> e (ERR_, ' ', err)
@@ -953,17 +696,12 @@ instance ProtocolEncoding BrokerMsg where
e = smpEncode
protocolP v = \case
MSG_ -> do
msgId <- _smpP
MSG <$> case v of
1 -> RcvMessage msgId <$> smpP <*> pure noMsgFlags <*> bodyP
2 -> RcvMessage msgId <$> smpP <*> smpP <*> (A.space *> bodyP)
_ -> RcvMessage msgId (MkSystemTime 0 0) noMsgFlags <$> bodyP
where
bodyP = EncRcvMsgBody . unTail <$> smpP
MSG_
| v == 1 -> MSG <$> _smpP <*> smpP <*> pure noMsgFlags <*> (unTail <$> smpP)
| otherwise -> MSG <$> _smpP <*> smpP <*> smpP <*> (unTail <$> _smpP)
IDS_ -> IDS <$> (QIK <$> _smpP <*> smpP <*> smpP)
NID_ -> NID <$> _smpP <*> smpP
NMSG_ -> NMSG <$> _smpP <*> smpP
NID_ -> NID <$> _smpP
NMSG_ -> pure NMSG
END_ -> pure END
OK_ -> pure OK
ERR_ -> ERR <$> _smpP
@@ -1049,51 +787,20 @@ instance Encoding CommandError where
_ -> fail "bad command error type"
-- | Send signed SMP transmission to TCP transport.
tPut :: Transport c => THandle c -> NonEmpty (SentRawTransmission) -> IO (NonEmpty (Either TransportError ()))
tPut th trs
| batch th = tPutBatch [] $ L.map tEncode trs
| otherwise = forM trs $ tPutBlock th . tEncode
where
tPutBatch :: [Either TransportError ()] -> NonEmpty ByteString -> IO (NonEmpty (Either TransportError ()))
tPutBatch rs ts = do
let (n, s, ts_) = encodeBatch 0 "" ts
r <- if n == 0 then pure [Left TELargeMsg] else replicate n <$> tPutBlock th (lenEncode n `B.cons` s)
let rs' = rs <> r
case ts_ of
Just ts' -> tPutBatch rs' ts'
_ -> pure $ L.fromList rs'
encodeBatch :: Int -> ByteString -> NonEmpty ByteString -> (Int, ByteString, Maybe (NonEmpty ByteString))
encodeBatch n s ts@(t :| ts_)
| n == 255 = (n, s, Just ts)
| otherwise =
let s' = s <> smpEncode (Large t)
n' = n + 1
in if B.length s' > blockSize th - 1
then (n,s,) $ if n == 0 then L.nonEmpty ts_ else Just ts
else case L.nonEmpty ts_ of
Just ts' -> encodeBatch n' s' ts'
_ -> (n', s', Nothing)
tEncode (sig, tr) = smpEncode (C.signatureBytes sig) <> tr
tPut :: Transport c => THandle c -> SentRawTransmission -> IO (Either TransportError ())
tPut th (sig, t) = tPutBlock th $ smpEncode (C.signatureBytes sig) <> t
encodeTransmission :: ProtocolEncoding c => Version -> ByteString -> Transmission c -> ByteString
encodeTransmission v sessionId (CorrId corrId, queueId, command) =
smpEncode (sessionId, corrId, queueId) <> encodeProtocol v command
-- | Receive and parse transmission from the TCP transport (ignoring any trailing padding).
tGetParse :: Transport c => THandle c -> IO (NonEmpty (Either TransportError RawTransmission))
tGetParse th
| batch th = either ((:| []) . Left) id <$> runExceptT getBatch
| otherwise = (:| []) . (parse transmissionP TEBadBlock =<<) <$> tGetBlock th
where
getBatch :: ExceptT TransportError IO (NonEmpty (Either TransportError RawTransmission))
getBatch = do
s <- ExceptT $ tGetBlock th
ts <- liftEither $ parse smpP TEBadBlock s
pure $ L.map (\(Large t) -> parse transmissionP TEBadBlock t) ts
tGetParse :: Transport c => THandle c -> IO (Either TransportError RawTransmission)
tGetParse th = (parse transmissionP TEBadBlock =<<) <$> tGetBlock th
-- | Receive client and server transmissions (determined by `cmd` type).
tGet :: forall cmd c m. (ProtocolEncoding cmd, Transport c, MonadIO m) => THandle c -> m (NonEmpty (SignedTransmission cmd))
tGet th@THandle {sessionId, thVersion = v} = liftIO (tGetParse th) >>= mapM decodeParseValidate
tGet :: forall cmd c m. (ProtocolEncoding cmd, Transport c, MonadIO m) => THandle c -> m (SignedTransmission cmd)
tGet th@THandle {sessionId, thVersion = v} = liftIO (tGetParse th) >>= decodeParseValidate
where
decodeParseValidate :: Either TransportError RawTransmission -> m (SignedTransmission cmd)
decodeParseValidate = \case
+161 -243
View File
@@ -6,7 +6,6 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE RankNTypes #-}
@@ -41,25 +40,22 @@ import Control.Monad.Except
import Control.Monad.IO.Unlift
import Control.Monad.Reader
import Crypto.Random
import Data.Bifunctor (first)
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Either (fromRight, partitionEithers)
import Data.Functor (($>))
import Data.List (intercalate)
import qualified Data.List.NonEmpty as L
import qualified Data.Map.Strict as M
import Data.Maybe (isNothing)
import Data.Set (Set)
import qualified Data.Set as S
import qualified Data.Text as T
import Data.Text.Encoding (decodeLatin1)
import Data.Time.Calendar.Month.Compat (pattern MonthDay)
import Data.Time.Calendar.OrdinalDate (mondayStartWeek)
import Data.Time.Clock (UTCTime (..), diffTimeToPicoseconds, getCurrentTime)
import Data.Time.Clock.System (SystemTime (..), getSystemTime)
import Data.Time.Format.ISO8601 (iso8601Show)
import Data.Type.Equality
import GHC.TypeLits (KnownNat)
import Network.Socket (ServiceName)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding (Encoding (smpEncode))
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol
import Simplex.Messaging.Server.Env.STM
@@ -68,15 +64,12 @@ import Simplex.Messaging.Server.MsgStore
import Simplex.Messaging.Server.MsgStore.STM (MsgQueue)
import Simplex.Messaging.Server.QueueStore
import Simplex.Messaging.Server.QueueStore.STM (QueueStore)
import Simplex.Messaging.Server.Stats
import Simplex.Messaging.Server.StoreLog
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.Server
import Simplex.Messaging.Util
import System.Exit (exitFailure)
import System.IO (hPutStrLn)
import System.Mem.Weak (deRefWeak)
import UnliftIO.Concurrent
import UnliftIO.Directory (doesFileExist, renameFile)
@@ -103,14 +96,13 @@ smpServer :: forall m. (MonadUnliftIO m, MonadReader Env m) => TMVar Bool -> m (
smpServer started = do
s <- asks server
cfg@ServerConfig {transports} <- asks config
restoreServerStats
restoreServerMessages
raceAny_
( serverThread s subscribedQ subscribers subscriptions cancelSub :
serverThread s ntfSubscribedQ notifiers ntfSubscriptions (\_ -> pure ()) :
map runServer transports <> expireMessagesThread_ cfg <> serverStatsThread_ cfg
)
`finally` (withLog closeStoreLog >> saveServerMessages >> saveServerStats)
`finally` (withLog closeStoreLog >> saveServerMessages)
where
runServer :: (ServiceName, ATransport) -> m ()
runServer (tcpPort, ATransport t) = do
@@ -143,7 +135,7 @@ smpServer started = do
endPreviousSubscriptions :: (QueueId, Client) -> m (Maybe s)
endPreviousSubscriptions (qId, c) = do
void . forkIO . atomically $
writeTBQueue (sndQ c) [(CorrId "", qId, END)]
writeTBQueue (sndQ c) (CorrId "", qId, END)
atomically $ TM.lookupDelete qId (clientSubs c)
expireMessagesThread_ :: ServerConfig -> [m ()]
@@ -164,30 +156,36 @@ smpServer started = do
>>= atomically . (`deleteExpiredMsgs` old)
serverStatsThread_ :: ServerConfig -> [m ()]
serverStatsThread_ ServerConfig {logStatsInterval = Just interval, logStatsStartTime, serverStatsLogFile} =
[logServerStats logStatsStartTime interval serverStatsLogFile]
serverStatsThread_ ServerConfig {logStatsInterval = Just interval, logStatsStartTime} =
[logServerStats logStatsStartTime interval]
serverStatsThread_ _ = []
logServerStats :: Int -> Int -> FilePath -> m ()
logServerStats startAt logInterval statsFilePath = do
logServerStats :: Int -> Int -> m ()
logServerStats startAt logInterval = do
initialDelay <- (startAt -) . fromIntegral . (`div` 1000000_000000) . diffTimeToPicoseconds . utctDayTime <$> liftIO getCurrentTime
liftIO $ putStrLn $ "server stats log enabled: " <> statsFilePath
logInfo $ "fromTime,qCreated,qSecured,qDeleted,msgSent,msgRecv,dayMsgQueues,weekMsgQueues,monthMsgQueues"
threadDelay $ 1000000 * (initialDelay + if initialDelay < 0 then 86400 else 0)
ServerStats {fromTime, qCreated, qSecured, qDeleted, msgSent, msgRecv, activeQueues} <- asks serverStats
ServerStats {fromTime, qCreated, qSecured, qDeleted, msgSent, msgRecv, dayMsgQueues, weekMsgQueues, monthMsgQueues} <- asks serverStats
let interval = 1000000 * logInterval
withFile statsFilePath AppendMode $ \h -> liftIO $ do
hSetBuffering h LineBuffering
forever $ do
ts <- getCurrentTime
fromTime' <- atomically $ swapTVar fromTime ts
qCreated' <- atomically $ swapTVar qCreated 0
qSecured' <- atomically $ swapTVar qSecured 0
qDeleted' <- atomically $ swapTVar qDeleted 0
msgSent' <- atomically $ swapTVar msgSent 0
msgRecv' <- atomically $ swapTVar msgRecv 0
ps <- atomically $ periodStatCounts activeQueues ts
hPutStrLn h $ intercalate "," [iso8601Show $ utctDay fromTime', show qCreated', show qSecured', show qDeleted', show msgSent', show msgRecv', dayCount ps, weekCount ps, monthCount ps]
threadDelay interval
forever $ do
ts <- liftIO getCurrentTime
fromTime' <- atomically $ swapTVar fromTime ts
qCreated' <- atomically $ swapTVar qCreated 0
qSecured' <- atomically $ swapTVar qSecured 0
qDeleted' <- atomically $ swapTVar qDeleted 0
msgSent' <- atomically $ swapTVar msgSent 0
msgRecv' <- atomically $ swapTVar msgRecv 0
let day = utctDay ts
(_, wDay) = mondayStartWeek day
MonthDay _ mDay = day
(dayMsgQueues', weekMsgQueues', monthMsgQueues') <-
atomically $ (,,) <$> periodCount 1 dayMsgQueues <*> periodCount wDay weekMsgQueues <*> periodCount mDay monthMsgQueues
logInfo . T.pack $ intercalate "," [show fromTime', show qCreated', show qSecured', show qDeleted', show msgSent', show msgRecv', show dayMsgQueues', weekMsgQueues', monthMsgQueues']
threadDelay interval
where
periodCount :: Int -> TVar (Set RecipientId) -> STM String
periodCount 1 pVar = show . S.size <$> swapTVar pVar S.empty
periodCount _ _ = pure ""
runClient :: Transport c => TProxy c -> c -> m ()
runClient _ h = do
@@ -198,10 +196,10 @@ smpServer started = do
Left _ -> pure ()
runClientTransport :: (Transport c, MonadUnliftIO m, MonadReader Env m) => THandle c -> m ()
runClientTransport th@THandle {thVersion, sessionId} = do
runClientTransport th@THandle {sessionId} = do
q <- asks $ tbqSize . config
ts <- liftIO getSystemTime
c <- atomically $ newClient q thVersion sessionId ts
c <- atomically $ newClient q sessionId ts
s <- asks server
expCfg <- asks $ inactiveClientExpiration . config
raceAny_ ([send th c, client c s, receive th c] <> disconnectThread_ c expCfg)
@@ -233,36 +231,26 @@ cancelSub sub =
Sub {subThread = SubThread t} -> liftIO $ deRefWeak t >>= mapM_ killThread
_ -> return ()
receive :: forall c m. (Transport c, MonadUnliftIO m, MonadReader Env m) => THandle c -> Client -> m ()
receive :: (Transport c, MonadUnliftIO m, MonadReader Env m) => THandle c -> Client -> m ()
receive th Client {rcvQ, sndQ, activeAt} = forever $ do
ts <- L.toList <$> tGet th
(sig, signed, (corrId, queueId, cmdOrError)) <- tGet th
atomically . writeTVar activeAt =<< liftIO getSystemTime
as <- partitionEithers <$> mapM cmdAction ts
write sndQ $ fst as
write rcvQ $ snd as
case cmdOrError of
Left e -> write sndQ (corrId, queueId, ERR e)
Right cmd -> do
verified <- verifyTransmission sig signed queueId cmd
if verified
then write rcvQ (corrId, queueId, cmd)
else write sndQ (corrId, queueId, ERR AUTH)
where
cmdAction :: SignedTransmission Cmd -> m (Either (Transmission BrokerMsg) (Maybe QueueRec, Transmission Cmd))
cmdAction (sig, signed, (corrId, queueId, cmdOrError)) =
case cmdOrError of
Left e -> pure $ Left (corrId, queueId, ERR e)
Right cmd -> verified <$> verifyTransmission sig signed queueId cmd
where
verified = \case
VRVerified qr -> Right (qr, (corrId, queueId, cmd))
VRFailed -> Left (corrId, queueId, ERR AUTH)
write q = mapM_ (atomically . writeTBQueue q) . L.nonEmpty
write q t = atomically $ writeTBQueue q t
send :: (Transport c, MonadUnliftIO m) => THandle c -> Client -> m ()
send h@THandle {thVersion = v} Client {sndQ, sessionId, activeAt} = forever $ do
ts <- atomically $ L.sortWith tOrder <$> readTBQueue sndQ
-- TODO the line below can return Lefts, but we ignore it and do not disconnect the client
void . liftIO . tPut h $ L.map ((Nothing,) . encodeTransmission v sessionId) ts
t <- atomically $ readTBQueue sndQ
-- TODO the line below can return Left, but we ignore it and do not disconnect the client
void . liftIO $ tPut h (Nothing, encodeTransmission v sessionId t)
atomically . writeTVar activeAt =<< liftIO getSystemTime
where
tOrder :: Transmission BrokerMsg -> Int
tOrder (_, _, cmd) = case cmd of
MSG {} -> 0
_ -> 1
disconnectTransport :: (Transport c, MonadUnliftIO m) => THandle c -> client -> (client -> TVar SystemTime) -> ExpirationConfig -> m ()
disconnectTransport THandle {connection} c activeAt expCfg = do
@@ -273,28 +261,23 @@ disconnectTransport THandle {connection} c activeAt expCfg = do
ts <- readTVarIO $ activeAt c
when (systemSeconds ts < old) $ closeConnection connection
data VerificationResult = VRVerified (Maybe QueueRec) | VRFailed
verifyTransmission ::
forall m. (MonadUnliftIO m, MonadReader Env m) => Maybe C.ASignature -> ByteString -> QueueId -> Cmd -> m VerificationResult
forall m. (MonadUnliftIO m, MonadReader Env m) => Maybe C.ASignature -> ByteString -> QueueId -> Cmd -> m Bool
verifyTransmission sig_ signed queueId cmd = do
case cmd of
Cmd SRecipient (NEW k _) -> pure $ Nothing `verified` verifyCmdSignature sig_ signed k
Cmd SRecipient (NEW k _) -> pure $ verifyCmdSignature sig_ signed k
Cmd SRecipient _ -> verifyCmd SRecipient $ verifyCmdSignature sig_ signed . recipientKey
Cmd SSender SEND {} -> verifyCmd SSender $ verifyMaybe . senderKey
Cmd SSender PING -> pure $ VRVerified Nothing
Cmd SNotifier NSUB -> verifyCmd SNotifier $ verifyMaybe . fmap notifierKey . notifier
Cmd SSender PING -> pure True
Cmd SNotifier NSUB -> verifyCmd SNotifier $ verifyMaybe . fmap snd . notifier
where
verifyCmd :: SParty p -> (QueueRec -> Bool) -> m VerificationResult
verifyCmd :: SParty p -> (QueueRec -> Bool) -> m Bool
verifyCmd party f = do
st <- asks queueStore
q_ <- atomically (getQueue st party queueId)
pure $ case q_ of
Right q -> Just q `verified` f q
_ -> maybe False (dummyVerifyCmd signed) sig_ `seq` VRFailed
q <- atomically $ getQueue st party queueId
pure $ either (const $ maybe False (dummyVerifyCmd signed) sig_ `seq` False) f q
verifyMaybe :: Maybe C.APublicVerifyKey -> Bool
verifyMaybe = maybe (isNothing sig_) $ verifyCmdSignature sig_ signed
verified q cond = if cond then VRVerified q else VRFailed
verifyCmdSignature :: Maybe C.ASignature -> ByteString -> C.APublicVerifyKey -> Bool
verifyCmdSignature sig_ signed key = maybe False (verify key) sig_
@@ -322,19 +305,19 @@ dummyKeyEd448 :: C.PublicKey 'C.Ed448
dummyKeyEd448 = "MEMwBQYDK2VxAzoA6ibQc9XpkSLtwrf7PLvp81qW/etiumckVFImCMRdftcG/XopbOSaq9qyLhrgJWKOLyNrQPNVvpMA"
client :: forall m. (MonadUnliftIO m, MonadReader Env m) => Client -> Server -> m ()
client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Server {subscribedQ, ntfSubscribedQ, notifiers} =
client clnt@Client {subscriptions, ntfSubscriptions, rcvQ, sndQ} Server {subscribedQ, ntfSubscribedQ, notifiers} =
forever $
atomically (readTBQueue rcvQ)
>>= mapM processCommand
>>= processCommand
>>= atomically . writeTBQueue sndQ
where
processCommand :: (Maybe QueueRec, Transmission Cmd) -> m (Transmission BrokerMsg)
processCommand (qr_, (corrId, queueId, cmd)) = do
processCommand :: Transmission Cmd -> m (Transmission BrokerMsg)
processCommand (corrId, queueId, cmd) = do
st <- asks queueStore
case cmd of
Cmd SSender command ->
case command of
SEND flags msgBody -> withQueue $ \qr -> sendMessage qr flags msgBody
SEND flags msgBody -> sendMessage st flags msgBody
PING -> pure (corrId, "", PONG)
Cmd SNotifier NSUB -> subscribeNotifications
Cmd SRecipient command ->
@@ -344,12 +327,11 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
(asks $ allowNewQueues . config)
(createQueue st rKey dhKey)
(pure (corrId, queueId, ERR AUTH))
SUB -> withQueue (`subscribeQueue` queueId)
GET -> withQueue getMessage
ACK msgId -> withQueue (`acknowledgeMsg` msgId)
SUB -> subscribeQueue queueId
GET -> getMessage
ACK msgId -> acknowledgeMsg msgId
KEY sKey -> secureQueue_ st sKey
NKEY nKey dhKey -> addQueueNotifier_ st nKey dhKey
NDEL -> deleteQueueNotifier_ st
NKEY nKey -> addQueueNotifier_ st nKey
OFF -> suspendQueue_ st
DEL -> delQueueAndMsgs st
where
@@ -376,15 +358,14 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
addQueueRetry n qik qRec = do
ids@(rId, _) <- getIds
-- create QueueRec record with these ids and keys
let qr = qRec ids
atomically (addQueue st qr) >>= \case
atomically (addQueue st $ qRec ids) >>= \case
Left DUPLICATE_ -> addQueueRetry (n - 1) qik qRec
Left e -> pure $ ERR e
Right _ -> do
withLog (`logCreateById` rId)
stats <- asks serverStats
atomically $ modifyTVar (qCreated stats) (+ 1)
subscribeQueue qr rId $> IDS (qik ids)
subscribeQueue rId $> IDS (qik ids)
logCreateById :: StoreLog 'WriteMode -> RecipientId -> IO ()
logCreateById s rId =
@@ -404,36 +385,27 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
atomically $ modifyTVar (qSecured stats) (+ 1)
atomically $ (corrId,queueId,) . either ERR (const OK) <$> secureQueue st queueId sKey
addQueueNotifier_ :: QueueStore -> NtfPublicVerifyKey -> RcvNtfPublicDhKey -> m (Transmission BrokerMsg)
addQueueNotifier_ st notifierKey dhKey = do
(rcvPublicDhKey, privDhKey) <- liftIO C.generateKeyPair'
let rcvNtfDhSecret = C.dh' dhKey privDhKey
(corrId,queueId,) <$> addNotifierRetry 3 rcvPublicDhKey rcvNtfDhSecret
addQueueNotifier_ :: QueueStore -> NtfPublicVerifyKey -> m (Transmission BrokerMsg)
addQueueNotifier_ st nKey = (corrId,queueId,) <$> addNotifierRetry 3
where
addNotifierRetry :: Int -> RcvNtfPublicDhKey -> RcvNtfDhSecret -> m BrokerMsg
addNotifierRetry 0 _ _ = pure $ ERR INTERNAL
addNotifierRetry n rcvPublicDhKey rcvNtfDhSecret = do
notifierId <- randomId =<< asks (queueIdBytes . config)
let ntfCreds = NtfCreds {notifierId, notifierKey, rcvNtfDhSecret}
atomically (addQueueNotifier st queueId ntfCreds) >>= \case
Left DUPLICATE_ -> addNotifierRetry (n - 1) rcvPublicDhKey rcvNtfDhSecret
addNotifierRetry :: Int -> m BrokerMsg
addNotifierRetry 0 = pure $ ERR INTERNAL
addNotifierRetry n = do
nId <- randomId =<< asks (queueIdBytes . config)
atomically (addQueueNotifier st queueId nId nKey) >>= \case
Left DUPLICATE_ -> addNotifierRetry $ n - 1
Left e -> pure $ ERR e
Right _ -> do
withLog $ \s -> logAddNotifier s queueId ntfCreds
pure $ NID notifierId rcvPublicDhKey
deleteQueueNotifier_ :: QueueStore -> m (Transmission BrokerMsg)
deleteQueueNotifier_ st = do
withLog (`logDeleteNotifier` queueId)
okResp <$> atomically (deleteQueueNotifier st queueId)
withLog $ \s -> logAddNotifier s queueId nId nKey
pure $ NID nId
suspendQueue_ :: QueueStore -> m (Transmission BrokerMsg)
suspendQueue_ st = do
withLog (`logDeleteQueue` queueId)
okResp <$> atomically (suspendQueue st queueId)
subscribeQueue :: QueueRec -> RecipientId -> m (Transmission BrokerMsg)
subscribeQueue qr rId =
subscribeQueue :: RecipientId -> m (Transmission BrokerMsg)
subscribeQueue rId =
atomically (TM.lookup rId subscriptions) >>= \case
Nothing ->
atomically newSub >>= deliver
@@ -455,10 +427,10 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
deliver sub = do
q <- getStoreMsgQueue rId
msg_ <- atomically $ tryPeekMsg q
deliverMessage qr rId sub q msg_
deliverMessage rId sub q msg_
getMessage :: QueueRec -> m (Transmission BrokerMsg)
getMessage qr =
getMessage :: m (Transmission BrokerMsg)
getMessage =
atomically (TM.lookup queueId subscriptions) >>= \case
Nothing ->
atomically newSub >>= getMessage_
@@ -481,14 +453,9 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
q <- getStoreMsgQueue queueId
atomically $
tryPeekMsg q >>= \case
Just msg ->
let encMsg = encryptMsg qr msg
in setDelivered s msg $> (corrId, queueId, MSG encMsg)
Just msg -> setDelivered s msg $> (corrId, queueId, msgCmd msg)
_ -> pure (corrId, queueId, OK)
withQueue :: (QueueRec -> m (Transmission BrokerMsg)) -> m (Transmission BrokerMsg)
withQueue action = maybe (pure $ err AUTH) action qr_
subscribeNotifications :: m (Transmission BrokerMsg)
subscribeNotifications = atomically $ do
unlessM (TM.member queueId ntfSubscriptions) $ do
@@ -496,8 +463,8 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
TM.insert queueId () ntfSubscriptions
pure ok
acknowledgeMsg :: QueueRec -> MsgId -> m (Transmission BrokerMsg)
acknowledgeMsg qr msgId = do
acknowledgeMsg :: MsgId -> m (Transmission BrokerMsg)
acknowledgeMsg msgId = do
atomically (TM.lookup queueId subscriptions) >>= \case
Nothing -> pure $ err NO_MSG
Just sub ->
@@ -512,80 +479,83 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
_ -> do
(msgDeleted, msg_) <- atomically $ tryDelPeekMsg q msgId
when msgDeleted updateStats
deliverMessage qr queueId sub q msg_
deliverMessage queueId sub q msg_
_ -> pure $ err NO_MSG
where
getDelivered :: TVar Sub -> STM (Maybe Sub)
getDelivered sub = do
s@Sub {delivered} <- readTVar sub
tryTakeTMVar delivered $>>= \msgId' ->
if msgId == msgId' || B.null msgId
if B.null msgId || msgId == msgId'
then pure $ Just s
else putTMVar delivered msgId' $> Nothing
updateStats :: m ()
updateStats = do
stats <- asks serverStats
atomically $ modifyTVar (msgRecv stats) (+ 1)
atomically $ updatePeriodStats (activeQueues stats) queueId
atomically $ updateActiveQueues stats queueId
sendMessage :: QueueRec -> MsgFlags -> MsgBody -> m (Transmission BrokerMsg)
sendMessage qr msgFlags msgBody
| B.length msgBody > maxMessageLength = pure $ err LARGE_MSG
| otherwise = case status qr of
QueueOff -> return $ err AUTH
QueueActive ->
mapM mkMessage (C.maxLenBS msgBody) >>= \case
Left _ -> pure $ err LARGE_MSG
Right msg -> do
ms <- asks msgStore
ServerConfig {messageExpiration, msgQueueQuota} <- asks config
old <- liftIO $ mapM expireBeforeEpoch messageExpiration
ntfNonceDrg <- asks idsDrg
resp@(_, _, sent) <- atomically $ do
q <- getMsgQueue ms (recipientId qr) msgQueueQuota
mapM_ (deleteExpiredMsgs q) old
ifM (isFull q) (pure $ err QUOTA) $ do
when (notification msgFlags) $ trySendNotification msg ntfNonceDrg
writeMsg q msg
pure ok
when (sent == OK) $ do
stats <- asks serverStats
atomically $ modifyTVar (msgSent stats) (+ 1)
atomically $ updatePeriodStats (activeQueues stats) (recipientId qr)
pure resp
updateActiveQueues :: ServerStats -> RecipientId -> STM ()
updateActiveQueues stats qId = do
updatePeriod dayMsgQueues
updatePeriod weekMsgQueues
updatePeriod monthMsgQueues
where
mkMessage :: C.MaxLenBS MaxMessageLen -> m Message
mkMessage body = do
msgId <- randomId =<< asks (msgIdBytes . config)
msgTs <- liftIO getSystemTime
pure $ Message msgId msgTs msgFlags body
updatePeriod pSel = modifyTVar (pSel stats) (S.insert qId)
trySendNotification :: Message -> TVar ChaChaDRG -> STM ()
trySendNotification msg ntfNonceDrg =
forM_ (notifier qr) $ \NtfCreds {notifierId, rcvNtfDhSecret} ->
mapM_ (writeNtf notifierId msg rcvNtfDhSecret ntfNonceDrg) =<< TM.lookup notifierId notifiers
sendMessage :: QueueStore -> MsgFlags -> MsgBody -> m (Transmission BrokerMsg)
sendMessage st flags msgBody
| B.length msgBody > maxMessageLength = pure $ err LARGE_MSG
| otherwise = do
qr <- atomically $ getQueue st SSender queueId
either (return . err) storeMessage qr
where
storeMessage :: QueueRec -> m (Transmission BrokerMsg)
storeMessage qr = case status qr of
QueueOff -> return $ err AUTH
QueueActive ->
mkMessage >>= \case
Left _ -> pure $ err LARGE_MSG
Right msg -> do
ms <- asks msgStore
ServerConfig {messageExpiration, msgQueueQuota} <- asks config
old <- liftIO $ mapM expireBeforeEpoch messageExpiration
resp@(_, _, sent) <- atomically $ do
q <- getMsgQueue ms (recipientId qr) msgQueueQuota
mapM_ (deleteExpiredMsgs q) old
ifM (isFull q) (pure $ err QUOTA) $ do
when (notification flags) trySendNotification
writeMsg q msg
pure ok
when (sent == OK) $ do
stats <- asks serverStats
atomically $ modifyTVar (msgSent stats) (+ 1)
atomically $ updateActiveQueues stats $ recipientId qr
pure resp
where
mkMessage :: m (Either C.CryptoError Message)
mkMessage = do
msgId <- randomId =<< asks (msgIdBytes . config)
ts <- liftIO getSystemTime
let c = C.cbEncrypt (rcvDhSecret qr) (C.cbNonce msgId) msgBody (maxMessageLength + 2)
pure $ Message msgId ts flags <$> c
writeNtf :: NotifierId -> Message -> RcvNtfDhSecret -> TVar ChaChaDRG -> Client -> STM ()
writeNtf nId msg rcvNtfDhSecret ntfNonceDrg Client {sndQ = q} =
unlessM (isFullTBQueue sndQ) $ do
(nmsgNonce, encNMsgMeta) <- mkMessageNotification msg rcvNtfDhSecret ntfNonceDrg
writeTBQueue q [(CorrId "", nId, NMSG nmsgNonce encNMsgMeta)]
trySendNotification :: STM ()
trySendNotification =
forM_ (notifier qr) $ \(nId, _) ->
mapM_ (writeNtf nId) =<< TM.lookup nId notifiers
mkMessageNotification :: Message -> RcvNtfDhSecret -> TVar ChaChaDRG -> STM (C.CbNonce, EncNMsgMeta)
mkMessageNotification Message {msgId, msgTs} rcvNtfDhSecret ntfNonceDrg = do
cbNonce <- C.pseudoRandomCbNonce ntfNonceDrg
let msgMeta = NMsgMeta {msgId, msgTs}
encNMsgMeta = C.cbEncrypt rcvNtfDhSecret cbNonce (smpEncode msgMeta) 128
pure . (cbNonce,) $ fromRight "" encNMsgMeta
writeNtf :: NotifierId -> Client -> STM ()
writeNtf nId Client {sndQ = q} =
unlessM (isFullTBQueue sndQ) $
writeTBQueue q (CorrId "", nId, NMSG)
deliverMessage :: QueueRec -> RecipientId -> TVar Sub -> MsgQueue -> Maybe Message -> m (Transmission BrokerMsg)
deliverMessage qr rId sub q msg_ = do
deliverMessage :: RecipientId -> TVar Sub -> MsgQueue -> Maybe Message -> m (Transmission BrokerMsg)
deliverMessage rId sub q msg_ =
readTVarIO sub >>= \case
s@Sub {subThread = NoSub} ->
case msg_ of
Just msg ->
let encMsg = encryptMsg qr msg
in atomically (setDelivered s msg) $> (corrId, rId, MSG encMsg)
Just msg -> atomically (setDelivered s msg) $> (corrId, rId, msgCmd msg)
_ -> forkSub $> ok
_ -> pure ok
where
@@ -596,24 +566,14 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
atomically . modifyTVar sub $ \case
s@Sub {subThread = SubPending} -> s {subThread = SubThread t}
s -> s
where
subscriber = atomically $ do
msg <- peekMsg q
let encMsg = encryptMsg qr msg
writeTBQueue sndQ [(CorrId "", rId, MSG encMsg)]
s <- readTVar sub
void $ setDelivered s msg
writeTVar sub s {subThread = NoSub}
encryptMsg :: QueueRec -> Message -> RcvMessage
encryptMsg qr Message {msgId, msgTs, msgFlags, msgBody}
| thVersion == 1 || thVersion == 2 = encrypt msgBody
| otherwise = encrypt $ encodeRcvMsgBody RcvMsgBody {msgTs, msgFlags, msgBody}
where
encrypt :: KnownNat i => C.MaxLenBS i -> RcvMessage
encrypt body =
let encBody = EncRcvMsgBody $ C.cbEncryptMaxLenBS (rcvDhSecret qr) (C.cbNonce msgId) body
in RcvMessage msgId msgTs msgFlags encBody
subscriber :: m ()
subscriber = atomically $ do
msg <- peekMsg q
writeTBQueue sndQ (CorrId "", rId, msgCmd msg)
s <- readTVar sub
void $ setDelivered s msg
writeTVar sub s {subThread = NoSub}
setDelivered :: Sub -> Message -> STM Bool
setDelivered s Message {msgId} = tryPutTMVar (delivered s) msgId
@@ -624,6 +584,9 @@ client clnt@Client {thVersion, subscriptions, ntfSubscriptions, rcvQ, sndQ} Serv
quota <- asks $ msgQueueQuota . config
atomically $ getMsgQueue ms rId quota
msgCmd :: Message -> BrokerMsg
msgCmd Message {msgId, ts, msgFlags, msgBody} = MSG msgId ts msgFlags msgBody
delQueueAndMsgs :: QueueStore -> m (Transmission BrokerMsg)
delQueueAndMsgs st = do
withLog (`logDeleteQueue` queueId)
@@ -654,78 +617,33 @@ randomId n = do
gVar <- asks idsDrg
atomically (C.pseudoRandomBytes n gVar)
saveServerMessages :: (MonadUnliftIO m, MonadReader Env m) => m ()
saveServerMessages :: forall m. (MonadUnliftIO m, MonadReader Env m) => m ()
saveServerMessages = asks (storeMsgsFile . config) >>= mapM_ saveMessages
where
saveMessages f = do
logInfo $ "saving messages to file " <> T.pack f
liftIO $ putStrLn $ "saving messages to file " <> f
ms <- asks msgStore
liftIO . withFile f WriteMode $ \h ->
readTVarIO ms >>= mapM_ (saveQueueMsgs ms h) . M.keys
logInfo "messages saved"
where
saveQueueMsgs ms h rId =
atomically (flushMsgQueue ms rId)
>>= mapM_ (B.hPutStrLn h . strEncode . MLRv3 rId)
>>= mapM_ (B.hPutStrLn h . strEncode . MsgLogRecord rId)
restoreServerMessages :: forall m. (MonadUnliftIO m, MonadReader Env m) => m ()
restoreServerMessages = asks (storeMsgsFile . config) >>= mapM_ restoreMessages
where
restoreMessages f = whenM (doesFileExist f) $ do
logInfo $ "restoring messages from file " <> T.pack f
st <- asks queueStore
liftIO $ putStrLn $ "restoring messages from file " <> f
ms <- asks msgStore
quota <- asks $ msgQueueQuota . config
runExceptT (liftIO (B.readFile f) >>= mapM_ (restoreMsg st ms quota) . B.lines) >>= \case
Left e -> do
logError . T.pack $ "error restoring messages: " <> e
liftIO exitFailure
_ -> do
renameFile f $ f <> ".bak"
logInfo "messages restored"
liftIO $ mapM_ (restoreMsg ms quota) . B.lines =<< B.readFile f
renameFile f $ f <> ".bak"
where
restoreMsg st ms quota s = do
r <- liftEither . first (msgErr "parsing") $ strDecode s
case r of
MLRv3 rId msg -> addToMsgQueue rId msg
MLRv1 rId encMsg -> do
qr <- liftEitherError (msgErr "queue unknown") . atomically $ getQueue st SRecipient rId
msg' <- updateMsgV1toV3 qr encMsg
addToMsgQueue rId msg'
where
addToMsgQueue rId msg = do
full <- atomically $ do
q <- getMsgQueue ms rId quota
ifM (isFull q) (pure True) (writeMsg q msg $> False)
when full . logError . decodeLatin1 $ "message queue " <> strEncode rId <> " is full, message not restored: " <> strEncode (msgId (msg :: Message))
updateMsgV1toV3 QueueRec {rcvDhSecret} RcvMessage {msgId, msgTs, msgFlags, msgBody = EncRcvMsgBody body} = do
let nonce = C.cbNonce msgId
msgBody <- liftEither . first (msgErr "v1 message decryption") $ C.maxLenBS =<< C.cbDecrypt rcvDhSecret nonce body
pure Message {msgId, msgTs, msgFlags, msgBody}
msgErr :: Show e => String -> e -> String
msgErr op e = op <> " error (" <> show e <> "): " <> B.unpack (B.take 100 s)
saveServerStats :: (MonadUnliftIO m, MonadReader Env m) => m ()
saveServerStats =
asks (serverStatsBackupFile . config)
>>= mapM_ (\f -> asks serverStats >>= atomically . getServerStatsData >>= liftIO . saveStats f)
where
saveStats f stats = do
logInfo $ "saving server stats to file " <> T.pack f
B.writeFile f $ strEncode stats
logInfo "server stats saved"
restoreServerStats :: (MonadUnliftIO m, MonadReader Env m) => m ()
restoreServerStats = asks (serverStatsBackupFile . config) >>= mapM_ restoreStats
where
restoreStats f = whenM (doesFileExist f) $ do
logInfo $ "restoring server stats from file " <> T.pack f
liftIO (strDecode <$> B.readFile f) >>= \case
Right d -> do
s <- asks serverStats
atomically $ setServerStats s d
renameFile f $ f <> ".bak"
logInfo "server stats restored"
Left e -> do
logInfo $ "error restoring server stats: " <> T.pack e
liftIO exitFailure
restoreMsg ms quota s = case strDecode s of
Left e -> B.putStrLn $ "message parsing error (" <> B.pack e <> "): " <> B.take 100 s
Right (MsgLogRecord rId msg) -> do
full <- atomically $ do
q <- getMsgQueue ms rId quota
ifM (isFull q) (pure True) (writeMsg q msg $> False)
when full . B.putStrLn $ "message queue " <> strEncode rId <> " is full, message not restored: " <> strEncode (msgId msg)
+3 -11
View File
@@ -23,8 +23,7 @@ import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Transport (ATransport (..), TLS, Transport (..))
import Simplex.Messaging.Transport.Server (loadFingerprint)
import Simplex.Messaging.Transport.WebSockets (WS)
import Simplex.Messaging.Util (whenM)
import System.Directory (createDirectoryIfMissing, doesDirectoryExist, doesFileExist, listDirectory, removeDirectoryRecursive, removePathForcibly)
import System.Directory (createDirectoryIfMissing, doesDirectoryExist, doesFileExist, removeDirectoryRecursive)
import System.Exit (exitFailure)
import System.FilePath (combine)
import System.IO (BufferMode (..), IOMode (..), hFlush, hGetLine, hSetBuffering, stderr, stdout, withFile)
@@ -148,7 +147,7 @@ cliCommandP ServerCLIConfig {cfgDir, logDir, iniFile} =
initializeServer :: ServerCLIConfig cfg -> InitOptions -> IO ()
initializeServer cliCfg InitOptions {enableStoreLog, signAlgorithm, ip, fqdn} = do
clearDirs cliCfg
cleanup cliCfg
createDirectoryIfMissing True cfgDir
createDirectoryIfMissing True logDir
createX509
@@ -277,14 +276,7 @@ cleanup ServerCLIConfig {cfgDir, logDir} = do
deleteDirIfExists cfgDir
deleteDirIfExists logDir
where
deleteDirIfExists path = whenM (doesDirectoryExist path) $ removeDirectoryRecursive path
clearDirs :: ServerCLIConfig cfg -> IO ()
clearDirs ServerCLIConfig {cfgDir, logDir} = do
clearDirIfExists cfgDir
clearDirIfExists logDir
where
clearDirIfExists path = whenM (doesDirectoryExist path) $ listDirectory path >>= mapM_ (removePathForcibly . combine path)
deleteDirIfExists path = doesDirectoryExist path >>= (`when` removeDirectoryRecursive path)
printServiceInfo :: ServerCLIConfig cfg -> ByteString -> IO ()
printServiceInfo ServerCLIConfig {serverVersion} fpStr = do
+36 -15
View File
@@ -2,6 +2,7 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TupleSections #-}
module Simplex.Messaging.Server.Env.STM where
@@ -9,10 +10,11 @@ import Control.Concurrent (ThreadId)
import Control.Monad.IO.Unlift
import Crypto.Random
import Data.ByteString.Char8 (ByteString)
import Data.List.NonEmpty (NonEmpty)
import Data.Map.Strict (Map)
import qualified Data.Map.Strict as M
import Data.Time.Clock (getCurrentTime)
import Data.Set (Set)
import qualified Data.Set as S
import Data.Time.Clock (UTCTime, getCurrentTime)
import Data.Time.Clock.System (SystemTime)
import Data.X509.Validation (Fingerprint (..))
import Network.Socket (ServiceName)
@@ -22,9 +24,8 @@ import Simplex.Messaging.Crypto (KeyHash (..))
import Simplex.Messaging.Protocol
import Simplex.Messaging.Server.Expiration
import Simplex.Messaging.Server.MsgStore.STM
import Simplex.Messaging.Server.QueueStore (NtfCreds (..), QueueRec (..))
import Simplex.Messaging.Server.QueueStore (QueueRec (..))
import Simplex.Messaging.Server.QueueStore.STM
import Simplex.Messaging.Server.Stats
import Simplex.Messaging.Server.StoreLog
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
@@ -56,10 +57,6 @@ data ServerConfig = ServerConfig
-- | time of the day when the stats are logged first, to log at consistent times,
-- irrespective of when the server is started (seconds from 00:00 UTC)
logStatsStartTime :: Int,
-- | file to log stats
serverStatsLogFile :: FilePath,
-- | file to save and restore stats
serverStatsBackupFile :: Maybe FilePath,
-- | CA certificate private key is not needed for initialization
caCertificateFile :: FilePath,
privateKeyFile :: FilePath,
@@ -104,14 +101,25 @@ data Server = Server
data Client = Client
{ subscriptions :: TMap RecipientId (TVar Sub),
ntfSubscriptions :: TMap NotifierId (),
rcvQ :: TBQueue (NonEmpty (Maybe QueueRec, Transmission Cmd)),
sndQ :: TBQueue (NonEmpty (Transmission BrokerMsg)),
thVersion :: Version,
rcvQ :: TBQueue (Transmission Cmd),
sndQ :: TBQueue (Transmission BrokerMsg),
sessionId :: ByteString,
connected :: TVar Bool,
activeAt :: TVar SystemTime
}
data ServerStats = ServerStats
{ qCreated :: TVar Int,
qSecured :: TVar Int,
qDeleted :: TVar Int,
msgSent :: TVar Int,
msgRecv :: TVar Int,
dayMsgQueues :: TVar (Set RecipientId),
weekMsgQueues :: TVar (Set RecipientId),
monthMsgQueues :: TVar (Set RecipientId),
fromTime :: TVar UTCTime
}
data SubscriptionThread = NoSub | SubPending | SubThread (Weak ThreadId) | ProhibitSub
data Sub = Sub
@@ -127,15 +135,28 @@ newServer qSize = do
notifiers <- TM.empty
return Server {subscribedQ, subscribers, ntfSubscribedQ, notifiers}
newClient :: Natural -> Version -> ByteString -> SystemTime -> STM Client
newClient qSize thVersion sessionId ts = do
newClient :: Natural -> ByteString -> SystemTime -> STM Client
newClient qSize sessionId ts = do
subscriptions <- TM.empty
ntfSubscriptions <- TM.empty
rcvQ <- newTBQueue qSize
sndQ <- newTBQueue qSize
connected <- newTVar True
activeAt <- newTVar ts
return Client {subscriptions, ntfSubscriptions, rcvQ, sndQ, thVersion, sessionId, connected, activeAt}
return Client {subscriptions, ntfSubscriptions, rcvQ, sndQ, sessionId, connected, activeAt}
newServerStats :: UTCTime -> STM ServerStats
newServerStats ts = do
qCreated <- newTVar 0
qSecured <- newTVar 0
qDeleted <- newTVar 0
msgSent <- newTVar 0
msgRecv <- newTVar 0
dayMsgQueues <- newTVar S.empty
weekMsgQueues <- newTVar S.empty
monthMsgQueues <- newTVar S.empty
fromTime <- newTVar ts
pure ServerStats {qCreated, qSecured, qDeleted, msgSent, msgRecv, dayMsgQueues, weekMsgQueues, monthMsgQueues, fromTime}
newSubscription :: SubscriptionThread -> STM Sub
newSubscription subThread = do
@@ -169,4 +190,4 @@ newEnv config@ServerConfig {caCertificateFile, certificateFile, privateKeyFile,
addNotifier :: QueueRec -> Map NotifierId RecipientId -> Map NotifierId RecipientId
addNotifier q = case notifier q of
Nothing -> id
Just NtfCreds {notifierId} -> M.insert notifierId (recipientId q)
Just (nId, _) -> M.insert nId (recipientId q)
+19 -9
View File
@@ -1,22 +1,32 @@
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE NamedFieldPuns #-}
module Simplex.Messaging.Server.MsgStore where
import Control.Applicative ((<|>))
import Data.Int (Int64)
import Data.Time.Clock.System (SystemTime)
import Numeric.Natural
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol (Message (..), MsgId, RcvMessage (..), RecipientId)
import Simplex.Messaging.Protocol (MsgBody, MsgFlags, MsgId, RecipientId)
data MsgLogRecord = MLRv3 RecipientId Message | MLRv1 RecipientId RcvMessage
data Message = Message
{ msgId :: MsgId,
ts :: SystemTime,
msgFlags :: MsgFlags,
msgBody :: MsgBody
}
instance StrEncoding Message where
strEncode Message {msgId, ts, msgFlags, msgBody} = strEncode (msgId, ts, msgFlags, msgBody)
strP = do
(msgId, ts, msgFlags, msgBody) <- strP
pure Message {msgId, ts, msgFlags, msgBody}
data MsgLogRecord = MsgLogRecord RecipientId Message
instance StrEncoding MsgLogRecord where
strEncode = \case
MLRv3 rId msg -> strEncode (Str "v3", rId, msg)
MLRv1 rId msg -> strEncode (rId, msg)
strP = "v3 " *> (MLRv3 <$> strP_ <*> strP) <|> MLRv1 <$> strP_ <*> strP
strEncode (MsgLogRecord rId msg) = strEncode (rId, msg)
strP = MsgLogRecord <$> strP_ <*> strP
class MonadMsgStore s q m | s -> q where
getMsgQueue :: s -> RecipientId -> Natural -> m q
+5 -6
View File
@@ -11,12 +11,11 @@ module Simplex.Messaging.Server.MsgStore.STM where
import Control.Concurrent.STM.TBQueue (flushTBQueue)
import Control.Monad (when)
import qualified Data.ByteString.Char8 as B
import Data.Functor (($>))
import Data.Int (Int64)
import Data.Time.Clock.System (SystemTime (systemSeconds))
import Numeric.Natural
import Simplex.Messaging.Protocol (Message (..), MsgId, RecipientId)
import Simplex.Messaging.Protocol (MsgId, RecipientId)
import Simplex.Messaging.Server.MsgStore
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
@@ -61,7 +60,7 @@ instance MonadMsgQueue MsgQueue STM where
tryDelMsg (MsgQueue q) msgId' =
tryPeekTBQueue q >>= \case
Just Message {msgId}
| msgId == msgId' || B.null msgId' -> tryReadTBQueue q $> True
| msgId == msgId' -> tryReadTBQueue q $> True
| otherwise -> pure False
_ -> pure False
@@ -70,7 +69,7 @@ instance MonadMsgQueue MsgQueue STM where
tryDelPeekMsg (MsgQueue q) msgId' =
tryPeekTBQueue q >>= \case
msg_@(Just Message {msgId})
| msgId == msgId' || B.null msgId' -> (True,) <$> (tryReadTBQueue q >> tryPeekTBQueue q)
| msgId == msgId' -> (True,) <$> (tryReadTBQueue q >> tryPeekTBQueue q)
| otherwise -> pure (False, msg_)
_ -> pure (False, Nothing)
@@ -78,6 +77,6 @@ instance MonadMsgQueue MsgQueue STM where
deleteExpiredMsgs (MsgQueue q) old = loop
where
loop = tryPeekTBQueue q >>= mapM_ delOldMsg
delOldMsg Message {msgTs} =
when (systemSeconds msgTs < old) $
delOldMsg Message {ts} =
when (systemSeconds ts < old) $
tryReadTBQueue q >> loop
+2 -18
View File
@@ -1,11 +1,9 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE NamedFieldPuns #-}
module Simplex.Messaging.Server.QueueStore where
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol
data QueueRec = QueueRec
@@ -14,31 +12,17 @@ data QueueRec = QueueRec
rcvDhSecret :: RcvDhSecret,
senderId :: SenderId,
senderKey :: Maybe SndPublicVerifyKey,
notifier :: Maybe NtfCreds,
notifier :: Maybe (NotifierId, NtfPublicVerifyKey),
status :: QueueStatus
}
deriving (Eq, Show)
data NtfCreds = NtfCreds
{ notifierId :: NotifierId,
notifierKey :: NtfPublicVerifyKey,
rcvNtfDhSecret :: RcvNtfDhSecret
}
deriving (Eq, Show)
instance StrEncoding NtfCreds where
strEncode NtfCreds {notifierId, notifierKey, rcvNtfDhSecret} = strEncode (notifierId, notifierKey, rcvNtfDhSecret)
strP = do
(notifierId, notifierKey, rcvNtfDhSecret) <- strP
pure NtfCreds {notifierId, notifierKey, rcvNtfDhSecret}
data QueueStatus = QueueActive | QueueOff deriving (Eq, Show)
class MonadQueueStore s m where
addQueue :: s -> QueueRec -> m (Either ErrorType ())
getQueue :: s -> SParty p -> QueueId -> m (Either ErrorType QueueRec)
secureQueue :: s -> RecipientId -> SndPublicVerifyKey -> m (Either ErrorType QueueRec)
addQueueNotifier :: s -> RecipientId -> NtfCreds -> m (Either ErrorType QueueRec)
deleteQueueNotifier :: s -> RecipientId -> m (Either ErrorType ())
addQueueNotifier :: s -> RecipientId -> NotifierId -> NtfPublicVerifyKey -> m (Either ErrorType QueueRec)
suspendQueue :: s -> RecipientId -> m (Either ErrorType ())
deleteQueue :: s -> RecipientId -> m (Either ErrorType ())
+10 -17
View File
@@ -61,23 +61,16 @@ instance MonadQueueStore QueueStore STM where
Just _ -> pure Nothing
_ -> writeTVar qVar q {senderKey = Just sKey} $> Just q
addQueueNotifier :: QueueStore -> RecipientId -> NtfCreds -> STM (Either ErrorType QueueRec)
addQueueNotifier QueueStore {queues, notifiers} rId ntfCreds@NtfCreds {notifierId = nId} = do
addQueueNotifier :: QueueStore -> RecipientId -> NotifierId -> NtfPublicVerifyKey -> STM (Either ErrorType QueueRec)
addQueueNotifier QueueStore {queues, notifiers} rId nId nKey = do
ifM (TM.member nId notifiers) (pure $ Left DUPLICATE_) $
withQueue rId queues $ \qVar -> do
q <- readTVar qVar
forM_ (notifier q) $ (`TM.delete` notifiers) . notifierId
writeTVar qVar q {notifier = Just ntfCreds}
TM.insert nId rId notifiers
pure $ Just q
deleteQueueNotifier :: QueueStore -> RecipientId -> STM (Either ErrorType ())
deleteQueueNotifier QueueStore {queues, notifiers} rId =
withQueue rId queues $ \qVar -> do
q <- readTVar qVar
forM_ (notifier q) $ \NtfCreds {notifierId} -> TM.delete notifierId notifiers
writeTVar qVar q {notifier = Nothing}
pure $ Just ()
withQueue rId queues $ \qVar ->
readTVar qVar >>= \q -> case notifier q of
Just _ -> pure Nothing
_ -> do
writeTVar qVar q {notifier = Just (nId, nKey)}
TM.insert nId rId notifiers
pure $ Just q
suspendQueue :: QueueStore -> RecipientId -> STM (Either ErrorType ())
suspendQueue QueueStore {queues} rId =
@@ -89,7 +82,7 @@ instance MonadQueueStore QueueStore STM where
Just qVar ->
readTVar qVar >>= \q -> do
TM.delete (senderId q) senders
forM_ (notifier q) $ \NtfCreds {notifierId} -> TM.delete notifierId notifiers
forM_ (notifier q) $ \(nId, _) -> TM.delete nId notifiers
pure $ Right ()
_ -> pure $ Left AUTH
-168
View File
@@ -1,168 +0,0 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Simplex.Messaging.Server.Stats where
import Control.Applicative (optional)
import qualified Data.Attoparsec.ByteString.Char8 as A
import qualified Data.ByteString.Char8 as B
import Data.Set (Set)
import qualified Data.Set as S
import Data.Time.Calendar.Month.Compat (pattern MonthDay)
import Data.Time.Calendar.OrdinalDate (mondayStartWeek)
import Data.Time.Clock (UTCTime (..))
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol (RecipientId)
import UnliftIO.STM
data ServerStats = ServerStats
{ fromTime :: TVar UTCTime,
qCreated :: TVar Int,
qSecured :: TVar Int,
qDeleted :: TVar Int,
msgSent :: TVar Int,
msgRecv :: TVar Int,
activeQueues :: PeriodStats RecipientId
}
data ServerStatsData = ServerStatsData
{ _fromTime :: UTCTime,
_qCreated :: Int,
_qSecured :: Int,
_qDeleted :: Int,
_msgSent :: Int,
_msgRecv :: Int,
_activeQueues :: PeriodStatsData RecipientId
}
newServerStats :: UTCTime -> STM ServerStats
newServerStats ts = do
fromTime <- newTVar ts
qCreated <- newTVar 0
qSecured <- newTVar 0
qDeleted <- newTVar 0
msgSent <- newTVar 0
msgRecv <- newTVar 0
activeQueues <- newPeriodStats
pure ServerStats {fromTime, qCreated, qSecured, qDeleted, msgSent, msgRecv, activeQueues}
getServerStatsData :: ServerStats -> STM ServerStatsData
getServerStatsData s = do
_fromTime <- readTVar $ fromTime s
_qCreated <- readTVar $ qCreated s
_qSecured <- readTVar $ qSecured s
_qDeleted <- readTVar $ qDeleted s
_msgSent <- readTVar $ msgSent s
_msgRecv <- readTVar $ msgRecv s
_activeQueues <- getPeriodStatsData $ activeQueues s
pure ServerStatsData {_fromTime, _qCreated, _qSecured, _qDeleted, _msgSent, _msgRecv, _activeQueues}
setServerStats :: ServerStats -> ServerStatsData -> STM ()
setServerStats s d = do
writeTVar (fromTime s) (_fromTime d)
writeTVar (qCreated s) (_qCreated d)
writeTVar (qSecured s) (_qSecured d)
writeTVar (qDeleted s) (_qDeleted d)
writeTVar (msgSent s) (_msgSent d)
writeTVar (msgRecv s) (_msgRecv d)
setPeriodStats (activeQueues s) (_activeQueues d)
instance StrEncoding ServerStatsData where
strEncode ServerStatsData {_fromTime, _qCreated, _qSecured, _qDeleted, _msgSent, _msgRecv, _activeQueues} =
B.unlines
[ "fromTime=" <> strEncode _fromTime,
"qCreated=" <> strEncode _qCreated,
"qSecured=" <> strEncode _qSecured,
"qDeleted=" <> strEncode _qDeleted,
"msgSent=" <> strEncode _msgSent,
"msgRecv=" <> strEncode _msgRecv,
"activeQueues:",
strEncode _activeQueues
]
strP = do
_fromTime <- "fromTime=" *> strP <* A.endOfLine
_qCreated <- "qCreated=" *> strP <* A.endOfLine
_qSecured <- "qSecured=" *> strP <* A.endOfLine
_qDeleted <- "qDeleted=" *> strP <* A.endOfLine
_msgSent <- "msgSent=" *> strP <* A.endOfLine
_msgRecv <- "msgRecv=" *> strP <* A.endOfLine
r <- optional ("activeQueues:" <* A.endOfLine)
_activeQueues <- case r of
Just _ -> strP <* optional A.endOfLine
_ -> do
_day <- "dayMsgQueues=" *> strP <* A.endOfLine
_week <- "weekMsgQueues=" *> strP <* A.endOfLine
_month <- "monthMsgQueues=" *> strP <* optional A.endOfLine
pure PeriodStatsData {_day, _week, _month}
pure ServerStatsData {_fromTime, _qCreated, _qSecured, _qDeleted, _msgSent, _msgRecv, _activeQueues}
data PeriodStats a = PeriodStats
{ day :: TVar (Set a),
week :: TVar (Set a),
month :: TVar (Set a)
}
newPeriodStats :: STM (PeriodStats a)
newPeriodStats = do
day <- newTVar S.empty
week <- newTVar S.empty
month <- newTVar S.empty
pure PeriodStats {day, week, month}
data PeriodStatsData a = PeriodStatsData
{ _day :: Set a,
_week :: Set a,
_month :: Set a
}
getPeriodStatsData :: PeriodStats a -> STM (PeriodStatsData a)
getPeriodStatsData s = do
_day <- readTVar $ day s
_week <- readTVar $ week s
_month <- readTVar $ month s
pure PeriodStatsData {_day, _week, _month}
setPeriodStats :: PeriodStats a -> PeriodStatsData a -> STM ()
setPeriodStats s d = do
writeTVar (day s) (_day d)
writeTVar (week s) (_week d)
writeTVar (month s) (_month d)
instance (Ord a, StrEncoding a) => StrEncoding (PeriodStatsData a) where
strEncode PeriodStatsData {_day, _week, _month} =
"day=" <> strEncode _day <> "\nweek=" <> strEncode _week <> "\nmonth=" <> strEncode _month
strP = do
_day <- "day=" *> strP <* A.endOfLine
_week <- "week=" *> strP <* A.endOfLine
_month <- "month=" *> strP
pure PeriodStatsData {_day, _week, _month}
data PeriodStatCounts = PeriodStatCounts
{ dayCount :: String,
weekCount :: String,
monthCount :: String
}
periodStatCounts :: forall a. PeriodStats a -> UTCTime -> STM PeriodStatCounts
periodStatCounts ps ts = do
let d = utctDay ts
(_, wDay) = mondayStartWeek d
MonthDay _ mDay = d
dayCount <- periodCount 1 $ day ps
weekCount <- periodCount wDay $ week ps
monthCount <- periodCount mDay $ month ps
pure PeriodStatCounts {dayCount, weekCount, monthCount}
where
periodCount :: Int -> TVar (Set a) -> STM String
periodCount 1 pVar = show . S.size <$> swapTVar pVar S.empty
periodCount _ _ = pure ""
updatePeriodStats :: Ord a => PeriodStats a -> a -> STM ()
updatePeriodStats stats pId = do
updatePeriod day
updatePeriod week
updatePeriod month
where
updatePeriod pSel = modifyTVar (pSel stats) (S.insert pId)
+11 -23
View File
@@ -13,12 +13,10 @@ module Simplex.Messaging.Server.StoreLog
openReadStoreLog,
storeLogFilePath,
closeStoreLog,
writeStoreLogRecord,
logCreateQueue,
logSecureQueue,
logAddNotifier,
logDeleteQueue,
logDeleteNotifier,
readWriteStoreLog,
)
where
@@ -36,7 +34,7 @@ import Data.Map.Strict (Map)
import qualified Data.Map.Strict as M
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol
import Simplex.Messaging.Server.QueueStore (NtfCreds (..), QueueRec (..), QueueStatus (..))
import Simplex.Messaging.Server.QueueStore (QueueRec (..), QueueStatus (..))
import Simplex.Messaging.Transport (trimCR)
import System.Directory (doesFileExist)
import System.IO
@@ -50,9 +48,8 @@ data StoreLog (a :: IOMode) where
data StoreLogRecord
= CreateQueue QueueRec
| SecureQueue QueueId SndPublicVerifyKey
| AddNotifier QueueId NtfCreds
| AddNotifier QueueId NotifierId NtfPublicVerifyKey
| DeleteQueue QueueId
| DeleteNotifier QueueId
instance StrEncoding QueueRec where
strEncode QueueRec {recipientId, recipientKey, rcvDhSecret, senderId, senderKey, notifier} =
@@ -65,7 +62,7 @@ instance StrEncoding QueueRec where
]
<> maybe "" notifierStr notifier
where
notifierStr ntfCreds = " notifier=" <> strEncode ntfCreds
notifierStr (nId, nKey) = " nid=" <> strEncode nId <> " nk=" <> strEncode nKey
strP = do
recipientId <- "rid=" *> strP_
@@ -73,29 +70,24 @@ instance StrEncoding QueueRec where
rcvDhSecret <- "rdh=" *> strP_
senderId <- "sid=" *> strP_
senderKey <- "sk=" *> strP
notifier <- optional $ " notifier=" *> strP
notifier <- optional $ (,) <$> (" nid=" *> strP_) <*> ("nk=" *> strP)
pure QueueRec {recipientId, recipientKey, rcvDhSecret, senderId, senderKey, notifier, status = QueueActive}
instance StrEncoding StoreLogRecord where
strEncode = \case
CreateQueue q -> strEncode (Str "CREATE", q)
SecureQueue rId sKey -> strEncode (Str "SECURE", rId, sKey)
AddNotifier rId ntfCreds -> strEncode (Str "NOTIFIER", rId, ntfCreds)
AddNotifier rId nId nKey -> strEncode (Str "NOTIFIER", rId, nId, nKey)
DeleteQueue rId -> strEncode (Str "DELETE", rId)
DeleteNotifier rId -> strEncode (Str "NDELETE", rId)
strP =
"CREATE " *> (CreateQueue <$> strP)
<|> "SECURE " *> (SecureQueue <$> strP_ <*> strP)
<|> "NOTIFIER " *> (AddNotifier <$> strP_ <*> strP)
<|> "NOTIFIER " *> (AddNotifier <$> strP_ <*> strP_ <*> strP)
<|> "DELETE " *> (DeleteQueue <$> strP)
<|> "NDELETE " *> (DeleteNotifier <$> strP)
openWriteStoreLog :: FilePath -> IO (StoreLog 'WriteMode)
openWriteStoreLog f = do
h <- openFile f WriteMode
hSetBuffering h LineBuffering
pure $ WriteStoreLog f h
openWriteStoreLog f = WriteStoreLog f <$> openFile f WriteMode
openReadStoreLog :: FilePath -> IO (StoreLog 'ReadMode)
openReadStoreLog f = do
@@ -112,7 +104,7 @@ closeStoreLog = \case
WriteStoreLog _ h -> hClose h
ReadStoreLog _ h -> hClose h
writeStoreLogRecord :: StrEncoding r => StoreLog 'WriteMode -> r -> IO ()
writeStoreLogRecord :: StoreLog 'WriteMode -> StoreLogRecord -> IO ()
writeStoreLogRecord (WriteStoreLog _ h) r = do
B.hPutStrLn h $ strEncode r
hFlush h
@@ -123,15 +115,12 @@ logCreateQueue s = writeStoreLogRecord s . CreateQueue
logSecureQueue :: StoreLog 'WriteMode -> QueueId -> SndPublicVerifyKey -> IO ()
logSecureQueue s qId sKey = writeStoreLogRecord s $ SecureQueue qId sKey
logAddNotifier :: StoreLog 'WriteMode -> QueueId -> NtfCreds -> IO ()
logAddNotifier s qId ntfCreds = writeStoreLogRecord s $ AddNotifier qId ntfCreds
logAddNotifier :: StoreLog 'WriteMode -> QueueId -> NotifierId -> NtfPublicVerifyKey -> IO ()
logAddNotifier s qId nId nKey = writeStoreLogRecord s $ AddNotifier qId nId nKey
logDeleteQueue :: StoreLog 'WriteMode -> QueueId -> IO ()
logDeleteQueue s = writeStoreLogRecord s . DeleteQueue
logDeleteNotifier :: StoreLog 'WriteMode -> QueueId -> IO ()
logDeleteNotifier s = writeStoreLogRecord s . DeleteNotifier
readWriteStoreLog :: StoreLog 'ReadMode -> IO (Map RecipientId QueueRec, StoreLog 'WriteMode)
readWriteStoreLog s@(ReadStoreLog f _) = do
qs <- readQueues s
@@ -160,8 +149,7 @@ readQueues (ReadStoreLog _ h) = LB.hGetContents h >>= returnResult . procStoreLo
procLogRecord m = \case
CreateQueue q -> M.insert (recipientId q) q m
SecureQueue qId sKey -> M.adjust (\q -> q {senderKey = Just sKey}) qId m
AddNotifier qId ntfCreds -> M.adjust (\q -> q {notifier = Just ntfCreds}) qId m
AddNotifier qId nId nKey -> M.adjust (\q -> q {notifier = Just (nId, nKey)}) qId m
DeleteQueue qId -> M.delete qId m
DeleteNotifier qId -> M.adjust (\q -> q {notifier = Nothing}) qId m
printError :: LogParsingError -> IO ()
printError (e, s) = B.putStrLn $ "Error parsing log: " <> B.pack e <> " - " <> s
+6 -15
View File
@@ -46,15 +46,12 @@ module Simplex.Messaging.Transport
-- * SMP transport
THandle (..),
TransportError (..),
HandshakeError (..),
smpServerHandshake,
smpClientHandshake,
tPutBlock,
tGetBlock,
serializeTransportError,
transportErrorP,
sendHandshake,
getHandshake,
-- * Trim trailing CR
trimCR,
@@ -96,10 +93,10 @@ smpBlockSize :: Int
smpBlockSize = 16384
supportedSMPServerVRange :: VersionRange
supportedSMPServerVRange = mkVersionRange 1 4
supportedSMPServerVRange = mkVersionRange 1 2
simplexMQVersion :: String
simplexMQVersion = "3.2.0"
simplexMQVersion = "2.3.1"
-- * Transport connection class
@@ -258,10 +255,7 @@ data THandle c = THandle
sessionId :: SessionId,
blockSize :: Int,
-- | agreed server protocol version
thVersion :: Version,
-- | send multiple transmissions in a single block
-- based on protocol and protocol version
batch :: Bool
thVersion :: Version
}
-- | TLS-unique channel binding
@@ -367,7 +361,7 @@ smpServerHandshake c kh smpVRange = do
| keyHash /= kh ->
throwE $ TEHandshake IDENTITY
| smpVersion `isCompatible` smpVRange -> do
pure $ smpThHandle th smpVersion
pure (th :: THandle c) {thVersion = smpVersion}
| otherwise -> throwE $ TEHandshake VERSION
-- | Client SMP transport handshake.
@@ -382,12 +376,9 @@ smpClientHandshake c keyHash smpVRange = do
else case smpVersionRange `compatibleVersion` smpVRange of
Just (Compatible smpVersion) -> do
sendHandshake th $ ClientHandshake {smpVersion, keyHash}
pure $ smpThHandle th smpVersion
pure (th :: THandle c) {thVersion = smpVersion}
Nothing -> throwE $ TEHandshake VERSION
smpThHandle :: forall c. THandle c -> Version -> THandle c
smpThHandle th v = (th :: THandle c) {thVersion = v, batch = v >= 4}
sendHandshake :: (Transport c, Encoding smp) => THandle c -> smp -> ExceptT TransportError IO ()
sendHandshake th = ExceptT . tPutBlock th . smpEncode
@@ -395,4 +386,4 @@ getHandshake :: (Transport c, Encoding smp) => THandle c -> ExceptT TransportErr
getHandshake th = ExceptT $ (parse smpP (TEHandshake PARSE) =<<) <$> tGetBlock th
smpTHandle :: Transport c => c -> THandle c
smpTHandle c = THandle {connection = c, sessionId = tlsUnique c, blockSize = smpBlockSize, thVersion = 0, batch = False}
smpTHandle c = THandle {connection = c, sessionId = tlsUnique c, blockSize = smpBlockSize, thVersion = 0}
+15 -124
View File
@@ -1,121 +1,44 @@
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Simplex.Messaging.Transport.Client
( runTransportClient,
runTLSTransportClient,
smpClientHandshake,
defaultSMPPort,
defaultSocksProxy,
SocksProxy,
TransportHost (..),
TransportHosts (..),
TransportHosts_ (..),
)
where
import Control.Applicative (optional)
import Control.Monad.Except
import Control.Monad.IO.Unlift
import Data.Aeson (FromJSON (..), ToJSON (..))
import qualified Data.Attoparsec.ByteString.Char8 as A
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Default (def)
import Data.List.NonEmpty (NonEmpty (..))
import qualified Data.List.NonEmpty as L
import Data.Maybe (fromMaybe)
import Data.String
import Data.Word (Word8)
import qualified Data.X509 as X
import qualified Data.X509.CertificateStore as XS
import Data.X509.Validation (Fingerprint (..))
import qualified Data.X509.Validation as XV
import GHC.IO.Exception (IOErrorType (..))
import Network.Socket
import Network.Socks5
import qualified Network.TLS as T
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers (parseAll, parseString)
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.KeepAlive
import Simplex.Messaging.Util (bshow, (<$?>))
import System.IO.Error
import Text.Read (readMaybe)
import UnliftIO.Exception (IOException)
import qualified UnliftIO.Exception as E
data TransportHost
= THIPv4 (Word8, Word8, Word8, Word8)
| THOnionHost ByteString
| THDomainName HostName
deriving (Eq, Ord, Show)
instance Encoding TransportHost where
smpEncode = smpEncode . strEncode
smpP = parseAll strP <$?> smpP
instance StrEncoding TransportHost where
strEncode = \case
THIPv4 (a1, a2, a3, a4) -> B.intercalate "." $ map bshow [a1, a2, a3, a4]
THOnionHost host -> host
THDomainName host -> B.pack host
strP =
A.choice
[ THIPv4 <$> ((,,,) <$> ipNum <*> ipNum <*> ipNum <*> A.decimal),
THOnionHost <$> ((<>) <$> A.takeTill (== '.') <*> A.string ".onion"),
THDomainName . B.unpack <$> A.takeWhile1 (A.notInClass ":#,;/ ")
]
where
ipNum = A.decimal <* A.char '.'
instance ToJSON TransportHost where
toEncoding = strToJEncoding
toJSON = strToJSON
newtype TransportHosts = TransportHosts {thList :: NonEmpty TransportHost}
instance StrEncoding TransportHosts where
strEncode = strEncodeList . L.toList . thList
strP = TransportHosts . L.fromList <$> strP `A.sepBy1'` A.char ','
newtype TransportHosts_ = TransportHosts_ {thList_ :: [TransportHost]}
instance StrEncoding TransportHosts_ where
strEncode = strEncodeList . thList_
strP = TransportHosts_ <$> strP `A.sepBy'` A.char ','
instance IsString TransportHost where fromString = parseString strDecode
instance IsString (NonEmpty TransportHost) where fromString = parseString strDecode
-- | Connect to passed TCP host:port and pass handle to the client.
runTransportClient :: (Transport c, MonadUnliftIO m) => Maybe SocksProxy -> TransportHost -> ServiceName -> Maybe C.KeyHash -> Maybe KeepAliveOpts -> (c -> m a) -> m a
runTransportClient :: (Transport c, MonadUnliftIO m) => HostName -> ServiceName -> Maybe C.KeyHash -> Maybe KeepAliveOpts -> (c -> m a) -> m a
runTransportClient = runTLSTransportClient supportedParameters Nothing
runTLSTransportClient :: (Transport c, MonadUnliftIO m) => T.Supported -> Maybe XS.CertificateStore -> Maybe SocksProxy -> TransportHost -> ServiceName -> Maybe C.KeyHash -> Maybe KeepAliveOpts -> (c -> m a) -> m a
runTLSTransportClient tlsParams caStore_ socksProxy_ host port keyHash keepAliveOpts client = do
let clientParams = mkTLSClientParams tlsParams caStore_ (B.unpack $ strEncode host) port keyHash
connectTCP = case socksProxy_ of
Just proxy -> connectSocksClient proxy $ hostAddr host
_ -> connectTCPClient . B.unpack $ strEncode host
c <- liftIO $ do
sock <- connectTCP port
mapM_ (setSocketKeepAlive sock) keepAliveOpts
connectTLS clientParams sock >>= getClientConnection
runTLSTransportClient :: (Transport c, MonadUnliftIO m) => T.Supported -> Maybe XS.CertificateStore -> HostName -> ServiceName -> Maybe C.KeyHash -> Maybe KeepAliveOpts -> (c -> m a) -> m a
runTLSTransportClient tlsParams caStore_ host port keyHash keepAliveOpts client = do
let clientParams = mkTLSClientParams tlsParams caStore_ host port keyHash
c <- liftIO $ startTCPClient host port clientParams keepAliveOpts
client c `E.finally` liftIO (closeConnection c)
where
hostAddr = \case
THIPv4 addr -> SocksAddrIPV4 $ tupleToHostAddress addr
THOnionHost h -> SocksAddrDomainName h
THDomainName h -> SocksAddrDomainName $ B.pack h
connectTCPClient :: HostName -> ServiceName -> IO Socket
connectTCPClient host port = withSocketsDo $ resolve >>= tryOpen err
startTCPClient :: forall c. Transport c => HostName -> ServiceName -> T.ClientParams -> Maybe KeepAliveOpts -> IO c
startTCPClient host port clientParams keepAliveOpts = withSocketsDo $ resolve >>= tryOpen err
where
err :: IOException
err = mkIOError NoSuchThing "no address" Nothing Nothing
@@ -125,52 +48,20 @@ connectTCPClient host port = withSocketsDo $ resolve >>= tryOpen err
let hints = defaultHints {addrSocketType = Stream}
in getAddrInfo (Just hints) (Just host) (Just port)
tryOpen :: IOException -> [AddrInfo] -> IO Socket
tryOpen :: IOException -> [AddrInfo] -> IO c
tryOpen e [] = E.throwIO e
tryOpen _ (addr : as) =
E.try (open addr) >>= either (`tryOpen` as) pure
open :: AddrInfo -> IO Socket
open :: AddrInfo -> IO c
open addr = do
sock <- socket (addrFamily addr) (addrSocketType addr) (addrProtocol addr)
connect sock $ addrAddress addr
pure sock
mapM_ (setSocketKeepAlive sock) keepAliveOpts
ctx <- connectTLS clientParams sock
getClientConnection ctx
defaultSMPPort :: PortNumber
defaultSMPPort = 5223
connectSocksClient :: SocksProxy -> SocksHostAddress -> ServiceName -> IO Socket
connectSocksClient (SocksProxy addr) hostAddr _port = do
let port = if null _port then defaultSMPPort else fromMaybe defaultSMPPort $ readMaybe _port
fst <$> socksConnect (defaultSocksConf addr) (SocksAddress hostAddr port)
defaultSocksHost :: HostAddress
defaultSocksHost = tupleToHostAddress (127, 0, 0, 1)
defaultSocksProxy :: SocksProxy
defaultSocksProxy = SocksProxy $ SockAddrInet 9050 defaultSocksHost
newtype SocksProxy = SocksProxy SockAddr
deriving (Eq)
instance Show SocksProxy where show (SocksProxy addr) = show addr
instance StrEncoding SocksProxy where
strEncode = B.pack . show
strP = do
host <- maybe defaultSocksHost tupleToHostAddress <$> optional ipv4P
port <- fromMaybe 9050 <$> optional (A.char ':' *> (fromInteger <$> A.decimal))
pure . SocksProxy $ SockAddrInet port host
where
ipv4P = (,,,) <$> ipNum <*> ipNum <*> ipNum <*> A.decimal
ipNum = A.decimal <* A.char '.'
instance ToJSON SocksProxy where
toJSON = strToJSON
toEncoding = strToJEncoding
instance FromJSON SocksProxy where
parseJSON = strParseJSON "SocksProxy"
-- readCertificateStore :: FilePath -> IO (Maybe CertificateStore)
mkTLSClientParams :: T.Supported -> Maybe XS.CertificateStore -> HostName -> ServiceName -> Maybe C.KeyHash -> T.ClientParams
mkTLSClientParams supported caStore_ host port keyHash_ = do
@@ -193,7 +84,7 @@ validateCertificateChain (C.KeyHash kh) host port cc@(X.CertificateChain sc@[_,
x509validate = XV.validate X.HashSHA256 hooks checks certStore cache serviceID cc
where
hooks = XV.defaultHooks
checks = XV.defaultChecks {XV.checkFQHN = False}
checks = XV.defaultChecks
certStore = XS.makeCertificateStore sc
cache = XV.exceptionValidationCache [] -- we manually check fingerprint only of the identity certificate (ca.crt)
serviceID = (host, port)
@@ -19,7 +19,7 @@ import qualified Network.HTTP2.Client as H
import Network.Socket (HostName, ServiceName)
import qualified Network.TLS as T
import Numeric.Natural (Natural)
import Simplex.Messaging.Transport.Client (TransportHost (..), runTLSTransportClient)
import Simplex.Messaging.Transport.Client (runTLSTransportClient)
import Simplex.Messaging.Transport.HTTP2 (http2TLSParams, withTlsConfig)
import Simplex.Messaging.Transport.KeepAlive (KeepAliveOpts)
import UnliftIO.STM
@@ -122,7 +122,7 @@ sendRequest HTTP2Client {reqQ, config} req = do
runHTTP2Client :: T.Supported -> Maybe XS.CertificateStore -> HostName -> ServiceName -> Maybe KeepAliveOpts -> ((Request -> (Response -> IO ()) -> IO ()) -> IO ()) -> IO ()
runHTTP2Client tlsParams caStore host port keepAliveOpts client =
runTLSTransportClient tlsParams caStore Nothing (THDomainName host) port Nothing keepAliveOpts $ \c ->
runTLSTransportClient tlsParams caStore host port Nothing keepAliveOpts $ \c ->
withTlsConfig c 16384 (`run` client)
where
run = H.run $ ClientConfig "https" (B.pack host) 20
+1 -8
View File
@@ -1,15 +1,10 @@
{-# LANGUAGE CApiFFI #-}
{-# LANGUAGE CPP #-}
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE NamedFieldPuns #-}
module Simplex.Messaging.Transport.KeepAlive where
import Data.Aeson (FromJSON (..), ToJSON (..))
import qualified Data.Aeson as J
import Foreign.C (CInt (..))
import GHC.Generics (Generic)
import Network.Socket
data KeepAliveOpts = KeepAliveOpts
@@ -17,9 +12,7 @@ data KeepAliveOpts = KeepAliveOpts
keepIntvl :: Int,
keepCnt :: Int
}
deriving (Eq, Show, Generic, FromJSON)
instance ToJSON KeepAliveOpts where toEncoding = J.genericToEncoding J.defaultOptions
deriving (Show)
defaultKeepAliveOpts :: KeepAliveOpts
defaultKeepAliveOpts =
+1 -1
View File
@@ -54,7 +54,7 @@ runTCPServer started port server = do
(closeServer started clients)
$ \sock -> forever . E.bracketOnError (accept sock) (close . fst) $ \(conn, _peer) -> do
-- catchAll_ is needed here in case the connection was closed earlier
cId <- atomically $ stateTVar clientId $ \cId -> let cId' = cId + 1 in (cId', cId')
cId <- atomically $ stateTVar clientId $ \cId -> (cId + 1, cId + 1)
let closeConn _ = atomically (TM.delete cId clients) >> gracefulClose conn 5000 `catchAll_` pure ()
tId <- mkWeakThreadId =<< server conn `forkFinally` closeConn
atomically $ TM.insert cId tId clients
-11
View File
@@ -1,4 +1,3 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
@@ -11,8 +10,6 @@ import Control.Monad.Trans.Except
import Data.Bifunctor (first)
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Text (Text)
import qualified Data.Text as T
import UnliftIO.Async
raceAny_ :: MonadUnliftIO m => [m a] -> m ()
@@ -35,10 +32,6 @@ bshow :: Show a => a -> ByteString
bshow = B.pack . show
{-# INLINE bshow #-}
tshow :: Show a => a -> Text
tshow = T.pack . show
{-# INLINE tshow #-}
maybeWord :: (a -> ByteString) -> Maybe a -> ByteString
maybeWord f = maybe "" $ B.cons ' ' . f
{-# INLINE maybeWord #-}
@@ -63,10 +56,6 @@ tryE :: Monad m => ExceptT e m a -> ExceptT e m (Either e a)
tryE m = (Right <$> m) `catchE` (pure . Left)
{-# INLINE tryE #-}
liftE :: (e -> e') -> ExceptT e IO a -> ExceptT e' IO a
liftE f a = ExceptT $ first f <$> runExceptT a
{-# INLINE liftE #-}
ifM :: Monad m => m Bool -> m a -> m a -> m a
ifM ba t f = ba >>= \b -> if b then t else f
{-# INLINE ifM #-}
-4
View File
@@ -15,7 +15,6 @@ module Simplex.Messaging.Version
pattern Compatible,
mkVersionRange,
safeVersionRange,
versionToRange,
isCompatible,
proveCompatible,
compatibleVersion,
@@ -54,9 +53,6 @@ safeVersionRange v1 v2
| v1 <= v2 = Just $ VRange v1 v2
| otherwise = Nothing
versionToRange :: Version -> VersionRange
versionToRange v = VRange v v
instance Encoding VersionRange where
smpEncode (VRange v1 v2) = smpEncode (v1, v2)
smpP =
+1 -1
View File
@@ -39,7 +39,7 @@ extra-deps:
- network-3.1.2.7@sha256:e3d78b13db9512aeb106e44a334ab42b7aa48d26c097299084084cb8be5c5568,4888
- simple-logger-0.1.0@sha256:be8ede4bd251a9cac776533bae7fb643369ebd826eb948a9a18df1a8dd252ff8,1079
- tls-1.6.0@sha256:7ae39373fd2de27fb80e90f76d22aeeb9a074a0ddd120cbd02c9c52f516a9e55,6987
# below dependencies are to update Aeson to 2.0.3
# below dependancies are to update Aeson to 2.0.3
- OneTuple-0.3.1@sha256:a848c096c9d29e82ffdd30a9998aa2931cbccb3a1bc137539d80f6174d31603e,2262
- attoparsec-0.14.4@sha256:79584bdada8b730cb5138fca8c35c76fbef75fc1d1e01e6b1d815a5ee9843191,5810
- hashable-1.4.0.2@sha256:0cddd0229d1aac305ea0404409c0bbfab81f075817bd74b8b2929eff58333e55,5005
+25 -24
View File
@@ -29,6 +29,8 @@ import Simplex.Messaging.Protocol (ErrorType (..), MsgBody)
import Simplex.Messaging.Transport (ATransport (..), TProxy (..), Transport (..))
import Simplex.Messaging.Util (bshow)
import System.Directory (removeFile)
import System.Posix.Files (stdFileMode)
import System.Posix.Semaphore (OpenSemFlags (..), semOpen, semPost, semThreadWait)
import System.Timeout
import Test.Hspec
@@ -42,6 +44,7 @@ agentTests (ATransport t) = do
describe "SQLite schema dump" schemaDumpTest
describe "SMP agent protocol syntax" $ syntaxTests t
describe "Establishing duplex connection" $ do
it "test semaphore" testSemaphore
it "should connect via one server and one agent" $
smpAgentTest2_1_1 $ testDuplexConnection t
it "should connect via one server and one agent (random IDs)" $
@@ -78,17 +81,9 @@ agentTests (ATransport t) = do
it "should deliver messages if one of connections has quota exceeded" $
smpAgentTest2_2_1 $ testMsgDeliveryQuotaExceeded t
tGetAgent :: Transport c => c -> IO (ATransmissionOrError 'Agent)
tGetAgent h = do
t@(_, _, cmd) <- tGet SAgent h
case cmd of
Right CONNECT {} -> tGetAgent h
Right DISCONNECT {} -> tGetAgent h
_ -> pure t
-- | receive message to handle `h`
(<#:) :: Transport c => c -> IO (ATransmissionOrError 'Agent)
(<#:) = tGetAgent
(<#:) = tGet SAgent
-- | send transmission `t` to handle `h` and get response
(#:) :: Transport c => c -> (ByteString, ByteString, ByteString) -> IO (ATransmissionOrError 'Agent)
@@ -122,19 +117,25 @@ h <#= p = (h <#:) >>= (`shouldSatisfy` p . correctTransmission)
h #:# err = tryGet `shouldReturn` ()
where
tryGet =
10000 `timeout` tGetAgent h >>= \case
10000 `timeout` tGet SAgent h >>= \case
Just _ -> error err
_ -> return ()
pattern Msg :: MsgBody -> ACommand 'Agent
pattern Msg msgBody <- MSG MsgMeta {integrity = MsgOk} _ msgBody
testSemaphore :: IO ()
testSemaphore = do
sem <- semOpen "/test" OpenSemFlags {semCreate = True, semExclusive = False} stdFileMode 1
semThreadWait sem
semPost sem
testDuplexConnection :: Transport c => TProxy c -> c -> c -> IO ()
testDuplexConnection _ alice bob = do
("1", "bob", Right (INV cReq)) <- alice #: ("1", "bob", "NEW INV")
let cReq' = strEncode cReq
bob #: ("11", "alice", "JOIN " <> cReq' <> " 14\nbob's connInfo") #> ("11", "alice", OK)
("", "bob", Right (CONF confId _ "bob's connInfo")) <- (alice <#:)
("", "bob", Right (CONF confId "bob's connInfo")) <- (alice <#:)
alice #: ("2", "bob", "LET " <> confId <> " 16\nalice's connInfo") #> ("2", "bob", OK)
bob <# ("", "alice", INFO "alice's connInfo")
bob <# ("", "alice", CON)
@@ -167,7 +168,7 @@ testDuplexConnRandomIds _ alice bob = do
("1", bobConn, Right (INV cReq)) <- alice #: ("1", "", "NEW INV")
let cReq' = strEncode cReq
("11", aliceConn, Right OK) <- bob #: ("11", "", "JOIN " <> cReq' <> " 14\nbob's connInfo")
("", bobConn', Right (CONF confId _ "bob's connInfo")) <- (alice <#:)
("", bobConn', Right (CONF confId "bob's connInfo")) <- (alice <#:)
bobConn' `shouldBe` bobConn
alice #: ("2", bobConn, "LET " <> confId <> " 16\nalice's connInfo") =#> \case ("2", c, OK) -> c == bobConn; _ -> False
bob <# ("", aliceConn, INFO "alice's connInfo")
@@ -201,9 +202,9 @@ testContactConnection _ alice bob tom = do
let cReq' = strEncode cReq
bob #: ("11", "alice", "JOIN " <> cReq' <> " 14\nbob's connInfo") #> ("11", "alice", OK)
("", "alice_contact", Right (REQ aInvId _ "bob's connInfo")) <- (alice <#:)
("", "alice_contact", Right (REQ aInvId "bob's connInfo")) <- (alice <#:)
alice #: ("2", "bob", "ACPT " <> aInvId <> " 16\nalice's connInfo") #> ("2", "bob", OK)
("", "alice", Right (CONF bConfId _ "alice's connInfo")) <- (bob <#:)
("", "alice", Right (CONF bConfId "alice's connInfo")) <- (bob <#:)
bob #: ("12", "alice", "LET " <> bConfId <> " 16\nbob's connInfo 2") #> ("12", "alice", OK)
alice <# ("", "bob", INFO "bob's connInfo 2")
alice <# ("", "bob", CON)
@@ -214,9 +215,9 @@ testContactConnection _ alice bob tom = do
bob #: ("13", "alice", "ACK 4") #> ("13", "alice", OK)
tom #: ("21", "alice", "JOIN " <> cReq' <> " 14\ntom's connInfo") #> ("21", "alice", OK)
("", "alice_contact", Right (REQ aInvId' _ "tom's connInfo")) <- (alice <#:)
("", "alice_contact", Right (REQ aInvId' "tom's connInfo")) <- (alice <#:)
alice #: ("4", "tom", "ACPT " <> aInvId' <> " 16\nalice's connInfo") #> ("4", "tom", OK)
("", "alice", Right (CONF tConfId _ "alice's connInfo")) <- (tom <#:)
("", "alice", Right (CONF tConfId "alice's connInfo")) <- (tom <#:)
tom #: ("22", "alice", "LET " <> tConfId <> " 16\ntom's connInfo 2") #> ("22", "alice", OK)
alice <# ("", "tom", INFO "tom's connInfo 2")
alice <# ("", "tom", CON)
@@ -232,11 +233,11 @@ testContactConnRandomIds _ alice bob = do
let cReq' = strEncode cReq
("11", aliceConn, Right OK) <- bob #: ("11", "", "JOIN " <> cReq' <> " 14\nbob's connInfo")
("", aliceContact', Right (REQ aInvId _ "bob's connInfo")) <- (alice <#:)
("", aliceContact', Right (REQ aInvId "bob's connInfo")) <- (alice <#:)
aliceContact' `shouldBe` aliceContact
("2", bobConn, Right OK) <- alice #: ("2", "", "ACPT " <> aInvId <> " 16\nalice's connInfo")
("", aliceConn', Right (CONF bConfId _ "alice's connInfo")) <- (bob <#:)
("", aliceConn', Right (CONF bConfId "alice's connInfo")) <- (bob <#:)
aliceConn' `shouldBe` aliceConn
bob #: ("12", aliceConn, "LET " <> bConfId <> " 16\nbob's connInfo 2") #> ("12", aliceConn, OK)
@@ -254,7 +255,7 @@ testRejectContactRequest _ alice bob = do
("1", "a_contact", Right (INV cReq)) <- alice #: ("1", "a_contact", "NEW CON")
let cReq' = strEncode cReq
bob #: ("11", "alice", "JOIN " <> cReq' <> " 10\nbob's info") #> ("11", "alice", OK)
("", "a_contact", Right (REQ aInvId _ "bob's info")) <- (alice <#:)
("", "a_contact", Right (REQ aInvId "bob's info")) <- (alice <#:)
-- RJCT must use correct contact connection
alice #: ("2a", "bob", "RJCT " <> aInvId) #> ("2a", "bob", ERR $ CONN NOT_FOUND)
alice #: ("2b", "a_contact", "RJCT " <> aInvId) #> ("2b", "a_contact", OK)
@@ -333,11 +334,11 @@ testServerConnectionAfterError t _ = do
bob #: ("1", "alice", "SUB") #> ("1", "alice", ERR (BROKER NETWORK))
alice #: ("1", "bob", "SUB") #> ("1", "bob", ERR (BROKER NETWORK))
withServer $ do
alice <#= \case ("", "bob", SENT 4) -> True; ("", "", UP s ["bob"]) -> s == server; _ -> False
alice <#= \case ("", "bob", SENT 4) -> True; ("", "", UP s ["bob"]) -> s == server; _ -> False
alice <# ("", "bob", SENT 4)
bob <# ("", "", UP server ["alice"])
bob <#= \case ("", "alice", Msg "hello") -> True; _ -> False
bob #: ("2", "alice", "ACK 4") #> ("2", "alice", OK)
alice <# ("", "", UP server ["bob"])
alice #: ("1", "bob", "SEND F 11\nhello again") #> ("1", "bob", MID 5)
alice <# ("", "bob", SENT 5)
bob <#= \case ("", "alice", Msg "hello again") -> True; _ -> False
@@ -395,7 +396,7 @@ testConcurrentMsgDelivery _ alice bob = do
("1", "bob2", Right (INV cReq)) <- alice #: ("1", "bob2", "NEW INV")
let cReq' = strEncode cReq
bob #: ("11", "alice2", "JOIN " <> cReq' <> " 14\nbob's connInfo") #> ("11", "alice2", OK)
("", "bob2", Right (CONF _confId _ "bob's connInfo")) <- (alice <#:)
("", "bob2", Right (CONF _confId "bob's connInfo")) <- (alice <#:)
-- below commands would be needed to accept bob's connection, but alice does not
-- alice #: ("2", "bob", "LET " <> _confId <> " 16\nalice's connInfo") #> ("2", "bob", OK)
-- bob <# ("", "alice", INFO "alice's connInfo")
@@ -434,7 +435,7 @@ connect (h1, name1) (h2, name2) = do
("c1", _, Right (INV cReq)) <- h1 #: ("c1", name2, "NEW INV")
let cReq' = strEncode cReq
h2 #: ("c2", name1, "JOIN " <> cReq' <> " 5\ninfo2") #> ("c2", name1, OK)
("", _, Right (CONF connId _ "info2")) <- (h1 <#:)
("", _, Right (CONF connId "info2")) <- (h1 <#:)
h1 #: ("c3", name2, "LET " <> connId <> " 5\ninfo1") #> ("c3", name2, OK)
h2 <# ("", name1, INFO "info1")
h2 <# ("", name1, CON)
@@ -455,7 +456,7 @@ sendMessage (h1, name1) (h2, name2) msg = do
-- ("c1", conn2, Right (INV cReq)) <- h1 #: ("c1", "", "NEW INV")
-- let cReq' = strEncode cReq
-- ("c2", conn1, Right OK) <- h2 #: ("c2", "", "JOIN " <> cReq' <> " 5\ninfo2")
-- ("", _, Right (REQ connId _ "info2")) <- (h1 <#:)
-- ("", _, Right (REQ connId "info2")) <- (h1 <#:)
-- h1 #: ("c3", conn2, "ACPT " <> connId <> " 5\ninfo1") =#> \case ("c3", c, OK) -> c == conn2; _ -> False
-- h2 <# ("", conn1, INFO "info1")
-- h2 <# ("", conn1, CON)
+32 -35
View File
@@ -11,7 +11,7 @@ import Simplex.Messaging.Agent.Protocol
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Crypto.Ratchet
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol (ProtocolServer (..), supportedSMPClientVRange)
import Simplex.Messaging.Protocol (ProtocolServer (..), smpClientVRange)
import Simplex.Messaging.Version
import Test.Hspec
@@ -19,24 +19,21 @@ uri :: String
uri = "smp.simplex.im"
srv :: SMPServer
srv = SMPServer "smp.simplex.im" "5223" (C.KeyHash "\215m\248\251")
queueAddr :: SMPQueueAddress
queueAddr =
SMPQueueAddress
{ smpServer = srv,
senderId = "\223\142z\251",
dhPublicKey = testDhKey
srv =
ProtocolServer
{ host = "smp.simplex.im",
port = "5223",
keyHash = C.KeyHash "\215m\248\251"
}
queueAddrNoPort :: SMPQueueAddress
queueAddrNoPort = queueAddr {smpServer = srv {port = ""}}
queue :: SMPQueueUri
queue = SMPQueueUri supportedSMPClientVRange queueAddr
queueV1 :: SMPQueueUri
queueV1 = SMPQueueUri (mkVersionRange 1 1) queueAddr
queue =
SMPQueueUri
{ smpServer = srv,
senderId = "\223\142z\251",
clientVRange = smpClientVRange,
dhPublicKey = testDhKey
}
testDhKey :: C.PublicKeyX25519
testDhKey = "MCowBQYDK2VuAyEAjiswwI3O/NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o="
@@ -52,7 +49,7 @@ connReqData =
ConnReqUriData
{ crScheme = simplexChat,
crAgentVRange = mkVersionRange 1 1,
crSmpQueues = [queueV1]
crSmpQueues = [queue]
}
testDhPubKey :: C.PublicKeyX448
@@ -73,40 +70,40 @@ connectionRequest12 :: AConnectionRequestUri
connectionRequest12 =
ACR SCMInvitation $
CRInvitationUri
connReqData {crAgentVRange = supportedSMPAgentVRange, crSmpQueues = [queueV1, queueV1]}
connReqData {crAgentVRange = supportedSMPAgentVRange, crSmpQueues = [queue, queue]}
testE2ERatchetParams13
connectionRequestTests :: Spec
connectionRequestTests =
describe "connection request parsing / serializing" $ do
it "should serialize SMP queue URIs" $ do
strEncode (queue :: SMPQueueUri) {queueAddress = queueAddrNoPort}
`shouldBe` "smp://[email protected]/3456-w==#/?v=1-2&dh=" <> testDhKeyStrUri
strEncode (queue :: SMPQueueUri) {smpServer = srv {port = ""}}
`shouldBe` "smp://[email protected]/3456-w==#" <> testDhKeyStr
strEncode queue {clientVRange = mkVersionRange 1 2}
`shouldBe` "smp://[email protected]:5223/3456-w==#/?v=1-2&dh=" <> testDhKeyStrUri
`shouldBe` "smp://[email protected]:5223/3456-w==#" <> testDhKeyStr
it "should parse SMP queue URIs" $ do
strDecode ("smp://[email protected]/3456-w==#/?v=1-2&dh=" <> testDhKeyStr)
`shouldBe` Right (queue :: SMPQueueUri) {queueAddress = queueAddrNoPort}
strDecode ("smp://[email protected]/3456-w==#/?v=1&dh=" <> testDhKeyStr)
`shouldBe` Right (queue :: SMPQueueUri) {smpServer = srv {port = ""}}
strDecode ("smp://[email protected]/3456-w==#" <> testDhKeyStr)
`shouldBe` Right (queueV1 :: SMPQueueUri) {queueAddress = queueAddrNoPort}
`shouldBe` Right (queue :: SMPQueueUri) {smpServer = srv {port = ""}}
strDecode ("smp://[email protected]:5223/3456-w==#" <> testDhKeyStr)
`shouldBe` Right queueV1
strDecode ("smp://[email protected]:5223/3456-w==#" <> testDhKeyStr <> "/?v=1-2&extra_param=abc")
`shouldBe` Right queue
strDecode ("smp://[email protected]:5223/3456-w==#/?extra_param=abc&v=1&dh=" <> testDhKeyStr)
`shouldBe` Right queueV1
strDecode ("smp://[email protected]:5223/3456-w==#" <> testDhKeyStr <> "/?v=1&extra_param=abc")
`shouldBe` Right queueV1
`shouldBe` Right queue
strDecode ("smp://[email protected]:5223/3456-w==#/?extra_param=abc&v=1-2&dh=" <> testDhKeyStr)
`shouldBe` Right queue {clientVRange = mkVersionRange 1 2}
strDecode ("smp://[email protected]:5223/3456-w==#" <> testDhKeyStr <> "/?v=1-2&extra_param=abc")
`shouldBe` Right queue {clientVRange = mkVersionRange 1 2}
it "should serialize connection requests" $ do
strEncode connectionRequest
`shouldBe` "https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23%2F%3Fv%3D1%26dh%3D"
<> urlEncode True testDhKeyStrUri
`shouldBe` "https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23"
<> testDhKeyStrUri
<> "&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D"
strEncode connectionRequest12
`shouldBe` "https://simplex.chat/invitation#/?v=1-2&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23%2F%3Fv%3D1%26dh%3D"
<> urlEncode True testDhKeyStrUri
<> "%2Csmp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23%2F%3Fv%3D1%26dh%3D"
<> urlEncode True testDhKeyStrUri
`shouldBe` "https://simplex.chat/invitation#/?v=1-2&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23"
<> testDhKeyStrUri
<> "%2Csmp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23"
<> testDhKeyStrUri
<> "&e2e=v%3D1-3%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D"
it "should parse connection requests" $ do
strDecode
+27 -183
View File
@@ -1,5 +1,4 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
@@ -9,7 +8,6 @@
module AgentTests.FunctionalAPITests
( functionalAPITests,
makeConnection,
get,
(##>),
(=##>),
@@ -17,20 +15,15 @@ module AgentTests.FunctionalAPITests
)
where
import Control.Concurrent (killThread, threadDelay)
import Control.Monad
import Control.Concurrent (threadDelay)
import Control.Monad.Except (ExceptT, runExceptT)
import Control.Monad.IO.Unlift
import Data.Int (Int64)
import qualified Data.Map as M
import qualified Data.Set as S
import Data.Time.Clock.System (SystemTime (..), getSystemTime)
import SMPAgentClient
import SMPClient (cfg, testPort, testPort2, testStoreLogFile2, withSmpServer, withSmpServerConfigOn, withSmpServerStoreLogOn, withSmpServerStoreMsgLogOn)
import SMPClient (cfg, testPort, withSmpServer, withSmpServerConfigOn, withSmpServerStoreLogOn, withSmpServerStoreMsgLogOn)
import Simplex.Messaging.Agent
import Simplex.Messaging.Agent.Env.SQLite (AgentConfig (..))
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Client (ProtocolClientConfig (..))
import Simplex.Messaging.Protocol (ErrorType (..), MsgBody)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Server.Env.STM (ServerConfig (..))
@@ -47,21 +40,13 @@ a ##> t = a >>= \t' -> liftIO (t' `shouldBe` t)
a =##> p = a >>= \t -> liftIO (t `shouldSatisfy` p)
get :: MonadIO m => AgentClient -> m (ATransmission 'Agent)
get c = do
t@(_, _, cmd) <- atomically (readTBQueue $ subQ c)
case cmd of
CONNECT {} -> get c
DISCONNECT {} -> get c
_ -> pure t
get c = atomically (readTBQueue $ subQ c)
pattern Msg :: MsgBody -> ACommand 'Agent
pattern Msg msgBody <- MSG MsgMeta {integrity = MsgOk} _ msgBody
smpCfgV1 :: ProtocolClientConfig
smpCfgV1 = (smpCfg agentCfg) {smpServerVRange = mkVersionRange 1 1}
agentCfgV1 :: AgentConfig
agentCfgV1 = agentCfg {smpAgentVRange = mkVersionRange 1 1, smpClientVRange = mkVersionRange 1 1, smpCfg = smpCfgV1}
agentCfgV1 = agentCfg {smpAgentVersion = 1, smpAgentVRange = mkVersionRange 1 1}
functionalAPITests :: ATransport -> Spec
functionalAPITests t = do
@@ -104,16 +89,6 @@ functionalAPITests t = do
testInactiveClientDisconnected t
it "should NOT disconnect active clients" $
testActiveClientNotDisconnected t
describe "Suspending agent" $ do
it "should update client when agent is suspended" $
withSmpServer t testSuspendingAgent
it "should complete sending messages when agent is suspended" $
testSuspendingAgentCompleteSending t
it "should suspend agent on timeout, even if pending messages not sent" $
testSuspendingAgentTimeout t
describe "Batching SMP commands" $ do
it "should subscribe to multiple subscriptions with batching" $
testBatchedSubscriptions t
testAgentClient :: IO ()
testAgentClient = do
@@ -168,7 +143,7 @@ runAgentClientTest alice bob baseId = do
Right () <- runExceptT $ do
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, CONF confId _ "bob's connInfo") <- get alice
("", _, CONF confId "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, INFO "alice's connInfo")
@@ -204,9 +179,9 @@ runAgentClientContactTest alice bob baseId = do
Right () <- runExceptT $ do
(_, qInfo) <- createConnection alice SCMContact
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, REQ invId _ "bob's connInfo") <- get alice
("", _, REQ invId "bob's connInfo") <- get alice
bobId <- acceptContact alice invId "alice's connInfo"
("", _, CONF confId _ "alice's connInfo") <- get bob
("", _, CONF confId "alice's connInfo") <- get bob
allowConnection bob aliceId confId "bob's connInfo"
get alice ##> ("", bobId, INFO "bob's connInfo")
get alice ##> ("", bobId, CON)
@@ -255,7 +230,7 @@ testAsyncInitiatingOffline = do
aliceId <- joinConnection bob cReq "bob's connInfo"
alice' <- liftIO $ getSMPAgentClient agentCfg initAgentServers
subscribeConnection alice' bobId
("", _, CONF confId _ "bob's connInfo") <- get alice'
("", _, CONF confId "bob's connInfo") <- get alice'
allowConnection alice' bobId confId "alice's connInfo"
get alice' ##> ("", bobId, CON)
get bob ##> ("", aliceId, INFO "alice's connInfo")
@@ -271,7 +246,7 @@ testAsyncJoiningOfflineBeforeActivation = do
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
disconnectAgentClient bob
("", _, CONF confId _ "bob's connInfo") <- get alice
("", _, CONF confId "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
bob' <- liftIO $ getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
subscribeConnection bob' aliceId
@@ -292,7 +267,7 @@ testAsyncBothOffline = do
disconnectAgentClient bob
alice' <- liftIO $ getSMPAgentClient agentCfg initAgentServers
subscribeConnection alice' bobId
("", _, CONF confId _ "bob's connInfo") <- get alice'
("", _, CONF confId "bob's connInfo") <- get alice'
allowConnection alice' bobId confId "alice's connInfo"
bob' <- liftIO $ getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
subscribeConnection bob' aliceId
@@ -321,7 +296,7 @@ testAsyncServerOffline t = do
srv1 `shouldBe` testSMPServer
conns1 `shouldBe` [bobId]
aliceId <- joinConnection bob cReq "bob's connInfo"
("", _, CONF confId _ "bob's connInfo") <- get alice
("", _, CONF confId "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, INFO "alice's connInfo")
@@ -346,11 +321,18 @@ testDuplicateMessage t = do
alice <- getSMPAgentClient agentCfg initAgentServers
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
(aliceId, bobId, bob1) <- withSmpServerStoreMsgLogOn t testPort $ \_ -> do
Right (aliceId, bobId) <- runExceptT $ makeConnection alice bob
Right () <- runExceptT $ do
Right (aliceId, bobId) <- runExceptT $ do
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, CONF confId "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, INFO "alice's connInfo")
get bob ##> ("", aliceId, CON)
4 <- sendMessage alice bobId SMP.noMsgFlags "hello"
get alice ##> ("", bobId, SENT 4)
get bob =##> \case ("", c, Msg "hello") -> c == aliceId; _ -> False
pure (aliceId, bobId)
disconnectAgentClient bob
-- if the agent user did not send ACK, the message will be delivered again
@@ -390,17 +372,6 @@ testDuplicateMessage t = do
get bob2 =##> \case ("", c, Msg "hello 3") -> c == aliceId; _ -> False
pure ()
makeConnection :: AgentClient -> AgentClient -> ExceptT AgentErrorType IO (ConnId, ConnId)
makeConnection alice bob = do
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, CONF confId _ "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, INFO "alice's connInfo")
get bob ##> ("", aliceId, CON)
pure (aliceId, bobId)
testInactiveClientDisconnected :: ATransport -> IO ()
testInactiveClientDisconnected t = do
let cfg' = cfg {inactiveClientExpiration = Just ExpirationConfig {ttl = 1, checkInterval = 1}}
@@ -439,140 +410,13 @@ testActiveClientNotDisconnected t = do
get alice ##> ("", "", DOWN testSMPServer [connId])
milliseconds ts = systemSeconds ts * 1000 + fromIntegral (systemNanoseconds ts `div` 1000000)
testSuspendingAgent :: IO ()
testSuspendingAgent = do
a <- getSMPAgentClient agentCfg initAgentServers
b <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right () <- runExceptT $ do
(aId, bId) <- makeConnection a b
4 <- sendMessage a bId SMP.noMsgFlags "hello"
get a ##> ("", bId, SENT 4)
get b =##> \case ("", c, Msg "hello") -> c == aId; _ -> False
ackMessage b aId 4
suspendAgent b 1000000
get b ##> ("", "", SUSPENDED)
5 <- sendMessage a bId SMP.noMsgFlags "hello 2"
get a ##> ("", bId, SENT 5)
Nothing <- 100000 `timeout` get b
activateAgent b
get b =##> \case ("", c, Msg "hello 2") -> c == aId; _ -> False
pure ()
testSuspendingAgentCompleteSending :: ATransport -> IO ()
testSuspendingAgentCompleteSending t = do
a <- getSMPAgentClient agentCfg initAgentServers
b <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right (aId, bId) <- withSmpServerStoreLogOn t testPort $ \_ -> runExceptT $ do
(aId, bId) <- makeConnection a b
4 <- sendMessage a bId SMP.noMsgFlags "hello"
get a ##> ("", bId, SENT 4)
get b =##> \case ("", c, Msg "hello") -> c == aId; _ -> False
ackMessage b aId 4
pure (aId, bId)
Right () <- runExceptT $ do
("", "", DOWN {}) <- get a
("", "", DOWN {}) <- get b
5 <- sendMessage b aId SMP.noMsgFlags "hello too"
6 <- sendMessage b aId SMP.noMsgFlags "how are you?"
liftIO $ threadDelay 100000
suspendAgent b 5000000
Right () <- withSmpServerStoreLogOn t testPort $ \_ -> runExceptT $ do
get b =##> \case ("", c, SENT 5) -> c == aId; ("", "", UP {}) -> True; _ -> False
get b =##> \case ("", c, SENT 5) -> c == aId; ("", "", UP {}) -> True; _ -> False
get b =##> \case ("", c, SENT 6) -> c == aId; ("", "", UP {}) -> True; _ -> False
("", "", SUSPENDED) <- get b
("", "", UP {}) <- get a
get a =##> \case ("", c, Msg "hello too") -> c == bId; _ -> False
ackMessage a bId 5
get a =##> \case ("", c, Msg "how are you?") -> c == bId; _ -> False
ackMessage a bId 6
pure ()
testSuspendingAgentTimeout :: ATransport -> IO ()
testSuspendingAgentTimeout t = do
a <- getSMPAgentClient agentCfg initAgentServers
b <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right (aId, _) <- withSmpServer t . runExceptT $ do
(aId, bId) <- makeConnection a b
4 <- sendMessage a bId SMP.noMsgFlags "hello"
get a ##> ("", bId, SENT 4)
get b =##> \case ("", c, Msg "hello") -> c == aId; _ -> False
ackMessage b aId 4
pure (aId, bId)
Right () <- runExceptT $ do
("", "", DOWN {}) <- get a
("", "", DOWN {}) <- get b
5 <- sendMessage b aId SMP.noMsgFlags "hello too"
6 <- sendMessage b aId SMP.noMsgFlags "how are you?"
suspendAgent b 100000
("", "", SUSPENDED) <- get b
pure ()
pure ()
testBatchedSubscriptions :: ATransport -> IO ()
testBatchedSubscriptions t = do
a <- getSMPAgentClient agentCfg initAgentServers2
b <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers2
Right conns <- runServers $ do
conns <- forM [1 .. 200 :: Int] . const $ makeConnection a b
forM_ conns $ \(aId, bId) -> exchangeGreetings a bId b aId
forM_ (take 10 conns) $ \(aId, bId) -> do
deleteConnection a bId
deleteConnection b aId
liftIO $ threadDelay 1000000
pure conns
("", "", DOWN {}) <- get a
("", "", DOWN {}) <- get a
("", "", DOWN {}) <- get b
("", "", DOWN {}) <- get b
Right () <- runServers $ do
("", "", UP {}) <- get a
("", "", UP {}) <- get a
("", "", UP {}) <- get b
("", "", UP {}) <- get b
liftIO $ threadDelay 1000000
subscribe a $ map snd conns
subscribe b $ map fst conns
forM_ (drop 10 conns) $ \(aId, bId) -> exchangeGreetingsMsgId 6 a bId b aId
pure ()
where
subscribe :: AgentClient -> [ConnId] -> ExceptT AgentErrorType IO ()
subscribe c cs = do
r <- subscribeConnections c cs
liftIO $ do
let dc = S.fromList $ take 10 cs
all (== Right ()) (M.withoutKeys r dc) `shouldBe` True
all (== Left (CONN NOT_FOUND)) (M.restrictKeys r dc) `shouldBe` True
M.keys r `shouldMatchList` cs
runServers :: ExceptT AgentErrorType IO a -> IO (Either AgentErrorType a)
runServers a = do
withSmpServerStoreLogOn t testPort $ \t1 -> do
res <- withSmpServerConfigOn t cfg {storeLogFile = Just testStoreLogFile2} testPort2 $ \t2 -> do
res <- runExceptT a
killThread t2
pure res
killThread t1
pure res
exchangeGreetings :: AgentClient -> ConnId -> AgentClient -> ConnId -> ExceptT AgentErrorType IO ()
exchangeGreetings = exchangeGreetingsMsgId 4
exchangeGreetingsMsgId :: Int64 -> AgentClient -> ConnId -> AgentClient -> ConnId -> ExceptT AgentErrorType IO ()
exchangeGreetingsMsgId msgId alice bobId bob aliceId = do
msgId1 <- sendMessage alice bobId SMP.noMsgFlags "hello"
liftIO $ msgId1 `shouldBe` msgId
get alice ##> ("", bobId, SENT msgId)
exchangeGreetings alice bobId bob aliceId = do
4 <- sendMessage alice bobId SMP.noMsgFlags "hello"
get alice ##> ("", bobId, SENT 4)
get bob =##> \case ("", c, Msg "hello") -> c == aliceId; _ -> False
ackMessage bob aliceId msgId
msgId2 <- sendMessage bob aliceId SMP.noMsgFlags "hello too"
let msgId' = msgId + 1
liftIO $ msgId2 `shouldBe` msgId'
get bob ##> ("", aliceId, SENT msgId')
ackMessage bob aliceId 4
5 <- sendMessage bob aliceId SMP.noMsgFlags "hello too"
get bob ##> ("", aliceId, SENT 5)
get alice =##> \case ("", c, Msg "hello too") -> c == bobId; _ -> False
ackMessage alice bobId msgId'
ackMessage alice bobId 5
+73 -291
View File
@@ -9,42 +9,35 @@ module AgentTests.NotificationTests where
-- import Control.Logger.Simple (LogConfig (..), LogLevel (..), setLogLevel, withGlobalLogging)
import AgentTests.FunctionalAPITests (get, makeConnection, (##>), (=##>), pattern Msg)
import Control.Concurrent (killThread, threadDelay)
import AgentTests.FunctionalAPITests (get, (##>), (=##>), pattern Msg)
import Control.Concurrent (threadDelay)
import Control.Monad.Except
import qualified Data.Aeson as J
import qualified Data.Aeson.Types as JT
import Data.Bifunctor (bimap, first)
import Data.Bifunctor (bimap)
import qualified Data.ByteString.Base64.URL as U
import Data.ByteString.Char8 (ByteString)
import Data.Text.Encoding (encodeUtf8)
import NtfClient
import SMPAgentClient (agentCfg, initAgentServers, testDB, testDB2)
import SMPClient (testPort, withSmpServer, withSmpServerStoreLogOn)
import SMPClient (withSmpServer)
import Simplex.Messaging.Agent
import Simplex.Messaging.Agent.Env.SQLite (AgentConfig (..))
import Simplex.Messaging.Agent.Protocol
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Push.APNS
import Simplex.Messaging.Notifications.Types (NtfToken (..))
import Simplex.Messaging.Protocol (ErrorType (AUTH), MsgFlags (MsgFlags), SMPMsgMeta (..))
import Simplex.Messaging.Protocol (ErrorType (AUTH))
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Transport (ATransport)
import Simplex.Messaging.Util (tryE)
import System.Directory (doesFileExist, removeFile)
import System.Directory (removeFile)
import Test.Hspec
import UnliftIO
removeFileIfExists :: FilePath -> IO ()
removeFileIfExists filePath = do
fileExists <- doesFileExist filePath
when fileExists $ removeFile filePath
notificationTests :: ATransport -> Spec
notificationTests t =
after_ (removeFile testDB >> removeFileIfExists testDB2) $ do
after_ (removeFile testDB) $ do
describe "Managing notification tokens" $ do
it "should register and verify notification token" $
withAPNSMockServer $ \apns ->
@@ -67,36 +60,20 @@ notificationTests t =
withSmpServer t $
withAPNSMockServer $ \apns ->
withNtfServer t $ testNotificationSubscriptionNewConnection apns
it "should change notifications mode" $ \_ ->
withSmpServer t $
withAPNSMockServer $ \apns ->
withNtfServer t $ testChangeNotificationsMode apns
it "should change token" $ \_ ->
withSmpServer t $
withAPNSMockServer $ \apns ->
withNtfServer t $ testChangeToken apns
describe "Notifications server store log" $
it "should save and restore tokens and subscriptions" $ \_ ->
withSmpServer t $
withAPNSMockServer $ \apns ->
testNotificationsStoreLog t apns
describe "Notifications after SMP server restart" $
it "should resume subscriptions after SMP server is restarted" $ \_ ->
withAPNSMockServer $ \apns ->
withNtfServer t $ testNotificationsSMPRestart t apns
testNotificationToken :: APNSMockServer -> IO ()
testNotificationToken APNSMockServer {apnsQ} = do
a <- getSMPAgentClient agentCfg initAgentServers
Right () <- runExceptT $ do
let tkn = DeviceToken PPApnsTest "abcd"
NTRegistered <- registerNtfToken a tkn NMPeriodic
let tkn = DeviceToken PPApns "abcd"
NTRegistered <- registerNtfToken a tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse} <-
atomically $ readTBQueue apnsQ
verification <- ntfData .-> "verification"
nonce <- C.cbNonce <$> ntfData .-> "nonce"
liftIO $ sendApnsResponse APNSRespOk
verifyNtfToken a tkn nonce verification
verifyNtfToken a tkn verification nonce
enableNtfCron a tkn 30
NTActive <- checkNtfToken a tkn
deleteNtfToken a tkn
-- agent deleted this token
@@ -118,21 +95,22 @@ testNtfTokenRepeatRegistration APNSMockServer {apnsQ} = do
-- withGlobalLogging logCfg $ do
a <- getSMPAgentClient agentCfg initAgentServers
Right () <- runExceptT $ do
let tkn = DeviceToken PPApnsTest "abcd"
NTRegistered <- registerNtfToken a tkn NMPeriodic
let tkn = DeviceToken PPApns "abcd"
NTRegistered <- registerNtfToken a tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse} <-
atomically $ readTBQueue apnsQ
verification <- ntfData .-> "verification"
nonce <- C.cbNonce <$> ntfData .-> "nonce"
liftIO $ sendApnsResponse APNSRespOk
NTRegistered <- registerNtfToken a tkn NMPeriodic
NTRegistered <- registerNtfToken a tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData'}, sendApnsResponse = sendApnsResponse'} <-
atomically $ readTBQueue apnsQ
_ <- ntfData' .-> "verification"
_ <- C.cbNonce <$> ntfData' .-> "nonce"
liftIO $ sendApnsResponse' APNSRespOk
-- can still use the first verification code, it is the same after decryption
verifyNtfToken a tkn nonce verification
verifyNtfToken a tkn verification nonce
enableNtfCron a tkn 30
NTActive <- checkNtfToken a tkn
pure ()
pure ()
@@ -144,16 +122,16 @@ testNtfTokenSecondRegistration APNSMockServer {apnsQ} = do
a <- getSMPAgentClient agentCfg initAgentServers
a' <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right () <- runExceptT $ do
let tkn = DeviceToken PPApnsTest "abcd"
NTRegistered <- registerNtfToken a tkn NMPeriodic
let tkn = DeviceToken PPApns "abcd"
NTRegistered <- registerNtfToken a tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse} <-
atomically $ readTBQueue apnsQ
verification <- ntfData .-> "verification"
nonce <- C.cbNonce <$> ntfData .-> "nonce"
liftIO $ sendApnsResponse APNSRespOk
verifyNtfToken a tkn nonce verification
verifyNtfToken a tkn verification nonce
NTRegistered <- registerNtfToken a' tkn NMPeriodic
NTRegistered <- registerNtfToken a' tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData'}, sendApnsResponse = sendApnsResponse'} <-
atomically $ readTBQueue apnsQ
verification' <- ntfData' .-> "verification"
@@ -164,22 +142,25 @@ testNtfTokenSecondRegistration APNSMockServer {apnsQ} = do
NTActive <- checkNtfToken a tkn
-- and the second is not yet verified
liftIO $ threadDelay 50000
NTConfirmed <- checkNtfToken a' tkn
r <- checkNtfToken a' tkn
liftIO . print $ "checkNtfToken a' tkn: " <> show r
unless (r == NTConfirmed) $ error "bad status"
-- now the second token registration is verified
verifyNtfToken a' tkn nonce' verification'
verifyNtfToken a' tkn verification' nonce'
-- the first registration is removed
Left (NTF AUTH) <- tryE $ checkNtfToken a tkn
-- and the second is active
NTActive <- checkNtfToken a' tkn
enableNtfCron a' tkn 30
pure ()
pure ()
testNtfTokenServerRestart :: ATransport -> APNSMockServer -> IO ()
testNtfTokenServerRestart t APNSMockServer {apnsQ} = do
a <- getSMPAgentClient agentCfg initAgentServers
let tkn = DeviceToken PPApnsTest "abcd"
let tkn = DeviceToken PPApns "abcd"
Right ntfData <- withNtfServer t . runExceptT $ do
NTRegistered <- registerNtfToken a tkn NMPeriodic
NTRegistered <- registerNtfToken a tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse} <-
atomically $ readTBQueue apnsQ
liftIO $ sendApnsResponse APNSRespOk
@@ -193,68 +174,51 @@ testNtfTokenServerRestart t APNSMockServer {apnsQ} = do
Right () <- withNtfServer t . runExceptT $ do
verification <- ntfData .-> "verification"
nonce <- C.cbNonce <$> ntfData .-> "nonce"
Left (NTF AUTH) <- tryE $ verifyNtfToken a' tkn nonce verification
Left (NTF AUTH) <- tryE $ verifyNtfToken a' tkn verification nonce
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData'}, sendApnsResponse = sendApnsResponse'} <-
atomically $ readTBQueue apnsQ
verification' <- ntfData' .-> "verification"
nonce' <- C.cbNonce <$> ntfData' .-> "nonce"
liftIO $ sendApnsResponse' APNSRespOk
verifyNtfToken a' tkn nonce' verification'
verifyNtfToken a' tkn verification' nonce'
NTActive <- checkNtfToken a' tkn
pure ()
enableNtfCron a' tkn 30
pure ()
testNotificationSubscriptionExistingConnection :: APNSMockServer -> IO ()
testNotificationSubscriptionExistingConnection APNSMockServer {apnsQ} = do
alice <- getSMPAgentClient agentCfg initAgentServers
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right (bobId, aliceId, nonce, message) <- runExceptT $ do
Right () <- runExceptT $ do
-- establish connection
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, CONF confId _ "bob's connInfo") <- get alice
("", _, CONF confId "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get bob ##> ("", aliceId, INFO "alice's connInfo")
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, CON)
-- register notification token
let tkn = DeviceToken PPApnsTest "abcd"
NTRegistered <- registerNtfToken alice tkn NMInstant
let tkn = DeviceToken PPApns "abcd"
NTRegistered <- registerNtfToken alice tkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse} <-
atomically $ readTBQueue apnsQ
verification <- ntfData .-> "verification"
vNonce <- C.cbNonce <$> ntfData .-> "nonce"
nonce <- C.cbNonce <$> ntfData .-> "nonce"
liftIO $ sendApnsResponse APNSRespOk
verifyNtfToken alice tkn vNonce verification
verifyNtfToken alice tkn verification nonce
NTActive <- checkNtfToken alice tkn
-- send message
liftIO $ threadDelay 50000
1 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello"
get bob ##> ("", aliceId, SENT $ baseId + 1)
-- notification
(nonce, message) <- messageNotification apnsQ
pure (bobId, aliceId, nonce, message)
-- alice client already has subscription for the connection
Left (CMD PROHIBITED) <- runExceptT $ getNotificationMessage alice nonce message
-- aliceNtf client doesn't have subscription and is allowed to get notification message
aliceNtf <- getSMPAgentClient agentCfg initAgentServers
Right () <- runExceptT $ do
(_, [SMPMsgMeta {msgFlags = MsgFlags True}]) <- getNotificationMessage aliceNtf nonce message
pure ()
disconnectAgentClient aliceNtf
Right () <- runExceptT $ do
messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 1
-- delete notification subscription
deleteNtfSub alice bobId
-- send message
2 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello again"
get bob ##> ("", aliceId, SENT $ baseId + 2)
-- no notifications should follow
noNotification apnsQ
-- no unexpected notifications should follow
500000 `timeout` atomically (readTBQueue apnsQ) >>= \case
Nothing -> pure ()
_ -> error "unexpected notification"
pure ()
where
baseId = 3
@@ -266,249 +230,67 @@ testNotificationSubscriptionNewConnection APNSMockServer {apnsQ} = do
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right () <- runExceptT $ do
-- alice registers notification token
_ <- registerTestToken alice "abcd" NMInstant apnsQ
let aliceTkn = DeviceToken PPApns "abcd"
NTRegistered <- registerNtfToken alice aliceTkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse} <-
atomically $ readTBQueue apnsQ
verification <- ntfData .-> "verification"
nonce <- C.cbNonce <$> ntfData .-> "nonce"
liftIO $ sendApnsResponse APNSRespOk
verifyNtfToken alice aliceTkn verification nonce
NTActive <- checkNtfToken alice aliceTkn
-- bob registers notification token
_ <- registerTestToken bob "bcde" NMInstant apnsQ
let bobTkn = DeviceToken PPApns "bcde"
NTRegistered <- registerNtfToken bob bobTkn
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData'}, sendApnsResponse = sendApnsResponse'} <-
atomically $ readTBQueue apnsQ
verification' <- ntfData' .-> "verification"
nonce' <- C.cbNonce <$> ntfData' .-> "nonce"
liftIO $ sendApnsResponse' APNSRespOk
verifyNtfToken bob bobTkn verification' nonce'
NTActive <- checkNtfToken bob bobTkn
-- establish connection
liftIO $ threadDelay 50000
(bobId, qInfo) <- createConnection alice SCMInvitation
liftIO $ threadDelay 500000
aliceId <- joinConnection bob qInfo "bob's connInfo"
liftIO $ print 0
void $ messageNotification apnsQ
("", _, CONF confId _ "bob's connInfo") <- get alice
messageNotification apnsQ
("", _, CONF confId "bob's connInfo") <- get alice
liftIO $ threadDelay 500000
allowConnection alice bobId confId "alice's connInfo"
liftIO $ print 1
void $ messageNotification apnsQ
messageNotification apnsQ
get bob ##> ("", aliceId, INFO "alice's connInfo")
liftIO $ print 2
void $ messageNotification apnsQ
messageNotification apnsQ
get alice ##> ("", bobId, CON)
liftIO $ print 3
void $ messageNotification apnsQ
messageNotification apnsQ
get bob ##> ("", aliceId, CON)
-- bob sends message
1 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello"
get bob ##> ("", aliceId, SENT $ baseId + 1)
liftIO $ print 4
void $ messageNotification apnsQ
messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 1
-- alice sends message
2 <- msgId <$> sendMessage alice bobId (SMP.MsgFlags True) "hey there"
get alice ##> ("", bobId, SENT $ baseId + 2)
liftIO $ print 5
void $ messageNotification apnsQ
messageNotification apnsQ
get bob =##> \case ("", c, Msg "hey there") -> c == aliceId; _ -> False
ackMessage bob aliceId $ baseId + 2
-- no unexpected notifications should follow
noNotification apnsQ
500000 `timeout` atomically (readTBQueue apnsQ) >>= \case
Nothing -> pure ()
_ -> error "unexpected notification"
pure ()
where
baseId = 3
msgId = subtract baseId
registerTestToken :: AgentClient -> ByteString -> NotificationsMode -> TBQueue APNSMockRequest -> ExceptT AgentErrorType IO DeviceToken
registerTestToken a token mode apnsQ = do
let tkn = DeviceToken PPApnsTest token
NTRegistered <- registerNtfToken a tkn mode
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData'}, sendApnsResponse = sendApnsResponse'} <-
atomically $ readTBQueue apnsQ
verification' <- ntfData' .-> "verification"
nonce' <- C.cbNonce <$> ntfData' .-> "nonce"
liftIO $ sendApnsResponse' APNSRespOk
verifyNtfToken a tkn nonce' verification'
NTActive <- checkNtfToken a tkn
pure tkn
testChangeNotificationsMode :: APNSMockServer -> IO ()
testChangeNotificationsMode APNSMockServer {apnsQ} = do
alice <- getSMPAgentClient agentCfg initAgentServers
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right () <- runExceptT $ do
-- establish connection
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, CONF confId _ "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get bob ##> ("", aliceId, INFO "alice's connInfo")
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, CON)
-- register notification token, set mode to NMInstant
tkn <- registerTestToken alice "abcd" NMInstant apnsQ
-- send message, receive notification
liftIO $ threadDelay 500000
1 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello"
get bob ##> ("", aliceId, SENT $ baseId + 1)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 1
-- set mode to NMPeriodic
NTActive <- registerNtfToken alice tkn NMPeriodic
-- send message, no notification
liftIO $ threadDelay 500000
2 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello again"
get bob ##> ("", aliceId, SENT $ baseId + 2)
noNotification apnsQ
get alice =##> \case ("", c, Msg "hello again") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 2
-- set mode to NMInstant
NTActive <- registerNtfToken alice tkn NMInstant
-- send message, receive notification
liftIO $ threadDelay 500000
3 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello there"
get bob ##> ("", aliceId, SENT $ baseId + 3)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello there") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 3
-- turn off notifications
deleteNtfToken alice tkn
-- send message, no notification
liftIO $ threadDelay 500000
4 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "why hello there"
get bob ##> ("", aliceId, SENT $ baseId + 4)
noNotification apnsQ
get alice =##> \case ("", c, Msg "why hello there") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 4
-- turn on notifications, set mode to NMInstant
void $ registerTestToken alice "abcd" NMInstant apnsQ
-- send message, receive notification
liftIO $ threadDelay 500000
5 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hey"
get bob ##> ("", aliceId, SENT $ baseId + 5)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hey") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 5
-- no notifications should follow
noNotification apnsQ
pure ()
where
baseId = 3
msgId = subtract baseId
testChangeToken :: APNSMockServer -> IO ()
testChangeToken APNSMockServer {apnsQ} = do
alice <- getSMPAgentClient agentCfg initAgentServers
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right (aliceId, bobId) <- runExceptT $ do
-- establish connection
(bobId, qInfo) <- createConnection alice SCMInvitation
aliceId <- joinConnection bob qInfo "bob's connInfo"
("", _, CONF confId _ "bob's connInfo") <- get alice
allowConnection alice bobId confId "alice's connInfo"
get bob ##> ("", aliceId, INFO "alice's connInfo")
get alice ##> ("", bobId, CON)
get bob ##> ("", aliceId, CON)
-- register notification token, set mode to NMInstant
void $ registerTestToken alice "abcd" NMInstant apnsQ
-- send message, receive notification
liftIO $ threadDelay 500000
1 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello"
get bob ##> ("", aliceId, SENT $ baseId + 1)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello") -> c == bobId; _ -> False
ackMessage alice bobId $ baseId + 1
pure (aliceId, bobId)
disconnectAgentClient alice
alice1 <- getSMPAgentClient agentCfg initAgentServers
Right () <- runExceptT $ do
subscribeConnection alice1 bobId
-- change notification token
void $ registerTestToken alice1 "bcde" NMInstant apnsQ
-- send message, receive notification
liftIO $ threadDelay 500000
2 <- msgId <$> sendMessage bob aliceId (SMP.MsgFlags True) "hello there"
get bob ##> ("", aliceId, SENT $ baseId + 2)
void $ messageNotification apnsQ
get alice1 =##> \case ("", c, Msg "hello there") -> c == bobId; _ -> False
ackMessage alice1 bobId $ baseId + 2
-- no notifications should follow
noNotification apnsQ
pure ()
where
baseId = 3
msgId = subtract baseId
testNotificationsStoreLog :: ATransport -> APNSMockServer -> IO ()
testNotificationsStoreLog t APNSMockServer {apnsQ} = do
alice <- getSMPAgentClient agentCfg initAgentServers
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right (aliceId, bobId) <- withNtfServerStoreLog t $ \threadId -> runExceptT $ do
(aliceId, bobId) <- makeConnection alice bob
_ <- registerTestToken alice "abcd" NMInstant apnsQ
liftIO $ threadDelay 250000
4 <- sendMessage bob aliceId (SMP.MsgFlags True) "hello"
get bob ##> ("", aliceId, SENT 4)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello") -> c == bobId; _ -> False
ackMessage alice bobId 4
liftIO $ killThread threadId
pure (aliceId, bobId)
liftIO $ threadDelay 250000
Right () <- withNtfServerStoreLog t $ \threadId -> runExceptT $ do
liftIO $ threadDelay 250000
5 <- sendMessage bob aliceId (SMP.MsgFlags True) "hello again"
get bob ##> ("", aliceId, SENT 5)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello again") -> c == bobId; _ -> False
liftIO $ killThread threadId
pure ()
testNotificationsSMPRestart :: ATransport -> APNSMockServer -> IO ()
testNotificationsSMPRestart t APNSMockServer {apnsQ} = do
alice <- getSMPAgentClient agentCfg initAgentServers
bob <- getSMPAgentClient agentCfg {dbFile = testDB2} initAgentServers
Right (aliceId, bobId) <- withSmpServerStoreLogOn t testPort $ \threadId -> runExceptT $ do
(aliceId, bobId) <- makeConnection alice bob
_ <- registerTestToken alice "abcd" NMInstant apnsQ
liftIO $ threadDelay 250000
4 <- sendMessage bob aliceId (SMP.MsgFlags True) "hello"
get bob ##> ("", aliceId, SENT 4)
void $ messageNotification apnsQ
get alice =##> \case ("", c, Msg "hello") -> c == bobId; _ -> False
ackMessage alice bobId 4
liftIO $ killThread threadId
pure (aliceId, bobId)
Right () <- runExceptT $ do
get alice =##> \case ("", "", DOWN _ [c]) -> c == bobId; _ -> False
get bob =##> \case ("", "", DOWN _ [c]) -> c == aliceId; _ -> False
Right () <- withSmpServerStoreLogOn t testPort $ \threadId -> runExceptT $ do
get alice =##> \case ("", "", UP _ [c]) -> c == bobId; _ -> False
get bob =##> \case ("", "", UP _ [c]) -> c == aliceId; _ -> False
liftIO $ threadDelay 1000000
5 <- sendMessage bob aliceId (SMP.MsgFlags True) "hello again"
get bob ##> ("", aliceId, SENT 5)
_ <- messageNotificationData alice apnsQ
get alice =##> \case ("", c, Msg "hello again") -> c == bobId; _ -> False
liftIO $ killThread threadId
pure ()
messageNotification :: TBQueue APNSMockRequest -> ExceptT AgentErrorType IO (C.CbNonce, ByteString)
messageNotification :: TBQueue APNSMockRequest -> ExceptT AgentErrorType IO ()
messageNotification apnsQ = do
1000000 `timeout` atomically (readTBQueue apnsQ) >>= \case
500000 `timeout` atomically (readTBQueue apnsQ) >>= \case
Nothing -> error "no notification"
Just APNSMockRequest {notification = APNSNotification {aps = APNSMutableContent {}, notificationData = Just ntfData}, sendApnsResponse} -> do
nonce <- C.cbNonce <$> ntfData .-> "nonce"
message <- ntfData .-> "message"
_ <- ntfData .-> "checkMessage"
_ <- C.cbNonce <$> ntfData .-> "nonce"
liftIO $ sendApnsResponse APNSRespOk
pure (nonce, message)
_ -> error "bad notification"
messageNotificationData :: AgentClient -> TBQueue APNSMockRequest -> ExceptT AgentErrorType IO PNMessageData
messageNotificationData c apnsQ = do
(nonce, message) <- messageNotification apnsQ
NtfToken {ntfDhSecret = Just dhSecret} <- getNtfTokenData c
Right pnMsgData <- liftEither . first INTERNAL $ Right . strDecode =<< first show (C.cbDecrypt dhSecret nonce message)
pure pnMsgData
noNotification :: TBQueue APNSMockRequest -> ExceptT AgentErrorType IO ()
noNotification apnsQ = do
500000 `timeout` atomically (readTBQueue apnsQ) >>= \case
Nothing -> pure ()
_ -> error "unexpected notification"
+174 -174
View File
@@ -11,6 +11,7 @@ module AgentTests.SQLiteTests (storeTests) where
import Control.Concurrent.Async (concurrently_)
import Control.Concurrent.STM
import Control.Monad (replicateM_)
import Control.Monad.Except (ExceptT, runExceptT)
import Crypto.Random (drgNew)
import Data.ByteString.Char8 (ByteString)
import qualified Data.Text as T
@@ -43,7 +44,7 @@ withStore2 = before connect2 . after (removeStore . fst)
connect2 :: IO (SQLiteStore, SQLiteStore)
connect2 = do
s1 <- createStore
s2 <- connectSQLiteStore (dbFilePath s1)
s2 <- connectSQLiteStore (dbFilePath s1) Nothing
pure (s1, s2)
createStore :: IO SQLiteStore
@@ -51,26 +52,32 @@ createStore = do
-- Randomize DB file name to avoid SQLite IO errors supposedly caused by asynchronous
-- IO operations on multiple similarly named files; error seems to be environment specific
r <- randomIO :: IO Word32
createSQLiteStore (testDB <> show r) Migrations.app True
createSQLiteStore (testDB <> show r) Nothing Migrations.app True
removeStore :: SQLiteStore -> IO ()
removeStore db = do
close db
removeFile $ dbFilePath db
removeStore store = do
close store
removeFile $ dbFilePath store
where
close :: SQLiteStore -> IO ()
close st = mapM_ DB.close =<< atomically (tryTakeTMVar $ dbConnection st)
returnsResult :: (Eq a, Eq e, Show a, Show e) => ExceptT e IO a -> a -> Expectation
action `returnsResult` r = runExceptT action `shouldReturn` Right r
throwsError :: (Eq a, Eq e, Show a, Show e) => ExceptT e IO a -> e -> Expectation
action `throwsError` e = runExceptT action `shouldReturn` Left e
-- TODO add null port tests
storeTests :: Spec
storeTests = do
withStore2 $ do
describe "stress test" testConcurrentWrites
withStore $ do
describe "db setup" $ do
describe "store setup" $ do
testCompiledThreadsafe
testForeignKeysEnabled
describe "db methods" $ do
describe "store methods" $ do
describe "Queue and Connection management" $ do
describe "createRcvConn" $ do
testCreateRcvConn
@@ -105,29 +112,28 @@ testConcurrentWrites :: SpecWith (SQLiteStore, SQLiteStore)
testConcurrentWrites =
it "should complete multiple concurrent write transactions w/t sqlite busy errors" $ \(s1, s2) -> do
g <- newTVarIO =<< drgNew
_ <- withTransaction s1 $ \db ->
createRcvConn db g cData1 rcvQueue1 SCMInvitation
_ <- runExceptT $ createRcvConn s1 g cData1 rcvQueue1 SCMInvitation
let ConnData {connId} = cData1
concurrently_ (runTest s1 connId) (runTest s2 connId)
where
runTest :: SQLiteStore -> ConnId -> IO ()
runTest st connId = replicateM_ 100 . withTransaction st $ \db -> do
(internalId, internalRcvId, _, _) <- updateRcvIds db connId
runTest :: SQLiteStore -> ConnId -> IO (Either StoreError ())
runTest store connId = runExceptT . replicateM_ 100 $ do
(internalId, internalRcvId, _, _) <- updateRcvIds store connId
let rcvMsgData = mkRcvMsgData internalId internalRcvId 0 "0" "hash_dummy"
createRcvMsg db connId rcvMsgData
createRcvMsg store connId rcvMsgData
testCompiledThreadsafe :: SpecWith SQLiteStore
testCompiledThreadsafe =
it "compiled sqlite library should be threadsafe" . withStoreTransaction $ \db -> do
it "compiled sqlite library should be threadsafe" . withStoreConnection $ \db -> do
compileOptions <- DB.query_ db "pragma COMPILE_OPTIONS;" :: IO [[T.Text]]
compileOptions `shouldNotContain` [["THREADSAFE=0"]]
withStoreTransaction :: (DB.Connection -> IO a) -> SQLiteStore -> IO a
withStoreTransaction = flip withTransaction
withStoreConnection :: (DB.Connection -> IO a) -> SQLiteStore -> IO a
withStoreConnection = flip withConnection
testForeignKeysEnabled :: SpecWith SQLiteStore
testForeignKeysEnabled =
it "foreign keys should be enabled" . withStoreTransaction $ \db -> do
it "foreign keys should be enabled" . withStoreConnection $ \db -> do
let inconsistentQuery =
[sql|
INSERT INTO snd_queues
@@ -161,8 +167,9 @@ rcvQueue1 =
e2eDhSecret = Nothing,
sndId = Just "2345",
status = New,
smpClientVersion = 1,
clientNtfCreds = Nothing
ntfPublicKey = Nothing,
ntfPrivateKey = Nothing,
notifierId = Nothing
}
sndQueue1 :: SndQueue
@@ -174,131 +181,130 @@ sndQueue1 =
sndPrivateKey = testPrivateSignKey,
e2ePubKey = Nothing,
e2eDhSecret = testDhSecret,
status = New,
smpClientVersion = 1
status = New
}
testCreateRcvConn :: SpecWith SQLiteStore
testCreateRcvConn =
it "should create RcvConnection and add SndQueue" . withStoreTransaction $ \db -> do
it "should create RcvConnection and add SndQueue" $ \store -> do
g <- newTVarIO =<< drgNew
createRcvConn db g cData1 rcvQueue1 SCMInvitation
`shouldReturn` Right "conn1"
getConn db "conn1"
`shouldReturn` Right (SomeConn SCRcv (RcvConnection cData1 rcvQueue1))
upgradeRcvConnToDuplex db "conn1" sndQueue1
`shouldReturn` Right ()
getConn db "conn1"
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1))
createRcvConn store g cData1 rcvQueue1 SCMInvitation
`returnsResult` "conn1"
getConn store "conn1"
`returnsResult` SomeConn SCRcv (RcvConnection cData1 rcvQueue1)
upgradeRcvConnToDuplex store "conn1" sndQueue1
`returnsResult` ()
getConn store "conn1"
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1)
testCreateRcvConnRandomId :: SpecWith SQLiteStore
testCreateRcvConnRandomId =
it "should create RcvConnection and add SndQueue with random ID" . withStoreTransaction $ \db -> do
it "should create RcvConnection and add SndQueue with random ID" $ \store -> do
g <- newTVarIO =<< drgNew
Right connId <- createRcvConn db g cData1 {connId = ""} rcvQueue1 SCMInvitation
getConn db connId
`shouldReturn` Right (SomeConn SCRcv (RcvConnection cData1 {connId} rcvQueue1))
upgradeRcvConnToDuplex db connId sndQueue1
`shouldReturn` Right ()
getConn db connId
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 {connId} rcvQueue1 sndQueue1))
Right connId <- runExceptT $ createRcvConn store g cData1 {connId = ""} rcvQueue1 SCMInvitation
getConn store connId
`returnsResult` SomeConn SCRcv (RcvConnection cData1 {connId} rcvQueue1)
upgradeRcvConnToDuplex store connId sndQueue1
`returnsResult` ()
getConn store connId
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 {connId} rcvQueue1 sndQueue1)
testCreateRcvConnDuplicate :: SpecWith SQLiteStore
testCreateRcvConnDuplicate =
it "should throw error on attempt to create duplicate RcvConnection" . withStoreTransaction $ \db -> do
it "should throw error on attempt to create duplicate RcvConnection" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
createRcvConn db g cData1 rcvQueue1 SCMInvitation
`shouldReturn` Left (SEConnDuplicate)
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
createRcvConn store g cData1 rcvQueue1 SCMInvitation
`throwsError` SEConnDuplicate
testCreateSndConn :: SpecWith SQLiteStore
testCreateSndConn =
it "should create SndConnection and add RcvQueue" . withStoreTransaction $ \db -> do
it "should create SndConnection and add RcvQueue" $ \store -> do
g <- newTVarIO =<< drgNew
createSndConn db g cData1 sndQueue1
`shouldReturn` Right "conn1"
getConn db "conn1"
`shouldReturn` Right (SomeConn SCSnd (SndConnection cData1 sndQueue1))
upgradeSndConnToDuplex db "conn1" rcvQueue1
`shouldReturn` Right ()
getConn db "conn1"
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1))
createSndConn store g cData1 sndQueue1
`returnsResult` "conn1"
getConn store "conn1"
`returnsResult` SomeConn SCSnd (SndConnection cData1 sndQueue1)
upgradeSndConnToDuplex store "conn1" rcvQueue1
`returnsResult` ()
getConn store "conn1"
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1)
testCreateSndConnRandomID :: SpecWith SQLiteStore
testCreateSndConnRandomID =
it "should create SndConnection and add RcvQueue with random ID" . withStoreTransaction $ \db -> do
it "should create SndConnection and add RcvQueue with random ID" $ \store -> do
g <- newTVarIO =<< drgNew
Right connId <- createSndConn db g cData1 {connId = ""} sndQueue1
getConn db connId
`shouldReturn` Right (SomeConn SCSnd (SndConnection cData1 {connId} sndQueue1))
upgradeSndConnToDuplex db connId rcvQueue1
`shouldReturn` Right ()
getConn db connId
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 {connId} rcvQueue1 sndQueue1))
Right connId <- runExceptT $ createSndConn store g cData1 {connId = ""} sndQueue1
getConn store connId
`returnsResult` SomeConn SCSnd (SndConnection cData1 {connId} sndQueue1)
upgradeSndConnToDuplex store connId rcvQueue1
`returnsResult` ()
getConn store connId
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 {connId} rcvQueue1 sndQueue1)
testCreateSndConnDuplicate :: SpecWith SQLiteStore
testCreateSndConnDuplicate =
it "should throw error on attempt to create duplicate SndConnection" . withStoreTransaction $ \db -> do
it "should throw error on attempt to create duplicate SndConnection" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createSndConn db g cData1 sndQueue1
createSndConn db g cData1 sndQueue1
`shouldReturn` Left (SEConnDuplicate)
_ <- runExceptT $ createSndConn store g cData1 sndQueue1
createSndConn store g cData1 sndQueue1
`throwsError` SEConnDuplicate
testGetRcvConn :: SpecWith SQLiteStore
testGetRcvConn =
it "should get connection using rcv queue id and server" . withStoreTransaction $ \db -> do
it "should get connection using rcv queue id and server" $ \store -> do
let smpServer = SMPServer "smp.simplex.im" "5223" testKeyHash
let recipientId = "1234"
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
getRcvConn db smpServer recipientId
`shouldReturn` Right (SomeConn SCRcv (RcvConnection cData1 rcvQueue1))
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
getRcvConn store smpServer recipientId
`returnsResult` SomeConn SCRcv (RcvConnection cData1 rcvQueue1)
testDeleteRcvConn :: SpecWith SQLiteStore
testDeleteRcvConn =
it "should create RcvConnection and delete it" . withStoreTransaction $ \db -> do
it "should create RcvConnection and delete it" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
getConn db "conn1"
`shouldReturn` Right (SomeConn SCRcv (RcvConnection cData1 rcvQueue1))
deleteConn db "conn1"
`shouldReturn` ()
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
getConn store "conn1"
`returnsResult` SomeConn SCRcv (RcvConnection cData1 rcvQueue1)
deleteConn store "conn1"
`returnsResult` ()
-- TODO check queues are deleted as well
getConn db "conn1"
`shouldReturn` Left (SEConnNotFound)
getConn store "conn1"
`throwsError` SEConnNotFound
testDeleteSndConn :: SpecWith SQLiteStore
testDeleteSndConn =
it "should create SndConnection and delete it" . withStoreTransaction $ \db -> do
it "should create SndConnection and delete it" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createSndConn db g cData1 sndQueue1
getConn db "conn1"
`shouldReturn` Right (SomeConn SCSnd (SndConnection cData1 sndQueue1))
deleteConn db "conn1"
`shouldReturn` ()
_ <- runExceptT $ createSndConn store g cData1 sndQueue1
getConn store "conn1"
`returnsResult` SomeConn SCSnd (SndConnection cData1 sndQueue1)
deleteConn store "conn1"
`returnsResult` ()
-- TODO check queues are deleted as well
getConn db "conn1"
`shouldReturn` Left (SEConnNotFound)
getConn store "conn1"
`throwsError` SEConnNotFound
testDeleteDuplexConn :: SpecWith SQLiteStore
testDeleteDuplexConn =
it "should create DuplexConnection and delete it" . withStoreTransaction $ \db -> do
it "should create DuplexConnection and delete it" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
_ <- upgradeRcvConnToDuplex db "conn1" sndQueue1
getConn db "conn1"
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1))
deleteConn db "conn1"
`shouldReturn` ()
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
_ <- runExceptT $ upgradeRcvConnToDuplex store "conn1" sndQueue1
getConn store "conn1"
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1)
deleteConn store "conn1"
`returnsResult` ()
-- TODO check queues are deleted as well
getConn db "conn1"
`shouldReturn` Left (SEConnNotFound)
getConn store "conn1"
`throwsError` SEConnNotFound
testUpgradeRcvConnToDuplex :: SpecWith SQLiteStore
testUpgradeRcvConnToDuplex =
it "should throw error on attempt to add SndQueue to SndConnection or DuplexConnection" . withStoreTransaction $ \db -> do
it "should throw error on attempt to add SndQueue to SndConnection or DuplexConnection" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createSndConn db g cData1 sndQueue1
_ <- runExceptT $ createSndConn store g cData1 sndQueue1
let anotherSndQueue =
SndQueue
{ server = SMPServer "smp.simplex.im" "5223" testKeyHash,
@@ -307,20 +313,19 @@ testUpgradeRcvConnToDuplex =
sndPrivateKey = testPrivateSignKey,
e2ePubKey = Nothing,
e2eDhSecret = testDhSecret,
status = New,
smpClientVersion = 1
status = New
}
upgradeRcvConnToDuplex db "conn1" anotherSndQueue
`shouldReturn` Left (SEBadConnType CSnd)
_ <- upgradeSndConnToDuplex db "conn1" rcvQueue1
upgradeRcvConnToDuplex db "conn1" anotherSndQueue
`shouldReturn` Left (SEBadConnType CDuplex)
upgradeRcvConnToDuplex store "conn1" anotherSndQueue
`throwsError` SEBadConnType CSnd
_ <- runExceptT $ upgradeSndConnToDuplex store "conn1" rcvQueue1
upgradeRcvConnToDuplex store "conn1" anotherSndQueue
`throwsError` SEBadConnType CDuplex
testUpgradeSndConnToDuplex :: SpecWith SQLiteStore
testUpgradeSndConnToDuplex =
it "should throw error on attempt to add RcvQueue to RcvConnection or DuplexConnection" . withStoreTransaction $ \db -> do
it "should throw error on attempt to add RcvQueue to RcvConnection or DuplexConnection" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
let anotherRcvQueue =
RcvQueue
{ server = SMPServer "smp.simplex.im" "5223" testKeyHash,
@@ -331,55 +336,56 @@ testUpgradeSndConnToDuplex =
e2eDhSecret = Nothing,
sndId = Just "4567",
status = New,
smpClientVersion = 1,
clientNtfCreds = Nothing
ntfPublicKey = Nothing,
ntfPrivateKey = Nothing,
notifierId = Nothing
}
upgradeSndConnToDuplex db "conn1" anotherRcvQueue
`shouldReturn` Left (SEBadConnType CRcv)
_ <- upgradeRcvConnToDuplex db "conn1" sndQueue1
upgradeSndConnToDuplex db "conn1" anotherRcvQueue
`shouldReturn` Left (SEBadConnType CDuplex)
upgradeSndConnToDuplex store "conn1" anotherRcvQueue
`throwsError` SEBadConnType CRcv
_ <- runExceptT $ upgradeRcvConnToDuplex store "conn1" sndQueue1
upgradeSndConnToDuplex store "conn1" anotherRcvQueue
`throwsError` SEBadConnType CDuplex
testSetRcvQueueStatus :: SpecWith SQLiteStore
testSetRcvQueueStatus =
it "should update status of RcvQueue" . withStoreTransaction $ \db -> do
it "should update status of RcvQueue" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
getConn db "conn1"
`shouldReturn` Right (SomeConn SCRcv (RcvConnection cData1 rcvQueue1))
setRcvQueueStatus db rcvQueue1 Confirmed
`shouldReturn` ()
getConn db "conn1"
`shouldReturn` Right (SomeConn SCRcv (RcvConnection cData1 rcvQueue1 {status = Confirmed}))
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
getConn store "conn1"
`returnsResult` SomeConn SCRcv (RcvConnection cData1 rcvQueue1)
setRcvQueueStatus store rcvQueue1 Confirmed
`returnsResult` ()
getConn store "conn1"
`returnsResult` SomeConn SCRcv (RcvConnection cData1 rcvQueue1 {status = Confirmed})
testSetSndQueueStatus :: SpecWith SQLiteStore
testSetSndQueueStatus =
it "should update status of SndQueue" . withStoreTransaction $ \db -> do
it "should update status of SndQueue" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createSndConn db g cData1 sndQueue1
getConn db "conn1"
`shouldReturn` Right (SomeConn SCSnd (SndConnection cData1 sndQueue1))
setSndQueueStatus db sndQueue1 Confirmed
`shouldReturn` ()
getConn db "conn1"
`shouldReturn` Right (SomeConn SCSnd (SndConnection cData1 sndQueue1 {status = Confirmed}))
_ <- runExceptT $ createSndConn store g cData1 sndQueue1
getConn store "conn1"
`returnsResult` SomeConn SCSnd (SndConnection cData1 sndQueue1)
setSndQueueStatus store sndQueue1 Confirmed
`returnsResult` ()
getConn store "conn1"
`returnsResult` SomeConn SCSnd (SndConnection cData1 sndQueue1 {status = Confirmed})
testSetQueueStatusDuplex :: SpecWith SQLiteStore
testSetQueueStatusDuplex =
it "should update statuses of RcvQueue and SndQueue in DuplexConnection" . withStoreTransaction $ \db -> do
it "should update statuses of RcvQueue and SndQueue in DuplexConnection" $ \store -> do
g <- newTVarIO =<< drgNew
_ <- createRcvConn db g cData1 rcvQueue1 SCMInvitation
_ <- upgradeRcvConnToDuplex db "conn1" sndQueue1
getConn db "conn1"
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1))
setRcvQueueStatus db rcvQueue1 Secured
`shouldReturn` ()
getConn db "conn1"
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 {status = Secured} sndQueue1))
setSndQueueStatus db sndQueue1 Confirmed
`shouldReturn` ()
getConn db "conn1"
`shouldReturn` Right (SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 {status = Secured} sndQueue1 {status = Confirmed}))
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
_ <- runExceptT $ upgradeRcvConnToDuplex store "conn1" sndQueue1
getConn store "conn1"
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 sndQueue1)
setRcvQueueStatus store rcvQueue1 Secured
`returnsResult` ()
getConn store "conn1"
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 {status = Secured} sndQueue1)
setSndQueueStatus store sndQueue1 Confirmed
`returnsResult` ()
getConn store "conn1"
`returnsResult` SomeConn SCDuplex (DuplexConnection cData1 rcvQueue1 {status = Secured} sndQueue1 {status = Confirmed})
hw :: ByteString
hw = encodeUtf8 "Hello world!"
@@ -405,24 +411,22 @@ mkRcvMsgData internalId internalRcvId externalSndId brokerId internalHash =
externalPrevSndHash = "hash_from_sender"
}
testCreateRcvMsg_ :: DB.Connection -> PrevExternalSndId -> PrevRcvMsgHash -> ConnId -> RcvMsgData -> Expectation
testCreateRcvMsg_ db expectedPrevSndId expectedPrevHash connId rcvMsgData@RcvMsgData {..} = do
testCreateRcvMsg_ :: SQLiteStore -> PrevExternalSndId -> PrevRcvMsgHash -> ConnId -> RcvMsgData -> Expectation
testCreateRcvMsg_ st expectedPrevSndId expectedPrevHash connId rcvMsgData@RcvMsgData {..} = do
let MsgMeta {recipient = (internalId, _)} = msgMeta
updateRcvIds db connId
`shouldReturn` (InternalId internalId, internalRcvId, expectedPrevSndId, expectedPrevHash)
createRcvMsg db connId rcvMsgData
`shouldReturn` ()
updateRcvIds st connId
`returnsResult` (InternalId internalId, internalRcvId, expectedPrevSndId, expectedPrevHash)
createRcvMsg st connId rcvMsgData
`returnsResult` ()
testCreateRcvMsg :: SpecWith SQLiteStore
testCreateRcvMsg =
it "should reserve internal ids and create a RcvMsg" $ \st -> do
g <- newTVarIO =<< drgNew
let ConnData {connId} = cData1
_ <- withTransaction st $ \db -> do
createRcvConn db g cData1 rcvQueue1 SCMInvitation
withTransaction st $ \db -> do
testCreateRcvMsg_ db 0 "" connId $ mkRcvMsgData (InternalId 1) (InternalRcvId 1) 1 "1" "hash_dummy"
testCreateRcvMsg_ db 1 "hash_dummy" connId $ mkRcvMsgData (InternalId 2) (InternalRcvId 2) 2 "2" "new_hash_dummy"
_ <- runExceptT $ createRcvConn st g cData1 rcvQueue1 SCMInvitation
testCreateRcvMsg_ st 0 "" connId $ mkRcvMsgData (InternalId 1) (InternalRcvId 1) 1 "1" "hash_dummy"
testCreateRcvMsg_ st 1 "hash_dummy" connId $ mkRcvMsgData (InternalId 2) (InternalRcvId 2) 2 "2" "new_hash_dummy"
mkSndMsgData :: InternalId -> InternalSndId -> MsgHash -> SndMsgData
mkSndMsgData internalId internalSndId internalHash =
@@ -437,36 +441,32 @@ mkSndMsgData internalId internalSndId internalHash =
prevMsgHash = internalHash
}
testCreateSndMsg_ :: DB.Connection -> PrevSndMsgHash -> ConnId -> SndMsgData -> Expectation
testCreateSndMsg_ db expectedPrevHash connId sndMsgData@SndMsgData {..} = do
updateSndIds db connId
`shouldReturn` (internalId, internalSndId, expectedPrevHash)
createSndMsg db connId sndMsgData
`shouldReturn` ()
testCreateSndMsg_ :: SQLiteStore -> PrevSndMsgHash -> ConnId -> SndMsgData -> Expectation
testCreateSndMsg_ store expectedPrevHash connId sndMsgData@SndMsgData {..} = do
updateSndIds store connId
`returnsResult` (internalId, internalSndId, expectedPrevHash)
createSndMsg store connId sndMsgData
`returnsResult` ()
testCreateSndMsg :: SpecWith SQLiteStore
testCreateSndMsg =
it "should create a SndMsg and return InternalId and PrevSndMsgHash" $ \st -> do
it "should create a SndMsg and return InternalId and PrevSndMsgHash" $ \store -> do
g <- newTVarIO =<< drgNew
let ConnData {connId} = cData1
_ <- withTransaction st $ \db -> do
createSndConn db g cData1 sndQueue1
withTransaction st $ \db -> do
testCreateSndMsg_ db "" connId $ mkSndMsgData (InternalId 1) (InternalSndId 1) "hash_dummy"
testCreateSndMsg_ db "hash_dummy" connId $ mkSndMsgData (InternalId 2) (InternalSndId 2) "new_hash_dummy"
_ <- runExceptT $ createSndConn store g cData1 sndQueue1
testCreateSndMsg_ store "" connId $ mkSndMsgData (InternalId 1) (InternalSndId 1) "hash_dummy"
testCreateSndMsg_ store "hash_dummy" connId $ mkSndMsgData (InternalId 2) (InternalSndId 2) "new_hash_dummy"
testCreateRcvAndSndMsgs :: SpecWith SQLiteStore
testCreateRcvAndSndMsgs =
it "should create multiple RcvMsg and SndMsg, correctly ordering internal Ids and returning previous state" $ \st -> do
it "should create multiple RcvMsg and SndMsg, correctly ordering internal Ids and returning previous state" $ \store -> do
g <- newTVarIO =<< drgNew
let ConnData {connId} = cData1
_ <- withTransaction st $ \db -> do
g <- newTVarIO =<< drgNew
createRcvConn db g cData1 rcvQueue1 SCMInvitation
withTransaction st $ \db -> do
_ <- upgradeRcvConnToDuplex db "conn1" sndQueue1
testCreateRcvMsg_ db 0 "" connId $ mkRcvMsgData (InternalId 1) (InternalRcvId 1) 1 "1" "rcv_hash_1"
testCreateRcvMsg_ db 1 "rcv_hash_1" connId $ mkRcvMsgData (InternalId 2) (InternalRcvId 2) 2 "2" "rcv_hash_2"
testCreateSndMsg_ db "" connId $ mkSndMsgData (InternalId 3) (InternalSndId 1) "snd_hash_1"
testCreateRcvMsg_ db 2 "rcv_hash_2" connId $ mkRcvMsgData (InternalId 4) (InternalRcvId 3) 3 "3" "rcv_hash_3"
testCreateSndMsg_ db "snd_hash_1" connId $ mkSndMsgData (InternalId 5) (InternalSndId 2) "snd_hash_2"
testCreateSndMsg_ db "snd_hash_2" connId $ mkSndMsgData (InternalId 6) (InternalSndId 3) "snd_hash_3"
_ <- runExceptT $ createRcvConn store g cData1 rcvQueue1 SCMInvitation
_ <- runExceptT $ upgradeRcvConnToDuplex store "conn1" sndQueue1
testCreateRcvMsg_ store 0 "" connId $ mkRcvMsgData (InternalId 1) (InternalRcvId 1) 1 "1" "rcv_hash_1"
testCreateRcvMsg_ store 1 "rcv_hash_1" connId $ mkRcvMsgData (InternalId 2) (InternalRcvId 2) 2 "2" "rcv_hash_2"
testCreateSndMsg_ store "" connId $ mkSndMsgData (InternalId 3) (InternalSndId 1) "snd_hash_1"
testCreateRcvMsg_ store 2 "rcv_hash_2" connId $ mkRcvMsgData (InternalId 4) (InternalRcvId 3) 3 "3" "rcv_hash_3"
testCreateSndMsg_ store "snd_hash_1" connId $ mkSndMsgData (InternalId 5) (InternalSndId 2) "snd_hash_2"
testCreateSndMsg_ store "snd_hash_2" connId $ mkSndMsgData (InternalId 6) (InternalSndId 3) "snd_hash_3"
+1 -1
View File
@@ -20,7 +20,7 @@ schemaDumpTest =
testVerifySchemaDump :: IO ()
testVerifySchemaDump = do
void $ createSQLiteStore testDB Migrations.app False
void $ createSQLiteStore testDB Nothing Migrations.app False
void $ readCreateProcess (shell $ "touch " <> schema) ""
savedSchema <- readFile schema
savedSchema `seq` pure ()
+12 -31
View File
@@ -6,7 +6,6 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StandaloneDeriving #-}
@@ -24,14 +23,12 @@ import qualified Data.Aeson as J
import qualified Data.Aeson.Types as JT
import Data.ByteString.Builder (lazyByteString)
import Data.ByteString.Char8 (ByteString)
import Data.List.NonEmpty (NonEmpty)
import Data.Text (Text)
import GHC.Generics (Generic)
import Network.HTTP.Types (Status)
import qualified Network.HTTP.Types as N
import qualified Network.HTTP2.Server as H
import Network.Socket
import Simplex.Messaging.Client (chooseTransportHost, defaultNetworkConfig)
import Simplex.Messaging.Client.Agent (defaultSMPClientAgentConfig)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
@@ -53,7 +50,7 @@ import qualified UnliftIO.Exception as E
import UnliftIO.STM
import UnliftIO.Timeout (timeout)
testHost :: NonEmpty TransportHost
testHost :: HostName
testHost = "localhost"
ntfTestPort :: ServiceName
@@ -65,13 +62,9 @@ apnsTestPort = "6010"
testKeyHash :: C.KeyHash
testKeyHash = "LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI="
ntfTestStoreLogFile :: FilePath
ntfTestStoreLogFile = "tests/tmp/ntf-server-store.log"
testNtfClient :: (Transport c, MonadUnliftIO m, MonadFail m) => (THandle c -> m a) -> m a
testNtfClient client = do
Right host <- pure $ chooseTransportHost defaultNetworkConfig testHost
runTransportClient Nothing host ntfTestPort (Just testKeyHash) (Just defaultKeepAliveOpts) $ \h ->
testNtfClient :: (Transport c, MonadUnliftIO m) => (THandle c -> m a) -> m a
testNtfClient client =
runTransportClient testHost ntfTestPort (Just testKeyHash) (Just defaultKeepAliveOpts) $ \h ->
liftIO (runExceptT $ ntfClientHandshake h testKeyHash supportedNTFServerVRange) >>= \case
Right th -> client th
Left e -> error $ show e
@@ -88,33 +81,21 @@ ntfServerCfg =
smpAgentCfg = defaultSMPClientAgentConfig,
apnsConfig =
defaultAPNSPushClientConfig
{ apnsPort = apnsTestPort,
{ apnsHost = "localhost",
apnsPort = apnsTestPort,
http2cfg = defaultHTTP2ClientConfig {caStoreFile = "tests/fixtures/ca.crt"}
},
inactiveClientExpiration = Just defaultInactiveClientExpiration,
storeLogFile = Nothing,
resubscribeDelay = 1000,
-- CA certificate private key is not needed for initialization
caCertificateFile = "tests/fixtures/ca.crt",
privateKeyFile = "tests/fixtures/server.key",
certificateFile = "tests/fixtures/server.crt",
-- stats config
logStatsInterval = Nothing,
logStatsStartTime = 0,
serverStatsLogFile = "tests/ntf-server-stats.daily.log",
serverStatsBackupFile = Nothing
certificateFile = "tests/fixtures/server.crt"
}
withNtfServerStoreLog :: (MonadUnliftIO m, MonadRandom m) => ATransport -> (ThreadId -> m a) -> m a
withNtfServerStoreLog t = withNtfServerCfg t ntfServerCfg {storeLogFile = Just ntfTestStoreLogFile}
withNtfServerThreadOn :: (MonadUnliftIO m, MonadRandom m) => ATransport -> ServiceName -> (ThreadId -> m a) -> m a
withNtfServerThreadOn t port' = withNtfServerCfg t ntfServerCfg {transports = [(port', t)]}
withNtfServerCfg :: (MonadUnliftIO m, MonadRandom m) => ATransport -> NtfServerConfig -> (ThreadId -> m a) -> m a
withNtfServerCfg t cfg =
withNtfServerThreadOn t port' =
serverBracket
(\started -> runNtfServerBlocking started cfg {transports = [(ntfTestPort, t)]})
(\started -> runNtfServerBlocking started ntfServerCfg {transports = [(port', t)]})
(pure ())
serverBracket :: MonadUnliftIO m => (TMVar Bool -> m ()) -> m () -> (ThreadId -> m a) -> m a
@@ -136,7 +117,7 @@ withNtfServerOn t port' = withNtfServerThreadOn t port' . const
withNtfServer :: (MonadUnliftIO m, MonadRandom m) => ATransport -> m a -> m a
withNtfServer t = withNtfServerOn t ntfTestPort
runNtfTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => (THandle c -> m a) -> m a
runNtfTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => (THandle c -> m a) -> m a
runNtfTest test = withNtfServer (transport @c) $ testNtfClient test
ntfServerTest ::
@@ -149,10 +130,10 @@ ntfServerTest _ t = runNtfTest $ \h -> tPut' h t >> tGet' h
where
tPut' h (sig, corrId, queueId, smp) = do
let t' = smpEncode (sessionId (h :: THandle c), corrId, queueId, smp)
[Right ()] <- tPut h [(sig, t')]
Right () <- tPut h (sig, t')
pure ()
tGet' h = do
[(Nothing, _, (CorrId corrId, qId, Right cmd))] <- tGet h
(Nothing, _, (CorrId corrId, qId, Right cmd)) <- tGet h
pure (Nothing, corrId, qId, cmd)
ntfTest :: Transport c => TProxy c -> (THandle c -> IO ()) -> Expectation
+16 -56
View File
@@ -1,5 +1,4 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
@@ -13,7 +12,6 @@ import Control.Concurrent (threadDelay)
import Control.Monad.Except (runExceptT)
import qualified Data.Aeson as J
import qualified Data.Aeson.Types as JT
import Data.Bifunctor (first)
import qualified Data.ByteString.Base64.URL as U
import Data.ByteString.Char8 (ByteString)
import Data.Text.Encoding (encodeUtf8)
@@ -25,20 +23,15 @@ import ServerTests
samplePubKey,
sampleSig,
signSendRecv,
tGet1,
tPut1,
(#==),
_SEND',
pattern Resp,
)
import qualified Simplex.Messaging.Agent.Protocol as AP
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Push.APNS
import qualified Simplex.Messaging.Notifications.Server.Push.APNS as APNS
import Simplex.Messaging.Parsers (parse, parseAll)
import Simplex.Messaging.Protocol hiding (notification)
import Simplex.Messaging.Transport
import Test.Hspec
@@ -54,9 +47,9 @@ ntfSyntaxTests (ATransport t) = do
it "unknown command" $ ("", "abcd", "1234", ('H', 'E', 'L', 'L', 'O')) >#> ("", "abcd", "1234", ERR $ CMD UNKNOWN)
describe "NEW" $ do
it "no parameters" $ (sampleSig, "bcda", "", TNEW_) >#> ("", "bcda", "", ERR $ CMD SYNTAX)
it "many parameters" $ (sampleSig, "cdab", "", (TNEW_, (' ', '\x01', 'A'), ('T', 'A', 'T', "abcd" :: ByteString), samplePubKey, sampleDhPubKey)) >#> ("", "cdab", "", ERR $ CMD SYNTAX)
it "no signature" $ ("", "dabc", "", (TNEW_, ' ', ('T', 'A', 'T', "abcd" :: ByteString), samplePubKey, sampleDhPubKey)) >#> ("", "dabc", "", ERR $ CMD NO_AUTH)
it "token ID" $ (sampleSig, "abcd", "12345678", (TNEW_, ' ', ('T', 'A', 'T', "abcd" :: ByteString), samplePubKey, sampleDhPubKey)) >#> ("", "abcd", "12345678", ERR $ CMD HAS_AUTH)
it "many parameters" $ (sampleSig, "cdab", "", (TNEW_, (' ', '\x01', 'A'), ('T', 'A', "abcd" :: ByteString), samplePubKey, sampleDhPubKey)) >#> ("", "cdab", "", ERR $ CMD SYNTAX)
it "no signature" $ ("", "dabc", "", (TNEW_, ' ', ('T', 'A', "abcd" :: ByteString), samplePubKey, sampleDhPubKey)) >#> ("", "dabc", "", ERR $ CMD NO_AUTH)
it "token ID" $ (sampleSig, "abcd", "12345678", (TNEW_, ' ', ('T', 'A', "abcd" :: ByteString), samplePubKey, sampleDhPubKey)) >#> ("", "abcd", "12345678", ERR $ CMD HAS_AUTH)
where
(>#>) ::
Encoding smp =>
@@ -71,15 +64,15 @@ pattern RespNtf corrId queueId command <- (_, _, (corrId, queueId, Right command
sendRecvNtf :: forall c e. (Transport c, NtfEntityI e) => THandle c -> (Maybe C.ASignature, ByteString, ByteString, NtfCommand e) -> IO (SignedTransmission NtfResponse)
sendRecvNtf h@THandle {thVersion, sessionId} (sgn, corrId, qId, cmd) = do
let t = encodeTransmission thVersion sessionId (CorrId corrId, qId, cmd)
Right () <- tPut1 h (sgn, t)
tGet1 h
Right () <- tPut h (sgn, t)
tGet h
signSendRecvNtf :: forall c e. (Transport c, NtfEntityI e) => THandle c -> C.APrivateSignKey -> (ByteString, ByteString, NtfCommand e) -> IO (SignedTransmission NtfResponse)
signSendRecvNtf h@THandle {thVersion, sessionId} pk (corrId, qId, cmd) = do
let t = encodeTransmission thVersion sessionId (CorrId corrId, qId, cmd)
Right sig <- runExceptT $ C.sign pk t
Right () <- tPut1 h (Just sig, t)
tGet1 h
Right () <- tPut h (Just sig, t)
tGet h
(.->) :: J.Value -> J.Key -> Either String ByteString
v .-> key =
@@ -93,12 +86,12 @@ testNotificationSubscription (ATransport t) =
(nPub, nKey) <- C.generateSignatureKeyPair C.SEd25519
(tknPub, tknKey) <- C.generateSignatureKeyPair C.SEd25519
(dhPub, dhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
let tkn = DeviceToken PPApnsTest "abcd"
let tkn = DeviceToken PPApns "abcd"
withAPNSMockServer $ \APNSMockServer {apnsQ} ->
smpTest2 t $ \rh sh ->
ntfTest t $ \nh -> do
-- create queue
(sId, rId, rKey, rcvDhSecret) <- createAndSecureQueue rh sPub
(sId, rId, rKey, _dhShared) <- createAndSecureQueue rh sPub
-- register and verify token
RespNtf "1" "" (NRTknId tId ntfDh) <- signSendRecvNtf nh tknKey ("1", "", TNEW $ NewNtfTkn tkn tknPub dhPub)
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData}, sendApnsResponse = send} <-
@@ -111,11 +104,9 @@ testNotificationSubscription (ATransport t) =
RespNtf "2" _ NROk <- signSendRecvNtf nh tknKey ("2", tId, TVFY code)
RespNtf "2a" _ (NRTkn NTActive) <- signSendRecvNtf nh tknKey ("2a", tId, TCHK)
-- enable queue notifications
(rcvNtfPubDhKey, rcvNtfPrivDhKey) <- C.generateKeyPair'
Resp "3" _ (NID nId rcvNtfSrvPubDhKey) <- signSendRecv rh rKey ("3", rId, NKEY nPub rcvNtfPubDhKey)
Resp "3" _ (NID nId) <- signSendRecv rh rKey ("3", rId, NKEY nPub)
let srv = SMPServer SMP.testHost SMP.testPort SMP.testKeyHash
q = SMPQueueNtf srv nId
rcvNtfDhSecret = C.dh' rcvNtfSrvPubDhKey rcvNtfPrivDhKey
RespNtf "4" _ (NRSubId _subId) <- signSendRecvNtf nh tknKey ("4", "", SNEW $ NewNtfSub tId q nKey)
-- send message
threadDelay 50000
@@ -123,45 +114,14 @@ testNotificationSubscription (ATransport t) =
-- receive notification
APNSMockRequest {notification, sendApnsResponse = send'} <- atomically $ readTBQueue apnsQ
let APNSNotification {aps = APNSMutableContent {}, notificationData = Just ntfData'} = notification
Right checkMessage = ntfData' .-> "checkMessage"
Right nonce' = C.cbNonce <$> ntfData' .-> "nonce"
Right message = ntfData' .-> "message"
Right ntfDataDecrypted = C.cbDecrypt dhSecret nonce' message
Right APNS.PNMessageData {smpQueue = SMPQueueNtf {smpServer, notifierId}, nmsgNonce, encNMsgMeta} =
parse strP (AP.INTERNAL "error parsing PNMessageData") ntfDataDecrypted
Right nMsgMeta = C.cbDecrypt rcvNtfDhSecret nmsgNonce encNMsgMeta
Right NMsgMeta {msgId, msgTs} = parse smpP (AP.INTERNAL "error parsing NMsgMeta") nMsgMeta
smpServer `shouldBe` srv
notifierId `shouldBe` nId
Right smpQueueURI = C.cbDecrypt dhSecret nonce' checkMessage
smpQueueURI `shouldBe` strEncode srv <> "/" <> strEncode nId
send' APNSRespOk
-- receive message
Resp "" _ (MSG RcvMessage {msgId = mId1, msgBody = EncRcvMsgBody body}) <- tGet1 rh
Right ClientRcvMsgBody {msgTs = mTs, msgBody} <- pure $ parseAll clientRcvMsgBodyP =<< first show (C.cbDecrypt rcvDhSecret (C.cbNonce mId1) body)
mId1 `shouldBe` msgId
mTs `shouldBe` msgTs
(msgBody, "hello") #== "delivered from queue"
let dec _nonce = C.cbDecrypt _dhShared (C.cbNonce _nonce)
Resp "" _ (MSG mId1 _ _ msg1) <- tGet rh
(dec mId1 msg1, Right "hello") #== "delivered from queue"
Resp "6" _ OK <- signSendRecv rh rKey ("6", rId, ACK mId1)
pure ()
-- replace token
let tkn' = DeviceToken PPApnsTest "efgh"
RespNtf "7" tId' NROk <- signSendRecvNtf nh tknKey ("7", tId, TRPL tkn')
tId `shouldBe` tId'
APNSMockRequest {notification = APNSNotification {aps = APNSBackground _, notificationData = Just ntfData2}, sendApnsResponse = send2} <-
atomically $ readTBQueue apnsQ
send2 APNSRespOk
let Right verification2 = ntfData2 .-> "verification"
Right nonce2 = C.cbNonce <$> ntfData2 .-> "nonce"
Right code2 = NtfRegCode <$> C.cbDecrypt dhSecret nonce2 verification2
RespNtf "8" _ NROk <- signSendRecvNtf nh tknKey ("8", tId, TVFY code2)
RespNtf "8a" _ (NRTkn NTActive) <- signSendRecvNtf nh tknKey ("8a", tId, TCHK)
-- send message
Resp "9" _ OK <- signSendRecv sh sKey ("9", sId, _SEND' "hello 2")
APNSMockRequest {notification = notification3, sendApnsResponse = send3} <- atomically $ readTBQueue apnsQ
let APNSNotification {aps = APNSMutableContent {}, notificationData = Just ntfData3} = notification3
Right nonce3 = C.cbNonce <$> ntfData3 .-> "nonce"
Right message3 = ntfData3 .-> "message"
Right ntfDataDecrypted3 = C.cbDecrypt dhSecret nonce3 message3
Right APNS.PNMessageData {smpQueue = SMPQueueNtf {smpServer = smpServer3, notifierId = notifierId3}} =
parse strP (AP.INTERNAL "error parsing PNMessageData") ntfDataDecrypted3
smpServer3 `shouldBe` srv
notifierId3 `shouldBe` nId
send3 APNSRespOk
+16 -34
View File
@@ -1,5 +1,4 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
@@ -10,9 +9,8 @@ module SMPAgentClient where
import Control.Monad.IO.Unlift
import Crypto.Random
import qualified Data.ByteString.Char8 as B
import Data.List.NonEmpty (NonEmpty)
import qualified Data.List.NonEmpty as L
import Network.Socket (ServiceName)
import Network.Socket (HostName, ServiceName)
import NtfClient (ntfTestPort)
import SMPClient
( serverBracket,
@@ -27,8 +25,7 @@ import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Agent.RetryInterval
import Simplex.Messaging.Agent.Server (runSMPAgentBlocking)
import Simplex.Messaging.Client (ProtocolClientConfig (..), chooseTransportHost, defaultClientConfig, defaultNetworkConfig)
import Simplex.Messaging.Parsers (parseAll)
import Simplex.Messaging.Client (ProtocolClientConfig (..), defaultClientConfig)
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.Client
import Simplex.Messaging.Transport.KeepAlive
@@ -36,7 +33,7 @@ import Test.Hspec
import UnliftIO.Concurrent
import UnliftIO.Directory
agentTestHost :: NonEmpty TransportHost
agentTestHost :: HostName
agentTestHost = "localhost"
agentTestPort :: ServiceName
@@ -58,21 +55,14 @@ testDB3 :: String
testDB3 = "tests/tmp/smp-agent3.test.protocol.db"
smpAgentTest :: forall c. Transport c => TProxy c -> ARawTransmission -> IO ARawTransmission
smpAgentTest _ cmd = runSmpAgentTest $ \(h :: c) -> tPutRaw h cmd >> get h
where
get h = do
t@(_, _, cmdStr) <- tGetRaw h
case parseAll commandP cmdStr of
Right (ACmd SAgent CONNECT {}) -> get h
Right (ACmd SAgent DISCONNECT {}) -> get h
_ -> pure t
smpAgentTest _ cmd = runSmpAgentTest $ \(h :: c) -> tPutRaw h cmd >> tGetRaw h
runSmpAgentTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => (c -> m a) -> m a
runSmpAgentTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => (c -> m a) -> m a
runSmpAgentTest test = withSmpServer t . withSmpAgent t $ testSMPAgentClient test
where
t = transport @c
runSmpAgentServerTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => ((ThreadId, ThreadId) -> c -> m a) -> m a
runSmpAgentServerTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => ((ThreadId, ThreadId) -> c -> m a) -> m a
runSmpAgentServerTest test =
withSmpServerThreadOn t testPort $
\server -> withSmpAgentThreadOn t (agentTestPort, testPort, testDB) $
@@ -83,7 +73,7 @@ runSmpAgentServerTest test =
smpAgentServerTest :: Transport c => ((ThreadId, ThreadId) -> c -> IO ()) -> Expectation
smpAgentServerTest test' = runSmpAgentServerTest test' `shouldReturn` ()
runSmpAgentTestN :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => [(ServiceName, ServiceName, String)] -> ([c] -> m a) -> m a
runSmpAgentTestN :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => [(ServiceName, ServiceName, String)] -> ([c] -> m a) -> m a
runSmpAgentTestN agents test = withSmpServer t $ run agents []
where
run :: [(ServiceName, ServiceName, String)] -> [c] -> m a
@@ -91,7 +81,7 @@ runSmpAgentTestN agents test = withSmpServer t $ run agents []
run (a@(p, _, _) : as) hs = withSmpAgentOn t a $ testSMPAgentClientOn p $ \h -> run as (h : hs)
t = transport @c
runSmpAgentTestN_1 :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => Int -> ([c] -> m a) -> m a
runSmpAgentTestN_1 :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => Int -> ([c] -> m a) -> m a
runSmpAgentTestN_1 nClients test = withSmpServer t . withSmpAgent t $ run nClients []
where
run :: Int -> [c] -> m a
@@ -168,31 +158,25 @@ smpAgentTest1_1_1 test' =
testSMPServer :: SMPServer
testSMPServer = "smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@localhost:5001"
testSMPServer2 :: SMPServer
testSMPServer2 = "smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@localhost:5002"
initAgentServers :: InitialAgentServers
initAgentServers =
InitialAgentServers
{ smp = L.fromList [testSMPServer],
ntf = ["ntf://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@localhost:6001"],
netCfg = defaultNetworkConfig {tcpTimeout = 500_000}
ntf = ["smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@localhost:6001"]
}
initAgentServers2 :: InitialAgentServers
initAgentServers2 = initAgentServers {smp = L.fromList [testSMPServer, testSMPServer2]}
agentCfg :: AgentConfig
agentCfg =
defaultAgentConfig
{ tcpPort = agentTestPort,
tbqSize = 4,
tbqSize = 1,
dbFile = testDB,
dbSemName = Nothing, -- Just "/simplex-agent",
smpCfg =
defaultClientConfig
{ qSize = 1,
defaultTransport = (testPort, transport @TLS),
networkConfig = defaultNetworkConfig {tcpTimeout = 500_000}
tcpTimeout = 500_000
},
ntfCfg =
defaultClientConfig
@@ -200,8 +184,7 @@ agentCfg =
defaultTransport = (ntfTestPort, transport @TLS)
},
reconnectInterval = defaultReconnectInterval {initialInterval = 50_000},
ntfWorkerDelay = 1000,
ntfSMPWorkerDelay = 1000,
ntfWorkerThrottle = 1000,
caCertificateFile = "tests/fixtures/ca.crt",
privateKeyFile = "tests/fixtures/server.key",
certificateFile = "tests/fixtures/server.crt"
@@ -224,15 +207,14 @@ withSmpAgentOn t (port', smpPort', db') = withSmpAgentThreadOn t (port', smpPort
withSmpAgent :: (MonadUnliftIO m, MonadRandom m) => ATransport -> m a -> m a
withSmpAgent t = withSmpAgentOn t (agentTestPort, testPort, testDB)
testSMPAgentClientOn :: (Transport c, MonadUnliftIO m, MonadFail m) => ServiceName -> (c -> m a) -> m a
testSMPAgentClientOn :: (Transport c, MonadUnliftIO m) => ServiceName -> (c -> m a) -> m a
testSMPAgentClientOn port' client = do
Right useHost <- pure $ chooseTransportHost defaultNetworkConfig agentTestHost
runTransportClient Nothing useHost port' (Just testKeyHash) (Just defaultKeepAliveOpts) $ \h -> do
runTransportClient agentTestHost port' (Just testKeyHash) (Just defaultKeepAliveOpts) $ \h -> do
line <- liftIO $ getLn h
if line == "Welcome to SMP agent v" <> B.pack simplexMQVersion
then client h
else do
error $ "wrong welcome message: " <> B.unpack line
testSMPAgentClient :: (Transport c, MonadUnliftIO m, MonadFail m) => (c -> m a) -> m a
testSMPAgentClient :: (Transport c, MonadUnliftIO m) => (c -> m a) -> m a
testSMPAgentClient = testSMPAgentClientOn agentTestPort
+10 -29
View File
@@ -3,7 +3,6 @@
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NumericUnderscores #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
@@ -14,9 +13,7 @@ import Control.Monad.Except (runExceptT)
import Control.Monad.IO.Unlift
import Crypto.Random
import Data.ByteString.Char8 (ByteString)
import Data.List.NonEmpty (NonEmpty)
import Network.Socket
import Simplex.Messaging.Client (chooseTransportHost, defaultNetworkConfig)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Protocol
@@ -25,14 +22,13 @@ import Simplex.Messaging.Server.Env.STM
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.Client
import Simplex.Messaging.Transport.KeepAlive
import Simplex.Messaging.Version
import Test.Hspec
import UnliftIO.Concurrent
import qualified UnliftIO.Exception as E
import UnliftIO.STM (TMVar, atomically, newEmptyTMVarIO, takeTMVar)
import UnliftIO.Timeout (timeout)
testHost :: NonEmpty TransportHost
testHost :: HostName
testHost = "localhost"
testPort :: ServiceName
@@ -47,26 +43,16 @@ testKeyHash = "LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI="
testStoreLogFile :: FilePath
testStoreLogFile = "tests/tmp/smp-server-store.log"
testStoreLogFile2 :: FilePath
testStoreLogFile2 = "tests/tmp/smp-server-store.log.2"
testStoreMsgsFile :: FilePath
testStoreMsgsFile = "tests/tmp/smp-server-messages.log"
testServerStatsBackupFile :: FilePath
testServerStatsBackupFile = "tests/tmp/smp-server-stats.log"
testSMPClient :: (Transport c, MonadUnliftIO m, MonadFail m) => (THandle c -> m a) -> m a
testSMPClient client = do
Right useHost <- pure $ chooseTransportHost defaultNetworkConfig testHost
runTransportClient Nothing useHost testPort (Just testKeyHash) (Just defaultKeepAliveOpts) $ \h ->
testSMPClient :: (Transport c, MonadUnliftIO m) => (THandle c -> m a) -> m a
testSMPClient client =
runTransportClient testHost testPort (Just testKeyHash) (Just defaultKeepAliveOpts) $ \h ->
liftIO (runExceptT $ smpClientHandshake h testKeyHash supportedSMPServerVRange) >>= \case
Right th -> client th
Left e -> error $ show e
cfgV2 :: ServerConfig
cfgV2 = cfg {smpServerVRange = mkVersionRange 1 2}
cfg :: ServerConfig
cfg =
ServerConfig
@@ -83,22 +69,17 @@ cfg =
inactiveClientExpiration = Just defaultInactiveClientExpiration,
logStatsInterval = Nothing,
logStatsStartTime = 0,
serverStatsLogFile = "tests/smp-server-stats.daily.log",
serverStatsBackupFile = Nothing,
caCertificateFile = "tests/fixtures/ca.crt",
privateKeyFile = "tests/fixtures/server.key",
certificateFile = "tests/fixtures/server.crt",
smpServerVRange = supportedSMPServerVRange
}
withSmpServerStoreMsgLogOnV2 :: (MonadUnliftIO m, MonadRandom m) => ATransport -> ServiceName -> (ThreadId -> m a) -> m a
withSmpServerStoreMsgLogOnV2 t = withSmpServerConfigOn t cfgV2 {storeLogFile = Just testStoreLogFile, storeMsgsFile = Just testStoreMsgsFile}
withSmpServerStoreMsgLogOn :: (MonadUnliftIO m, MonadRandom m) => ATransport -> ServiceName -> (ThreadId -> m a) -> m a
withSmpServerStoreMsgLogOn t = withSmpServerConfigOn t cfg {storeLogFile = Just testStoreLogFile, storeMsgsFile = Just testStoreMsgsFile, serverStatsBackupFile = Just testServerStatsBackupFile}
withSmpServerStoreMsgLogOn t = withSmpServerConfigOn t cfg {storeLogFile = Just testStoreLogFile, storeMsgsFile = Just testStoreMsgsFile}
withSmpServerStoreLogOn :: (MonadUnliftIO m, MonadRandom m) => ATransport -> ServiceName -> (ThreadId -> m a) -> m a
withSmpServerStoreLogOn t = withSmpServerConfigOn t cfg {storeLogFile = Just testStoreLogFile, serverStatsBackupFile = Just testServerStatsBackupFile}
withSmpServerStoreLogOn t = withSmpServerConfigOn t cfg {storeLogFile = Just testStoreLogFile}
withSmpServerConfigOn :: (MonadUnliftIO m, MonadRandom m) => ATransport -> ServerConfig -> ServiceName -> (ThreadId -> m a) -> m a
withSmpServerConfigOn t cfg' port' =
@@ -128,10 +109,10 @@ withSmpServerOn t port' = withSmpServerThreadOn t port' . const
withSmpServer :: (MonadUnliftIO m, MonadRandom m) => ATransport -> m a -> m a
withSmpServer t = withSmpServerOn t testPort
runSmpTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => (THandle c -> m a) -> m a
runSmpTest :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => (THandle c -> m a) -> m a
runSmpTest test = withSmpServer (transport @c) $ testSMPClient test
runSmpTestN :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m, MonadFail m) => Int -> ([THandle c] -> m a) -> m a
runSmpTestN :: forall c m a. (Transport c, MonadUnliftIO m, MonadRandom m) => Int -> ([THandle c] -> m a) -> m a
runSmpTestN nClients test = withSmpServer (transport @c) $ run nClients []
where
run :: Int -> [THandle c] -> m a
@@ -148,10 +129,10 @@ smpServerTest _ t = runSmpTest $ \h -> tPut' h t >> tGet' h
where
tPut' h (sig, corrId, queueId, smp) = do
let t' = smpEncode (sessionId (h :: THandle c), corrId, queueId, smp)
[Right ()] <- tPut h [(sig, t')]
Right () <- tPut h (sig, t')
pure ()
tGet' h = do
[(Nothing, _, (CorrId corrId, qId, Right cmd))] <- tGet h
(Nothing, _, (CorrId corrId, qId, Right cmd)) <- tGet h
pure (Nothing, corrId, qId, cmd)
smpTest :: Transport c => TProxy c -> (THandle c -> IO ()) -> Expectation
+75 -257
View File
@@ -1,9 +1,7 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE ScopedTypeVariables #-}
@@ -15,8 +13,6 @@ import Control.Concurrent (ThreadId, killThread, threadDelay)
import Control.Concurrent.STM
import Control.Exception (SomeException, try)
import Control.Monad.Except (forM, forM_, runExceptT)
import Control.Monad.IO.Class
import Data.Bifunctor (first)
import Data.ByteString.Base64
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
@@ -24,7 +20,6 @@ import SMPClient
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers (parseAll)
import Simplex.Messaging.Protocol
import Simplex.Messaging.Server.Env.STM (ServerConfig (..))
import Simplex.Messaging.Server.Expiration
@@ -39,8 +34,7 @@ serverTests :: ATransport -> Spec
serverTests t@(ATransport t') = do
describe "SMP syntax" $ syntaxTests t
describe "SMP queues" $ do
describe "NEW and KEY commands, SEND messages (v2)" $ testCreateSecureV2 t'
describe "NEW and KEY commands, SEND messages (v3)" $ testCreateSecure t
describe "NEW and KEY commands, SEND messages" $ testCreateSecure t
describe "NEW, OFF and DEL commands, SEND messages" $ testCreateDelete t
describe "Stress test" $ stressTest t
describe "allowNewQueues setting" $ testAllowNewQueues t'
@@ -51,7 +45,6 @@ serverTests t@(ATransport t') = do
describe "GET & SUB commands" $ testGetSubCommands t'
describe "Store log" $ testWithStoreLog t
describe "Restore messages" $ testRestoreMessages t
describe "Restore messages (v2)" $ testRestoreMessagesV2 t
describe "Timing of AUTH error" $ testTiming t
describe "Message notifications" $ testMessageNotifications t
describe "Message expiration" $ do
@@ -65,31 +58,18 @@ pattern Resp corrId queueId command <- (_, _, (corrId, queueId, Right command))
pattern Ids :: RecipientId -> SenderId -> RcvPublicDhKey -> BrokerMsg
pattern Ids rId sId srvDh <- IDS (QIK rId sId srvDh)
pattern Msg :: MsgId -> MsgBody -> BrokerMsg
pattern Msg msgId body <- MSG RcvMessage {msgId, msgBody = EncRcvMsgBody body}
sendRecv :: forall c p. (Transport c, PartyI p) => THandle c -> (Maybe C.ASignature, ByteString, ByteString, Command p) -> IO (SignedTransmission BrokerMsg)
sendRecv h@THandle {thVersion, sessionId} (sgn, corrId, qId, cmd) = do
let t = encodeTransmission thVersion sessionId (CorrId corrId, qId, cmd)
Right () <- tPut1 h (sgn, t)
tGet1 h
Right () <- tPut h (sgn, t)
tGet h
signSendRecv :: forall c p. (Transport c, PartyI p) => THandle c -> C.APrivateSignKey -> (ByteString, ByteString, Command p) -> IO (SignedTransmission BrokerMsg)
signSendRecv h@THandle {thVersion, sessionId} pk (corrId, qId, cmd) = do
let t = encodeTransmission thVersion sessionId (CorrId corrId, qId, cmd)
Right sig <- runExceptT $ C.sign pk t
Right () <- tPut1 h (Just sig, t)
tGet1 h
tPut1 :: Transport c => THandle c -> SentRawTransmission -> IO (Either TransportError ())
tPut1 h t = do
[r] <- tPut h [t]
pure r
tGet1 :: (ProtocolEncoding cmd, Transport c, MonadIO m, MonadFail m) => THandle c -> m (SignedTransmission cmd)
tGet1 h = do
[r] <- tGet h
pure r
Right () <- tPut h (Just sig, t)
tGet h
(#==) :: (HasCallStack, Eq a, Show a) => (a, a) -> String -> Assertion
(actual, expected) #== message = assertEqual message expected actual
@@ -100,76 +80,6 @@ _SEND = SEND noMsgFlags
_SEND' :: MsgBody -> Command 'Sender
_SEND' = SEND MsgFlags {notification = True}
decryptMsgV2 :: C.DhSecret 'C.X25519 -> ByteString -> ByteString -> Either C.CryptoError ByteString
decryptMsgV2 dhShared = C.cbDecrypt dhShared . C.cbNonce
decryptMsgV3 :: C.DhSecret 'C.X25519 -> ByteString -> ByteString -> Either String MsgBody
decryptMsgV3 dhShared nonce body = do
ClientRcvMsgBody {msgBody} <- parseAll clientRcvMsgBodyP =<< first show (C.cbDecrypt dhShared (C.cbNonce nonce) body)
pure msgBody
testCreateSecureV2 :: forall c. Transport c => TProxy c -> Spec
testCreateSecureV2 _ =
it "should create (NEW) and secure (KEY) queue" $
withSmpServerConfigOn (transport @c) cfgV2 testPort $ \_ -> testSMPClient @c $ \h -> do
(rPub, rKey) <- C.generateSignatureKeyPair C.SEd448
(dhPub, dhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" rId1 (Ids rId sId srvDh) <- signSendRecv h rKey ("abcd", "", NEW rPub dhPub)
let dec = decryptMsgV2 $ C.dh' srvDh dhPriv
(rId1, "") #== "creates queue"
Resp "bcda" sId1 ok1 <- sendRecv h ("", "bcda", sId, _SEND "hello")
(ok1, OK) #== "accepts unsigned SEND"
(sId1, sId) #== "same queue ID in response 1"
Resp "" _ (Msg mId1 msg1) <- tGet1 h
(dec mId1 msg1, Right "hello") #== "delivers message"
Resp "cdab" _ ok4 <- signSendRecv h rKey ("cdab", rId, ACK mId1)
(ok4, OK) #== "replies OK when message acknowledged if no more messages"
Resp "dabc" _ err6 <- signSendRecv h rKey ("dabc", rId, ACK mId1)
(err6, ERR NO_MSG) #== "replies ERR when message acknowledged without messages"
(sPub, sKey) <- C.generateSignatureKeyPair C.SEd448
Resp "abcd" sId2 err1 <- signSendRecv h sKey ("abcd", sId, _SEND "hello")
(err1, ERR AUTH) #== "rejects signed SEND"
(sId2, sId) #== "same queue ID in response 2"
Resp "bcda" _ err2 <- sendRecv h (sampleSig, "bcda", rId, KEY sPub)
(err2, ERR AUTH) #== "rejects KEY with wrong signature"
Resp "cdab" _ err3 <- signSendRecv h rKey ("cdab", sId, KEY sPub)
(err3, ERR AUTH) #== "rejects KEY with sender's ID"
Resp "dabc" rId2 ok2 <- signSendRecv h rKey ("dabc", rId, KEY sPub)
(ok2, OK) #== "secures queue"
(rId2, rId) #== "same queue ID in response 3"
Resp "abcd" _ err4 <- signSendRecv h rKey ("abcd", rId, KEY sPub)
(err4, ERR AUTH) #== "rejects KEY if already secured"
Resp "bcda" _ ok3 <- signSendRecv h sKey ("bcda", sId, _SEND "hello again")
(ok3, OK) #== "accepts signed SEND"
Resp "" _ (Msg mId2 msg2) <- tGet1 h
(dec mId2 msg2, Right "hello again") #== "delivers message 2"
Resp "cdab" _ ok5 <- signSendRecv h rKey ("cdab", rId, ACK mId2)
(ok5, OK) #== "replies OK when message acknowledged 2"
Resp "dabc" _ err5 <- sendRecv h ("", "dabc", sId, _SEND "hello")
(err5, ERR AUTH) #== "rejects unsigned SEND"
let maxAllowedMessage = B.replicate maxMessageLength '-'
Resp "bcda" _ OK <- signSendRecv h sKey ("bcda", sId, _SEND maxAllowedMessage)
Resp "" _ (Msg mId3 msg3) <- tGet1 h
(dec mId3 msg3, Right maxAllowedMessage) #== "delivers message of max size"
let biggerMessage = B.replicate (maxMessageLength + 1) '-'
Resp "bcda" _ (ERR LARGE_MSG) <- signSendRecv h sKey ("bcda", sId, _SEND biggerMessage)
pure ()
testCreateSecure :: ATransport -> Spec
testCreateSecure (ATransport t) =
it "should create (NEW) and secure (KEY) queue" $
@@ -177,14 +87,14 @@ testCreateSecure (ATransport t) =
(rPub, rKey) <- C.generateSignatureKeyPair C.SEd448
(dhPub, dhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" rId1 (Ids rId sId srvDh) <- signSendRecv h rKey ("abcd", "", NEW rPub dhPub)
let dec = decryptMsgV3 $ C.dh' srvDh dhPriv
let dec nonce = C.cbDecrypt (C.dh' srvDh dhPriv) (C.cbNonce nonce)
(rId1, "") #== "creates queue"
Resp "bcda" sId1 ok1 <- sendRecv h ("", "bcda", sId, _SEND "hello")
(ok1, OK) #== "accepts unsigned SEND"
(sId1, sId) #== "same queue ID in response 1"
Resp "" _ (Msg mId1 msg1) <- tGet1 h
Resp "" _ (MSG mId1 _ _ msg1) <- tGet h
(dec mId1 msg1, Right "hello") #== "delivers message"
Resp "cdab" _ ok4 <- signSendRecv h rKey ("cdab", rId, ACK mId1)
@@ -214,7 +124,7 @@ testCreateSecure (ATransport t) =
Resp "bcda" _ ok3 <- signSendRecv h sKey ("bcda", sId, _SEND "hello again")
(ok3, OK) #== "accepts signed SEND"
Resp "" _ (Msg mId2 msg2) <- tGet1 h
Resp "" _ (MSG mId2 _ _ msg2) <- tGet h
(dec mId2 msg2, Right "hello again") #== "delivers message 2"
Resp "cdab" _ ok5 <- signSendRecv h rKey ("cdab", rId, ACK mId2)
@@ -223,15 +133,6 @@ testCreateSecure (ATransport t) =
Resp "dabc" _ err5 <- sendRecv h ("", "dabc", sId, _SEND "hello")
(err5, ERR AUTH) #== "rejects unsigned SEND"
let maxAllowedMessage = B.replicate maxMessageLength '-'
Resp "bcda" _ OK <- signSendRecv h sKey ("bcda", sId, _SEND maxAllowedMessage)
Resp "" _ (Msg mId3 msg3) <- tGet1 h
(dec mId3 msg3, Right maxAllowedMessage) #== "delivers message of max size"
let biggerMessage = B.replicate (maxMessageLength + 1) '-'
Resp "bcda" _ (ERR LARGE_MSG) <- signSendRecv h sKey ("bcda", sId, _SEND biggerMessage)
pure ()
testCreateDelete :: ATransport -> Spec
testCreateDelete (ATransport t) =
it "should create (NEW), suspend (OFF) and delete (DEL) queue" $
@@ -239,7 +140,7 @@ testCreateDelete (ATransport t) =
(rPub, rKey) <- C.generateSignatureKeyPair C.SEd25519
(dhPub, dhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" rId1 (Ids rId sId srvDh) <- signSendRecv rh rKey ("abcd", "", NEW rPub dhPub)
let dec = decryptMsgV3 $ C.dh' srvDh dhPriv
let dec nonce = C.cbDecrypt (C.dh' srvDh dhPriv) (C.cbNonce nonce)
(rId1, "") #== "creates queue"
(sPub, sKey) <- C.generateSignatureKeyPair C.SEd25519
@@ -252,7 +153,7 @@ testCreateDelete (ATransport t) =
Resp "dabc" _ ok7 <- signSendRecv sh sKey ("dabc", sId, _SEND "hello 2")
(ok7, OK) #== "accepts signed SEND 2 - this message is not delivered because the first is not ACKed"
Resp "" _ (Msg mId1 msg1) <- tGet1 rh
Resp "" _ (MSG mId1 _ _ msg1) <- tGet rh
(dec mId1 msg1, Right "hello") #== "delivers message"
Resp "abcd" _ err1 <- sendRecv rh (sampleSig, "abcd", rId, OFF)
@@ -274,7 +175,7 @@ testCreateDelete (ATransport t) =
Resp "bcda" _ ok4 <- signSendRecv rh rKey ("bcda", rId, OFF)
(ok4, OK) #== "accepts OFF when suspended"
Resp "cdab" _ (Msg mId2 msg2) <- signSendRecv rh rKey ("cdab", rId, SUB)
Resp "cdab" _ (MSG mId2 _ _ msg2) <- signSendRecv rh rKey ("cdab", rId, SUB)
(dec mId2 msg2, Right "hello") #== "accepts SUB when suspended and delivers the message again (because was not ACKed)"
Resp "dabc" _ err5 <- sendRecv rh (sampleSig, "dabc", rId, DEL)
@@ -308,7 +209,7 @@ stressTest (ATransport t) =
smpTest3 t $ \h1 h2 h3 -> do
(rPub, rKey) <- C.generateSignatureKeyPair C.SEd25519
(dhPub, _ :: C.PrivateKeyX25519) <- C.generateKeyPair'
rIds <- forM ([1 .. 50] :: [Int]) . const $ do
rIds <- forM [1 .. 50 :: Int] . const $ do
Resp "" "" (Ids rId _ _) <- signSendRecv h1 rKey ("", "", NEW rPub dhPub)
pure rId
let subscribeQueues h = forM_ rIds $ \rId -> do
@@ -336,14 +237,14 @@ testDuplex (ATransport t) =
(arPub, arKey) <- C.generateSignatureKeyPair C.SEd448
(aDhPub, aDhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" _ (Ids aRcv aSnd aSrvDh) <- signSendRecv alice arKey ("abcd", "", NEW arPub aDhPub)
let aDec = decryptMsgV3 $ C.dh' aSrvDh aDhPriv
let aDec nonce = C.cbDecrypt (C.dh' aSrvDh aDhPriv) (C.cbNonce nonce)
-- aSnd ID is passed to Bob out-of-band
(bsPub, bsKey) <- C.generateSignatureKeyPair C.SEd448
Resp "bcda" _ OK <- sendRecv bob ("", "bcda", aSnd, _SEND $ "key " <> strEncode bsPub)
-- "key ..." is ad-hoc, not a part of SMP protocol
Resp "" _ (Msg mId1 msg1) <- tGet1 alice
Resp "" _ (MSG mId1 _ _ msg1) <- tGet alice
Resp "cdab" _ OK <- signSendRecv alice arKey ("cdab", aRcv, ACK mId1)
Right ["key", bobKey] <- pure $ B.words <$> aDec mId1 msg1
(bobKey, strEncode bsPub) #== "key received from Bob"
@@ -352,11 +253,11 @@ testDuplex (ATransport t) =
(brPub, brKey) <- C.generateSignatureKeyPair C.SEd448
(bDhPub, bDhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" _ (Ids bRcv bSnd bSrvDh) <- signSendRecv bob brKey ("abcd", "", NEW brPub bDhPub)
let bDec = decryptMsgV3 $ C.dh' bSrvDh bDhPriv
let bDec nonce = C.cbDecrypt (C.dh' bSrvDh bDhPriv) (C.cbNonce nonce)
Resp "bcda" _ OK <- signSendRecv bob bsKey ("bcda", aSnd, _SEND $ "reply_id " <> encode bSnd)
-- "reply_id ..." is ad-hoc, not a part of SMP protocol
Resp "" _ (Msg mId2 msg2) <- tGet1 alice
Resp "" _ (MSG mId2 _ _ msg2) <- tGet alice
Resp "cdab" _ OK <- signSendRecv alice arKey ("cdab", aRcv, ACK mId2)
Right ["reply_id", bId] <- pure $ B.words <$> aDec mId2 msg2
(bId, encode bSnd) #== "reply queue ID received from Bob"
@@ -365,7 +266,7 @@ testDuplex (ATransport t) =
Resp "dabc" _ OK <- sendRecv alice ("", "dabc", bSnd, _SEND $ "key " <> strEncode asPub)
-- "key ..." is ad-hoc, not a part of SMP protocol
Resp "" _ (Msg mId3 msg3) <- tGet1 bob
Resp "" _ (MSG mId3 _ _ msg3) <- tGet bob
Resp "abcd" _ OK <- signSendRecv bob brKey ("abcd", bRcv, ACK mId3)
Right ["key", aliceKey] <- pure $ B.words <$> bDec mId3 msg3
(aliceKey, strEncode asPub) #== "key received from Alice"
@@ -373,13 +274,13 @@ testDuplex (ATransport t) =
Resp "cdab" _ OK <- signSendRecv bob bsKey ("cdab", aSnd, _SEND "hi alice")
Resp "" _ (Msg mId4 msg4) <- tGet1 alice
Resp "" _ (MSG mId4 _ _ msg4) <- tGet alice
Resp "dabc" _ OK <- signSendRecv alice arKey ("dabc", aRcv, ACK mId4)
(aDec mId4 msg4, Right "hi alice") #== "message received from Bob"
Resp "abcd" _ OK <- signSendRecv alice asKey ("abcd", bSnd, _SEND "how are you bob")
Resp "" _ (Msg mId5 msg5) <- tGet1 bob
Resp "" _ (MSG mId5 _ _ msg5) <- tGet bob
Resp "bcda" _ OK <- signSendRecv bob brKey ("bcda", bRcv, ACK mId5)
(bDec mId5 msg5, Right "how are you bob") #== "message received from alice"
@@ -390,27 +291,27 @@ testSwitchSub (ATransport t) =
(rPub, rKey) <- C.generateSignatureKeyPair C.SEd448
(dhPub, dhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" _ (Ids rId sId srvDh) <- signSendRecv rh1 rKey ("abcd", "", NEW rPub dhPub)
let dec = decryptMsgV3 $ C.dh' srvDh dhPriv
let dec nonce = C.cbDecrypt (C.dh' srvDh dhPriv) (C.cbNonce nonce)
Resp "bcda" _ ok1 <- sendRecv sh ("", "bcda", sId, _SEND "test1")
(ok1, OK) #== "sent test message 1"
Resp "cdab" _ ok2 <- sendRecv sh ("", "cdab", sId, _SEND "test2, no ACK")
(ok2, OK) #== "sent test message 2"
Resp "" _ (Msg mId1 msg1) <- tGet1 rh1
Resp "" _ (MSG mId1 _ _ msg1) <- tGet rh1
(dec mId1 msg1, Right "test1") #== "test message 1 delivered to the 1st TCP connection"
Resp "abcd" _ (Msg mId2 msg2) <- signSendRecv rh1 rKey ("abcd", rId, ACK mId1)
Resp "abcd" _ (MSG mId2 _ _ msg2) <- signSendRecv rh1 rKey ("abcd", rId, ACK mId1)
(dec mId2 msg2, Right "test2, no ACK") #== "test message 2 delivered, no ACK"
Resp "bcda" _ (Msg mId2' msg2') <- signSendRecv rh2 rKey ("bcda", rId, SUB)
Resp "bcda" _ (MSG mId2' _ _ msg2') <- signSendRecv rh2 rKey ("bcda", rId, SUB)
(dec mId2' msg2', Right "test2, no ACK") #== "same simplex queue via another TCP connection, tes2 delivered again (no ACK in 1st queue)"
Resp "cdab" _ OK <- signSendRecv rh2 rKey ("cdab", rId, ACK mId2')
Resp "" _ end <- tGet1 rh1
Resp "" _ end <- tGet rh1
(end, END) #== "unsubscribed the 1st TCP connection"
Resp "dabc" _ OK <- sendRecv sh ("", "dabc", sId, _SEND "test3")
Resp "" _ (Msg mId3 msg3) <- tGet1 rh2
Resp "" _ (MSG mId3 _ _ msg3) <- tGet rh2
(dec mId3 msg3, Right "test3") #== "delivered to the 2nd TCP connection"
Resp "abcd" _ err <- signSendRecv rh1 rKey ("abcd", rId, ACK mId3)
@@ -433,9 +334,9 @@ testGetCommand t =
atomically . putTMVar queue =<< createAndSecureQueue rh sPub
testSMPClient @c $ \rh -> do
(sId, rId, rKey, dhShared) <- atomically $ takeTMVar queue
let dec = decryptMsgV3 dhShared
let dec nonce = C.cbDecrypt dhShared (C.cbNonce nonce)
Resp "1" _ OK <- signSendRecv sh sKey ("1", sId, _SEND "hello")
Resp "2" _ (Msg mId1 msg1) <- signSendRecv rh rKey ("2", rId, GET)
Resp "2" _ (MSG mId1 _ _ msg1) <- signSendRecv rh rKey ("2", rId, GET)
(dec mId1 msg1, Right "hello") #== "retrieved from queue"
Resp "3" _ OK <- signSendRecv rh rKey ("3", rId, ACK mId1)
Resp "4" _ OK <- signSendRecv rh rKey ("4", rId, GET)
@@ -447,14 +348,14 @@ testGetSubCommands t =
(sPub, sKey) <- C.generateSignatureKeyPair C.SEd25519
smpTest3 t $ \rh1 rh2 sh -> do
(sId, rId, rKey, dhShared) <- createAndSecureQueue rh1 sPub
let dec = decryptMsgV3 dhShared
let dec nonce = C.cbDecrypt dhShared (C.cbNonce nonce)
Resp "1" _ OK <- signSendRecv sh sKey ("1", sId, _SEND "hello 1")
Resp "1a" _ OK <- signSendRecv sh sKey ("1a", sId, _SEND "hello 2")
Resp "1b" _ OK <- signSendRecv sh sKey ("1b", sId, _SEND "hello 3")
Resp "1c" _ OK <- signSendRecv sh sKey ("1c", sId, _SEND "hello 4")
-- both get the same if not ACK'd
Resp "" _ (Msg mId1 msg1) <- tGet1 rh1
Resp "2" _ (Msg mId1' msg1') <- signSendRecv rh2 rKey ("2", rId, GET)
Resp "" _ (MSG mId1 _ _ msg1) <- tGet rh1
Resp "2" _ (MSG mId1' _ _ msg1') <- signSendRecv rh2 rKey ("2", rId, GET)
(dec mId1 msg1, Right "hello 1") #== "received from queue via SUB"
(dec mId1' msg1', Right "hello 1") #== "retrieved from queue with GET"
mId1 `shouldBe` mId1'
@@ -463,27 +364,27 @@ testGetSubCommands t =
Resp "3" _ (ERR (CMD PROHIBITED)) <- signSendRecv rh1 rKey ("3", rId, GET)
Resp "3a" _ (ERR (CMD PROHIBITED)) <- signSendRecv rh2 rKey ("3a", rId, SUB)
-- ACK for SUB delivers the next message
Resp "4" _ (Msg mId2 msg2) <- signSendRecv rh1 rKey ("4", rId, ACK mId1)
Resp "4" _ (MSG mId2 _ _ msg2) <- signSendRecv rh1 rKey ("4", rId, ACK mId1)
(dec mId2 msg2, Right "hello 2") #== "received from queue via SUB"
-- bad msgId returns error
Resp "5" _ (ERR NO_MSG) <- signSendRecv rh2 rKey ("5", rId, ACK "1234")
-- already ACK'd by subscriber, but still returns OK when msgId matches
Resp "5a" _ OK <- signSendRecv rh2 rKey ("5a", rId, ACK mId1)
-- msg2 is not lost - even if subscriber does not ACK it, it is delivered to getter
Resp "6" _ (Msg mId2' msg2') <- signSendRecv rh2 rKey ("6", rId, GET)
Resp "6" _ (MSG mId2' _ _ msg2') <- signSendRecv rh2 rKey ("6", rId, GET)
(dec mId2' msg2', Right "hello 2") #== "retrieved from queue with GET"
mId2 `shouldBe` mId2'
msg2 `shouldBe` msg2'
-- getter ACK returns OK, even though there is the next message
Resp "7" _ OK <- signSendRecv rh2 rKey ("7", rId, ACK mId2')
Resp "8" _ (Msg mId3 msg3) <- signSendRecv rh2 rKey ("8", rId, GET)
Resp "8" _ (MSG mId3 _ _ msg3) <- signSendRecv rh2 rKey ("8", rId, GET)
(dec mId3 msg3, Right "hello 3") #== "retrieved from queue with GET"
-- subscriber ACK does not lose message
Resp "9" _ (Msg mId3' msg3') <- signSendRecv rh1 rKey ("9", rId, ACK mId2')
Resp "9" _ (MSG mId3' _ _ msg3') <- signSendRecv rh1 rKey ("9", rId, ACK mId2')
(dec mId3' msg3', Right "hello 3") #== "retrieved from queue with GET"
mId3 `shouldBe` mId3'
msg3 `shouldBe` msg3'
Resp "10" _ (Msg mId4 msg4) <- signSendRecv rh1 rKey ("10", rId, ACK mId3)
Resp "10" _ (MSG mId4 _ _ msg4) <- signSendRecv rh1 rKey ("10", rId, ACK mId3)
(dec mId4 msg4, Right "hello 4") #== "retrieved from queue with GET"
Resp "11" _ OK <- signSendRecv rh1 rKey ("11", rId, ACK mId4)
-- no more messages for getter too
@@ -505,8 +406,7 @@ testWithStoreLog at@(ATransport t) =
withSmpServerStoreLogOn at testPort . runTest t $ \h -> runClient t $ \h1 -> do
(sId1, rId1, rKey1, dhShared) <- createAndSecureQueue h sPub1
(rcvNtfPubDhKey, _) <- C.generateKeyPair'
Resp "abcd" _ (NID nId _) <- signSendRecv h rKey1 ("abcd", rId1, NKEY nPub rcvNtfPubDhKey)
Resp "abcd" _ (NID nId) <- signSendRecv h rKey1 ("abcd", rId1, NKEY nPub)
atomically $ do
writeTVar recipientId1 rId1
writeTVar recipientKey1 $ Just rKey1
@@ -515,15 +415,15 @@ testWithStoreLog at@(ATransport t) =
writeTVar notifierId nId
Resp "dabc" _ OK <- signSendRecv h1 nKey ("dabc", nId, NSUB)
Resp "bcda" _ OK <- signSendRecv h sKey1 ("bcda", sId1, _SEND' "hello")
Resp "" _ (Msg mId1 msg1) <- tGet1 h
(decryptMsgV3 dhShared mId1 msg1, Right "hello") #== "delivered from queue 1"
Resp "" _ (NMSG _ _) <- tGet1 h1
Resp "" _ (MSG mId1 _ _ msg1) <- tGet h
(C.cbDecrypt dhShared (C.cbNonce mId1) msg1, Right "hello") #== "delivered from queue 1"
Resp "" _ NMSG <- tGet h1
(sId2, rId2, rKey2, dhShared2) <- createAndSecureQueue h sPub2
atomically $ writeTVar senderId2 sId2
Resp "cdab" _ OK <- signSendRecv h sKey2 ("cdab", sId2, _SEND "hello too")
Resp "" _ (Msg mId2 msg2) <- tGet1 h
(decryptMsgV3 dhShared2 mId2 msg2, Right "hello too") #== "delivered from queue 2"
Resp "" _ (MSG mId2 _ _ msg2) <- tGet h
(C.cbDecrypt dhShared2 (C.cbNonce mId2) msg2, Right "hello too") #== "delivered from queue 2"
Resp "dabc" _ OK <- signSendRecv h rKey2 ("dabc", rId2, DEL)
pure ()
@@ -545,9 +445,9 @@ testWithStoreLog at@(ATransport t) =
nId <- readTVarIO notifierId
Resp "dabc" _ OK <- signSendRecv h1 nKey ("dabc", nId, NSUB)
Resp "bcda" _ OK <- signSendRecv h sKey1 ("bcda", sId1, _SEND' "hello")
Resp "cdab" _ (Msg mId3 msg3) <- signSendRecv h rKey1 ("cdab", rId1, SUB)
(decryptMsgV3 dh1 mId3 msg3, Right "hello") #== "delivered from restored queue"
Resp "" _ (NMSG _ _) <- tGet1 h1
Resp "cdab" _ (MSG mId3 _ _ msg3) <- signSendRecv h rKey1 ("cdab", rId1, SUB)
(C.cbDecrypt dh1 (C.cbNonce mId3) msg3, Right "hello") #== "delivered from restored queue"
Resp "" _ NMSG <- tGet h1
-- this queue is removed - not restored
sId2 <- readTVarIO senderId2
Resp "cdab" _ (ERR AUTH) <- signSendRecv h sKey2 ("cdab", sId2, _SEND "hello too")
@@ -588,9 +488,9 @@ testRestoreMessages at@(ATransport t) =
writeTVar dhShared $ Just dh
writeTVar senderId sId
Resp "1" _ OK <- signSendRecv h sKey ("1", sId, _SEND "hello")
Resp "" _ (Msg mId1 msg1) <- tGet1 h1
Resp "" _ (MSG mId1 _ _ msg1) <- tGet h1
Resp "1a" _ OK <- signSendRecv h1 rKey ("1a", rId, ACK mId1)
(decryptMsgV3 dh mId1 msg1, Right "hello") #== "message delivered"
(C.cbDecrypt dh (C.cbNonce mId1) msg1, Right "hello") #== "message delivered"
-- messages below are delivered after server restart
sId <- readTVarIO senderId
Resp "2" _ OK <- signSendRecv h sKey ("2", sId, _SEND "hello 2")
@@ -605,13 +505,12 @@ testRestoreMessages at@(ATransport t) =
rId <- readTVarIO recipientId
Just rKey <- readTVarIO recipientKey
Just dh <- readTVarIO dhShared
let dec = decryptMsgV3 dh
Resp "2" _ (Msg mId2 msg2) <- signSendRecv h rKey ("2", rId, SUB)
(dec mId2 msg2, Right "hello 2") #== "restored message delivered"
Resp "3" _ (Msg mId3 msg3) <- signSendRecv h rKey ("3", rId, ACK mId2)
(dec mId3 msg3, Right "hello 3") #== "restored message delivered"
Resp "4" _ (Msg mId4 msg4) <- signSendRecv h rKey ("4", rId, ACK mId3)
(dec mId4 msg4, Right "hello 4") #== "restored message delivered"
Resp "2" _ (MSG mId2 _ _ msg2) <- signSendRecv h rKey ("2", rId, SUB)
(C.cbDecrypt dh (C.cbNonce mId2) msg2, Right "hello 2") #== "restored message delivered"
Resp "3" _ (MSG mId3 _ _ msg3) <- signSendRecv h rKey ("3", rId, ACK mId2)
(C.cbDecrypt dh (C.cbNonce mId3) msg3, Right "hello 3") #== "restored message delivered"
Resp "4" _ (MSG mId4 _ _ msg4) <- signSendRecv h rKey ("4", rId, ACK mId3)
(C.cbDecrypt dh (C.cbNonce mId4) msg4, Right "hello 4") #== "restored message delivered"
logSize testStoreLogFile `shouldReturn` 1
-- the last message is not removed because it was not ACK'd
@@ -621,78 +520,9 @@ testRestoreMessages at@(ATransport t) =
rId <- readTVarIO recipientId
Just rKey <- readTVarIO recipientKey
Just dh <- readTVarIO dhShared
Resp "4" _ (Msg mId4 msg4) <- signSendRecv h rKey ("4", rId, SUB)
Resp "4" _ (MSG mId4 _ _ msg4) <- signSendRecv h rKey ("4", rId, SUB)
Resp "5" _ OK <- signSendRecv h rKey ("5", rId, ACK mId4)
(decryptMsgV3 dh mId4 msg4, Right "hello 4") #== "restored message delivered"
logSize testStoreLogFile `shouldReturn` 1
logSize testStoreMsgsFile `shouldReturn` 0
removeFile testStoreLogFile
removeFile testStoreMsgsFile
where
runTest :: Transport c => TProxy c -> (THandle c -> IO ()) -> ThreadId -> Expectation
runTest _ test' server = do
testSMPClient test' `shouldReturn` ()
killThread server
runClient :: Transport c => TProxy c -> (THandle c -> IO ()) -> Expectation
runClient _ test' = testSMPClient test' `shouldReturn` ()
testRestoreMessagesV2 :: ATransport -> Spec
testRestoreMessagesV2 at@(ATransport t) =
it "should store messages on exit and restore on start" $ do
(sPub, sKey) <- C.generateSignatureKeyPair C.SEd25519
recipientId <- newTVarIO ""
recipientKey <- newTVarIO Nothing
dhShared <- newTVarIO Nothing
senderId <- newTVarIO ""
withSmpServerStoreMsgLogOnV2 at testPort . runTest t $ \h -> do
runClient t $ \h1 -> do
(sId, rId, rKey, dh) <- createAndSecureQueue h1 sPub
atomically $ do
writeTVar recipientId rId
writeTVar recipientKey $ Just rKey
writeTVar dhShared $ Just dh
writeTVar senderId sId
Resp "1" _ OK <- signSendRecv h sKey ("1", sId, _SEND "hello")
Resp "" _ (Msg mId1 msg1) <- tGet1 h1
Resp "1a" _ OK <- signSendRecv h1 rKey ("1a", rId, ACK mId1)
(decryptMsgV2 dh mId1 msg1, Right "hello") #== "message delivered"
-- messages below are delivered after server restart
sId <- readTVarIO senderId
Resp "2" _ OK <- signSendRecv h sKey ("2", sId, _SEND "hello 2")
Resp "3" _ OK <- signSendRecv h sKey ("3", sId, _SEND "hello 3")
Resp "4" _ OK <- signSendRecv h sKey ("4", sId, _SEND "hello 4")
pure ()
logSize testStoreLogFile `shouldReturn` 2
logSize testStoreMsgsFile `shouldReturn` 3
withSmpServerStoreMsgLogOnV2 at testPort . runTest t $ \h -> do
rId <- readTVarIO recipientId
Just rKey <- readTVarIO recipientKey
Just dh <- readTVarIO dhShared
let dec = decryptMsgV2 dh
Resp "2" _ (Msg mId2 msg2) <- signSendRecv h rKey ("2", rId, SUB)
(dec mId2 msg2, Right "hello 2") #== "restored message delivered"
Resp "3" _ (Msg mId3 msg3) <- signSendRecv h rKey ("3", rId, ACK mId2)
(dec mId3 msg3, Right "hello 3") #== "restored message delivered"
Resp "4" _ (Msg mId4 msg4) <- signSendRecv h rKey ("4", rId, ACK mId3)
(dec mId4 msg4, Right "hello 4") #== "restored message delivered"
logSize testStoreLogFile `shouldReturn` 1
-- the last message is not removed because it was not ACK'd
logSize testStoreMsgsFile `shouldReturn` 1
withSmpServerStoreMsgLogOnV2 at testPort . runTest t $ \h -> do
rId <- readTVarIO recipientId
Just rKey <- readTVarIO recipientKey
Just dh <- readTVarIO dhShared
Resp "4" _ (Msg mId4 msg4) <- signSendRecv h rKey ("4", rId, SUB)
Resp "5" _ OK <- signSendRecv h rKey ("5", rId, ACK mId4)
(decryptMsgV2 dh mId4 msg4, Right "hello 4") #== "restored message delivered"
(C.cbDecrypt dh (C.cbNonce mId4) msg4, Right "hello 4") #== "restored message delivered"
logSize testStoreLogFile `shouldReturn` 1
logSize testStoreMsgsFile `shouldReturn` 0
@@ -722,15 +552,14 @@ testTiming :: ATransport -> Spec
testTiming (ATransport t) =
it "should have similar time for auth error, whether queue exists or not, for all key sizes" $
smpTest2 t $ \rh sh ->
mapM_ (testSameTiming rh sh) timingTests
mapM_
(testSameTiming rh sh)
[ (32, 32, 200),
(32, 57, 100),
(57, 32, 200),
(57, 57, 100)
]
where
timingTests :: [(Int, Int, Int)]
timingTests =
[ (32, 32, 200),
(32, 57, 100),
(57, 32, 200),
(57, 57, 100)
]
timeRepeat n = fmap fst . timeItT . forM_ (replicate n ()) . const
similarTime t1 t2 = abs (t2 / t1 - 1) < 0.25 `shouldBe` True
testSameTiming :: Transport c => THandle c -> THandle c -> (Int, Int, Int) -> Expectation
@@ -738,7 +567,7 @@ testTiming (ATransport t) =
(rPub, rKey) <- generateKeys goodKeySize
(dhPub, dhPriv :: C.PrivateKeyX25519) <- C.generateKeyPair'
Resp "abcd" "" (Ids rId sId srvDh) <- signSendRecv rh rKey ("abcd", "", NEW rPub dhPub)
let dec = decryptMsgV3 $ C.dh' srvDh dhPriv
let dec nonce = C.cbDecrypt (C.dh' srvDh dhPriv) (C.cbNonce nonce)
Resp "cdab" _ OK <- signSendRecv rh rKey ("cdab", rId, SUB)
(_, badKey) <- generateKeys badKeySize
@@ -748,7 +577,7 @@ testTiming (ATransport t) =
Resp "dabc" _ OK <- signSendRecv rh rKey ("dabc", rId, KEY sPub)
Resp "bcda" _ OK <- signSendRecv sh sKey ("bcda", sId, _SEND "hello")
Resp "" _ (Msg mId msg) <- tGet1 rh
Resp "" _ (MSG mId _ _ msg) <- tGet rh
(dec mId msg, Right "hello") #== "delivered from queue"
runTimingTest sh badKey sId $ _SEND "hello"
@@ -780,34 +609,23 @@ testMessageNotifications (ATransport t) =
(nPub, nKey) <- C.generateSignatureKeyPair C.SEd25519
smpTest4 t $ \rh sh nh1 nh2 -> do
(sId, rId, rKey, dhShared) <- createAndSecureQueue rh sPub
let dec = decryptMsgV3 dhShared
(rcvNtfPubDhKey, _) <- C.generateKeyPair'
Resp "1" _ (NID nId' _) <- signSendRecv rh rKey ("1", rId, NKEY nPub rcvNtfPubDhKey)
Resp "1a" _ (NID nId _) <- signSendRecv rh rKey ("1a", rId, NKEY nPub rcvNtfPubDhKey)
nId' `shouldNotBe` nId
let dec nonce = C.cbDecrypt dhShared (C.cbNonce nonce)
Resp "1" _ (NID nId) <- signSendRecv rh rKey ("1", rId, NKEY nPub)
Resp "2" _ OK <- signSendRecv nh1 nKey ("2", nId, NSUB)
Resp "3" _ OK <- signSendRecv sh sKey ("3", sId, _SEND' "hello")
Resp "" _ (Msg mId1 msg1) <- tGet1 rh
Resp "" _ (MSG mId1 _ _ msg1) <- tGet rh
(dec mId1 msg1, Right "hello") #== "delivered from queue"
Resp "3a" _ OK <- signSendRecv rh rKey ("3a", rId, ACK mId1)
Resp "" _ (NMSG _ _) <- tGet1 nh1
Resp "" _ NMSG <- tGet nh1
Resp "4" _ OK <- signSendRecv nh2 nKey ("4", nId, NSUB)
Resp "" _ END <- tGet1 nh1
Resp "" _ END <- tGet nh1
Resp "5" _ OK <- signSendRecv sh sKey ("5", sId, _SEND' "hello again")
Resp "" _ (Msg mId2 msg2) <- tGet1 rh
Resp "5a" _ OK <- signSendRecv rh rKey ("5a", rId, ACK mId2)
Resp "" _ (MSG mId2 _ _ msg2) <- tGet rh
(dec mId2 msg2, Right "hello again") #== "delivered from queue again"
Resp "" _ (NMSG _ _) <- tGet1 nh2
Resp "" _ NMSG <- tGet nh2
1000 `timeout` tGet @BrokerMsg nh1 >>= \case
Nothing -> pure ()
Nothing -> return ()
Just _ -> error "nothing else should be delivered to the 1st notifier's TCP connection"
Resp "6" _ OK <- signSendRecv rh rKey ("6", rId, NDEL)
Resp "7" _ OK <- signSendRecv sh sKey ("7", sId, _SEND' "hello there")
Resp "" _ (Msg mId3 msg3) <- tGet1 rh
(dec mId3 msg3, Right "hello there") #== "delivered from queue again"
1000 `timeout` tGet @BrokerMsg nh2 >>= \case
Nothing -> pure ()
Just _ -> error "nothing else should be delivered to the 2nd notifier's TCP connection"
testMsgExpireOnSend :: forall c. Transport c => TProxy c -> Spec
testMsgExpireOnSend t =
@@ -817,12 +635,12 @@ testMsgExpireOnSend t =
withSmpServerConfigOn (ATransport t) cfg' testPort $ \_ ->
testSMPClient @c $ \sh -> do
(sId, rId, rKey, dhShared) <- testSMPClient @c $ \rh -> createAndSecureQueue rh sPub
let dec = decryptMsgV3 dhShared
let dec nonce = C.cbDecrypt dhShared (C.cbNonce nonce)
Resp "1" _ OK <- signSendRecv sh sKey ("1", sId, _SEND "hello (should expire)")
threadDelay 2500000
Resp "2" _ OK <- signSendRecv sh sKey ("2", sId, _SEND "hello (should NOT expire)")
testSMPClient @c $ \rh -> do
Resp "3" _ (Msg mId msg) <- signSendRecv rh rKey ("3", rId, SUB)
Resp "3" _ (MSG mId _ _ msg) <- signSendRecv rh rKey ("3", rId, SUB)
(dec mId msg, Right "hello (should NOT expire)") #== "delivered"
1000 `timeout` tGet @BrokerMsg rh >>= \case
Nothing -> return ()
@@ -852,11 +670,11 @@ testMsgNOTExpireOnInterval t =
withSmpServerConfigOn (ATransport t) cfg' testPort $ \_ ->
testSMPClient @c $ \sh -> do
(sId, rId, rKey, dhShared) <- testSMPClient @c $ \rh -> createAndSecureQueue rh sPub
let dec = decryptMsgV3 dhShared
let dec nonce = C.cbDecrypt dhShared (C.cbNonce nonce)
Resp "1" _ OK <- signSendRecv sh sKey ("1", sId, _SEND "hello (should NOT expire)")
threadDelay 2500000
testSMPClient @c $ \rh -> do
Resp "2" _ (Msg mId msg) <- signSendRecv rh rKey ("2", rId, SUB)
Resp "2" _ (MSG mId _ _ msg) <- signSendRecv rh rKey ("2", rId, SUB)
(dec mId msg, Right "hello (should NOT expire)") #== "delivered"
1000 `timeout` tGet @BrokerMsg rh >>= \case
Nothing -> return ()