Instead of removing the --transport-loglevel flag, we will mark it
as deprecated. It was no longer used in any logging capacity but
removing the flag will cause existing cloudflared clients who set
the flag to fail to start.
The dedicated transport logger was never used for
log emission, so remove its construction and plumbing through tunnel
configuration, the supervisor, and the connection observer.
Remove the unused --transport-loglevel flag and its legacy aliases and
environment variables. The dedicated transport logger was never used for
log emission, so remove its construction and plumbing through tunnel
configuration, the supervisor, and the connection observer.
QUIC previously replaced control-stream registration failures with generic
service errors, preventing the supervisor from identifying edge responses such
as EDUPCONN and retryable registration failures. This could repeatedly select
the same edge address or terminate initial registration unexpectedly.
Preserve underlying causes through QUIC service wrappers, including failures
opening the control stream, while returning context cancellation directly.
Represent duplicate registration with a sentinel and use errors.Is and
errors.AsType to classify wrapped errors consistently.
Keep registration permanence available to the supervisor, rotate edge
addresses after EDUPCONN, and classify QUIC dial errors before nested idle
timeouts so connectivity retries can still trigger protocol fallback. Remove
duplicate connection-error logging and suppress expected shutdown noise.
Add coverage for duplicate registration mapping, QUIC error propagation,
cancellation, startup retry policy, edge-address rotation, and wrapped QUIC
dial timeouts.
Stop fetching protocol rollout percentages from DNS TXT records.
Remove the obsolete remote selector, percentage-fetching API, and
related tests, and simplify protocol selector construction.
Delete the unused tunnelsForHA implementations and stale edge
discovery test mocks. Remove unpopulated metrics and simplify
the stream copy path by dropping its permanently disabled debug
implementation.
Remove the hidden stdin-control flag and ReconnectSignal plumbing used to
force tunnel reconnections through stdin. Delete the associated component
test and supervisor control implementation.
Serve QUIC and HTTP/2 connections directly with the caller context instead
of wrapping them in an additional errgroup for reconnect signals. Automatic
reconnection after genuine transport failures remains unchanged.
This removal should help simplify the supervisor loop a bit and make way
for better improvements in this area.
Closes TUN-10820
Introduce a new crypto/ package as the single source of truth for TLS
curve preferences used on every edge-facing connection, and adopt
X25519MLKEM768 as the primary post-quantum key exchange for both QUIC
and HTTP/2:
PQ Prefer (default): X25519MLKEM768, P256Kyber768Draft00, CurveP256
PQ Strict (--post-quantum): X25519MLKEM768, P256Kyber768Draft00
The curve list is identical under FIPS and non-FIPS builds, so
crypto.GetCurvePreferences takes only a features.PostQuantumMode and
returns a fresh slice on every call.
HTTP/2 now applies these curve preferences the same way QUIC does. The
previous PostQuantumStrict rejection in serveHTTP2 and the forced
QUIC-only selection in NewProtocolSelector are removed since both
transports support the same post-quantum curves; the needPQ parameter
is dropped from NewProtocolSelector accordingly.
Also fix a shared tls.Config race: both the QUIC and HTTP/2 paths now
Clone() the per-protocol entry from TunnelConfig.EdgeTLSConfigs before
mutating CurvePreferences instead of writing through the shared map
entry.
Legacy Kyber draft curve X25519Kyber768Draft00
and the unused removeDuplicates helper are removed along with the old
supervisor/pqtunnels.go / _test.go files.
AGENTS.md is updated with guidance on the new crypto/ package, the
cfdcrypto import alias, the tls.Config cloning rule, and the lint
workflow implications of .golangci.yaml's whole-files: true setting.
Final run method, which runs cloudlflared pre-checks for both the normal startup procedure, as well as cloudflared diag.
For cloudflared diag, this produces a new json output to which is added to the final zip file.
Also added in a new flag to prevent this from running all the time, at least for now until we are 100% sure this works as intended. We will later remove this flag, only leaving in `--no-prechecks`, so this runs by default for everyone using cloudflared.
Tested pre-checks locally with origintunneld. The results show all pre-checks succeeding. In this case, it ran with only 1 region, since locally we run it with `--edge origintunneld1:7844`.
{width=900 height=217}
Corrects the pattern of using errgroup's and context cancellation to simplify the logic for canceling extra routines for the QUIC connection. This is because the extra context cancellation is redundant with the fact that the errgroup also cancels it's own provided context when a routine returns (error or not).
For the datagram handler specifically, since it can respond faster to a context cancellation from the QUIC connection, we wrap the error before surfacing it outside of the QUIC connection scope to the supervisor. Additionally, the supervisor will look for this error type to check if it should retry the QUIC connection. These two operations are required because the supervisor does not look for a context canceled error when deciding to retry a connection. If a context canceled from the datagram handler were to be returned up to the supervisor on the initial connection, the cloudflared application would exit. We want to ensure that cloudflared maintains connection attempts even if any of the services on-top of a QUIC connection fail (datagram handler in this case).
Additional logging is also introduced along these paths to help with understanding the error conditions from the specific handlers on-top of a QUIC connection.
Related CUSTESC-53681
Closes TUN-9610
Remove P256Kyber768Draft00PQKex curve from nonFips curve preferences and add tests to verify that the advertised curves are the same as the curve preferences we set.
Closes TUN-9161
Adds an OriginDialerService that takes over the roles of both DialUDP and DialTCP
towards the origin. This provides the possibility to leverage dialer "middleware"
to inject virtual origins, such as the DNS resolver service.
DNS Resolver service also gains access to the DialTCP operation to service TCP
DNS requests.
Minor refactoring includes changes to remove the needs previously provided by
the warp-routing configuration. This configuration cannot be disabled by cloudflared
so many of the references have been adjusted or removed.
Closes TUN-9470
Introduces a new `UDPOriginProxy` interface and `UDPOriginService`
to standardize how UDP connections are dialed to origins. Allows for
future overrides of the ingress service for specific dial destinations.
Simplifies dependency injection for UDP dialing throughout both datagram
v2 and v3 by using the same ingress service. Previous invocations called
into a DialUDP function in the ingress package that was a light
wrapper over `net.DialUDP`. Now a reference is passed into both datagram
controllers that allows more control over the DialUDP method.
Closes TUN-9469
Make sure to enforce snapshots of features and client information for each connection
so that the feature information can change in the background. This allows for new features
to only be applied to a connection if it completely disconnects and attempts a reconnect.
Updates the feature refresh time to 1 hour from previous cloudflared versions which
refreshed every 6 hours.
Closes TUN-9319
## Summary
Within the work of FEDRamp it is necessary to change the HA SD lookup to use as srv `fed-v2-origintunneld`
This work assumes that the tunnel token has an optional endpoint field which will be used to modify the behaviour of the HA SD lookup.
Finally, the presence of the endpoint will override region to _fed_ and fail if any value is passed for the flag region.
Closes TUN-9007
## Summary
Nowadays, Cloudflared only supports X25519Kyber768Draft00 (0x6399,25497) but older versions may use different preferences.
For FIPS compliance we are required to use P256Kyber768Draft00 (0xfe32,65074) which is supported in our internal fork of [Go-Boring-1.22.10](https://bitbucket.cfdata.org/projects/PLAT/repos/goboring/browse?at=refs/heads/go-boring/1.22.10 "Follow link").
In the near future, Go will support by default the X25519MLKEM768 (0x11ec,4588) given this we may drop the usage of our public fork of GO.
To summarise:
* Cloudflared FIPS: QUIC_CURVE_PREFERENCES=65074
* Cloudflared non-FIPS: QUIC_CURVE_PREFERENCES=4588
Closes TUN-8855
## Summary
Session is the concept used for UDP flows. Therefore, to make
the session limiter ambiguous for both TCP and UDP, this commit
renames it to flow limiter.
Closes TUN-8861
## Summary
In order to make cloudflared behavior more predictable and
prevent an exhaustion of resources, we have decided to add
session limits that can be configured by the user. This first
commit introduces the session limiter and adds it to the UDP
handling path. For now the limiter is set to run only in
unlimited mode.
Support rolling out the `support_datagram_v3` feature via remote feature rollout (DNS TXT record) with `dv3` key.
Consolidated some of the feature evaluation code into the features module to simplify the lookup of available features at runtime.
Reduced complexity for management logs feature lookup since it's a default feature.
Closes TUN-8807
A new ICMPResponder interface is introduced to provide different
implementations of how the ICMP flows should return to the QUIC
connection muxer.
Improves usages of netip.AddrPort to leverage the embedded zone
field for IPv6 addresses.
Closes TUN-8640
When a registration response from cloudflared gets lost on it's way back to the edge, the edge service will retry and send another registration request. Since cloudflared already has bound the local UDP socket for the provided request id, we want to re-send the registration response.
There are three types of retries that the edge will send:
1. A retry from the same QUIC connection index; cloudflared will just respond back with a registration response and reset the idle timer for the session.
2. A retry from a different QUIC connection index; cloudflared will need to migrate the current session connection to this new QUIC connection and reset the idle timer for the session.
3. A retry to a different cloudflared connector; cloudflared will eventually time the session out since no further packets will arrive to the session at the original connector.
Closes TUN-8709
The current supervisor serves the quic connection by performing all of the following in one method:
1. Dial QUIC edge connection
2. Initialize datagram muxer for UDP sessions and ICMP
3. Wrap all together in a single struct to serve the process loops
In an effort to better support modularity, each of these steps were broken out into their own separate methods that the supervisor will compose together to create the TunnelConnection and run its `Serve` method.
This also provides us with the capability to better interchange the functionality supported by the datagram session manager in the future with a new mechanism.
Closes TUN-8661
For macOS, we want to set the DF bit for the UDP packets used by the QUIC
connection; to achieve this, you need to explicitly set the network
to either "udp4" or "udp6". When determining which network type to pick
we need to use the edge IP address chosen to align with what the local
IP family interface we will use. This means we want cloudflared to bind
to local interfaces for a random port, so we provide a zero IP and 0 port
number (ex. 0.0.0.0:0). However, instead of providing the zero IP, we
can leave the value as nil and let the kernel decide which interface and
pick a random port as defined by the target edge IP family.
This was previously broken for IPv6-only edge connectivity on macOS and
all other operating systems should be unaffected because the network type
was left as default "udp" which will rely on the provided local or remote
IP for selection.
Closes TUN-8688
Some more legacy h2mux code to be cleaned up and moved out of the way.
The h2mux.Header used in the serialization for http2 proxied headers is moved to connection module. Additionally, the booleanfuse structure is also moved to supervisor as it is also needed. Both of these structures could be evaluated later for removal/updates, however, the intent of the proposed changes here is to remove the dependencies on the h2mux code and removal.
Approved-by: Chung-Ting Huang <[email protected]>
Approved-by: Luis Neto <[email protected]>
Approved-by: Gonçalo Garcia <[email protected]>
MR: https://gitlab.cfdata.org/cloudflare/tun/cloudflared/-/merge_requests/1576
Whenever cloudflared receives a SIGTERM or SIGINT it goes into graceful shutdown mode, which unregisters the connection and closes the control stream. Unregistering makes it so we no longer receive any new requests and makes the edge close the connection, allowing in-flight requests to finish (within a 3 minute period).
This was working fine for http2 connections, but the quic proxy was cancelling the context as soon as the controls stream ended, forcing the process to stop immediately.
This commit changes the behavior so that we wait the full grace period before cancelling the request
Revert "TUN-8621: Fix cloudflared version in change notes."
Revert "PPIP-2310: Update quick tunnel disclaimer"
Revert "TUN-8621: Prevent QUIC connection from closing before grace period after unregistering"
Revert "TUN-8484: Print response when QuickTunnel can't be unmarshalled"
Revert "TUN-8592: Use metadata from the edge to determine if request body is empty for QUIC transport"
Whenever cloudflared receives a SIGTERM or SIGINT it goes into graceful shutdown mode, which unregisters the connection and closes the control stream. Unregistering makes it so we no longer receive any new requests and makes the edge close the connection, allowing in-flight requests to finish (within a 3 minute period).
This was working fine for http2 connections, but the quic proxy was cancelling the context as soon as the controls stream ended, forcing the process to stop immediately.
This commit changes the behavior so that we wait the full grace period before cancelling the request
Since legacy tunnels have been removed for a while now, we can remove
many of the capnp rpc interfaces that are no longer leveraged by the
legacy tunnel registration and authentication mechanisms.
A clock structure was used to help support unit testing timetravel
but it is a globally shared object and is likely unsafe to share
across tests. Reordering of the tests seemed to have intermittent
failures for the TestWaitForBackoffFallback specifically on windows
builds.
Adjusting this to be a shim inside the BackoffHandler struct should
resolve shared object overrides in unit testing.
Additionally, added the reset retries functionality to be inline with
the ResetNow function of the BackoffHandler to align better with
expected functionality of the method.
Removes unused reconnectCredentialManager.
Combines the tunnelrpc and quic/schema capnp files into the same module.
To help reduce future issues with capnp id generation, capnpids are
provided in the capnp files from the existing capnp struct ids generated
in the go files.
Reduces the overall interface of the Capnp methods to the rest of
the code by providing an interface that will handle the quic protocol
selection.
Introduces a new `rpc-timeout` config that will allow all of the
SessionManager and ConfigurationManager RPC requests to have a timeout.
The timeout for these values is set to 5 seconds as non of these operations
for the managers should take a long time to complete.
Removed the RPC-specific logger as it never provided good debugging value
as the RPC method names were not visible in the logs.
## Summary
To prevent bad eyeballs and severs to be able to exhaust the quic
control flows we are adding the possibility of having a timeout
for a write operation to be acknowledged. This will prevent hanging
connections from exhausting the quic control flows, creating a DDoS.
Also update golang.org/x/net and google.golang.org/grpc to fix vulnerabilities,
although cloudflared is using them in a way that is not exposed to those risks
Devin Carr