This removes the last few places in lib/auth/* that were
consuming modules with modules.GetModules and setting
them via modulestest.SetTestModules.
Contributes to https://github.com/gravitational/teleport/issues/62799.
The library has had a number of CVEs recently and uses shady
dependencies. https://github.com/gravitational/teleport.e/pull/8513
removed the dependency pulled in via teleport.e. This updates the
e_imports.go to reflect that change which moved buger/jsonparser to
an indirect dependency.
To remove the dependency entirely, mark3labs/mcp-go was updated to
the latest version. In that version they have replaced
github.com/invopop/jsonschema, which brough in buger/jsonparser,
with github.com/google/jsonschema-go. As a result buger/jsonparser
is no longer in our dependency tree.
* Detect EC2 join failures and surface via readyz
* Replace readyz join check with systemd log capture
* Add JoinFailureTimeout constant
The installer's readyz polling timeout is referenced by both
the autodiscover installer and the SSM waiter. A shared constant
avoids drift between the two values and keeps the user-facing
status message in sync with the actual wait duration.
* Replace two-attempt join check with poll loop
The join health check now polls the readyz endpoint in a loop
until the node reports ready or a five-minute timeout elapses.
Previously the check ran at most twice with a fixed delay of
thirty seconds, which missed nodes that needed longer to join
and could not distinguish a slow start from a permanent failure.
On timeout the error includes a best-effort systemd service
snapshot gathered via systemctl show. This replaces the earlier
systemctl is-active gate that blocked readyz from being checked
at all when the service was in a non-active or transient state.
The readyz function returns a simple (ready, error) pair instead
of the earlier four-value tuple, and per-attempt timeouts that
hit deadline-exceeded are treated as transient so that polling
continues. A clockwork.Clock field enables deterministic time
control in tests.
* Align SSM waiter timeout with poll timeout
The SSM command waiter used a hardcoded 100-second wait that was
inherited from the v1 SDK default. The installer now polls readyz
for up to five minutes, so the SSM waiter must outlast that to
avoid reporting a stale in-progress state as the final outcome.
The new timeout is the join-failure timeout plus a ten-minute pad.
* Update docs for readyz poll loop errors
Error patterns, status messages, and the example event code now
reflect the polling-based join health check. The event code moves
from TDS01E to TDS00W, and the user task description no longer
assumes an IAM join handshake rejection as the sole failure mode.
* Trim SSM stdout/stderr in audit events
SSMRun audit events can carry unbounded stdout/stderr from
command invocations, risking the 64 KB stream message limit.
Cap each field at 24 000 characters (matching the per-field
cap in the event schema) by keeping only the trailing portion
and aligning the cut to a full-line boundary so diagnostics
stay readable.
* Simplify output.
Co-authored-by: Roman Tkachenko <[email protected]>
* Expand RedactFlagArgs doc comment
* Add IsConnectionError to utils package
Move the socket-level connection check from the installer package
into the shared utils package. The readyz checker needs this to
distinguish transient socket errors from application failures,
and the existing helpers in utils already cover related network
error classification.
* Remove install-script binary-exists guard
The shell installer no longer skips the download when a Teleport
binary already exists on disk. Previously the guard would bypass
reinstallation, but the installer is now expected to always run so
that package-managed upgrades and configuration changes take effect.
* Remove readiness handler tests from lib/service
These tests covered the HTTP readyz handler that is no longer
exercised through the service-level test surface. Readyz check
behaviour is now validated through the installer readyz checker
tests, making these redundant.
* Simplify join-failure error output
The error writer no longer unpacks trace message slices or strips
standalone sentinel lines before printing. Errors now flow through
as plain strings, which is sufficient because the upstream assembly
was simplified to produce one coherent error message.
* Extract readyz checker and diagnostics helpers
Split readyz check logic and systemd diagnostics out of the large
autodiscover file into dedicated files. The readyz checker becomes
a standalone struct with its own per-attempt timeout, connection
error classification, and test suite. Diagnostics helpers such as
journal capture, service state gathering, and the systemctl runner
move into a diagnostics file alongside ErrJoinFailure and related
constants.
The poll loop in checkJoinHealth is restructured to perform an
immediate first check before entering the timer loop, and error
assembly is inlined instead of threading through append helpers.
Tests now use a fake clock and goroutines to exercise the timer
driven polling flow deterministically.
* Move JoinFailureTimeout to installer package
Relocate JoinFailureTimeout from installstatus/exitcodes.go
into installer/autodiscover.go where the readyz polling
logic that consumes it lives. The exit-code String() method
now returns a generic timeout message; ssm_install.go
appends the concrete duration for the join-failure case.
* Reuse poll timer in readyz health check loop
The readyz poll loop allocated a new timer on every iteration and
checked readiness only after waiting. The loop now runs the first
check immediately before entering the poll cycle and reuses a single
timer via Reset, reducing per-iteration allocations. The ready state
drives the loop condition directly, moving the success return after
the loop so the function ends with the diagnostic-failure path at
the bottom.
* Replace binary mocks with function overrides
Join-health timeout tests relied on bintest process mocks for both
systemctl and journalctl, requiring filesystem setup, path wiring,
and post-run invocation assertions for every test case. Diagnostics
and journal capture now use optional function-level overrides that
are injected through the installer config, with thin wrapper methods
that fall through to the real implementation when no override is set.
Previously each test needed six to eight lines of mock orchestration
that obscured the actual test logic.
* Update lib/srv/server/installer/autodiscover_test.go
Scope context to test.
Co-authored-by: Chris Thach <[email protected]>
* Update lib/srv/server/installer/autodiscover_test.go
Scope context to test.
Co-authored-by: Chris Thach <[email protected]>
* Update lib/srv/server/installer/autodiscover_test.go
Scope context to test.
Co-authored-by: Chris Thach <[email protected]>
* Update lib/srv/server/installer/autodiscover_test.go
nstaller/autodiscover_test.go
Co-authored-by: Chris Thach <[email protected]>
* Simplify arg redaction loop to use range
Co-authored-by: Chris Thach <[email protected]>
* Simplify override fields to defaults in config
Co-authored-by: Tiago Silva <[email protected]>
* Add JoinFailureError structured error type
Replace the ErrJoinFailure sentinel with a JoinFailureError struct
that carries message, service diagnostics, journal output, and last
readyz error as separate fields. Previously callers had to parse a
concatenated string to extract individual diagnostic sections, making
downstream formatting and programmatic inspection fragile.
* Surface unexpected readyz errors to caller
Return unexpected errors from the readyz check instead of swallowing
them and returning false. The polling loop in checkJoinHealth keeps
retrying on these errors but now records the last one so it appears
in the timeout diagnostics. Previously a transient HTTP or socket
error was silently discarded, making it impossible to distinguish
"not ready yet" from "endpoint unreachable."
* Replace clock and overrides with function fields
Remove the clockwork dependency from the join-health check and use
the standard time package directly. Promote journalOverride and the
diagnosticsOverride to first-class diagnostics and journal function
fields initialized in checkAndSetDefaults. Build a JoinFailureError
on timeout instead of concatenating string parts. Tests switch from
fakeClock and BlockUntilContext to synctest.Test with time.Sleep,
removing the need for manual clock advancing.
* Use JoinFailureError in CLI error output
Switch the CLI join-failure path from errors.Is with the old sentinel
to errors.As with JoinFailureError. The writer function now accepts
the typed error and prints each diagnostic field on its own line.
Previously the error was printed as a single opaque string, making it
harder to scan in SSM console output.
* Normalize dash prefix in flag redaction
Add a lookupRedactor helper that maps both single-dash flags to their
double-dash redactor registration and vice versa, so callers need not
register both variants. Previously a -token=secret argument passed
through unredacted because only --token was registered in the redactor
map.
* Improve trace msg
Co-authored-by: Gavin Frazar <[email protected]>
* Minor refactor
Co-authored-by: Gavin Frazar <[email protected]>
* Minor refactor
Co-authored-by: Gavin Frazar <[email protected]>
* Minor refactor
Co-authored-by: Gavin Frazar <[email protected]>
* Minor refactor
Co-authored-by: Gavin Frazar <[email protected]>
* Minor refactor
Co-authored-by: Gavin Frazar <[email protected]>
* Minor log improvement
Co-authored-by: Gavin Frazar <[email protected]>
* Remove redundant check
Co-authored-by: Gavin Frazar <[email protected]>
* Quote command+args
Co-authored-by: Gavin Frazar <[email protected]>
* Remove token-redaction utility and its usage
Log node-configure arguments directly instead of redacting the join
token. The token is a short-lived secret issued by the auth server
for EC2 auto-discovery; it expires before log entries are typically
reviewed, so the redaction layer added complexity without meaningful
security benefit. Previously every call to the node-configure step
copied the argument slice through a redactor map and a flag-parsing
loop.
Replace hardcoded five-second sleeps in health-check tests with the
installer's readyzPollInterval field so the tests stay correct when
the interval changes. Remove a stray closing brace left over from a
prior edit.
* Improve diagnostics error messages and comments
Add a comment explaining the systemctl-show output format that the
property parser expects. Quote "systemctl show" in the log message
for the invocation-ID retrieval failure so it renders as a distinct
command name rather than bare words. Switch the infrastructure error
path to use the %q format verb with the full command object so the
message includes the complete binary path instead of manually joining
the argument list.
* Use server clock for EC2 installation SyncTime
Set SyncTime to the discovery server's clock instead of the SSM run
event timestamp. Previously the field carried the event time, which
reflects when SSM dispatched the command rather than when the server
recorded the installation result, causing user-task entries to show
stale timestamps that did not advance on re-evaluation.
* Expand trimToRecentTail godoc comment
* Bound diagnostics collection after join timeout
Diagnostics and journal capture now run under a dedicated five-second
timeout derived from the parent context. Previously they inherited the
caller's context with no independent deadline, so a hung systemctl or
journalctl call could block indefinitely after the readyz poll already
determined the node failed to join.
Journal capture errors are downgraded from a hard return to a warning
log entry. A transient journalctl failure was masking the underlying
join-failure error, causing the caller to receive a journal-capture
error instead of the structured JoinFailureError that carries the
diagnostics payload.
* Document waitTimeoutPad tradeoff and TODO
* Replace systemctl subprocess with D-Bus API
Diagnostics collection now queries systemd properties through the
native D-Bus connection rather than shelling out to systemctl show
and parsing stdout. The D-Bus interface needs fully-qualified unit
names, so the unit-name builder appends a ".service" suffix to each
returned value. This eliminates the systemctl binary-path dependency
during diagnostics and avoids subprocess-parsing fragility around
exit-code versus infrastructure error disambiguation.
* Fix go mod
* Bump how many journal lines we output
* Extract systemd client into lib/systemd package
The systemd D-Bus property lookups, invocation ID retrieval, and the
journalctl capture now live in a reusable lib/systemd package that
exposes a Client with ReadServiceState, InvocationID, and CaptureJournal
methods. Previously these helpers lived as unexported methods on the
AutoDiscoverNodeInstaller type, so any other call site that needed a
systemd state snapshot had to either duplicate the logic or import the
installer package just to reach them. Moving them to a dedicated package
keeps the shared integration layer free of installer-specific policy
and makes the code reusable by unrelated callers.
The installer diagnostics file retains only installer-facing types and
the enableAndRestartTeleportService logic. JoinFailureError doc comments
no longer hardcode the 5m0s timeout in the example, since the timeout
is now configurable and changes across releases. Two clarifying comments
on runSystemctlCommand document why context errors are preferred over
exec errors, and why ExitError branches include the exit code and the
captured stdout and stderr streams in the wrapped error message.
The installer-side test file drops the helpers' tests because they now
live alongside the package they cover in lib/systemd. The remaining
TestJoinFailureErrorString case derives its expected message from
JoinFailureTimeout rather than hardcoding a duration literal.
* Bound join-health polling and diagnostics collection
The join-health check now drives polling with retryutils.NewRetryV2
against a dedicated pollCtx bounded by readyzPollTimeout, and the
diagnostics collection step races the diagnostics and journal goroutines
against diagCtx using channels and a select loop. Previously the poll
loop used a hand-rolled timer with a remaining-duration calculation, and
the diagnostics step waited on a sync.WaitGroup, so a stuck diagnostics
or journal implementation blocked the join failure report indefinitely
even after the parent context was canceled. With the new structure,
context cancellation and the diagnostic timeout both unblock the join
failure path, and the journal output field now carries an explanatory
message when collection times out or is canceled instead of being
silently empty.
JoinFailureTimeout is reduced from five to three minutes to surface
join failures sooner in SSM invocation results, where the full five
minute wait previously delayed operator feedback without improving the
odds of recovery. LastError is no longer populated when the error from
readyzChecker originates from the poll context already expiring, so the
join failure message reflects the actual join outcome rather than a
context cancellation observed only after the poll deadline.
The installer now delegates diagnostics collection and journal capture
to a teleportsystemd.Client constructed from the installer config,
rather than calling unexported systemd helpers on the installer itself.
The local dbusConn interface and its sole usage are removed in favor of
teleportsystemd.NewConnFunc, which is now the test seam for injecting
fake D-Bus connections into the diagnostics client.
The autodiscover tests cover the new behaviors: a stuck diagnostics or
journal implementation does not block join failure reporting when the
parent context is canceled, a poll timeout does not populate LastError,
and a diagnostics timeout produces a human-readable journal substitute.
Existing tests switch from context.Background() to t.Context() so they
inherit the per-test deadline configured by the testing package.
The EC2 troubleshooting docs and ssm_install_test fixtures stop using
a hardcoded five-minute duration and instead derive it from exported
installer.JoinFailureTimeout, so they track future timeout changes
without manual edits across docs, tests, and error message snippets.
* Use t.Context in readyz checker tests
* Reduce join-failure timeout default from 3m to 2m
---------
Co-authored-by: Roman Tkachenko <[email protected]>
Co-authored-by: Chris Thach <[email protected]>
Co-authored-by: Tiago Silva <[email protected]>
Co-authored-by: Gavin Frazar <[email protected]>
* Improve results layout
* Do not show matches in resource's secondary properties/labels if there's already a match on the main field
* Adjust stories and add tests
* Change `font-weight` to 400 for better redability
* Add tests for `getVisibleMatches`
* Remove unnecessary trailing spaces
Label expressions don't have a way to declare slices. This PR introduces
a function that we expose for other parsers.
This will support some work in access graph repo.
Signed-off-by: Tiago Silva <[email protected]>
* Return HostResult messages for agents in bound keypair joining
The current bound keypair implementation only returns BotResult from
its join implementation, despite being otherwise updated to account
for host joins. This mostly works as the only real difference between
bot and host results is the bound keypair result... as well as
`ImmutableLabels`, which was effectively lost. Certificates were
otherwise valid and returned intact. This only applies to scoped
token joining as `ImmutableLabels` are nil for standard
`ProvisionTokenV2`.
This would ultimately cause any host joined using a scoped token with
`ImmutableLabels` configured to be inaccessible to users attempting
to access that resource.
To fix the issue, this changes `HandleBoundKeypairJoin` to return
a generic `messages.Result` and constructs a `messages.HostResult`
appropriately for instances. The `joinclient` is also updated to
account for host joins.
* Update token ref after rebase
* Fix lint
* Add support for nested markdown lists
* Fix header content not parsing as markdown
* Add support for markdown tables
* Add max depth of 10 for parsing nested lists
* Only consume sibling list items with the same list marker
* Respect the start number on ordered lists
* Code review fixes
* Code review fixes
* Cap the max number of table columns to 100, rows to 1000
* Check max iterations before skipping blank lines
* Add support for delegation to `tbot`'s relevant services
* Explicitly prevent use of delegation with leaf clusters
* Make `WithRoles` and `WithDelegation` mutually-exclusive
* Make `delegation_session_id` and `roles` mutually-exclusive in tbot config
* Use nil-tolerant proto getter functions
* Move delegation cert generation into separate method
* Stop delegation service from returning CA certificates
Per @strideynet's request, to avoid replicating existing bad patterns around
conflating the user and host CAs.
* Fix tbot OpenSSH output in delegation sessions
All remaining reconcilers relying on the default
ReconcilerConfig.CompareResources function have been updated to
provide one explicitly. All direct uses have been replaced with
equivalent functionality via the appropriate IsEqual function.
The golangci-lint depguard rules have been updated to
remove the current exclusions.
* Fix list resources when TELEPORT_UNSTABLE_FORCE_IN_BAND_MFA=yes.
Signed-off-by: Chris Thach <[email protected]>
* Improve comments.
Signed-off-by: Chris Thach <[email protected]>
* Minify test name.
Signed-off-by: Chris Thach <[email protected]>
---------
Signed-off-by: Chris Thach <[email protected]>
* Begin restructuring installation docs
See RFD 246
The documentation in the Installation section is currently organized by
platform while documentation for deploying Teleport components is
organized elsewhere. This creates confusion for users, who visit the
Installation docs far more often than any other section, missing
critical instructions.
This change modifies the Installation section to include four children:
- Deploying Teleport Agents
- Installing Teleport on a single machine
- Uninstalling Teleport
- An external link to the Download page, the fastest way to download
Teleport client tools.
Implementation details:
- Remove the `amazon-ec2.mdx` installation guide. This has to do with
cluster installation, not single-machine installation. Move AMI
information into the EC2 Terraform guides, which a separate change
will add to the Installation section. Also remove `helm.mdx` from the
single-machine section, where it does not belong. A separate change
will add our Helm guides to the Installation section.
- Move `amazon-ecs.mdx` to the `agents` directory. It's a guide for
deploying Teleport Agents.
- Remove `windows.mdx`, which only pertains to clients, in favor of
making the Download page more visible in the Installation section.
This change also adds some structural improvements to the `linux.mdx`
guide:
- Organize section introductiosn to create a clearer hierarchy between
more general and more specific information. Readers now read more general
information first.
- Move the item re: recommending Cloud to the top of the page.
- Rework the control plane section. Remove the tab for Teleport Cloud,
which is unnecessary since the section isn't for cloud customers. Add
stepped instructions to consolidate the text a bit.
- Include clearer branching in the TAR and package download section
between self-hosted and agent use cases. Move the branching between
FIPS and standard installations higher in the section for clarity.
* Fix links after reorganizing installation docs
* Clean up installation instructions
Responds to **GavinFrazar** feedback:
- Make the Installation section description more precise.
- Extract the Teleport AMI list into a partial.
* Fix clarity issues in the Installation section
- In the landing page, provide a clearer route for users interested in
self-hosting Teleport Community Edition.
- Restore the Windows guide so we have consistent installation
instructions for all supported platforms. Accordingly, update the
`docs/pages/installation/single-machine` landing page to indicate that
this section also pertains to client tools.
- Place the Amazon EC2 guide in the Single Machine subsection and
include the partial that lists AMIs. Replace the single H2 with an
Admonition for clarity. Fix redirects accordingly.
- Move "Downloads" to the end of the Installation sidebar section.
Clarify the purpose of the link.
- Include `fdpass-teleport` in the table of installed binaries.
- Use "control plane" instead of "cluster" to describe the Auth Service
and Proxy Service in the single-machine.mdx index page.
* MWI Scopes[5]: Bot Instances
* Extend E2E test to check BotInstance
* Propagate scope to recreated bot instances
* Address Nic's test feedback
* Push authz filtering down to cache/backend to avoid partial pages
* MWI: Support scoped join tokens in the bound keypair join method
* Bot fields for scoped tokens
* Check bot scope
* Allow bound keypair tokens to pass scope validation
* Support bots
* Fix outstanding references to ProvisionTokenV2
* Zero check on MustRegisterBefore due to proto impl differences
* Properly pass through nil must_register_before
* Also fix nil timestamps in bound keypair status
* And fix rotate_after
* Add BotInternal certificate field
* Add 'bot' TokenUsageMode value and require for bot joins
* Fix mutator execution for scoped bound keypair joining
* Implement more ScopedToken StrongValidateToken checks for bots
Adds a few more of the bot-specific RFD validation checks for bots.
* Improve bot join checks for scoped joining
* Fix build after rebase and add additional RFD checks
This fixes builds after upstream updates and implements a few more
validations required by the RFD:
- AssignedScope field for bots must be empty
- Moved original AssignedScope check to validateNonBot() since it was
checked unconditionally before.
- BotScope must be a presumably-valid scope at creation time
* Use scope-aware access checker for initial bot certs
* Fix WeakValidateToken() for scoped tokens for bots
WeakValidateToken() was not updated to match the new checks in
StrongValidateToken(); this adds minimal bot-related exemptions
around `assigned_scope` for bots to allow it to be read without
error.
* Add tests for scoped bot tokens, fix role bug
This adds a set of additional token tests for scoped bot tokens, and
fixes an issue where WeakValidateToken() was erroneously rejecting
invalid system roles.
* Properly initialize bound keypair scoped tokens on creation/upsert
We weren't applying the same initialization logic for scoped tokens
that we did for ProvisionTokenV2; this is required for registration
secrets to work properly.
* Update CRDs
* Fix failing test
* Specify types for bound keypair timestamp fields
* Update terraform resources
* Fix lint
* Include bot scope in BotJoin audit events
This includes the bot scope in audit events. It includes the user
label bot scope for successful joins where the user has been fetched,
and otherwise includes the bot scope configured on the token.
* Reject `token` join method for scoped bot tokens
Bot joining should use `bound_keypair`, so reject scoped bot tokens
that attempt to join with the traditional `token` join method.
* Fix failing test due to BotInternal cert parameter
TestRegisterBotInstance wasn't updated to include the BotInternal
flag on initial identity, so this includes it.
* Actually include Scope in bot audit events
* Mutate correct token status in `patchToken`
The mutate result was discarded because the wrong status object was
passed to `mutateScopedToken`.
* Use ScopedAuthorizer in join/Server.authenticate()
`authenticate()` currently rejects scoped identities which prevented
clients from reauthenticating with an existing identity, which is
important for bot renewal and bound keypair joining.
* Pass `BoundHostID` through properly in `GetBoundKeypairStatus()`
* Add basic TestJoinBoundKeypair_ScopedToken test
* Fix lint
* Update join server comments, add note about feature flag edge case
* Target correct provision token for standard token mutations
* Return correct mutated token in patchToken()
* Hide bound keypair registration secrets when `WithSecrets` is unset
* Avoid possible nil pointer dereference
* Address proto comment suggestion from review
* Deduplicate bound keypair mutator validation logic
Validation logic has been pulled out into a `validate()` interface
function and unified for both token types; scoped token bound keypair
fields are converted to their equivalent ProvisionTokenV2 variant
using existing helper logic.
* Nil check in consumeRecoveryMutator.validate()
* Simplify lastRotatedAtMutator
This removes the confusing switch with fallthroughs in favor of 2
simple ifs and a helper function.
* Ensure bound keypair status is non-nil before mutating
* Note non-nil requirement for new lastRotatedAt value
* Update generated operator resources
* Also nil-check status for standard tokens
* Mirror secret censoring for bound keypair secrets in tctl
This mirrors the existing secret censoring in tctl. It doesn't
actually work since the conditional can never be true.
* Use StrongValidate() on BotScope in token StrongValidate()
`validateBotToken()` was incorrectly calling `scopes.WeakValidate()`
on the bot scope. This changes it to call `scopes.StrongValidate()`
instead.
* Mark scoped token bound keypair registration_secret field as sensitive
* Make lastRotatedAtMutator's `mutateScopedToken` nil-safe
* Tag mutator validation errors with a unique name of the mutator
Errors are tagged with the primary field that mutator modifies to
help identify the failing mutator without necessarily identifying
the specific failing check.
* Address code review feedback
* Wait for ScopedRoleAssignment to become available
SRAs don't always propagate immediately, so adds a
`require.EventuallyWithT()` check to ensure it exists before
continuing.
* Add test for scoped bot joining via kubernetes
* Fix formatting
* app: Preserve trailing slash in app URI redirect
path.Clean strips trailing slashes, so a configured URI like
http://backend:9000/dashboard/ produces a redirect to /dashboard
instead of /dashboard/. The upstream returns 404 because it
expects the trailing slash.
Preserve the trailing slash from the original URI after cleaning
by re-appending it when the administrator's URI ended with one.
Add TestNeedsPathRedirect to cover the redirect path logic.
* app: Assert empty location on no-redirect paths
Add an else branch to TestNeedsPathRedirect that verifies the
returned location string is empty when no redirect is expected.
This guards against future regressions where needsPathRedirect
might return a non-empty string alongside ok=false.
rosstimothy