In preparation for 19.0.0 this removes all references
to legacy or broken record keys. The background key
conversion will have already converted all legacy or broken
keys thus allowing the removal of the migration and the key types.
Caching has become an integral part in functionality provided
by teleport. Allowing users to disable the cache was never something
that was a recomended configuration - it was predominately used
as a stopgap to address an outage. However, over time the cache
has become more stable and the need to bypass it should never
arise.
Additionally, this toggle causes several architectural difficulties
that increase complexity and maintainability of teleport. By always
enforcing that the cache is on, we can eventually separate the gRPC,
backend, and caching layers - a frequent developer experience nightmare.
Moving to a world where the cache is always guaranteed to be present
and available we will be able to remove layers of abstraction and
increase performance.
We do still honor `teleport.cache.max_backoff`. While it was tempting
to remove all teleport.cache knobs, it seemed that still allowing users
to control the backoff interval could alleviate any caching related
issues that lead to or are contributing to an outage. This seemed rather
important to still offer now that the ability to disable caching no
longer exists.
https://github.com/gravitational/teleport/pull/69282 incorrectly moved the
columns for recusrive listings which resulted in the data being presented
incorrectly. Equivalent golden tests for non-recursive listings have been
added to catch and prevent future regressions.
1) Allow make gen-tfschema to run offline by removing -u from go list
2) Prevent stale codgen by deleting previously generated code prior to
executing go run ./gen/main.go
As a result of 2, some files that don't have an existing payload registered
in gen/main.go have been removed.
- Add deatils about SQNs
- Ensures resources are referenced by SQNs
- Updates IaC section to cover additional scoped resources.
- Fixes a reference to a scoped token with a ~ separator
- Add acl users to supported commands
- Fix workload_identity permissions
The IdentifierFromPath and IdentifierPolicy functions no longer rely on
positional arguments to improve type safety. Path inputs are now combined
into a single struct with named fields to better catch mistakes. Return
types from IdentifierPolicy functions are now a single struct with named
fields instead of returning multiple strings.
* Namespce scoped tokens
Scope tokens are now referred to by SQNs and presented as an SQN to
users in all flows. Collisions between unscoped and scoped tokens
are no longer possible. The ScopedToken Get and Delete APIs have been
updated to take a request object, and the IaC tooling has been updated
to exclusively consume namespaced scoped tokens.
* fix: validate static scoped tokens
* use new scopes.MaybeSQN to determine if an identier is a SQN or not
* fix: get scoped token secrets in TF
* fix: scoped events delete parsing
* fix: scoped token cursors
* add commentary and TODOs
* fix: bound key pair lock name
* fix: only consume legacy tokens when scopes are disabled
The currently supported set of scoped resources: ScopedRole, ScopedToken,
ScopedRoleAssignment, KubernetesCluster, and App have all been migrated
to the new generic terraform provider mechanism. This is required for those
resources to properly provide the name and scope pair which uniquely identifies
a scoped resource.
Note: not all scoped resources currently support being identifier by a the
(name, scope) pair. However, this conversion is a prereq for when they do.
At such a time, those resources should only require changing the identifier
type and plumbing the name and scope into the Get and Delete requests.
This is in preparation for migrating to non-template based
Teleport resource implementations of Terraform DataSourceTypes
and ResourceTypes. The existing files were moved from terraform/provider
to terraform/provider/internal/legacy.
The terraform driver has been decoupled from Teleport and all the
hacks that the templates used. Instead the driver uses a series
of optional interfaces to perform actions at specific points
of the terraform lifecycle. For now, only a subset of the hooks
from the legacy templates have been adapted to minimize scope.
The hooks chosen were selected out of necessity for the first
resources that will be migrated off the legacy templates.
The other main change introduced here is around how resources
are identified. The legacy templates originally assumed that
all resources were only identified by name. That broke when
access list members were introduced. That now also breaks for
scoped resources. Identifiers used by the driver are now an
interface over a generic type. There are several concrete
implementations that cover the current cases for Teleport
resources: A singleton identifier with a constant name, a
named identifer that maps to a Teleport resource metadata name,
a composite identifer that maps to a name and a prefix, and
now a scoped identifier that maps to a name and a scope. Each
DataSource and Resource implementation can chose the appropriate
one and the driver doesn't care. This is also extensible if
there comes along another resource with a different identifier shape.
The Kube Operator APIs were updated to support Get and Delete
operations that may operate on a name and a scope. For now
there is no support for actually providing or consuming the scope.
This simply updates the plumbing so that in a follow up change
resources that are scoped will be able to operate.
Note: The operator is still limited in that it cannot have two
resources with same names in the same scope in the same
Kubernetes namespace.
Specifies concurrency in GitHub workflows such that a PR can
only have one active set of workflows at a time. Prior to this,
if a PR had multiple successive commits pushed, CI would run to
completion for _all_ commits. Now CI will only runn for the latest
commit, any workflows running from previous commits will be terminated.
RFD 153 is often not followed, and only caught after the fact. In an effort to catch deviations sooner than later, AGENTS.md was updated in an attempt to get the review bot to catch these deviations.
The test was made more deterministic by using a single worker, removing
a shared context with a 10 second timeout for all subtests, using a
unique and per-subtest directory to avoid file collisions, simplifying
checkpoint assertions.
The tests were refactored to leverage synctest instead of a fake
clock, use a single worker for tests that aren't exercising scheduling,
and remove all require.Eventually uses.
The ScopeAwareService and ScopeAwareServiceWrapper no have an
option ScopedOnly flag. When set, the service ignores all items
in the unscoped key range. This is useful for _new_ scoped items
that will never have any resources exist in an unscoped key range,
i.e. ScopedTokens, ScopedRoles, and ScopedRoleAssignments.
In preparation for making ScopedTokens fully namespaced this adds
a scope field to GetScopedTokenRequest and DeleteScopedTokenRequest.
Additionally, the provision.Token interface was updated to include a
GetScope method that will be consumed to populate APIs that require
a scope in follow up changes.
The BatchWriteItem API can return a response with items that failed to
be written without returning an error. DeleteRange could silently
allow items to never be removed under certain conditions because it
never consumed the response.
A new batchWriteItem helper now manages interactions with the BatchWriteItem
API. Any responses with UnprocessedItems are now retried with backoff.
Resolves https://github.com/gravitational/pressure-washing/issues/137.
All of the changes here are mechanical conversions generated from
open2opaque rewrite -levels=red./.... This concludes the conversion of
teleport to consume the opaque APIs. The corresponding changes in
teleport.e must be completed prior to switching entirely to opaque codegen.
See https://protobuf.dev/reference/go/opaque-migration/ for more details.
The Lint job was also updated to prevent any Open API uses until we can
fully convert to the Opaque API.
Removes any externally visibile notions to end users that a
token name and token secret are two different entities. This
removes the need to output a token_name and token_secret value
from `tctl scoped tokens add` as well as removing the token_secret
from teleport.yaml and teleport CLIs.
Internally, the name of a scoped_token is still NOT a secret. The
secret is a separate field in the scoped_token.status. When the
token is presented to a user it is done so as `<token_name>:<base64(token_secret)`.
The leading token_name is not encoded to allow users to visually
dentify tokens they may have created with a specific name. The `~`
separator was chosen because it is neither a valid backend key, scope
separator, and does not need any special encoding if included in a URL.
The token_secret is base64 encoded so that it appears as opaque text
to users and can be included in a URL.
Additionally, the tctl scoped ref parsing has been updated such
that tctl get/edit/rm honors both `<token_name>` and `<token_name>:<base64(token_secret)>`
This does contain a few breaking changes. The token_secret has been removed
from the teleport.yaml file config and its equivalent CLI flags have been
removed. Since scopes are still under active development and this only impacts
scoped_tokens and not traditional tokens the impact of this change should be
minimal.
This pulls in the latest OCI SDK in order to interact
with recently introduced regions.
This includes the following regions:
- ap-kulai-2
- sa-riodejaneiro-1
- af-casablanca-1
- me-alrayyan-1
* Convert existing non-gogo codegen to the Hybrid API
Contributes to https://github.com/gravitational/teleport/issues/66776.
All existing protos explicitly set to API_OPEN have been change to
API_HBYRID. The new codegen was performed via make grpc. There are no
other functional changes to the code to start consuming the Hybrid API
those will come later. The intent is to get all Hybrid codegen in and
backported to ease the transition.
* Initial migration to the Opaque API
Contributes to https://github.com/gravitational/teleport/issues/66776.
All of the changes here are mechanical conversions generated from
`open2opaque rewrite -levels=green ./...`. There will be a follow up
to this in teleport.e which does the same. Once all changes have been
merged the process will be repeated with -levels=yellow followed by
-levels=red.
See https://protobuf.dev/reference/go/opaque-migration/ for more
details.
* [2] Migration to the Opaque API
Contributes to #66776.
All of the changes here are mechanical conversions generated from
open2opaque rewrite -levels=yellow./.... There will be a follow up
to this in teleport.e which does the same. Once all changes have been
merged the process will be repeated with -levels=red.
See https://protobuf.dev/reference/go/opaque-migration/ for more details.
* Convert existing non-gogo codegen to the Hybrid API
Contributes to https://github.com/gravitational/teleport/issues/66776.
All existing protos explicitly set to API_OPEN have been change to
API_HBYRID. The new codegen was performed via make grpc. There are no
other functional changes to the code to start consuming the Hybrid API
those will come later. The intent is to get all Hybrid codegen in and
backported to ease the transition.
* Initial migration to the Opaque API
Contributes to https://github.com/gravitational/teleport/issues/66776.
All of the changes here are mechanical conversions generated from
`open2opaque rewrite -levels=green ./...`. There will be a follow up
to this in teleport.e which does the same. Once all changes have been
merged the process will be repeated with -levels=yellow followed by
-levels=red.
See https://protobuf.dev/reference/go/opaque-migration/ for more
details.
Contributes to https://github.com/gravitational/teleport/issues/66776.
All existing protos explicitly set to API_OPEN have been change to
API_HBYRID. The new codegen was performed via make grpc. There are no
other functional changes to the code to start consuming the Hybrid API
those will come later. The intent is to get all Hybrid codegen in and
backported to ease the transition.
Relocates client.ParseLabelSpec to lib/utils/parse. This results in
a net 5MB reduction in the weight of lib/config. This may not
have a correlation in binary size since lib/client is heavily consumed
by other packages.
```bash
$ goda cut './lib/config:all' | rg '^github.com/gravitational/teleport/lib/client\b|^ID'
github.com/gravitational/teleport/lib/client InDegree=1 Cut.PackageCount=75 Cut.AllFiles.Size=5.3MB Cut.Go.Lines=129512
```
Replace the functions in the scopes package to infer if scoped features
are enabled with a concrete struct that is injected to consumers. This
permits tests to toggle scopes features without having to specify environment
variables.
This is purely mechanical change to inject the build type as a
depency instead of relying on modules.GetModules().BuildType().
In addition to contributing to https://github.com/gravitational/teleport/issues/62799
this also serves as a prerequisite to future work that will provide
a user supplied build type that is not based on the modules to
retrieve the correct image.
Prevent access requests in APPROVED, DENIED, or PROMOTED states from
being updated through SetAccessRequestState, including same-state
updates that could otherwise mutate resolution metadata.
Render `/webapi/scripts/installer/:name` responses with the text
template engine instead of the HTML template engine. The endpoint emits
shell script content, so HTML entity escaping can corrupt rendered values
containing characters such as `&`, `<`, or `>`.
Resolves https://github.com/gravitational/pressure-washing/issues/195.
Replaces the per-type clusterrole/secret handlers with a generic
in-memory store keyed by GVR and namespace/name. Registers
standard CRUD routes for serviceaccounts, configmaps, secrets,
services, roles, rolebindings, clusterroles, clusterrolebindings,
deployments, statefulsets, daemonsets, replicasets, jobs,
networkpolicies, and poddisruptionbudgets. Advertises the apps,
batch, networking.k8s.io, and policy API groups via discovery.
Updates handler errors metav1.StatusReason so client-go can
classify NotFound/AlreadyExists/BadRequest responses.
This adds a getting started guide with a a high level primer on
scopes, prequisites needed to test scopes today, and examples on how
to use the currently supported scoped features.
Ensures the backend table and events table use the same revision when
updating items during KeepAlive operations. Previously a revision was
generated for _both_ tables causing watchers and Get calls to observe
different revisions.
The backend test suite was updated to validate revisions in the event
stream match revisions in the backend.
Resolves https://github.com/gravitational/pressure-washing/issues/139.
Converts non-pointer protobuf generated Go types to pointers
via `open2opaque rewrite -levels=red ./...`. This contributes to
https://github.com/gravitational/teleport/issues/66776 and was done
separately to isolate per proto file migrations in the future.
Enable pagination tests for Resource153 cache resources by default and fix
resource-specific test setup that previously skipped or bypassed pagination.
This also fixes two issues found:
UserTask upstream fallback listing honors the requested page size and
token.
WorkloadIdentity cache listing forwards the page size and token instead
of infinitely loading the first page.
The testResourcePagination helper exercised fallback upstream reads by
setting the cache in an unhealthy state, but never restored the cache
to a healthy state before completing. This resulted in all tests that
supported pagination being exercised against the upstream backend instead
of the cache.
All existing protos have an explicit override for the Open API to
preserve behavior. These protos will be migrated one by one if it
is possible to do so.
Forcing all new APIs to use Opaque by default is a protection mechanism.
The code generated from the Opaque API does not export any of the
fields in the message. All interactions happens via Getters and Setters.
This eliminates a whole class of potential misuse leading to a nil
deference and a panic.
Teleport validates the contents of sudoers files by using visudo to
check the contents. The binary is invoked from disk using arguments
-c -f - which indicate to validate the content passed via stdin.
Ubuntu 25 and newer uses sudo-rs by default. This causes problems
because the version shipped in Ubuntu 25 does not contain the fix
for https://github.com/trifectatechfoundation/sudo-rs/issues/1358.
This causes visudo to attempt to read a file with the literal name
`-` instead of from stdin. As a result all Teleport sessions which
try to write a sudoers file entry get a file does not exist error
from visudo.
The sanest way to work around this issue without added burden to
end users is to stop using stdin with visudo. All sudoers entries
are first written to a temporary file on disk and that file is
provided to visudo.
The linker is now able to perform DCE for all binaries except tsh
and teleport. A change to enable DCE for teleport will come in the
future. DCE for tsh will be harder to achieve until kubectl dependencies
are capable of DCE.
This udpates e to include https://github.com/gravitational/teleport.e/pull/8447
which removes the last template usages in enterprise code. Now that
templates are no longer imported a depguard rule has been added to
prevent it from being imported in the future.
The last step to eliminate DCE was to add the grpcnotrace build tag
to all binaries.
This is a pass on replacing modulestest.SetTestModules with supplying modules
where already possible. Some additional uses of modules.GetModules were also
replaced with localized modules injection to permit the removal of
modulestest.SetTestModules in tests. Most of the changes here are mechanical,
the one notable change is in TestSSHCommands. The test was not passing locally
because the shell (fish) builtin echo behaves differently. The easiest way
to produce expected output was to change the command invocation to use
/bin/echo to bypass any shell builtins for consistent behavior.
Inject modules into tool tests
Inject modules into integration tests
Inject modules into join server
Remove superflous enterprise modules in TestAuthenticationConfig_Parse_deviceTrustPB
Inject modules into signature algorithm config tests
Inject modules into TestGithubConnectorCRUDEventsEmitted
Inject modules into TestGetHierarchyForUser
Inject modules into kube proxy tests
Inject modules into controller tests
make fix-imports
add integration/app/fixture as test package
All existing template usage was converted to make use of
github.com/DataDog/datadog-agent/pkg/template. The DataDog package
is a fork of the stdlib template packages with method calling
removed so that reflect.MethodByName does not prevent DCE.
There were a handful of method calls in our templates that were trivially
changed to use custom functions defined via FuncMaps or specifying the
calculated value to the template instead of calling a function in the template.
A future change will add linter rules to prevent new text/template or
html/imports from landing. They have not been included here so as
to not to break enterprise code while it is migrated.
* Update kingpin
* Convert CLI formatting to use kingpin UsageRenderers
Avoiding the legacy kingpin text/templates allows for DCE to be used
by the linker. The existing formatting has been matched with a
pure Go UsageRenderer.
Adds an additional accesslist.EqualAccessListsOption,
accesslist.WithIgnoreOktaUserManagedFields to support equality checking
which ignoring Spec.Owners, Spec.MembershipRequires, spec.OwnershipRequires,
and Spec.Audit. This will be used to replace the current go-cmp usage
to determine whether modification of Okta Access Lists are allowed.
(AccessListMember) IsEqual has been restored with a goderived implementation
which checks full equality. This was removed in https://github.com/gravitational/teleport/pull/64542
to prevent accidental use with a services.Reconciler. As of
https://github.com/gravitational/teleport/pull/65151, the Reconcilers
require an explicit comparison function instead of implicitly using
an IsEqual implementation the concerns which led to removing the IsEqual
are slightly mitigated.
This removes the last few places in lib/auth/* that were
consuming modules with modules.GetModules and setting
them via modulestest.SetTestModules.
Contributes to https://github.com/gravitational/teleport/issues/62799.
The library has had a number of CVEs recently and uses shady
dependencies. https://github.com/gravitational/teleport.e/pull/8513
removed the dependency pulled in via teleport.e. This updates the
e_imports.go to reflect that change which moved buger/jsonparser to
an indirect dependency.
To remove the dependency entirely, mark3labs/mcp-go was updated to
the latest version. In that version they have replaced
github.com/invopop/jsonschema, which brough in buger/jsonparser,
with github.com/google/jsonschema-go. As a result buger/jsonparser
is no longer in our dependency tree.
All remaining reconcilers relying on the default
ReconcilerConfig.CompareResources function have been updated to
provide one explicitly. All direct uses have been replaced with
equivalent functionality via the appropriate IsEqual function.
The golangci-lint depguard rules have been updated to
remove the current exclusions.
`(AccessRequestV3) IsEqual` was added with a partially goderived
implementation to remove go-cmp comparisons outside test files.
This was not as straightforward as other resources due to
goderive not being able to handle interfaces. Due to the
oneof in AccessRequestsV3 goderive would silently fail
with a partially generated equality function for access requests.
The TeleportEqual derive plugin was updated to ignore the details
field. They are manually validated in `(AccessRequestV3) IsEqual`
if all other fields match.
`resourceAccessIDEqual` manually validates the constraints fields and
relies on goderived equality functions for each concrete type.
Dependabot reviewers are determined by codeowners. Since there were
no owners defined for the examples modules Dependabot PRs created
for security alerts were going unnoticed because nobody got assigned
to review them.
Replaces global variable with struct fields on the auth.Server which
can be injected. The newTestAuthServer helper used by trusted cluster
tests specifies the limits directly so tests no longer have to modify
the globals causing races.
Closes https://github.com/gravitational/teleport/issues/65564.
Add a note to HSM guides that the signature_algorithm_suite in
the cluster_auth_preference must be set to hsm-v1 before
rotating CAs during the migration. Without doing this users
can get confusing errors about unsupported key algorithms for
Ed25519.
Replaces `(AccessListMember) IsEqual` with a local goderived
implementation in local/services. All reconcilers
have been previously updated to specify their own comparison
function to prevent this change from breaking enterprise in
https://github.com/gravitational/teleport.e/pull/8233.
IsEqual was not updated to use the goderived implementation because
different use cases have different ideas about equality. A single
API to support them all is untenable and confusing. As a result the
callers are now responsible for adhering to their own equality
definitions. If in the future another use case arises that requires
full equality IsEqual may be reconsidered.
The modules are not yet consumed, and default to modules.GetModules(),
if not provided. Their existence is meant to bridge the transitional
period without breaking enterprise. Once this lands, enterprise will
be updated to both inject the modules to Auth and specify them via either
modules.SetModules or modulestest.SetTestModules. This will ensure
that enterprise does not break when Auth exclusively consumes the
dependency injected modules over the global modules. After the
enterprise update lands a follow up will convert Auth.
`(UserV2) IsEqual` was added with a goderived implementation to
remove go-cmp comparisons outside test files. This was not as
straightforward as other resources due to a goderive not being able
to handle interfaces. Due to the Device oneof in LocalAuth goderive
would silently fail with a partially generated equality function for
users.
The TeleportEqual derive plugin was updated to ignore the devices
field. They are manually validated in `(UserV2) IsEqual` if all other
user fields match. `mfaDevicesEqual` manually validates device fields
and relies on goderived quality functions for each concrete device type.
Before services.CompareResources can be removed all places
currently relying on it must be provided an alternative. Some
resources already implemented an IsEqual method but did not
expose it in the resource interface. Other resources have
had new equality methods created which defer to a goderive
generated function.
The template includes an entry for the Changelog and Manual Test Plan
both to serve as a reminder that they should be included and also
so users don't have to go and look up the expected format.
The new workflow is runs on any code change and validates
the PR description contains a fully completed manual test plan.
The test plan is expected to be in the following format.
```
## Manual Test Plan
### Test Environment
Deployed changes on this branch to rjones.cloud.gravitational.io.
### Test Cases
- [x] foo
- [x] bar
- [x] baz
```
If a change is a noop, i.e. it only modifies _test.go
files, then a `no-test-plan` label can be applied to
the PR and the bot will be disarmed.
Part of an effort to consolidate overlapping dependencies. Since tctl
already makes heavy use of charm for TUIs (i.e. tctl top) lipgloss
was chosen over color.
rosstimothy