diff --git a/rfd/0219-k8s-rbac-simplification.md b/rfd/0219-k8s-rbac-simplification.md new file mode 100644 index 00000000000..9884275bd68 --- /dev/null +++ b/rfd/0219-k8s-rbac-simplification.md @@ -0,0 +1,281 @@ +--- +authors: Guillaume Charmes (guillaume.charmes@goteleport.com), Tim Ross (tim.ross@goteleport.com) +state: draft +--- + +# RFD 0219 - Kubernetes RBAC Simplification + +## Required Approvers + +- Engineering: (@zmb3 || @rjones || @r0mant) && (@hugoShaka || @tigrato) +Product: @klizhentas + +## What + +The purpose of this RFD is to propose a simplification of the Kubernetes +RBAC (Role-Based Access Control) within the Teleport project. + +The goal is to streamline the user experience and reduce the complexity of +getting up and running on day 0 and managing permissions later on. + +## Why + +Today, the Kubernetes RBAC in Teleport is complex and can be difficult for +users to manage. +A proper setup requires some external Kubernetes config and matching role +setup. +This complexity leads to misconfigurations and security issues. + +Simplifying the RBAC model will help improve usability and security. + +### References + +A common issue currently is the un-intuitive result of complex rule sets and +more critically, difficult to get setup on day 0. + +- @klizhentas struggling to setup a cluster following the `self-hosted` flow: + (internal) +- Customer confusion on role resolution: + - + - (internal) +- Unexpected `exec` grant on read-only user: + (internal) + +## Goal + +Allow preset roles to be used to get started quickly. + +## Proposal + +### Prior work + +While initially planned to be included in this RFD, due to customer requests, +the following has already been implemented and shipped: + +- Support arbitrary resources, including CRDs #54426 +- Simplify Namespace/Kind behavior #54938 +- Handle all known resources on the Teleport side #55099 + +Given this, as RBAC has already been simplified quite a lot, a scoped down +version of the proposal would be to focus on new installs without impacting +existing customers. + +### Preset roles + +#### Background + +Originally, Teleport preset roles were inspired by Kubernetes roles, +specifically, `edit`, `cluster-admin` and `view`. + +- `cluster-admin` allows everything everywhere. +- `edit` allows CRUD on most builtin resources, but not administrative + resources. + - get/list/watch pods (including attach, exec, portforward and proxy), + secrets and services/proxy + - impersonate service accounts + - create/delete/deletecollection/patch/update pods (including exec/attach/ + portforward/proxy) + - create pods/evictions + - create/delete/deletecollection/patch/update configmaps, events, + persistentvolumeclaims, replicationcontrollers (including scale), secrets, + serviceaccounts, services (including proxy) + - create serviceaccounts/token + - create/delete/deletecollection/patch/update deployments (including + rollback, scale), replicasets (including scale), statefulsets (including + scale), ingresses, networkpolicies + - create/delete/deletecollection/patch/update horizontalpodautoscalers, cronjobs, + jobs, poddisruptionbudgets, leases + - get/list/watch controllerrevisions, daemonsets, deployments, replicasets, + statefulsets, horizontalpodautoscalers, cronjobs, jobs, poddisruptionbudgets, + ingresses, networkpolicies, leases +- `view` allows read-only access to most builtin resources. + - get/list/watch pods, configmaps, endpoints, persistentvolumeclaims, + replicationcontrollers (including scale), serviceaccounts, services, + bindings, events, limitranges, namespaces, resourcequotas, endpointslices, + controllerrevisions, daemonset, deployments, replicasets, statefulsets, + horizontalpodautoscalers, cronjobs, jobs, poddisruptionbudgets, ingresses, + networkpolicies, leases + +This resulted in the following Teleport roles: + +- `editor`, which maps to `cluster-admin` and can do everything. +- `access`, which maps to `edit` and can do most CRUD operations. +- `auditor`, which maps to `view` and can do read-only operations. + +#### New Kubernetes Bindings + +In order to be consistent and allow for seamless use of Kubernetes within +Teleport, we will introduce the following new preset bindings: + +- `kubernetes_groups` -> `ClusterRole` +- `teleport:preset:edit` -> `edit` + Can CRUD most basic builtin resources and read some limited administrative resources. +- `teleport:preset:cluster-admin` -> `cluster-admin` + Provides full access to all Kubernetes resources, including CRDs. +- `teleport:preset:view` -> `view` + Provides read-only access to some Kubernetes resources. + +On the Kubernetes side, we will create the corresponding `ClusterRoleBindings` based on `kubernetes_groups`. + +While there is an existing binding for `cluster-admin` via `system:masters`, it +is not available everywhere and there are no binding for `edit` nor `view`. +For consistency and reliability, we will create one binding for each preset +role. + +Those bindings will be available to Teleport users via the user trait feature. + +Kubernetes might introduce new API endpoints that should be allowed or denied. +Vendoring those roles on Teleport would require us to update the list for every +Kubernetes version, and would potentially cause a permission issues if a user +deploys an old Teleport version in a modern Kubernetes cluster. Relying on the +Kubernetes built-in roles for RBAC frees us from the maintenance burden and +prevents security issues caused by version mismatch. + +Example: + +```bash +tctl users add hugo --logins root --kubernetes-group teleport:preset:edit --roles=access +``` + +### Provisioning + +To enable easy setup, provisioning methods will be updated to create the +required new ClusterRoleBindings. + +#### Helm Chart + +The Helm Chart will by default provision the new Kubernetes resources starting +in v19.0.0. An option will be added to the chart to allow opting out of creating +the default ClusterRoleBindings. All existing release branches will have this +flag set to disable the new functionality to minimize breaking changes. + + +Example: + +```yaml +roles: kube,app,discovery +authToken: foo +proxyAddr: example.devteleport.com:443 +kubeClusterName: myCluster +rbac: + defaultClusterRoleBindings: true +``` + +The ClusterRoleBindings created by the chart will be prefixed with the +chart release and the namespace to allow multiple deployments in the +same Kubernetes cluster not to conflict since ClusterRoleBinding resources +are global and not namespaced. + +```yaml + +{{- if .Values.rbac.defaultClusterRoleBindings -}} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ .Release.Name }}-{{ .Release.Namespace }}-preset-edit +``` + + +#### Provision Script + +For simplicity, the [provisioning script](https://github.com/gravitational/teleport/blob/89a13c7c7c6538e5b6afc1b820db0e2e3a9b87fb/examples/k8s-auth/get-kubeconfig.sh#L4) +will be updated to always create the Kubernetes resources, and will not provide a means for users to opt out of +the new behavior. + +#### Auto Discovery + +##### EKS / AKS + +For both EKS and AKS, auto-discovery will be updated to create the expected +RBAC resources. + +##### GKE + +For GKE, we will need to change the documented GCP IAM role to enable admin +privileges. +Teleport will attempt to impersonate a privileged group, but there is a risk +GKE users will not be able to use the new preset roles out of the box. + +A special attention to error management will be required to help users +understand the issue and how to resolve it, which can be easily done by +running the provision script for example. + +#### Auto-update + +Auto-update will not attempt to apply the new model, with the assumption that +existing setups are already working. + +### UX + +#### Resource enrollment UI + +On the Web UI, the initial page generates a _Helm_ command line. + +After enrollment, a test page is shown, prompting for a `kubernetes_groups` +value, which defaults to the user's trait and updates it if changed in this +page. + +The following page is about testing the connection, and also have a form to +populate a subset of the users/groups. + +The initial idea was to allow the user to update it's traits on the fly to not +have to logout/log back in, however, as there is no mention of traits, and +because the form is present on multiple pages with different behavior, and +because it requires the user to initially already have a role assigned that +contains the trait interpolation, we will remove both the 'Setup access' page +and the 'users/groups' form from the 'Test connection' page. + +This will result in using the user's auth to test the connection instead of +requiring the user to enter a subset or all of it's groups/users, knowing +that the user cannot make add nor change any. + +#### Error management + +The error messages when using `kubectl` will be improved to include a link to +the documentation and more details on what is expected. This will help with +initial custom setups that skipped the provided provisioning scripts. + +### User flow + +#### Current flow + +- Day 0 - Initial setup: + - User installs Teleport and sets up a Kubernetes cluster. + - User provisions the Kubernetes cluster using a script, discovery, or Helm + chart to create the impersonation ClusterRole/ClusterRoleBinding. + - User configures Kubernetes RBAC with a custom ClusterRole with some + permissions + - User configures Kubernetes RBAC with a custom ClusterRoleBinding, which + needs to be understood matches with the `kubernetes_groups` field in the + Teleport role. + - User configures Teleport role with `kubernetes_groups`, (unclear what + `kubernetes_users` does from the docs) fields to match the custom + ClusterRoleBinding. + - User configures Teleport role with `kubernetes_resources` and + `kubernetes_labels` to match to match or reduce the permissions granted by + the custom ClusterRole. +- Day 1 - Ongoing management: + - User needs to understand the interaction between the Teleport role and the + underlying Kubernetes RBAC. + - User may need to update the Teleport role if the underlying Kubernetes + RBAC changes. + - No clear guidance on how to which permission should be set where (Teleport + or Kubernetes). + - User may need to troubleshoot unexpected permissions or access issues. + +#### Proposed flow + +- Day 0 - Initial setup, prospect and hackers: + - Prospect installs Teleport + - Prospect creates its first Teleport local user with the trait `kubernetes_groups: "teleport:preset:cluster-admin"` + - Prospect deploys a Teleport agent in their Kubernetes cluster + - Prospect can access the Kubernetes cluster as administrator with their local Teleport user +- Day 1 - Multiple users, late stage prospect or new Teleport adopter + - Admin has some Kubernetes agents connected, SSO connector set up, and several users + - Admin creates per-team roles with different `spec.allow. kubernetes_labels` to select each team's cluster + - Admin sets the `teleport:preset:edit` or `teleport:preset:view` as `spec.allow.kubernetes_groups` + - Admin grants the role to users using SSO connector or Access Lists + - users from each team can access their Kubernetes clusters +- Day 2 - More granular access, advanced Teleport users + - Admin can create custom Kubernetes rolebindings, deploy them in their clusters and use them in their Teleport roles instead of the presets + - Admin can scope down read and edit roles to select namespaces or resources by using `kubernetes_resources` in their roles diff --git a/rfd/cspell.json b/rfd/cspell.json index 9e54e2757b8..81666616818 100644 --- a/rfd/cspell.json +++ b/rfd/cspell.json @@ -129,11 +129,16 @@ "CEST", "CGNAT", "checkmodule", + "charmes", "Cieślak", "ciphertext", "ckms", "CLIPRDR", "clusterawsadm", + "clusterrole", + "clusterroles", + "clusterrolebinding", + "clusterrolebindings", "cmdline", "cmpopts", "cockroachdb", @@ -147,6 +152,7 @@ "configurator", "connectionupgrade", "containerregistry", + "controllerrevisions", "Corado", "crashloop", "creack", @@ -162,6 +168,7 @@ "customtype", "CUVE", "cwallet", + "daemonset", "Datalog", "datapoint", "datapoints", @@ -173,6 +180,7 @@ "deconfigures", "decryptable", "DEFAA", + "deletecollection", "delims", "depguard", "deploymentname", @@ -180,6 +188,7 @@ "devel", "devpay", "devs", + "devteleport", "diable", "diceware", "Dinis", @@ -211,6 +220,7 @@ "eksctl", "Elastisearch", "elif", + "endpointslices", "enduml", "Enes", "Entra", @@ -280,6 +290,7 @@ "grpcserver", "Grzegorz", "gserviceaccount", + "guillaume", "gvisor", "gzdunek", "gzipped", @@ -287,6 +298,7 @@ "headlessauthn", "healthcheck", "healthcheckconfig", + "healthchecks", "healthz", "heartbeated", "heartbeating", @@ -296,6 +308,7 @@ "HGETALL", "HKIA", "HMAC", + "horizontalpodautoscalers", "hostable", "hostip", "hostnames", @@ -388,6 +401,7 @@ "libpq", "libudev", "libykcs", + "limitranges", "listprofiles", "litebk", "livez", @@ -466,6 +480,7 @@ "namespacing", "netbox", "nethealth", + "networkpolicies", "nindent", "nistp", "nklaassen", @@ -530,6 +545,7 @@ "Pawel", "pchar", "PCKS", + "persistentvolumeclaims", "pgbouncer", "pgconn", "PGHOST", @@ -552,6 +568,7 @@ "plantuml", "plpgsql", "pluggable", + "poddisruptionbudgets", "podname", "poka", "policycoreutils", @@ -640,11 +657,14 @@ "replaceall", "replayable", "replicaset", + "replicasets", + "replicationcontrollers", "reprepro", "requeries", "requestable", "resolv", "resourcegroups", + "resourcequotas", "restclient", "reversetunnel", "reversetunnelv",