Authentication and Authorization in Kubernetes
Posted on June 23, 2021 by Adrian Wyssmann ‐ 11 min read
Authentication (who am I) and authorization (what I am allowed to do) are essential and thus having a basic understanding on how Kubernetes handles this, is very useful.
Authentication
Kubernetes offers different authentication strategies like client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth to authenticate API requests through authentication plugins. However first we have to understand user and service accounts.
User and Service accounts
All Kubernetes clusters have two categories of users:
- User Accounts:
- are meant for humans
- are global (unique name across all namespaces)
- might be synced from a corporate database
- not managed by Kubernetes
admin
is the default one
- Service Accounts (sa):
- are used for processes running in pods
- are namespaced
- managed by Kubernetes
- could be created by cluster users for specific tasks
default
is assigned to objects if no sa is defined
Support for authorization and user accounts is planned but incomplete.
User Accounts
It’s important to note that
Kubernetes does not have objects which represent normal user accounts. Normal users cannot be added to a cluster through an API call.
Any user that presents a valid certificate signed by the cluster’s certificate authority (CA) is considered authenticated, whereas the username is determined from the common name field in the ‘subject’ of the cert (e.g., /CN=bob
). Kubernetes then uses RBAC to authorize a user to perform more operations on a resource.
Certificate Signing Requests for Normal Users explains in details on how a normal user can get a valid certificate. However often you would have a management layer on top of Kubernetes (e.g. Rancher) which takes care of this.
Service Accounts
Service Accounts are used access the API from inside a pod, using automatically mounted service account credentials. To make this happen, there are several components in place:
- ServiceAccount Admission Controller which is an Adminisson Controller which modify pods as they are created or updated:
- If the pod does not have a
ServiceAccount
set, it sets theServiceAccount
to default. - It ensures that the
ServiceAccount
referenced by the pod exists, and otherwise rejects it. - It adds a
volume
to the pod which contains a token for API access if neither the ServiceAccountautomountServiceAccountToken
nor the Pod’sautomountServiceAccountToken
is set to false. - It adds a
volumeSource
to each container of the pod mounted at/var/run/secrets/kubernetes.io/serviceaccount
,if the previous step has created a volume for ServiceAccount token. - If the pod does not contain any
imagePullSecrets
, thenimagePullSecrets
of theServiceAccount
are added to the pod
- If the pod does not have a
- [TokenController] runs as part of
kube-controller-manager
and asynchronously does- watch ServiceAccount creation and creates a corresponding ServiceAccount token Secret to allow API access.
- watch ServiceAccount deletion and deletes all corresponding ServiceAccount token Secrets.
- watch ServiceAccount token Secret addition, and ensures the referenced ServiceAccount exists, and adds a token to the Secret if needed.
- watch Secret deletion and removes a reference from the corresponding ServiceAccount if needed.
- ServiceAccount controller manages the ServiceAccounts inside namespaces, and ensures a ServiceAccount named
default
exists in every active namespace.
Admission Controllers
Admission Controllers is piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. They are used by advanced kubernetes features and thus the recommended admission controllers are enabled by default but additional ones can be enabled:
Disabling is possible, but be aware that some features may not work then.
Admission controllers may be either or even both of
- “mutating” - They may modify objects they admit. They run before validating admission controllers.
- “validating” - The validate objects they admit, but do not modify.
Admission controllers limit requests to create, delete, modify or connect to (proxy) but do not support read requests.
There are two special controllers:
- MutatingAdmissionWebhook: Calls any mutating webhooks which match the request. Matching webhooks are called in serial; each one may modify the object if it desires. Use caution when authoring and installing mutating webhooks
- ValidatingAdmissionWebhook: Calls any validating webhooks which match the request. Matching webhooks are called in parallel; if any of them rejects the request, the request fails.
Authorization
Authorization in Kubernetes determines whether a request to the kube-apiserver is allowed or denied. By default permissions are denied and thus requests must be explicitly allowed.
Although Kubernetes uses the API server, access controls and policies that depend on specific fields of specific kinds of objects are handled by Admission Controllers.)
Requests can be
- Non-resource requests - Requests to endpoints other than
/api/v1/...
or/apis/<group>/<version>/...
. They use the lower-cased HTTP method of the request as the verb e.g.get
insteadGET
. - Resource requests - Requests to endpoints other than
/api/v1/...
or/apis/<group>/<version>/...
.
The following API request attributes are reviewed for each request:
user
- The user string provided during authentication.group
- The list of group names to which the authenticated user belongs.extra
- A map of arbitrary string keys to string values, provided by the authentication layer.API
- Indicates whether the request is for an API resource.Request path
- Path to miscellaneous non-resource endpoints like /api or /healthz.API request verb
- API verbs likeget
,list
,create
,update
,patch
,watch
,delete
, anddeletecollection
are used for resource requests. To determine the request verb for a resource API endpoint, see Determine the request verb.HTTP request verb
- Lowercased HTTP methods likeget
,post
,put
, anddelete
are used for non-resource requests.Resource
- The ID or name of the resource that is being accessed (for resource requests only) – For resource requests usingget
,update
,patch
, anddelete
verbs, you must provide the resource name.Subresource
- The subresource that is being accessed (for resource requests only).Namespace
- The namespace of the object that is being accessed (for namespaced resource requests only).API group
- The API Group being accessed (for resource requests only). An empty string designates the core API group
Authorization Modes
Kubernetes supports different Authorization Modes, which have to be enabled on the kube-apiserver using the --authorization-mode=XXX
flag:
Node
- A special-purpose authorization mode that grants permissions to kubelets based on the pods they are scheduled to run - see Node Authorization.ABAC
- Attribute-based access control (ABAC) grants access rights through the use of policies which combine attributes (user attributes, resource attributes, object, environment attributes, etc) together - see [ABAC][ABAC Mode]. Compared to RBAC, ABAC is permissive which includes granting full API access to all service accounts.RBAC
- Role-based access control (RBAC) grants access rights to API endpoints for users to perform specific tasks (view, create, modify)Webhook
- A WebHook is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST. A web application implementing WebHooks will POST a message to a URL when certain things happen - see Webhook Mode.
Today, mostly RBAC is preferred over ABAC, so let’s have a detailed look at it.
Role-based access control (RBAC)
Role-based access control (RBAC) is a mechanism to regulate access to resources using roles and privileges. To use Kubernetes RBAC the kube-apiserver has to be started with --authorization-mode=RBAC
, the one can use the rbac.authorization.k8s.io
API Group. Three primary rules are defined for RBAC:
- Role assignment: A subject can exercise a permission only if the subject has selected or been assigned a role.
- Role authorization: A subject’s active role must be authorized for the subject. With rule 1 above, this rule ensures that users can take on only roles for which they are authorized.
- Permission authorization: A subject can exercise a permission only if the permission is authorized for the subject’s active role. With rules 1 and 2, this rule ensures that users can exercise only permissions for which they are authorized.
These rules are implemented in Kubernetes using these objects:
Term | Description |
---|---|
Role | Set of permissions for a namespace |
ClusterRole | Cluster-wide (non-namespaced) set of permissions.Reuse them within multiple namespaces using RoleBinding or ClusterRoleBinding |
RoleBinding | Grants the permissions defined in a Role or Cluster Role to a user or set of users in a namespace. |
ClusterRoleBinding | Grants the permissions defined in a Role or ClusterRole to a user or set of users across a whole cluster. |
Roles and ClusterRoles
If you want to define a role within a namespace, use a Role, else use ClusterRole to define a role cluster-wide. For example role pod-reader
grants read access to pods in the default
namespace:
A ClusterRole can be used to grant the same permissions as a Role, but also can grant access to:
- cluster-scoped resources (like nodes)
- non-resource endpoints (like
/healthz
) - namespaced resources (like Pods), across all namespaces
Example of a ClusterRole that can be used to grant read access to secrets in any particular namespace, or across all namespaces, depending on how it is bound:
resources
are referenced using the string representation of their object name, whereas the name should match exactly the one in the URL for the relevant API endpoint e.g.pod
as the endpoint is/api/v1/namespaces/{namespace}/pods/{name}
. Subresource e.g./api/v1/namespaces/{namespace}/pods/{name}/log
are using a slash (/
) to delimit the resource and subresource i.e.pod/log
.By using
resourceNames
list, you can restrict to individual instances of a resource.You can aggregate several ClusterRoles into one combined ClusterRole using
aggregationRule
. The following example aggregates all CusterRoles with the labelrbac.example.com/aggregate-to-monitoring: "true"
RoleBinding and ClusterRoleBinding
So what means binding? Above we have an example of the Role pod-reader
. The example below binds the role to the user jane
within the default
namespace, allowing her to read pods in namespace default
:
The following example allows any user in the group manager
to read secrets in any namespace - namespaces are cluster-scoped objects:
- The
roleRef
for an existing binding cannot be changed. subjects
an be groups, users or ServiceAccounts. Usernames are represented as strings and can be of format plain names, email-style names,or numeric user IDs.- Groups, like users, are represented as strings, and that string has no format requirements, other than that the prefix
system:
is reserved. system:
means that the resource is directly managed by the cluster control plane
Checkout the role binding examples and the default roles and role bindings, as default ClusterRole and ClusterRoleBinding objects (prefixed with system:
).
Some more things to know:
- Auto-reconciliation: At each start-up, the API server updates default cluster roles with any missing permissions, and updates default cluster role bindings with any missing subjects.
- API discovery roles: Default role bindings authorize unauthenticated and authenticated users to read API information that is deemed safe to be publicly accessible (including CustomResourceDefinitions). This can be disabled.
- User-facing roles: Some of the default ClusterRoles are not
system:
prefixed as they are intended to be user-facing roles, including include super-user rolescluster-admin
,admin
,edit
andview
. - The RBAC API prevents users from escalating privileges by editing roles or role bindings, even when the RBAC authorizer is not in use. See also Restrictions on role creation or update and Restrictions on role binding creation or update
- ServiceAccount permissions: Default RBAC policies grant scoped permissions to control-plane components, nodes, and controllers, but grant no permissions to service accounts outside the
kube-system
namespace (beyond discovery permissions given to all authenticated users).
Access to the cluster
Accessing Clusters is usually done via kubectl
, but it requires a kuebconfig
which contains the details on how to access the cluster. Example:
The above uses X509 Client Certs where the common name of the subject is used as the user name for the request. These certificates have to be created and distributed and then provided in the kubeconf
as client-certificate
and a client-key
]. Alternatively one can use tokens (e.g. Static Token File, Service Account Tokens, OpenID Connect Tokens) or username
and password
.
But how to get there? At first, you create a cluster (using kubeadm
) the credential information is created on the control plane under /etc/kubernetes/admin.conf
Kubeadm signs the certificate in the
admin.conf
to have Subject: O = system:masters, CN = kubernetes-admin. system:masters
is a break-glass, super user group that bypasses the authorization layer (e.g. RBAC).
Still as a super admin you can use this config file for further actions. For other users you would generate a user-specific kubeconfig file with kubeadm kubeconfig user
command.
You may decide to use a platform like Rancher which abstracts this activities and makes it easier to integrate with authentication services, manage privileges and provide credentials for users.