Authentication and Authorization in Kubernetes

Authentication (who am I) and authorization (what I am allowed to do) are essential and thus having a basic understanding on how Kubernetes handles this, is very useful.

Authentication #

Kubernetes offers different authentication strategies like client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth to authenticate API requests through authentication plugins. However first we have to understand user and service accounts.

User and Service accounts #

All Kubernetes clusters have two categories of users:

• User Accounts:
  • are meant for humans
  • are global (unique name across all namespaces)
  • might be synced from a corporate database
  • not managed by Kubernetes
  • admin is the default one
• Service Accounts (sa):
  • are used for processes running in pods
  • are namespaced
  • managed by Kubernetes
  • could be created by cluster users for specific tasks
  • default is assigned to objects if no sa is defined

Support for authorization and user accounts is planned but incomplete.

User Accounts #

It’s important to note that

Kubernetes does not have objects which represent normal user accounts. Normal users cannot be added to a cluster through an API call.

Any user that presents a valid certificate signed by the cluster’s certificate authority (CA) is considered authenticated, whereas the username is determined from the common name field in the ‘subject’ of the cert (e.g., /CN=bob). Kubernetes then uses RBAC to authorize a user to perform more operations on a resource.

Certificate Signing Requests for Normal Users explains in details on how a normal user can get a valid certificate. However often you would have a management layer on top of Kubernetes (e.g. Rancher) which takes care of this.

Service Accounts #

Service Accounts are used access the API from inside a pod, using automatically mounted service account credentials. To make this happen, there are several components in place:

• ServiceAccount Admission Controller which is an Adminisson Controller which modify pods as they are created or updated:
  • If the pod does not have a ServiceAccount set, it sets the ServiceAccount to default.
  • It ensures that the ServiceAccount referenced by the pod exists, and otherwise rejects it.
  • It adds a volume to the pod which contains a token for API access if neither the ServiceAccount automountServiceAccountToken nor the Pod’s automountServiceAccountToken is set to false.
  • It adds a volumeSource to each container of the pod mounted at /var/run/secrets/kubernetes.io/serviceaccount,if the previous step has created a volume for ServiceAccount token.
  • If the pod does not contain any imagePullSecrets, then imagePullSecrets of the ServiceAccount are added to the pod
• [TokenController] runs as part of kube-controller-manager and asynchronously does
  • watch ServiceAccount creation and creates a corresponding ServiceAccount token Secret to allow API access.
  • watch ServiceAccount deletion and deletes all corresponding ServiceAccount token Secrets.
  • watch ServiceAccount token Secret addition, and ensures the referenced ServiceAccount exists, and adds a token to the Secret if needed.
  • watch Secret deletion and removes a reference from the corresponding ServiceAccount if needed.
• ServiceAccount controller manages the ServiceAccounts inside namespaces, and ensures a ServiceAccount named default exists in every active namespace.

Admission Controllers #

Admission Controllers is piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. They are used by advanced kubernetes features and thus the recommended admission controllers are enabled by default but additional ones can be enabled:

kube-apiserver --enable-admission-plugins=NamespaceLifecycle,LimitRanger ...

Disabling is possible, but be aware that some features may not work then.

kube-apiserver --disable-admission-plugins=PodNodeSelector,AlwaysDeny ...

Admission controllers may be either or even both of

• “mutating” - They may modify objects they admit. They run before validating admission controllers.
• “validating” - The validate objects they admit, but do not modify.

Admission controllers limit requests to create, delete, modify or connect to (proxy) but do not support read requests.

There are two special controllers:

• MutatingAdmissionWebhook: Calls any mutating webhooks which match the request. Matching webhooks are called in serial; each one may modify the object if it desires. Use caution when authoring and installing mutating webhooks
• ValidatingAdmissionWebhook: Calls any validating webhooks which match the request. Matching webhooks are called in parallel; if any of them rejects the request, the request fails.

Authorization #

Authorization in Kubernetes determines whether a request to the kube-apiserver is allowed or denied. By default permissions are denied and thus requests must be explicitly allowed.

Although Kubernetes uses the API server, access controls and policies that depend on specific fields of specific kinds of objects are handled by Admission Controllers.)

Requests can be

• Non-resource requests - Requests to endpoints other than /api/v1/... or /apis/<group>/<version>/.... They use the lower-cased HTTP method of the request as the verb e.g. get instead GET.
• Resource requests - Requests to endpoints other than /api/v1/... or /apis/<group>/<version>/....

The following API request attributes are reviewed for each request:

• user - The user string provided during authentication.
• group - The list of group names to which the authenticated user belongs.
• extra - A map of arbitrary string keys to string values, provided by the authentication layer.
• API - Indicates whether the request is for an API resource.
• Request path - Path to miscellaneous non-resource endpoints like /api or /healthz.
• API request verb - API verbs like get, list, create, update, patch, watch, delete, and deletecollection are used for resource requests. To determine the request verb for a resource API endpoint, see Determine the request verb.
• HTTP request verb - Lowercased HTTP methods like get, post, put, and delete are used for non-resource requests.
• Resource - The ID or name of the resource that is being accessed (for resource requests only) – For resource requests using get, update, patch, and delete verbs, you must provide the resource name.
• Subresource - The subresource that is being accessed (for resource requests only).
• Namespace - The namespace of the object that is being accessed (for namespaced resource requests only).
• API group - The API Group being accessed (for resource requests only). An empty string designates the core API group

Authorization Modes #

Kubernetes supports different Authorization Modes, which have to be enabled on the kube-apiserver using the --authorization-mode=XXX flag:

• Node - A special-purpose authorization mode that grants permissions to kubelets based on the pods they are scheduled to run - see Node Authorization.
• ABAC - Attribute-based access control (ABAC) grants access rights through the use of policies which combine attributes (user attributes, resource attributes, object, environment attributes, etc) together - see [ABAC][ABAC Mode]. Compared to RBAC, ABAC is permissive which includes granting full API access to all service accounts.
• RBAC - Role-based access control (RBAC) grants access rights to API endpoints for users to perform specific tasks (view, create, modify)
• Webhook - A WebHook is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST. A web application implementing WebHooks will POST a message to a URL when certain things happen - see Webhook Mode.

Today, mostly RBAC is preferred over ABAC, so let’s have a detailed look at it.

Role-based access control (RBAC) #

Role-based access control (RBAC) is a mechanism to regulate access to resources using roles and privileges. To use Kubernetes RBAC the kube-apiserver has to be started with --authorization-mode=RBAC, the one can use the rbac.authorization.k8s.io API Group. Three primary rules are defined for RBAC:

• Role assignment: A subject can exercise a permission only if the subject has selected or been assigned a role.
• Role authorization: A subject’s active role must be authorized for the subject. With rule 1 above, this rule ensures that users can take on only roles for which they are authorized.
• Permission authorization: A subject can exercise a permission only if the permission is authorized for the subject’s active role. With rules 1 and 2, this rule ensures that users can exercise only permissions for which they are authorized.

These rules are implemented in Kubernetes using these objects:

Roles and ClusterRoles #

If you want to define a role within a namespace, use a Role, else use ClusterRole to define a role cluster-wide. For example role pod-reader grants read access to pods in the default namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

A ClusterRole can be used to grant the same permissions as a Role, but also can grant access to:

• cluster-scoped resources (like nodes)
• non-resource endpoints (like /healthz)
• namespaced resources (like Pods), across all namespaces

Example of a ClusterRole that can be used to grant read access to secrets in any particular namespace, or across all namespaces, depending on how it is bound:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" omitted since ClusterRoles are not namespaced
  name: secret-reader
rules:
- apiGroups: [""]
  #
  # at the HTTP level, the name of the resource for accessing Secret
  # objects is "secrets"
  resources: ["secrets"]
  # resourceNames: ["my-secret"]
  verbs: ["get", "watch", "list"]

• resources are referenced using the string representation of their object name, whereas the name should match exactly the one in the URL for the relevant API endpoint e.g. pod as the endpoint is /api/v1/namespaces/{namespace}/pods/{name}. Subresource e.g. /api/v1/namespaces/{namespace}/pods/{name}/log are using a slash (/) to delimit the resource and subresource i.e. pod/log.

• By using resourceNames list, you can restrict to individual instances of a resource.

• You can aggregate several ClusterRoles into one combined ClusterRole using aggregationRule. The following example aggregates all CusterRoles with the label rbac.example.com/aggregate-to-monitoring: "true"

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: monitoring
  aggregationRule:
    clusterRoleSelectors:
    - matchLabels:
        rbac.example.com/aggregate-to-monitoring: "true"
  rules: [] # The control plane automatically fills in the rules
  

RoleBinding and ClusterRoleBinding #

So what means binding? Above we have an example of the Role pod-reader. The example below binds the role to the user jane within the default namespace, allowing her to read pods in namespace default:

apiVersion: rbac.authorization.k8s.io/v1
# This role binding allows "jane" to read pods in the "default" namespace.
# You need to already have a Role named "pod-reader" in that namespace.
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
# You can specify more than one "subject"
- kind: User
  name: jane # "name" is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  # "roleRef" specifies the binding to a Role / ClusterRole
  kind: Role #this must be Role or ClusterRole
  name: pod-reader # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io

The following example allows any user in the group manager to read secrets in any namespace - namespaces are cluster-scoped objects:

apiVersion: rbac.authorization.k8s.io/v1
# This cluster role binding allows anyone in the "manager" group to read secrets in any namespace.
kind: ClusterRoleBinding
metadata:
  name: read-secrets-global
subjects:
- kind: Group
  name: manager # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

• The roleRef for an existing binding cannot be changed.
• subjects an be groups, users or ServiceAccounts. Usernames are represented as strings and can be of format plain names, email-style names,or numeric user IDs.
• Groups, like users, are represented as strings, and that string has no format requirements, other than that the prefix system: is reserved.
• system: means that the resource is directly managed by the cluster control plane

Checkout the role binding examples and the default roles and role bindings, as default ClusterRole and ClusterRoleBinding objects (prefixed with system:).

Some more things to know:

• Auto-reconciliation: At each start-up, the API server updates default cluster roles with any missing permissions, and updates default cluster role bindings with any missing subjects.
• API discovery roles: Default role bindings authorize unauthenticated and authenticated users to read API information that is deemed safe to be publicly accessible (including CustomResourceDefinitions). This can be disabled.
• User-facing roles: Some of the default ClusterRoles are not system: prefixed as they are intended to be user-facing roles, including include super-user roles cluster-admin, admin, edit and view.
• The RBAC API prevents users from escalating privileges by editing roles or role bindings, even when the RBAC authorizer is not in use. See also Restrictions on role creation or update and Restrictions on role binding creation or update
• ServiceAccount permissions: Default RBAC policies grant scoped permissions to control-plane components, nodes, and controllers, but grant no permissions to service accounts outside the kube-system namespace (beyond discovery permissions given to all authenticated users).

Access to the cluster #

Accessing Clusters is usually done via kubectl, but it requires a kuebconfig which contains the details on how to access the cluster. Example:

apiVersion: v1
clusters:
- cluster:
    certificate-authority: fake-ca-file
    server: https://1.2.3.4
  name: development
contexts:
- context:
    cluster: development
    namespace: frontend
    user: developer
  name: dev-frontend
current-context: dev-frontend
kind: Config
preferences: {}
users:
- name: developer
  user:
    client-certificate: fake-cert-file
    client-key: fake-key-file

The above uses X509 Client Certs where the common name of the subject is used as the user name for the request. These certificates have to be created and distributed and then provided in the kubeconf as client-certificate and a client-key]. Alternatively one can use tokens (e.g. Static Token File, Service Account Tokens, OpenID Connect Tokens) or username and password.

But how to get there? At first, you create a cluster (using kubeadm) the credential information is created on the control plane under /etc/kubernetes/admin.conf

Kubeadm signs the certificate in the admin.conf to have Subject: O = system:masters, CN = kubernetes-admin. system:masters is a break-glass, super user group that bypasses the authorization layer (e.g. RBAC).

Still as a super admin you can use this config file for further actions. For other users you would generate a user-specific kubeconfig file with kubeadm kubeconfig user command.

You may decide to use a platform like Rancher which abstracts this activities and makes it easier to integrate with authentication services, manage privileges and provide credentials for users.