When working with Kubernetes CRDs is something that you will stumble upon, so you should know what it is. But this does not go without understanding the main principles of the Kubernetes API

Kubernetes Objects

Kubernetes objects are persistent entities in the Kubernetes system used to represent the state of your cluster. Most of them include two nested object fields:

• spec - the desired state (characteristics) of the object you create
• status - the current state of the object

The control plane is actively updating the status while it managing to keep the desired state. As you have seen objects are described as yaml files, which contain these fields:

• apiVersion - Which version of the Kubernetes API you’re using to create this object
• kind - What kind of object you want to create
• metadata - Data that helps uniquely identify the object, including a name string, UID, and optional namespace
• spec - What state you desire for the object

Have a look at the Kubernetes API Reference for details on these fields for the different objects.

The API

As mentioned in post “Kubernetes - a brief introduction to container orchestration” one of the main components of the control plane is the kube-apiserver. It exposes an HTTP API (OpenAPI) endpoint at /openapi/v2 that let end users, different parts of your cluster, and external components communicate with one another.

The Kubernetes API lets you query and manipulate the state of API objects in Kubernetes (for example: Pods, Namespaces, ConfigMaps, and Events).

Although you could use REST calls, a user usually uses kubectl.

In order to keep the objects persistent, the serialized state of objects is stored in the etcd - a key value store for backing store of all cluster data

API Groups and versioning

Different API versions are supported by providing different path such as /api/v1 or /apis/rbac.authorization.k8s.io/v1alpha1, which can have different levels of stability:

• Alpha:
  • The version names contain alpha (for example, v1alpha1).
  • The software may contain bugs. Enabling a feature may expose bugs. A feature may be disabled by default.
  • The support for a feature may be dropped at any time without notice.
  • The API may change in incompatible ways in a later software release without notice.
  • The software is recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support.
• Beta:
  • The version names contain beta (for example, v2beta3).
  • The software is well tested. Enabling a feature is considered safe. Features are enabled by default.
  • The support for a feature will not be dropped, though the details may change.
  • The schema and/or semantics of objects may change in incompatible ways in a subsequent beta or stable release. When this happens, migration instructions are provided. Schema changes may require deleting, editing, and re-creating API objects. The editing process may not be straightforward. The migration may require downtime for applications that rely on the feature.
  • The software is not recommended for production uses. Subsequent releases may introduce incompatible changes. If you have multiple clusters which can be upgraded independently, you may be able to relax this restriction.
• Stable:
  • The version name is vX where X is an integer.
  • The stable versions of features appear in released software for many subsequent versions

The version of the api is usually related to the Kubernetes version, as they evolve along with Kubernetes and usually should not break - see levels above.

In addition to the versioning, APIs are grouped and is specified by the REST path (/apis/$GROUP_NAME/$VERSION) and the apiVersion field of a serialized object (e.g. apiVersion: batch/v1). Pne exception ios the core (or legacy) group which is found at REST path /api/v1 and no core group is specified in the version field i.e. apiVersion: v1. See Kubernetes API reference for more details.

The design decision also allows to individually enable and disable api gropups:

• to disable batch/v1, set --runtime-config=batch/v1=false
• to enable batch/v2alpha1, set --runtime-config=batch/v2alpha1

Extend the Kubernetes API with CustomResourceDefinitions

The Kubernetes API can be extended in one of two ways:

• Custom resources: Simple and can be created without any programming.
• aggregation layers: requires programming, but allows more control over API behaviors like how data is stored and conversion between API versions.

Aggregation Layers

aggregation layers allows you to register an APIService object, which “claims” the URL path in the Kubernetes API e.g. /apis/myextension.mycompany.io/v1/…. Request to this path are then proxied.

The most common way to implement the APIService is to run an extension API server in Pod(s) that run in your cluster. If you’re using the extension API server to manage resources in your cluster, the extension API server (also written as “extension-apiserver”) is typically paired with one or more controllers. The apiserver-builder library provides a skeleton for both extension API servers and the associated controller(s).

Custom Resources

Custom Resources makes the kube-apiserver recognise new kinds of object. You need to know two terms

Kubernetes has the CustomResourceDefinitions API which allows you to creates a new RESTful resource path by using a defined structural schema. The example below will create a new endpoint at /apis/stable.example.com/v1/namespaces/*/crontabs/...

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must match the spec fields below, and be in the form: <plural>.<group>
  name: crontabs.stable.example.com
spec:
  # group name to use for REST API: /apis/<group>/<version>
  group: stable.example.com
  # list of versions supported by this CustomResourceDefinition
  versions:
    - name: v1
      # Each version can be enabled/disabled by Served flag.
      served: true
      # One and only one version must be marked as the storage version.
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                cronSpec:
                  type: string
                image:
                  type: string
                replicas:
                  type: integer
  # either Namespaced or Cluster
  scope: Namespaced
  names:
    # plural name to be used in the URL: /apis/<group>/<version>/<plural>
    plural: crontabs
    # singular name to be used as an alias on the CLI and for display
    singular: crontab
    # kind is normally the CamelCased singular type. Your resource manifests use this.
    kind: CronTab
    # shortNames allow shorter string to match your resource on the CLI
    shortNames:
    - ct

Once you applied this definition to your cluster - the defintion is cluster wide and thus not namespaced - you can create an object - namespaced or cluster-scoped.

apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
  name: my-new-cron-object
spec:
  cronSpec: "* * * * */5"
  image: my-awesome-cron-image

Checkout the official Documentation for more details.