Custom Resource Definitions (CRDs) are Kubernetes API extensions that can define new object types. Pods, ReplicaSets, ConfigMaps, and Ingresses are examples of common internal resources. CRDs allow you to add entirely new types to this list, then manage them using familiar Kubernetes tools like Kubectl.
The CRD mechanism is intentionally abstract and can be used in countless ways to store data and create new functionality. You can find dedicated resources in many popular community tools: Cert-Manager defines objects that represent SSL Certificates and Issuers, and Helm represents Charts as its CRD.
What does the resource do?
Kubernetes resources define the types of data you can store in your cluster. They are accessed through the Kubernetes API, which provides endpoints for creating, enumerating, and modifying elements within each resource type.
You can add custom resources to store your arbitrary data in your cluster. The elements you create will be stored by the etcd control plane component along with instances of internal resources. Custom resources are automatically exposed by the API, so you don’t need to build your own tools to create element instances.
By default, CRDs act as simple data structures. Although CRDs in nature often have their own behavior, this is not provided by the CRD mechanism. Custom Kubernetes controllers and operators can be used to implement functionality around specific resources. Without a controller, CRD elements will always exist as static in-cluster data that you can interact with through the CRUD endpoints provided by the Kubernetes API.
CRDs are dynamic components that can be created and removed at any time. Some of the object types included in Kubernetes are also implemented as CRDs, providing more modularity at the core of the cluster.
Creation of CRD
CRDs themselves are a type of Kubernetes object. By writing a YAML file and deploying it to your cluster, you create them like any other resource:
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: custom-apps.crds.example.com spec: group: crds.example.com versions: - name: v1 served: true storage: true schema: openAPIV3Schema: type: object properties: spec: type: object properties: app-name: type: string app-version: type: string release-count: type: integer scope: Namespaced names: plural: custom-apps singular: custom-app kind: CustomApp
Use Kubectl to add the CustomApp CRD to your cluster:
$ kubectl apply -f custom-apps-crd.yaml customresourcedefinition.apiextensions.k8s.io/custom-apps.crds.example.com created
A YAML file defines a CRD that can be used to store information about applications. CRDs are needed
spec.group field in exact format: group takes the form of the subdomain to which the CRD belongs. The same subdomain must be included in the CRDs
metadata.name . value
names.plural is written from scratch as a new subdomain component to build final
spec.names field defines how to access the CRD when using the Kubernetes API and Kubectl commands. In this example, you will be able to run
kubectl get custom-apps and
kubectl get custom-app/example-app Interacting with objects that use CRD. As a YAML file, you must specify when you create a new object
kind: CustomApp making it an instance of CRD.
Configured by CRD as a namespace-level object type
scope field. You can use it
Cluster as an alternative value for this field to create objects that exist at the cluster level outside of any namespace.
Information related to your individual objects is defined
spec.versions.schema.openAPIV3Schema field. Each “version” listed creates a new version of CRD’s API that you can refer to
apiVersion resource field in your YAML files. CRD data is configured using OpenAPI properties; here, every “custom application” in your cluster should be
release-count properties are set in its YAML
Use the CRD
It may take a few minutes to provision the new CRD’s API endpoints. Check the progress by getting the details of the CRD with kubectl:
$ kubectl describe crd custom-apps.crds.example.com ... Status: Accepted Names: Kind: CustomApp List Kind: CustomAppList Plural: custom-apps Singular: custom-app Conditions: Last Transition Time: 2022-04-04T13:29:24Z Message: no conflicts found Reason: NoConflicts Status: True Type: NamesAccepted Last Transition Time: 2022-04-04T13:29:24Z Message: the initial names have been accepted Reason: InitialNamesAccepted Status: True Type: Established ...
The CRD is ready to use when you see it
Type: Established near the end of the team’s performance. Kubernetes will accept requests to CRD’s API endpoint. In this case, the API base URL would be:
Now you can use Kubectl to view CRD-created objects:
$ kubectl get custom-apps No resources found in default namespace.
Although there is no object yet, Kubernetes now knows that it has a named resource type
Write a new YAML file to create a “Custom Application” object
kind: CustomApp. The
apiVersion Must be set to the group name and API version provided by the CRD. inside
spec section, enter the properties you defined in the CRD’s schema.
apiVersion: crds.example.com/v1 kind: CustomApp metadata: name: demo-app-1 spec: app-name: demo-app app-version: 1.1.0 release-count: 5
Use Kubectl to add the object to your cluster:
$ kubectl apply -f custom-app.yaml customapp.crds.example.com/demo-app created
Now you can get the object details using the familiar Kubectl commands:
$ kubectl describe custom-app/demo-app-1 Name: demo-app Namespace: default Labels: <none> Annotations: <none> API Version: crds.example.com/v1 Kind: CustomApp ... Spec: App - Name: demo-app App - Version: 1.1.0 Release - Count: 5 ... Events: <none>
You now have a working custom resource in your Kubernetes cluster that stores some data. You can remove the CRD by removing it in Kubectl; this will automatically clean up all objects that use it.
$ kubectl delete crd custom-apps.crds.example.com customresourcedefinition.apiextensions.k8s.io "custom-apps.crds.example.com" deleted
Building Declarative APIs with CRDs
This CRD does not add any functionality to the cluster. It stores data, provides an API to interact with it, and can be referenced by other objects. CRDs are more powerful when combined with a dedicated controller that can be responsible for managing them.
Controllers monitor resources and take actions in response to events. Writing a controller for your CRDs allows you to turn them into declarative APIs that make a real difference in your cluster. Your objects can represent desirable state instead of the exact current state.
Cert-Manager uses this model to automatically obtain SSL certificates when new CertificateRequest objects appear in your cluster. In the Kubernetes core, nodes capture and run container images in response to the appearance of Pods. Controllers allow you to add behavior to your own CRDs, so adding a “custom application” can cause its configuration to be fetched from an external service. You can start building controllers using the Go SDK to integrate your own code with the Kubernetes runtime.
When to use CRDs?
CRDs are best used for internal data management for your organization, team, or project. They are designed to represent clearly defined schemas either as static values or as a declarative API supported by a custom controller implementation.
Advanced features allow you to perform validation procedures for fields in your schema. You can also use finalizers to handle object deletion and adopt a versioning system to handle changes to your CRD definitions.
CRDs sometimes overlap with Kubernetes ConfigMaps. These are built-in objects to store general configuration information related to your applications. A ConfigMap is appropriate when you consume values at a specific location in your cluster, such as a Pod, that accesses database settings as environment variables entered from a configmap.
CRDs are intended to be used when data needs to be a first-class citizen in your cluster. You can create multiple instances of a CRD’s resource type, interact with them directly using Kubectl, and build your own structured schemas that guide users to enter the correct values. They may be a better choice when the data exists independently of any other element in your cluster.
Kubernetes Custom Resource Definitions (CRDs) define new object types that you can use with the Kubernetes API and Kubectl. Each CRD gets its own versioned API, has a structured schema, and can be used to implement new in-cluster functionality when supported by a supporting controller.
CRDs can sometimes seem complicated and are reserved for advanced situations. It doesn’t have to be this way. Simple CRDs to store static values in your cluster are easy to create, as shown in the “custom program” example above. They get first-class treatment within Kubernetes, so they can be used to store independent cluster data.
It is also important to understand that CRDs are not very compatible. Built-in objects such as ConfigMaps and Secrets are intended to house most forms of configuration that will be used directly by your application’s Nodes. Writing a CRD that defines your application’s configuration file schema is usually unnecessary and becomes more difficult to maintain over time because you won’t benefit from ConfigMap features like automatic rolling updates and environment variable injection.