blob: b98c4a2444ce7f097b6ae6e2bcac8fb1caff20bf [file] [log] [blame] [view]
Giorgi Lekveishviliec6b3cc2023-12-01 16:30:04 +04001# CoreDNS
2
3[CoreDNS](https://coredns.io/) is a DNS server that chains plugins and provides DNS Services
4
5# TL;DR;
6
7```console
8$ helm repo add coredns https://coredns.github.io/helm
9$ helm --namespace=kube-system install coredns coredns/coredns
10```
11
12## Introduction
13
14This chart bootstraps a [CoreDNS](https://github.com/coredns/coredns) deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. This chart will provide DNS Services and can be deployed in multiple configuration to support various scenarios listed below:
15
16- CoreDNS as a cluster dns service and a drop-in replacement for Kube/SkyDNS. This is the default mode and CoreDNS is deployed as cluster-service in kube-system namespace. This mode is chosen by setting `isClusterService` to true.
17- CoreDNS as an external dns service. In this mode CoreDNS is deployed as any kubernetes app in user specified namespace. The CoreDNS service can be exposed outside the cluster by using using either the NodePort or LoadBalancer type of service. This mode is chosen by setting `isClusterService` to false.
18- CoreDNS as an external dns provider for kubernetes federation. This is a sub case of 'external dns service' which uses etcd plugin for CoreDNS backend. This deployment mode as a dependency on `etcd-operator` chart, which needs to be pre-installed.
19
20## Prerequisites
21
22- Kubernetes 1.10 or later
23
24## Installing the Chart
25
26The chart can be installed as follows:
27
28```console
29$ helm repo add coredns https://coredns.github.io/helm
30$ helm --namespace=kube-system install coredns coredns/coredns
31```
32
33The command deploys CoreDNS on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists various ways to override default configuration during deployment.
34
35> **Tip**: List all releases using `helm list --all-namespaces`
36
37## Uninstalling the Chart
38
39To uninstall/delete the `coredns` deployment:
40
41```console
42$ helm uninstall coredns
43```
44
45The command removes all the Kubernetes components associated with the chart and deletes the release.
46
47## Configuration
48
49| Parameter | Description | Default |
50| :--------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------- |
51| `image.repository` | The image repository to pull from | coredns/coredns |
52| `image.tag` | The image tag to pull from (derived from Chart.yaml) | `` |
53| `image.pullPolicy` | Image pull policy | IfNotPresent |
54| `image.pullSecrets` | Specify container image pull secrets | `[]` |
55| `replicaCount` | Number of replicas | 1 |
56| `resources.limits.cpu` | Container maximum CPU | `100m` |
57| `resources.limits.memory` | Container maximum memory | `128Mi` |
58| `resources.requests.cpu` | Container requested CPU | `100m` |
59| `resources.requests.memory` | Container requested memory | `128Mi` |
60| `serviceType` | Kubernetes Service type | `ClusterIP` |
61| `prometheus.service.enabled` | Set this to `true` to create Service for Prometheus metrics | `false` |
62| `prometheus.service.annotations` | Annotations to add to the metrics Service | `{prometheus.io/scrape: "true", prometheus.io/port: "9153"}` |
63| `prometheus.monitor.enabled` | Set this to `true` to create ServiceMonitor for Prometheus operator | `false` |
64| `prometheus.monitor.additionalLabels` | Additional labels that can be used so ServiceMonitor will be discovered by Prometheus | {} |
65| `prometheus.monitor.namespace` | Selector to select which namespaces the Endpoints objects are discovered from. | `""` |
66| `prometheus.monitor.interval` | Scrape interval for polling the metrics endpoint. (E.g. "30s") | `""` |
67| `service.clusterIP` | IP address to assign to service | `""` |
68| `service.clusterIPs` | IP addresses to assign to service | `[]` |
69| `service.loadBalancerIP` | IP address to assign to load balancer (if supported) | `""` |
70| `service.externalIPs` | External IP addresses | [] |
71| `service.externalTrafficPolicy` | Enable client source IP preservation | [] |
72| `service.ipFamilyPolicy` | Service dual-stack policy | `""` |
73| `service.annotations` | Annotations to add to service | {} |
74| `serviceAccount.create` | If true, create & use serviceAccount | false |
75| `serviceAccount.name` | If not set & create is true, use template fullname | |
76| `rbac.create` | If true, create & use RBAC resources | true |
77| `rbac.pspEnable` | Specifies whether a PodSecurityPolicy should be created. | `false` |
78| `isClusterService` | Specifies whether chart should be deployed as cluster-service or normal k8s app. | true |
79| `priorityClassName` | Name of Priority Class to assign pods | `""` |
80| `securityContext` | securityContext definition for pods | capabilities.add.NET_BIND_SERVICE |
81| `servers` | Configuration for CoreDNS and plugins | See values.yml |
82| `livenessProbe.enabled` | Enable/disable the Liveness probe | `true` |
83| `livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `60` |
84| `livenessProbe.periodSeconds` | How often to perform the probe | `10` |
85| `livenessProbe.timeoutSeconds` | When the probe times out | `5` |
86| `livenessProbe.failureThreshold` | Minimum consecutive failures for the probe to be considered failed after having succeeded. | `5` |
87| `livenessProbe.successThreshold` | Minimum consecutive successes for the probe to be considered successful after having failed. | `1` |
88| `readinessProbe.enabled` | Enable/disable the Readiness probe | `true` |
89| `readinessProbe.initialDelaySeconds` | Delay before readiness probe is initiated | `30` |
90| `readinessProbe.periodSeconds` | How often to perform the probe | `10` |
91| `readinessProbe.timeoutSeconds` | When the probe times out | `5` |
92| `readinessProbe.failureThreshold` | Minimum consecutive failures for the probe to be considered failed after having succeeded. | `5` |
93| `readinessProbe.successThreshold` | Minimum consecutive successes for the probe to be considered successful after having failed. | `1` |
94| `affinity` | Affinity settings for pod assignment | {} |
95| `nodeSelector` | Node labels for pod assignment | {} |
96| `tolerations` | Tolerations for pod assignment | [] |
97| `zoneFiles` | Configure custom Zone files | [] |
98| `extraContainers` | Optional array of sidecar containers | [] |
99| `extraVolumes` | Optional array of volumes to create | [] |
100| `extraVolumeMounts` | Optional array of volumes to mount inside the CoreDNS container | [] |
101| `extraSecrets` | Optional array of secrets to mount inside the CoreDNS container | [] |
102| `customLabels` | Optional labels for Deployment(s), Pod, Service, ServiceMonitor objects | {} |
103| `customAnnotations` | Optional annotations for Deployment(s), Pod, Service, ServiceMonitor objects |
104| `rollingUpdate.maxUnavailable` | Maximum number of unavailable replicas during rolling update | `1` |
105| `rollingUpdate.maxSurge` | Maximum number of pods created above desired number of pods | `25%` |
106| `podDisruptionBudget` | Optional PodDisruptionBudget | {} |
107| `podAnnotations` | Optional Pod only Annotations | {} |
108| `terminationGracePeriodSeconds` | Optional duration in seconds the pod needs to terminate gracefully. | 30 |
109| `hpa.enabled` | Enable Hpa autoscaler instead of proportional one | `false` |
110| `hpa.minReplicas` | Hpa minimum number of CoreDNS replicas | `1` |
111| `hpa.maxReplicas` | Hpa maximum number of CoreDNS replicas | `2` |
112| `hpa.metrics` | Metrics definitions used by Hpa to scale up and down | {} |
113| `autoscaler.enabled` | Optionally enabled a cluster-proportional-autoscaler for CoreDNS | `false` |
114| `autoscaler.coresPerReplica` | Number of cores in the cluster per CoreDNS replica | `256` |
115| `autoscaler.nodesPerReplica` | Number of nodes in the cluster per CoreDNS replica | `16` |
116| `autoscaler.min` | Min size of replicaCount | 0 |
117| `autoscaler.max` | Max size of replicaCount | 0 (aka no max) |
118| `autoscaler.includeUnschedulableNodes` | Should the replicas scale based on the total number or only schedulable nodes | `false` |
119| `autoscaler.preventSinglePointFailure` | If true does not allow single points of failure to form | `true` |
120| `autoscaler.customFlags` | A list of custom flags to pass into cluster-proportional-autoscaler | (no args) |
121| `autoscaler.image.repository` | The image repository to pull autoscaler from | registry.k8s.io/cpa/cluster-proportional-autoscaler |
122| `autoscaler.image.tag` | The image tag to pull autoscaler from | `1.8.5` |
123| `autoscaler.image.pullPolicy` | Image pull policy for the autoscaler | IfNotPresent |
124| `autoscaler.image.pullSecrets` | Specify container image pull secrets | `[]` |
125| `autoscaler.priorityClassName` | Optional priority class for the autoscaler pod. `priorityClassName` used if not set. | `""` |
126| `autoscaler.affinity` | Affinity settings for pod assignment for autoscaler | {} |
127| `autoscaler.nodeSelector` | Node labels for pod assignment for autoscaler | {} |
128| `autoscaler.tolerations` | Tolerations for pod assignment for autoscaler | [] |
129| `autoscaler.resources.limits.cpu` | Container maximum CPU for cluster-proportional-autoscaler | `20m` |
130| `autoscaler.resources.limits.memory` | Container maximum memory for cluster-proportional-autoscaler | `10Mi` |
131| `autoscaler.resources.requests.cpu` | Container requested CPU for cluster-proportional-autoscaler | `20m` |
132| `autoscaler.resources.requests.memory` | Container requested memory for cluster-proportional-autoscaler | `10Mi` |
133| `autoscaler.configmap.annotations` | Annotations to add to autoscaler config map. For example to stop CI renaming them | {} |
134| `autoscaler.livenessProbe.enabled` | Enable/disable the Liveness probe | `true` |
135| `autoscaler.livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `10` |
136| `autoscaler.livenessProbe.periodSeconds` | How often to perform the probe | `5` |
137| `autoscaler.livenessProbe.timeoutSeconds` | When the probe times out | `5` |
138| `autoscaler.livenessProbe.failureThreshold` | Minimum consecutive failures for the probe to be considered failed after having succeeded. | `3` |
139| `autoscaler.livenessProbe.successThreshold` | Minimum consecutive successes for the probe to be considered successful after having failed. | `1` |
140| `autoscaler.extraContainers` | Optional array of sidecar containers | [] |
141| `deployment.enabled` | Optionally disable the main deployment and its respective resources. | `true` |
142| `deployment.name` | Name of the deployment if `deployment.enabled` is true. Otherwise the name of an existing deployment for the autoscaler or HPA to target. | `""` |
143| `deployment.annotations` | Annotations to add to the main deployment | `{}` |
144
145See `values.yaml` for configuration notes. Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
146
147```console
148$ helm install coredns \
149 coredns/coredns \
150 --set rbac.create=false
151```
152
153The above command disables automatic creation of RBAC rules.
154
155Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example,
156
157```console
158$ helm install coredns coredns/coredns -f values.yaml
159```
160
161> **Tip**: You can use the default [values.yaml](/charts/coredns/values.yaml)
162
163## Caveats
164
165The chart will automatically determine which protocols to listen on based on
166the protocols you define in your zones. This means that you could potentially
167use both "TCP" and "UDP" on a single port.
168Some cloud environments like "GCE" or "Azure container service" cannot
169create external loadbalancers with both "TCP" and "UDP" protocols. So
170When deploying CoreDNS with `serviceType="LoadBalancer"` on such cloud
171environments, make sure you do not attempt to use both protocols at the same
172time.
173
174## Autoscaling
175
176By setting `autoscaler.enabled = true` a
177[cluster-proportional-autoscaler](https://github.com/kubernetes-incubator/cluster-proportional-autoscaler)
178will be deployed. This will default to a coredns replica for every 256 cores, or
17916 nodes in the cluster. These can be changed with `autoscaler.coresPerReplica`
180and `autoscaler.nodesPerReplica`. When cluster is using large nodes (with more
181cores), `coresPerReplica` should dominate. If using small nodes,
182`nodesPerReplica` should dominate.
183
184This also creates a ServiceAccount, ClusterRole, and ClusterRoleBinding for
185the autoscaler deployment.
186
187`replicaCount` is ignored if this is enabled.
188
189By setting `hpa.enabled = true` a [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)
190is enabled for Coredns deployment. This can scale number of replicas based on meitrics
191like CpuUtilization, MemoryUtilization or Custom ones.
192
193## Adopting existing CoreDNS resources
194
195If you do not want to delete the existing CoreDNS resources in your cluster, you can adopt the resources into a release as of Helm 3.2.0.
196
197You will also need to annotate and label your existing resources to allow Helm to assume control of them. See: https://github.com/helm/helm/pull/7649
198
199```
200annotations:
201 meta.helm.sh/release-name: your-release-name
202 meta.helm.sh/release-namespace: your-release-namespace
203label:
204 app.kubernetes.io/managed-by: Helm
205```
206
207Once you have annotated and labeled all the resources this chart specifies, you may need to locally template the chart and compare against existing manifest to ensure there are no changes/diffs.s If
208you have been careful this should not diff and leave all the resources unmodified and now under management of helm.
209
210Some values to investigate to help adopt your existing manifests to the Helm release are:
211
212- k8sAppLabelOverride
213- service.name
214- customLabels
215
216In some cases, you will need to orphan delete your existing deployment since selector labels are immutable.
217
218```
219kubectl delete deployment coredns --cascade=orphan
220```
221
222This will delete the deployment and leave the replicaset to ensure no downtime in the cluster. You will need to manually delete the replicaset AFTER Helm has released a new deployment.
223
224Here is an example script to modify the annotations and labels of existing resources:
225
226WARNING: Substitute YOUR_HELM_RELEASE_NAME_HERE with the name of your helm release.
227
228```
229#!/usr/bin/env bash
230
231set -euo pipefail
232
233for kind in config service serviceAccount; do
234 echo "setting annotations and labels on $kind/coredns"
235 kubectl -n kube-system annotate --overwrite $kind coredns meta.helm.sh/release-name=YOUR_HELM_RELEASE_NAME_HERE
236 kubectl -n kube-system annotate --overwrite $kind coredns meta.helm.sh/release-namespace=kube-system
237 kubectl -n kube-system label --overwrite $kind coredns app.kubernetes.io/managed-by=Helm
238done
239```
240
241NOTE: Sometimes, previous deployments of kube-dns that have been migrated to CoreDNS still use kube-dns for the service name as well.
242
243```
244echo "setting annotations and labels on service/kube-dns"
245kubectl -n kube-system annotate --overwrite service kube-dns meta.helm.sh/release-name=YOUR_HELM_RELEASE_NAME_HERE
246kubectl -n kube-system annotate --overwrite service kube-dns meta.helm.sh/release-namespace=kube-system
247kubectl -n kube-system label --overwrite service kube-dns app.kubernetes.io/managed-by=Helm
248```