The Kubernetes (K8S) provider is used to interact with the resources supported by Kubernetes. The provider needs to be configured with the proper credentials before it can be used.
Use the navigation to the left to read about the available resources.
provider "kubernetes" {
config_path = "~/.kube/config"
config_context = "my-context"
}
resource "kubernetes_namespace" "example" {
metadata {
name = "my-first-namespace"
}
}
Both backward and forward compatibility with Kubernetes API is mostly defined by the official K8S Go library (prior to 1.1
release) and client Go library which we ship with Terraform. Below are versions of the library bundled with given versions of Terraform.
<= 0.9.6
(prior to provider split) - Kubernetes 1.5.4
0.9.7
(prior to provider split) < 1.1
(provider version) - Kubernetes 1.6.1
1.1+
- Kubernetes 1.7
Terraform providers for various cloud providers feature resources to spin up managed Kubernetes clusters on services such as EKS, AKS and GKE. Such resources (or data-sources) will have attributes that expose the credentials needed for the Kubernetes provider to connect to these clusters.
To use these credentials with the Kubernetes provider, they can be interpolated into the respective attributes of the Kubernetes provider configuration block.
The most reliable way to configure the Kubernetes provider is to ensure that the cluster itself and the Kubernetes provider resources can be managed with separate apply
operations. Data-sources can be used to convey values between the two stages as needed.
For specific usage examples, see the guides for AKS, EKS, and GKE.
The Kubernetes provider can get its configuration in two ways:
The provider always first tries to load a config file from a given location when config_path
or config_paths
(or their equivalent environment variables) are set. Depending on whether you have a current context set this may require config_context_auth_info
and/or config_context_cluster
and/or config_context
.
For a full list of supported provider authentication arguments and their corresponding environment variables, see the argument reference below.
The easiest way is to supply a path to your kubeconfig file using the config_path
attribute or using the KUBE_CONFIG_PATH
environment variable. A kubeconfig file may have multiple contexts. If config_context
is not specified, the provider will use the default
context.
provider "kubernetes" {
config_path = "~/.kube/config"
}
The provider also supports multiple paths in the same way that kubectl does using the config_paths
attribute or KUBE_CONFIG_PATHS
environment variable.
provider "kubernetes" {
config_paths = [
"/path/to/config_a.yaml",
"/path/to/config_b.yaml"
]
}
You can also configure the host, basic auth credentials, and client certificate authentication explicitly or through environment variables.
provider "kubernetes" {
host = "https://cluster_endpoint:port"
client_certificate = file("~/.kube/client-cert.pem")
client_key = file("~/.kube/client-key.pem")
cluster_ca_certificate = file("~/.kube/cluster-ca-cert.pem")
}
The provider uses the KUBERNETES_SERVICE_HOST
and KUBERNETES_SERVICE_PORT
environment variables to detect when it is running inside a cluster, so in this case you do not need to specify any attributes in the provider block if you want to connect to the local kubernetes cluster.
If you want to connect to a different cluster than the one terraform is running inside, configure the provider as above.
Find more comprehensive in-cluster
config example here.
Some cloud providers have short-lived authentication tokens that can expire relatively quickly. To ensure the Kubernetes provider is receiving valid credentials, an exec-based plugin can be used to fetch a new token before each Terraform operation. For example, on EKS, the command eks get-token
can be used:
provider "kubernetes" {
host = var.cluster_endpoint
cluster_ca_certificate = base64decode(var.cluster_ca_cert)
exec {
api_version = "client.authentication.k8s.io/v1beta1"
args = ["eks", "get-token", "--cluster-name", var.cluster_name]
command = "aws"
}
}
For further reading, see these examples which demonstrate different approaches to keeping the cluster credentials up to date: AKS, EKS, and GKE.
In certain cases, external systems can add and modify resources annotations and labels for their own purposes. However, Terraform will remove them since they are not presented in the code. It also might be hard to update code accordingly to stay tuned with the changes that come outside. In order to address this ignore_annotations
and ignore_labels
attributes were introduced on the provider level. They allow Terraform to ignore certain annotations and labels across all resources.
Both attributes support RegExp to match metadata objects more effectively.
Please keep in mind that all data sources remain unaffected, and the provider always returns all labels and annotations, despite the ignore_annotations
and ignore_labels
settings. The same applies to the pod and job definitions that fall under templates. To ignore certain annotations and/or labels on the template level, please use the ignore_changes
feature of the lifecycle meta-argument.
The following example demonstrates how to ignore changes related to the kubectl.kubernetes.io/restartedAt
annotation that were made in the upstream Kubernetes object:
resource "kubernetes_deployment_v1" "this" {
// omit the resource config
lifecycle {
ignore_changes = [
spec[0].template[0].metadata[0].annotations["kubectl.kubernetes.io/restartedAt"],
]
}
}
The following example demonstrates how to ignore particular annotation keys:
provider "kubernetes" {
ignore_annotations = [
"cni\\.projectcalico\\.org\\/podIP",
"cni\\.projectcalico\\.org\\/podIPs",
]
}
Next example demonstrates how to ignore AWS load balancer annotations:
provider "kubernetes" {
ignore_annotations = [
"^service\\.beta\\.kubernetes\\.io\\/aws-load-balancer.*",
]
}
Since dot .
, forward slash /
, and some other symbols have special meaning in RegExp, they should be escaped by adding a double backslash in front of them if you want to use them as they are.
The following arguments are supported:
host
- (Optional) The hostname (in form of URI) of the Kubernetes API. Can be sourced from KUBE_HOST
.username
- (Optional) The username to use for HTTP basic authentication when accessing the Kubernetes API. Can be sourced from KUBE_USER
.password
- (Optional) The password to use for HTTP basic authentication when accessing the Kubernetes API. Can be sourced from KUBE_PASSWORD
.insecure
- (Optional) Whether the server should be accessed without verifying the TLS certificate. Can be sourced from KUBE_INSECURE
. Defaults to false
.tls_server_name
- (Optional) Server name passed to the server for SNI and is used in the client to check server certificates against. Can be sourced from KUBE_TLS_SERVER_NAME
.client_certificate
- (Optional) PEM-encoded client certificate for TLS authentication. Can be sourced from KUBE_CLIENT_CERT_DATA
.client_key
- (Optional) PEM-encoded client certificate key for TLS authentication. Can be sourced from KUBE_CLIENT_KEY_DATA
.cluster_ca_certificate
- (Optional) PEM-encoded root certificates bundle for TLS authentication. Can be sourced from KUBE_CLUSTER_CA_CERT_DATA
.config_path
- (Optional) A path to a kube config file. Can be sourced from KUBE_CONFIG_PATH
.config_paths
- (Optional) A list of paths to the kube config files. Can be sourced from KUBE_CONFIG_PATHS
.config_context
- (Optional) Context to choose from the config file. Can be sourced from KUBE_CTX
.config_context_auth_info
- (Optional) Authentication info context of the kube config (name of the kubeconfig user, --user
flag in kubectl
). Can be sourced from KUBE_CTX_AUTH_INFO
.config_context_cluster
- (Optional) Cluster context of the kube config (name of the kubeconfig cluster, --cluster
flag in kubectl
). Can be sourced from KUBE_CTX_CLUSTER
.token
- (Optional) Token of your service account. Can be sourced from KUBE_TOKEN
.proxy_url
- (Optional) URL to the proxy to be used for all API requests. URLs with "http", "https", and "socks5" schemes are supported. Can be sourced from KUBE_PROXY_URL
.exec
- (Optional) Configuration block to use an [exec-based credential plugin] (https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins), e.g. call an external command to receive user credentials.
api_version
- (Required) API version to use when decoding the ExecCredentials resource, e.g. client.authentication.k8s.io/v1beta1
.command
- (Required) Command to execute.args
- (Optional) List of arguments to pass when executing the plugin.env
- (Optional) Map of environment variables to set when executing the plugin.ignore_annotations
- (Optional) List of Kubernetes metadata annotations to ignore across all resources handled by this provider for situations where external systems are managing certain resource annotations. This option does not affect annotations within a template block. Each item is a regular expression.ignore_labels
- (Optional) List of Kubernetes metadata labels to ignore across all resources handled by this provider for situations where external systems are managing certain resource labels. This option does not affect annotations within a template block. Each item is a regular expression.