Kubernetes Provider

The Kubernetes (K8S) provider is used to interact with the resources supported by Kubernetes. The provider needs to be configured with the proper credentials before it can be used.

Use the navigation to the left to read about the available resources.

Example Usage

provider "kubernetes" {
  config_path    = "~/.kube/config"
  config_context = "my-context"
}

resource "kubernetes_namespace" "example" {
  metadata {
    name = "my-first-namespace"
  }
}

Kubernetes versions

Both backward and forward compatibility with Kubernetes API is mostly defined by the official K8S Go library (prior to 1.1 release) and client Go library which we ship with Terraform. Below are versions of the library bundled with given versions of Terraform.

Stacking with managed Kubernetes cluster resources

Terraform providers for various cloud providers feature resources to spin up managed Kubernetes clusters on services such as EKS, AKS and GKE. Such resources (or data-sources) will have attributes that expose the credentials needed for the Kubernetes provider to connect to these clusters.

To use these credentials with the Kubernetes provider, they can be interpolated into the respective attributes of the Kubernetes provider configuration block.

The most reliable way to configure the Kubernetes provider is to ensure that the cluster itself and the Kubernetes provider resources can be managed with separate apply operations. Data-sources can be used to convey values between the two stages as needed.

For specific usage examples, see the guides for AKS, EKS, and GKE.

Authentication

The Kubernetes provider can get its configuration in two ways:

  1. Explicitly by supplying attributes to the provider block. This includes:
  2. Implicitly through environment variables. This includes:

The provider always first tries to load a config file from a given location when config_path or config_paths (or their equivalent environment variables) are set. Depending on whether you have a current context set this may require config_context_auth_info and/or config_context_cluster and/or config_context.

For a full list of supported provider authentication arguments and their corresponding environment variables, see the argument reference below.

File config

The easiest way is to supply a path to your kubeconfig file using the config_path attribute or using the KUBE_CONFIG_PATH environment variable. A kubeconfig file may have multiple contexts. If config_context is not specified, the provider will use the default context.

provider "kubernetes" {
  config_path = "~/.kube/config"
}

The provider also supports multiple paths in the same way that kubectl does using the config_paths attribute or KUBE_CONFIG_PATHS environment variable.

provider "kubernetes" {
  config_paths = [
    "/path/to/config_a.yaml",
    "/path/to/config_b.yaml"
  ]
}

Credentials config

You can also configure the host, basic auth credentials, and client certificate authentication explicitly or through environment variables.

provider "kubernetes" {
  host = "https://cluster_endpoint:port"

  client_certificate     = file("~/.kube/client-cert.pem")
  client_key             = file("~/.kube/client-key.pem")
  cluster_ca_certificate = file("~/.kube/cluster-ca-cert.pem")
}

In-cluster Config

The provider uses the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables to detect when it is running inside a cluster, so in this case you do not need to specify any attributes in the provider block if you want to connect to the local kubernetes cluster.

If you want to connect to a different cluster than the one terraform is running inside, configure the provider as above.

Find more comprehensive in-cluster config example here.

Exec plugins

Some cloud providers have short-lived authentication tokens that can expire relatively quickly. To ensure the Kubernetes provider is receiving valid credentials, an exec-based plugin can be used to fetch a new token before each Terraform operation. For example, on EKS, the command eks get-token can be used:

provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca_cert)
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
    command     = "aws"
  }
}

Examples

For further reading, see these examples which demonstrate different approaches to keeping the cluster credentials up to date: AKS, EKS, and GKE.

Ignore Kubernetes annotations and labels

In certain cases, external systems can add and modify resources annotations and labels for their own purposes. However, Terraform will remove them since they are not presented in the code. It also might be hard to update code accordingly to stay tuned with the changes that come outside. In order to address this ignore_annotations and ignore_labels attributes were introduced on the provider level. They allow Terraform to ignore certain annotations and labels across all resources.

Both attributes support RegExp to match metadata objects more effectively.

Please keep in mind that all data sources remain unaffected, and the provider always returns all labels and annotations, despite the ignore_annotations and ignore_labels settings. The same applies to the pod and job definitions that fall under templates. To ignore certain annotations and/or labels on the template level, please use the ignore_changes feature of the lifecycle meta-argument.

Examples

The following example demonstrates how to ignore changes related to the kubectl.kubernetes.io/restartedAt annotation that were made in the upstream Kubernetes object:

resource "kubernetes_deployment_v1" "this" {
  // omit the resource config
  lifecycle {
    ignore_changes = [
      spec[0].template[0].metadata[0].annotations["kubectl.kubernetes.io/restartedAt"],
    ]
  }
}

The following example demonstrates how to ignore particular annotation keys:

provider "kubernetes" {
  ignore_annotations = [
    "cni\\.projectcalico\\.org\\/podIP",
    "cni\\.projectcalico\\.org\\/podIPs",
  ]
}

Next example demonstrates how to ignore AWS load balancer annotations:

provider "kubernetes" {
  ignore_annotations = [
    "^service\\.beta\\.kubernetes\\.io\\/aws-load-balancer.*",
  ]
}

Since dot ., forward slash /, and some other symbols have special meaning in RegExp, they should be escaped by adding a double backslash in front of them if you want to use them as they are.

Argument Reference

The following arguments are supported: