databricks_spark_version Data Source

Gets Databricks Runtime (DBR) version that could be used for spark_version parameter in databricks_cluster and other resources that fits search criteria, like specific Spark or Scala version, ML or Genomics runtime, etc., similar to executing databricks clusters spark-versions, and filters it to return the latest version that matches criteria. Often used along databricks_node_type data source.

Example Usage

data "databricks_node_type" "with_gpu" {
  local_disk  = true
  min_cores   = 16
  gb_per_core = 1
  min_gpus    = 1
}

data "databricks_spark_version" "gpu_ml" {
  gpu = true
  ml  = true
}

resource "databricks_cluster" "research" {
  cluster_name            = "Research Cluster"
  spark_version           = data.databricks_spark_version.gpu_ml.id
  node_type_id            = data.databricks_node_type.with_gpu.id
  autotermination_minutes = 20
  autoscale {
    min_workers = 1
    max_workers = 50
  }
}

Argument Reference

Data source allows you to pick groups by the following attributes:

Attribute Reference

Data source exposes the following attributes:

The following resources are used in the same context: