Gets Databricks Runtime (DBR) version that could be used for spark_version
parameter in databricks_cluster and other resources that fits search criteria, like specific Spark or Scala version, ML or Genomics runtime, etc., similar to executing databricks clusters spark-versions
, and filters it to return the latest version that matches criteria. Often used along databricks_node_type data source.
data "databricks_node_type" "with_gpu" {
local_disk = true
min_cores = 16
gb_per_core = 1
min_gpus = 1
}
data "databricks_spark_version" "gpu_ml" {
gpu = true
ml = true
}
resource "databricks_cluster" "research" {
cluster_name = "Research Cluster"
spark_version = data.databricks_spark_version.gpu_ml.id
node_type_id = data.databricks_node_type.with_gpu.id
autotermination_minutes = 20
autoscale {
min_workers = 1
max_workers = 50
}
}
Data source allows you to pick groups by the following attributes:
latest
- (boolean, optional) if we should return only the latest version if there is more than one result. Default to true
. If set to false
and multiple versions are matching, throws an error.long_term_support
- (boolean, optional) if we should limit the search only to LTS (long term support) & ESR (extended support) versions. Default to false
.ml
- (boolean, optional) if we should limit the search only to ML runtimes. Default to false
.genomics
- (boolean, optional) if we should limit the search only to Genomics (HLS) runtimes. Default to false
.gpu
- (boolean, optional) if we should limit the search only to runtimes that support GPUs. Default to false
.beta
- (boolean, optional) if we should limit the search only to runtimes that are in Beta stage. Default to false
.scala
- (string, optional) if we should limit the search only to runtimes that are based on specific Scala version. Default to 2.12
.spark_version
- (string, optional) if we should limit the search only to runtimes that are based on specific Spark version. Default to empty string. It could be specified as 3
, or 3.0
, or full version, like, 3.0.1
.photon
- (boolean, optional) if we should limit the search only to Photon runtimes. Default to false
. Deprecated with DBR 14.0 release. Specify runtime_engine=\"PHOTON\"
in the cluster configuration instead!graviton
- (boolean, optional) if we should limit the search only to runtimes supporting AWS Graviton CPUs. Default to false
. _Deprecated with DBR 14.0 release. DBR version compiled for Graviton will be automatically installed when nodes with Graviton CPUs are specified in the cluster configuration._Data source exposes the following attributes:
id
- Databricks Runtime version, that can be used as spark_version
field in databricks_job, databricks_cluster, or databricks_instance_pool.The following resources are used in the same context: