This is a resource that lets you manage relatively small files on Databricks File System (DBFS). The best use cases are libraries for databricks_cluster or databricks_job. You can also use databricks_dbfs_file and databricks_dbfs_file_paths data sources.
In order to manage a file on Databricks File System with Terraform, you must specify the source
attribute containing the full path to the file on the local filesystem.
resource "databricks_dbfs_file" "this" {
source = "${path.module}/main.tf"
path = "/tmp/main.tf"
}
Alternatively, you can create DBFS files with custom content, using filesystem functions.
resource "databricks_dbfs_file" "this" {
content_base64 = base64encode(<<-EOT
Hello, world!
Module is ${abspath(path.module)}
EOT
)
path = "/tmp/this.txt"
}
Install databricks_library on all databricks_clusters:
data "databricks_clusters" "all" {
}
resource "databricks_dbfs_file" "app" {
source = "${path.module}/baz.whl"
path = "/FileStore/baz.whl"
}
resource "databricks_library" "app" {
for_each = data.databricks_clusters.all.ids
cluster_id = each.key
whl = databricks_dbfs_file.app.dbfs_path
}
The following arguments are supported:
source
- The full absolute path to the file. Conflicts with content_base64
.content_base64
- Encoded file contents. Conflicts with source
. Use of content_base64
is discouraged, as it's increasing memory footprint of Terraform state and should only be used in exceptional circumstances, like creating a data pipeline configuration file.path
- (Required) The path of the file in which you wish to save.In addition to all arguments above, the following attributes are exported:
id
- Same as path
.file_size
- The file size of the file that is being tracked by this resource in bytes.dbfs_path
- Path, but with dbfs:
prefix.The resource dbfs file can be imported using the path of the file:
terraform import databricks_dbfs_file.this <path>
The following resources are often used in the same context:
dbfs:/mnt/name
.