Represents a Data Fusion instance.
To get more information about Instance, see:
resource "google_data_fusion_instance" "basic_instance" {
name = "my-instance"
region = "us-central1"
type = "BASIC"
}
resource "google_data_fusion_instance" "extended_instance" {
name = "my-instance"
description = "My Data Fusion instance"
display_name = "My Data Fusion instance"
region = "us-central1"
type = "BASIC"
enable_stackdriver_logging = true
enable_stackdriver_monitoring = true
private_instance = true
dataproc_service_account = data.google_app_engine_default_service_account.default.email
labels = {
example_key = "example_value"
}
network_config {
network = "default"
ip_allocation = "${google_compute_global_address.private_ip_alloc.address}/${google_compute_global_address.private_ip_alloc.prefix_length}"
}
accelerators {
accelerator_type = "CDC"
state = "ENABLED"
}
}
data "google_app_engine_default_service_account" "default" {
}
resource "google_compute_network" "network" {
name = "datafusion-full-network"
}
resource "google_compute_global_address" "private_ip_alloc" {
name = "datafusion-ip-alloc"
address_type = "INTERNAL"
purpose = "VPC_PEERING"
prefix_length = 22
network = google_compute_network.network.id
}
resource "google_data_fusion_instance" "cmek" {
name = "my-instance"
region = "us-central1"
type = "BASIC"
crypto_key_config {
key_reference = google_kms_crypto_key.crypto_key.id
}
depends_on = [google_kms_crypto_key_iam_member.crypto_key_member]
}
resource "google_kms_crypto_key" "crypto_key" {
name = "my-instance"
key_ring = google_kms_key_ring.key_ring.id
}
resource "google_kms_key_ring" "key_ring" {
name = "my-instance"
location = "us-central1"
}
resource "google_kms_crypto_key_iam_member" "crypto_key_member" {
crypto_key_id = google_kms_crypto_key.crypto_key.id
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-datafusion.iam.gserviceaccount.com"
}
data "google_project" "project" {}
resource "google_data_fusion_instance" "enterprise_instance" {
name = "my-instance"
region = "us-central1"
type = "ENTERPRISE"
enable_rbac = true
}
resource "google_data_fusion_instance" "event" {
name = "my-instance"
region = "us-central1"
type = "BASIC"
event_publish_config {
enabled = true
topic = google_pubsub_topic.event.id
}
}
resource "google_pubsub_topic" "event" {
name = "my-instance"
}
resource "google_data_fusion_instance" "zone" {
name = "my-instance"
region = "us-central1"
zone = "us-central1-a"
type = "DEVELOPER"
}
The following arguments are supported:
name
-
(Required)
The ID of the instance or a fully qualified identifier for the instance.
type
-
(Required)
Represents the type of Data Fusion instance. Each type is configured with
the default settings for processing and memory.
BASIC
, ENTERPRISE
, DEVELOPER
.description
-
(Optional)
An optional description of the instance.
enable_stackdriver_logging
-
(Optional)
Option to enable Stackdriver Logging.
enable_stackdriver_monitoring
-
(Optional)
Option to enable Stackdriver Monitoring.
enable_rbac
-
(Optional)
Option to enable granular role-based access control.
labels
-
(Optional)
The resource labels for instance to use to annotate any related underlying resources,
such as Compute Engine VMs.
Note: This field is non-authoritative, and will only manage the labels present in your configuration.
Please refer to the field effective_labels
for all of the labels present on the resource.
options
-
(Optional)
Map of additional options used to configure the behavior of Data Fusion instance.
version
-
(Optional)
Current version of the Data Fusion.
private_instance
-
(Optional)
Specifies whether the Data Fusion instance should be private. If set to
true, all Data Fusion nodes will have private IP addresses and will not be
able to access the public internet.
dataproc_service_account
-
(Optional)
User-managed service account to set on Dataproc when Cloud Data Fusion creates Dataproc to run data processing pipelines.
network_config
-
(Optional)
Network configuration options. These are required when a private Data Fusion instance is to be created.
Structure is documented below.
zone
-
(Optional)
Name of the zone in which the Data Fusion instance will be created. Only DEVELOPER instances use this field.
display_name
-
(Optional)
Display name for an instance.
crypto_key_config
-
(Optional)
The crypto key configuration. This field is used by the Customer-Managed Encryption Keys (CMEK) feature.
Structure is documented below.
event_publish_config
-
(Optional)
Option to enable and pass metadata for event publishing.
Structure is documented below.
accelerators
-
(Optional)
List of accelerators enabled for this CDF instance.
If accelerators are enabled it is possible a permadiff will be created with the Options field.
Users will need to either manually update their state file to include these diffed options, or include the field in a lifecycle ignore changes block.
Structure is documented below.
region
-
(Optional)
The region of the Data Fusion instance.
project
- (Optional) The ID of the project in which the resource belongs.
If it is not provided, the provider project is used.
The network_config
block supports:
ip_allocation
-
(Required)
The IP range in CIDR notation to use for the managed Data Fusion instance
nodes. This range must not overlap with any other ranges used in the Data Fusion instance network.
network
-
(Required)
Name of the network in the project with which the tenant project
will be peered for executing pipelines. In case of shared VPC where the network resides in another host
project the network should specified in the form of projects/{host-project-id}/global/networks/{network}
The crypto_key_config
block supports:
key_reference
-
(Required)
The name of the key which is used to encrypt/decrypt customer data. For key in Cloud KMS, the key should be in the format of projects//locations//keyRings//cryptoKeys/.The event_publish_config
block supports:
enabled
-
(Required)
Option to enable Event Publishing.
topic
-
(Required)
The resource name of the Pub/Sub topic. Format: projects/{projectId}/topics/{topic_id}
The accelerators
block supports:
accelerator_type
-
(Required)
The type of an accelator for a CDF instance.
Possible values are: CDC
, HEALTHCARE
, CCAI_INSIGHTS
.
state
-
(Required)
The type of an accelator for a CDF instance.
Possible values are: ENABLED
, DISABLED
.
In addition to the arguments listed above, the following computed attributes are exported:
id
- an identifier for the resource with format projects/{{project}}/locations/{{region}}/instances/{{name}}
create_time
-
The time the instance was created in RFC3339 UTC "Zulu" format, accurate to nanoseconds.
update_time
-
The time the instance was last updated in RFC3339 UTC "Zulu" format, accurate to nanoseconds.
state
-
The current state of this Data Fusion instance.
state_message
-
Additional information about the current state of this Data Fusion instance if available.
service_endpoint
-
Endpoint on which the Data Fusion UI and REST APIs are accessible.
service_account
-
(Beta, Deprecated)
Service account which will be used to access resources in the customer project.
~> Warning: service_account
is deprecated and will be removed in a future major release. Instead, use tenant_project_id
to extract the tenant project ID.
tenant_project_id
-
The name of the tenant project.
gcs_bucket
-
Cloud Storage bucket generated by Data Fusion in the customer project.
api_endpoint
-
Endpoint on which the REST APIs is accessible.
p4_service_account
-
P4 service account for the customer project.
terraform_labels
-
The combination of labels configured directly on the resource
and default labels configured on the provider.
effective_labels
-
All of labels (key/value pairs) present on the resource in GCP, including the labels configured through Terraform, other clients and services.
This resource provides the following Timeouts configuration options:
create
- Default is 90 minutes.update
- Default is 25 minutes.delete
- Default is 50 minutes.Instance can be imported using any of these accepted formats:
projects/{{project}}/locations/{{region}}/instances/{{name}}
{{project}}/{{region}}/{{name}}
{{region}}/{{name}}
{{name}}
In Terraform v1.5.0 and later, use an import
block to import Instance using one of the formats above. For example:
import {
id = "projects/{{project}}/locations/{{region}}/instances/{{name}}"
to = google_data_fusion_instance.default
}
When using the terraform import
command, Instance can be imported using one of the formats above. For example:
$ terraform import google_data_fusion_instance.default projects/{{project}}/locations/{{region}}/instances/{{name}}
$ terraform import google_data_fusion_instance.default {{project}}/{{region}}/{{name}}
$ terraform import google_data_fusion_instance.default {{region}}/{{name}}
$ terraform import google_data_fusion_instance.default {{name}}
This resource supports User Project Overrides.