Resource: aws_glue_job

Provides a Glue Job resource.

Example Usage

Python Job

resource "aws_glue_job" "example" {
  name     = "example"
  role_arn = aws_iam_role.example.arn

  command {
    script_location = "s3://${aws_s3_bucket.example.bucket}/example.py"
  }
}

Ray Job

resource "aws_glue_job" "example" {
  name         = "example"
  role_arn     = aws_iam_role.example.arn
  glue_version = "4.0"
  worker_type  = "Z.2X"

  command {
    name            = "glueray"
    python_version  = "3.9"
    runtime         = "Ray2.4"
    script_location = "s3://${aws_s3_bucket.example.bucket}/example.py"
  }
}

Scala Job

resource "aws_glue_job" "example" {
  name     = "example"
  role_arn = aws_iam_role.example.arn

  command {
    script_location = "s3://${aws_s3_bucket.example.bucket}/example.scala"
  }

  default_arguments = {
    "--job-language" = "scala"
  }
}

Streaming Job

resource "aws_glue_job" "example" {
  name     = "example streaming job"
  role_arn = aws_iam_role.example.arn

  command {
    name            = "gluestreaming"
    script_location = "s3://${aws_s3_bucket.example.bucket}/example.script"
  }
}

Enabling CloudWatch Logs and Metrics

resource "aws_cloudwatch_log_group" "example" {
  name              = "example"
  retention_in_days = 14
}

resource "aws_glue_job" "example" {
  # ... other configuration ...

  default_arguments = {
    # ... potentially other arguments ...
    "--continuous-log-logGroup"          = aws_cloudwatch_log_group.example.name
    "--enable-continuous-cloudwatch-log" = "true"
    "--enable-continuous-log-filter"     = "true"
    "--enable-metrics"                   = ""
  }
}

Argument Reference

This resource supports the following arguments:

command Argument Reference

execution_property Argument Reference

notification_property Argument Reference

Attribute Reference

This resource exports the following attributes in addition to the arguments above:

Import

In Terraform v1.5.0 and later, use an import block to import Glue Jobs using name. For example:

import {
  to = aws_glue_job.MyJob
  id = "MyJob"
}

Using terraform import, import Glue Jobs using name. For example:

% terraform import aws_glue_job.MyJob MyJob