@aws-cdk_aws-glue-alpha.Job

class Job (construct) ๐Ÿ”น

LanguageType name
.NETAmazon.CDK.AWS.Glue.Alpha.Job
Gogithub.com/aws/aws-cdk-go/awscdkgluealpha/v2#Job
Javasoftware.amazon.awscdk.services.glue.alpha.Job
Pythonaws_cdk.aws_glue_alpha.Job
TypeScript (source)@aws-cdk/aws-glue-alpha ยป Job

Implements IConstruct, IDependable, IResource, IJob, IGrantable

A Glue Job.

Example

declare const bucket: s3.Bucket;
new glue.Job(this, 'ScalaSparkEtlJob', {
  executable: glue.JobExecutable.scalaEtl({
    glueVersion: glue.GlueVersion.V4_0,
    script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),
    className: 'com.example.HelloWorld',
    extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],
  }),
  workerType: glue.WorkerType.G_8X,
  description: 'an example Scala ETL job',
});

Initializer

new Job(scope: Construct, id: string, props: JobProps)

Parameters

  • scope Construct
  • id string
  • props JobProps

Construct Props

NameTypeDescription
executable๐Ÿ”นJobExecutableThe job's executable properties.
connections?๐Ÿ”นIConnection[]The Connections used for this job.
continuousLogging?๐Ÿ”นContinuousLoggingPropsEnables continuous logging with the specified props.
defaultArguments?๐Ÿ”น{ [string]: string }The default arguments for this job, specified as name-value pairs.
description?๐Ÿ”นstringThe description of the job.
enableProfilingMetrics?๐Ÿ”นbooleanEnables the collection of metrics for job profiling.
executionClass?๐Ÿ”นExecutionClassThe ExecutionClass whether the job is run with a standard or flexible execution class.
jobName?๐Ÿ”นstringThe name of the job.
maxCapacity?๐Ÿ”นnumberThe number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
maxConcurrentRuns?๐Ÿ”นnumberThe maximum number of concurrent runs allowed for the job.
maxRetries?๐Ÿ”นnumberThe maximum number of times to retry this job after a job run fails.
notifyDelayAfter?๐Ÿ”นDurationThe number of minutes to wait after a job run starts, before sending a job run delay notification.
role?๐Ÿ”นIRoleThe IAM role assumed by Glue to run this job.
securityConfiguration?๐Ÿ”นISecurityConfigurationThe SecurityConfiguration to use for this job.
sparkUI?๐Ÿ”นSparkUIPropsEnables the Spark UI debugging and monitoring with the specified props.
tags?๐Ÿ”น{ [string]: string }The tags to add to the resources on which the job runs.
timeout?๐Ÿ”นDurationThe maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
workerCount?๐Ÿ”นnumberThe number of workers of a defined WorkerType that are allocated when a job runs.
workerType?๐Ÿ”นWorkerTypeThe type of predefined worker that is allocated when a job runs.

executable๐Ÿ”น

Type: JobExecutable

The job's executable properties.


connections?๐Ÿ”น

Type: IConnection[] (optional, default: [] - no connections are added to the job)

The Connections used for this job.

Connections are used to connect to other AWS Service or resources within a VPC.


continuousLogging?๐Ÿ”น

Type: ContinuousLoggingProps (optional, default: continuous logging is disabled.)

Enables continuous logging with the specified props.

See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html


defaultArguments?๐Ÿ”น

Type: { [string]: string } (optional, default: no arguments)

The default arguments for this job, specified as name-value pairs.

See also: [https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters)


description?๐Ÿ”น

Type: string (optional, default: no value)

The description of the job.


enableProfilingMetrics?๐Ÿ”น

Type: boolean (optional, default: no profiling metrics emitted.)

Enables the collection of metrics for job profiling.

See also: [ --enable-metrics at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html]( --enable-metrics at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)


executionClass?๐Ÿ”น

Type: ExecutionClass (optional, default: STANDARD)

The ExecutionClass whether the job is run with a standard or flexible execution class.

See also: https://docs.aws.amazon.com/glue/latest/dg/add-job.html


jobName?๐Ÿ”น

Type: string (optional, default: a name is automatically generated)

The name of the job.


maxCapacity?๐Ÿ”น

Type: number (optional, default: 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell)

The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.

Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.


maxConcurrentRuns?๐Ÿ”น

Type: number (optional, default: 1)

The maximum number of concurrent runs allowed for the job.

An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.


maxRetries?๐Ÿ”น

Type: number (optional, default: 0)

The maximum number of times to retry this job after a job run fails.


notifyDelayAfter?๐Ÿ”น

Type: Duration (optional, default: no delay notifications)

The number of minutes to wait after a job run starts, before sending a job run delay notification.


role?๐Ÿ”น

Type: IRole (optional, default: a role is automatically generated)

The IAM role assumed by Glue to run this job.

If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.

See also: https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html


securityConfiguration?๐Ÿ”น

Type: ISecurityConfiguration (optional, default: no security configuration.)

The SecurityConfiguration to use for this job.


sparkUI?๐Ÿ”น

Type: SparkUIProps (optional, default: Spark UI debugging and monitoring is disabled.)

Enables the Spark UI debugging and monitoring with the specified props.

See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html


tags?๐Ÿ”น

Type: { [string]: string } (optional, default: {} - no tags)

The tags to add to the resources on which the job runs.


timeout?๐Ÿ”น

Type: Duration (optional, default: cdk.Duration.hours(48))

The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.


workerCount?๐Ÿ”น

Type: number (optional, default: differs based on specific Glue version/worker type)

The number of workers of a defined WorkerType that are allocated when a job runs.


workerType?๐Ÿ”น

Type: WorkerType (optional, default: differs based on specific Glue version)

The type of predefined worker that is allocated when a job runs.

Properties

NameTypeDescription
env๐Ÿ”นResourceEnvironmentThe environment this resource belongs to.
grantPrincipal๐Ÿ”นIPrincipalThe principal this Glue Job is running as.
jobArn๐Ÿ”นstringThe ARN of the job.
jobName๐Ÿ”นstringThe name of the job.
node๐Ÿ”นNodeThe tree node.
role๐Ÿ”นIRoleThe IAM role Glue assumes to run this job.
stack๐Ÿ”นStackThe stack in which this resource is defined.
sparkUILoggingLocation?๐Ÿ”นSparkUILoggingLocationThe Spark UI logs location if Spark UI monitoring and debugging is enabled.

env๐Ÿ”น

Type: ResourceEnvironment

The environment this resource belongs to.

For resources that are created and managed by the CDK (generally, those created by creating new class instances like Role, Bucket, etc.), this is always the same as the environment of the stack they belong to; however, for imported resources (those obtained from static methods like fromRoleArn, fromBucketName, etc.), that might be different than the stack they were imported into.


grantPrincipal๐Ÿ”น

Type: IPrincipal

The principal this Glue Job is running as.


jobArn๐Ÿ”น

Type: string

The ARN of the job.


jobName๐Ÿ”น

Type: string

The name of the job.


node๐Ÿ”น

Type: Node

The tree node.


role๐Ÿ”น

Type: IRole

The IAM role Glue assumes to run this job.


stack๐Ÿ”น

Type: Stack

The stack in which this resource is defined.


sparkUILoggingLocation?๐Ÿ”น

Type: SparkUILoggingLocation (optional)

The Spark UI logs location if Spark UI monitoring and debugging is enabled.

See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Methods

NameDescription
applyRemovalPolicy(policy)๐Ÿ”นApply the given removal policy to this resource.
metric(metricName, type, props?)๐Ÿ”นCreate a CloudWatch metric.
metricFailure(props?)๐Ÿ”นReturn a CloudWatch Metric indicating job failure.
metricSuccess(props?)๐Ÿ”นReturn a CloudWatch Metric indicating job success.
metricTimeout(props?)๐Ÿ”นReturn a CloudWatch Metric indicating job timeout.
onEvent(id, options?)๐Ÿ”นCreate a CloudWatch Event Rule for this Glue Job when it's in a given state.
onFailure(id, options?)๐Ÿ”นReturn a CloudWatch Event Rule matching FAILED state.
onStateChange(id, jobState, options?)๐Ÿ”นCreate a CloudWatch Event Rule for the transition into the input jobState.
onSuccess(id, options?)๐Ÿ”นCreate a CloudWatch Event Rule matching JobState.SUCCEEDED.
onTimeout(id, options?)๐Ÿ”นReturn a CloudWatch Event Rule matching TIMEOUT state.
toString()๐Ÿ”นReturns a string representation of this construct.
static fromJobAttributes(scope, id, attrs)๐Ÿ”นCreates a Glue Job.

applyRemovalPolicy(policy)๐Ÿ”น

public applyRemovalPolicy(policy: RemovalPolicy): void

Parameters

  • policy RemovalPolicy

Apply the given removal policy to this resource.

The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced.

The resource can be deleted (RemovalPolicy.DESTROY), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN).


metric(metricName, type, props?)๐Ÿ”น

public metric(metricName: string, type: MetricType, props?: MetricOptions): Metric

Parameters

  • metricName string โ€” name of the metric typically prefixed with glue.driver., glue.<executorId>. or glue.ALL..
  • type MetricType โ€” the metric type.
  • props MetricOptions โ€” metric options.

Returns

  • Metric

Create a CloudWatch metric.

See also: https://docs.aws.amazon.com/glue/latest/dg/monitoring-awsglue-with-cloudwatch-metrics.html


metricFailure(props?)๐Ÿ”น

public metricFailure(props?: MetricOptions): Metric

Parameters

  • props MetricOptions

Returns

  • Metric

Return a CloudWatch Metric indicating job failure.

This metric is based on the Rule returned by no-args onFailure() call.


metricSuccess(props?)๐Ÿ”น

public metricSuccess(props?: MetricOptions): Metric

Parameters

  • props MetricOptions

Returns

  • Metric

Return a CloudWatch Metric indicating job success.

This metric is based on the Rule returned by no-args onSuccess() call.


metricTimeout(props?)๐Ÿ”น

public metricTimeout(props?: MetricOptions): Metric

Parameters

  • props MetricOptions

Returns

  • Metric

Return a CloudWatch Metric indicating job timeout.

This metric is based on the Rule returned by no-args onTimeout() call.


onEvent(id, options?)๐Ÿ”น

public onEvent(id: string, options?: OnEventOptions): Rule

Parameters

  • id string โ€” construct id.
  • options OnEventOptions โ€” event options.

Returns

  • Rule

Create a CloudWatch Event Rule for this Glue Job when it's in a given state.

See also: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types


onFailure(id, options?)๐Ÿ”น

public onFailure(id: string, options?: OnEventOptions): Rule

Parameters

  • id string โ€” construct id.
  • options OnEventOptions โ€” optional event options.

Returns

  • Rule

Return a CloudWatch Event Rule matching FAILED state.


onStateChange(id, jobState, options?)๐Ÿ”น

public onStateChange(id: string, jobState: JobState, options?: OnEventOptions): Rule

Parameters

  • id string โ€” construct id.
  • jobState JobState โ€” the job state.
  • options OnEventOptions โ€” optional event options.

Returns

  • Rule

Create a CloudWatch Event Rule for the transition into the input jobState.


onSuccess(id, options?)๐Ÿ”น

public onSuccess(id: string, options?: OnEventOptions): Rule

Parameters

  • id string โ€” construct id.
  • options OnEventOptions โ€” optional event options.

Returns

  • Rule

Create a CloudWatch Event Rule matching JobState.SUCCEEDED.


onTimeout(id, options?)๐Ÿ”น

public onTimeout(id: string, options?: OnEventOptions): Rule

Parameters

  • id string โ€” construct id.
  • options OnEventOptions โ€” optional event options.

Returns

  • Rule

Return a CloudWatch Event Rule matching TIMEOUT state.


toString()๐Ÿ”น

public toString(): string

Returns

  • string

Returns a string representation of this construct.


static fromJobAttributes(scope, id, attrs)๐Ÿ”น

public static fromJobAttributes(scope: Construct, id: string, attrs: JobAttributes): IJob

Parameters

  • scope Construct โ€” The scope creating construct (usually this).
  • id string โ€” The construct's id.
  • attrs JobAttributes โ€” Import attributes.

Returns

  • IJob

Creates a Glue Job.