@aws-cdk_aws-glue-alpha.JobProps

interface JobProps ๐Ÿ”น

LanguageType name
.NETAmazon.CDK.AWS.Glue.Alpha.JobProps
Gogithub.com/aws/aws-cdk-go/awscdkgluealpha/v2#JobProps
Javasoftware.amazon.awscdk.services.glue.alpha.JobProps
Pythonaws_cdk.aws_glue_alpha.JobProps
TypeScript (source)@aws-cdk/aws-glue-alpha ยป JobProps

Construction properties for Job.

Example

declare const bucket: s3.Bucket;
new glue.Job(this, 'ScalaSparkEtlJob', {
  executable: glue.JobExecutable.scalaEtl({
    glueVersion: glue.GlueVersion.V4_0,
    script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),
    className: 'com.example.HelloWorld',
    extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],
  }),
  workerType: glue.WorkerType.G_8X,
  description: 'an example Scala ETL job',
});

Properties

NameTypeDescription
executable๐Ÿ”นJobExecutableThe job's executable properties.
connections?๐Ÿ”นIConnection[]The Connections used for this job.
continuousLogging?๐Ÿ”นContinuousLoggingPropsEnables continuous logging with the specified props.
defaultArguments?๐Ÿ”น{ [string]: string }The default arguments for this job, specified as name-value pairs.
description?๐Ÿ”นstringThe description of the job.
enableProfilingMetrics?๐Ÿ”นbooleanEnables the collection of metrics for job profiling.
executionClass?๐Ÿ”นExecutionClassThe ExecutionClass whether the job is run with a standard or flexible execution class.
jobName?๐Ÿ”นstringThe name of the job.
maxCapacity?๐Ÿ”นnumberThe number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
maxConcurrentRuns?๐Ÿ”นnumberThe maximum number of concurrent runs allowed for the job.
maxRetries?๐Ÿ”นnumberThe maximum number of times to retry this job after a job run fails.
notifyDelayAfter?๐Ÿ”นDurationThe number of minutes to wait after a job run starts, before sending a job run delay notification.
role?๐Ÿ”นIRoleThe IAM role assumed by Glue to run this job.
securityConfiguration?๐Ÿ”นISecurityConfigurationThe SecurityConfiguration to use for this job.
sparkUI?๐Ÿ”นSparkUIPropsEnables the Spark UI debugging and monitoring with the specified props.
tags?๐Ÿ”น{ [string]: string }The tags to add to the resources on which the job runs.
timeout?๐Ÿ”นDurationThe maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
workerCount?๐Ÿ”นnumberThe number of workers of a defined WorkerType that are allocated when a job runs.
workerType?๐Ÿ”นWorkerTypeThe type of predefined worker that is allocated when a job runs.

executable๐Ÿ”น

Type: JobExecutable

The job's executable properties.


connections?๐Ÿ”น

Type: IConnection[] (optional, default: [] - no connections are added to the job)

The Connections used for this job.

Connections are used to connect to other AWS Service or resources within a VPC.


continuousLogging?๐Ÿ”น

Type: ContinuousLoggingProps (optional, default: continuous logging is disabled.)

Enables continuous logging with the specified props.

See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html


defaultArguments?๐Ÿ”น

Type: { [string]: string } (optional, default: no arguments)

The default arguments for this job, specified as name-value pairs.

See also: [https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters)


description?๐Ÿ”น

Type: string (optional, default: no value)

The description of the job.


enableProfilingMetrics?๐Ÿ”น

Type: boolean (optional, default: no profiling metrics emitted.)

Enables the collection of metrics for job profiling.

See also: [ --enable-metrics at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html]( --enable-metrics at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)


executionClass?๐Ÿ”น

Type: ExecutionClass (optional, default: STANDARD)

The ExecutionClass whether the job is run with a standard or flexible execution class.

See also: https://docs.aws.amazon.com/glue/latest/dg/add-job.html


jobName?๐Ÿ”น

Type: string (optional, default: a name is automatically generated)

The name of the job.


maxCapacity?๐Ÿ”น

Type: number (optional, default: 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell)

The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.

Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.


maxConcurrentRuns?๐Ÿ”น

Type: number (optional, default: 1)

The maximum number of concurrent runs allowed for the job.

An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.


maxRetries?๐Ÿ”น

Type: number (optional, default: 0)

The maximum number of times to retry this job after a job run fails.


notifyDelayAfter?๐Ÿ”น

Type: Duration (optional, default: no delay notifications)

The number of minutes to wait after a job run starts, before sending a job run delay notification.


role?๐Ÿ”น

Type: IRole (optional, default: a role is automatically generated)

The IAM role assumed by Glue to run this job.

If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.

See also: https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html


securityConfiguration?๐Ÿ”น

Type: ISecurityConfiguration (optional, default: no security configuration.)

The SecurityConfiguration to use for this job.


sparkUI?๐Ÿ”น

Type: SparkUIProps (optional, default: Spark UI debugging and monitoring is disabled.)

Enables the Spark UI debugging and monitoring with the specified props.

See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html


tags?๐Ÿ”น

Type: { [string]: string } (optional, default: {} - no tags)

The tags to add to the resources on which the job runs.


timeout?๐Ÿ”น

Type: Duration (optional, default: cdk.Duration.hours(48))

The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.


workerCount?๐Ÿ”น

Type: number (optional, default: differs based on specific Glue version/worker type)

The number of workers of a defined WorkerType that are allocated when a job runs.


workerType?๐Ÿ”น

Type: WorkerType (optional, default: differs based on specific Glue version)

The type of predefined worker that is allocated when a job runs.