@aws-cdk_aws-glue-alpha.PythonSparkJobExecutableProps

interface PythonSparkJobExecutableProps ๐Ÿ”น

LanguageType name
.NETAmazon.CDK.AWS.Glue.Alpha.PythonSparkJobExecutableProps
Gogithub.com/aws/aws-cdk-go/awscdkgluealpha/v2#PythonSparkJobExecutableProps
Javasoftware.amazon.awscdk.services.glue.alpha.PythonSparkJobExecutableProps
Pythonaws_cdk.aws_glue_alpha.PythonSparkJobExecutableProps
TypeScript (source)@aws-cdk/aws-glue-alpha ยป PythonSparkJobExecutableProps

Props for creating a Python Spark (ETL or Streaming) job executable.

Example

new glue.Job(this, 'PythonSparkStreamingJob', {
  executable: glue.JobExecutable.pythonStreaming({
    glueVersion: glue.GlueVersion.V4_0,
    pythonVersion: glue.PythonVersion.THREE,
    script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),
  }),
  description: 'an example Python Streaming job',
});

Properties

NameTypeDescription
glueVersion๐Ÿ”นGlueVersionGlue version.
pythonVersion๐Ÿ”นPythonVersionThe Python version to use.
script๐Ÿ”นCodeThe script that executes a job.
extraFiles?๐Ÿ”นCode[]Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
extraJars?๐Ÿ”นCode[]Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported.
extraJarsFirst?๐Ÿ”นbooleanSetting this value to true prioritizes the customer's extra JAR files in the classpath.
extraPythonFiles?๐Ÿ”นCode[]Additional Python files that AWS Glue adds to the Python path before executing your script.
runtime?๐Ÿ”นRuntimeRuntime.

glueVersion๐Ÿ”น

Type: GlueVersion

Glue version.

See also: https://docs.aws.amazon.com/glue/latest/dg/release-notes.html


pythonVersion๐Ÿ”น

Type: PythonVersion

The Python version to use.


script๐Ÿ”น

Type: Code

The script that executes a job.


extraFiles?๐Ÿ”น

Type: Code[] (optional, default: [] - no extra files are copied to the working directory)

Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.

Only individual files are supported, directories are not supported.

See also: [ --extra-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html]( --extra-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)


extraJars?๐Ÿ”น

Type: Code[] (optional, default: [] - no extra jars are added to the classpath)

Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported.

See also: [ --extra-jars in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html]( --extra-jars in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)


extraJarsFirst?๐Ÿ”น

Type: boolean (optional, default: false - priority is not given to user-provided jars)

Setting this value to true prioritizes the customer's extra JAR files in the classpath.

See also: [ --user-jars-first in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html]( --user-jars-first in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)


extraPythonFiles?๐Ÿ”น

Type: Code[] (optional, default: no extra python files and argument is not set)

Additional Python files that AWS Glue adds to the Python path before executing your script.

Only individual files are supported, directories are not supported.

See also: [ --extra-py-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html]( --extra-py-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)


runtime?๐Ÿ”น

Type: Runtime (optional)

Runtime.

It is required for Ray jobs.