aws-cdk-lib.aws_stepfunctions_tasks.Classification

class Classification

LanguageType name
.NETAmazon.CDK.AWS.StepFunctions.Tasks.Classification
Gogithub.com/aws/aws-cdk-go/awscdk/v2/awsstepfunctionstasks#Classification
Javasoftware.amazon.awscdk.services.stepfunctions.tasks.Classification
Pythonaws_cdk.aws_stepfunctions_tasks.Classification
TypeScript (source)aws-cdk-lib » aws_stepfunctions_tasks » Classification

The classification within a EMR Containers application configuration.

Class can be extended to add other classifications. For example, new Classification('xxx-yyy');

Example

new tasks.EmrContainersStartJobRun(this, 'EMR Containers Start Job Run', {
  virtualCluster: tasks.VirtualClusterInput.fromVirtualClusterId('de92jdei2910fwedz'),
  releaseLabel: tasks.ReleaseLabel.EMR_6_2_0,
  jobName: 'EMR-Containers-Job',
  jobDriver: {
    sparkSubmitJobDriver: {
      entryPoint: sfn.TaskInput.fromText('local:///usr/lib/spark/examples/src/main/python/pi.py'),
    },
  },
  applicationConfig: [{
    classification: tasks.Classification.SPARK_DEFAULTS,
    properties: {
      'spark.executor.instances': '1',
      'spark.executor.memory': '512M',
    },
  }],
});

Initializer

new Classification(classificationStatement: string)

Parameters

  • classificationStatement string — A literal string in case a new EMR classification is released, if not already defined.

Creates a new Classification.

Properties

NameTypeDescription
classificationStatementstringA literal string in case a new EMR classification is released, if not already defined.
static SPARKClassificationSets the maximizeResourceAllocation property to true or false.
static SPARK_DEFAULTSClassificationSets values in the spark-defaults.conf file.
static SPARK_ENVClassificationSets values in the spark-env.sh file.
static SPARK_HIVE_SITEClassificationSets values in the hive-site.xml for Spark.
static SPARK_LOG4JClassificationSets values in the log4j.properties file.
static SPARK_METRICSClassificationSets values in the metrics.properties file.

classificationStatement

Type: string

A literal string in case a new EMR classification is released, if not already defined.


static SPARK

Type: Classification

Sets the maximizeResourceAllocation property to true or false.

When true, Amazon EMR automatically configures spark-defaults properties based on cluster hardware configuration.

For more info:

See also: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-configure.html#emr-spark-maximizeresourceallocation


static SPARK_DEFAULTS

Type: Classification

Sets values in the spark-defaults.conf file.

For more info:

See also: https://spark.apache.org/docs/latest/configuration.html


static SPARK_ENV

Type: Classification

Sets values in the spark-env.sh file.

For more info:

See also: https://spark.apache.org/docs/latest/configuration.html#environment-variables


static SPARK_HIVE_SITE

Type: Classification

Sets values in the hive-site.xml for Spark.


static SPARK_LOG4J

Type: Classification

Sets values in the log4j.properties file.

For more settings and info:

See also: https://github.com/apache/spark/blob/master/conf/log4j.properties.template


static SPARK_METRICS

Type: Classification

Sets values in the metrics.properties file.

For more settings and info:

See also: https://github.com/apache/spark/blob/master/conf/metrics.properties.template