Resource: aws_kinesis_analytics_application

Provides a Kinesis Analytics Application resource. Kinesis Analytics is a managed service that allows processing and analyzing streaming data using standard SQL.

For more details, see the Amazon Kinesis Analytics Documentation.

Example Usage

Kinesis Stream Input

resource "aws_kinesis_stream" "test_stream" {
  name        = "terraform-kinesis-test"
  shard_count = 1
}

resource "aws_kinesis_analytics_application" "test_application" {
  name = "kinesis-analytics-application-test"

  inputs {
    name_prefix = "test_prefix"

    kinesis_stream {
      resource_arn = aws_kinesis_stream.test_stream.arn
      role_arn     = aws_iam_role.test.arn
    }

    parallelism {
      count = 1
    }

    schema {
      record_columns {
        mapping  = "$.test"
        name     = "test"
        sql_type = "VARCHAR(8)"
      }

      record_encoding = "UTF-8"

      record_format {
        mapping_parameters {
          json {
            record_row_path = "$"
          }
        }
      }
    }
  }
}

Starting An Application

resource "aws_cloudwatch_log_group" "example" {
  name = "analytics"
}

resource "aws_cloudwatch_log_stream" "example" {
  name           = "example-kinesis-application"
  log_group_name = aws_cloudwatch_log_group.example.name
}

resource "aws_kinesis_stream" "example" {
  name        = "example-kinesis-stream"
  shard_count = 1
}

resource "aws_kinesis_firehose_delivery_stream" "example" {
  name        = "example-kinesis-delivery-stream"
  destination = "extended_s3"

  extended_s3_configuration {
    bucket_arn = aws_s3_bucket.example.arn
    role_arn   = aws_iam_role.example.arn
  }
}

resource "aws_kinesis_analytics_application" "test" {
  name = "example-application"

  cloudwatch_logging_options {
    log_stream_arn = aws_cloudwatch_log_stream.example.arn
    role_arn       = aws_iam_role.example.arn
  }

  inputs {
    name_prefix = "example_prefix"

    schema {
      record_columns {
        name     = "COLUMN_1"
        sql_type = "INTEGER"
      }

      record_format {
        mapping_parameters {
          csv {
            record_column_delimiter = ","
            record_row_delimiter    = "|"
          }
        }
      }
    }

    kinesis_stream {
      resource_arn = aws_kinesis_stream.example.arn
      role_arn     = aws_iam_role.example.arn
    }

    starting_position_configuration {
      starting_position = "NOW"
    }
  }

  outputs {
    name = "OUTPUT_1"

    schema {
      record_format_type = "CSV"
    }

    kinesis_firehose {
      resource_arn = aws_kinesis_firehose_delivery_stream.example.arn
      role_arn     = aws_iam_role.example.arn
    }
  }

  start_application = true
}

Argument Reference

This resource supports the following arguments:

CloudWatch Logging Options

Configure a CloudWatch Log Stream to monitor application errors.

The cloudwatch_logging_options block supports the following:

Inputs

Configure an Input for the Kinesis Analytics Application. You can only have 1 Input configured.

The inputs block supports the following:

Outputs

Configure Output destinations for the Kinesis Analytics Application. You can have a maximum of 3 destinations configured.

The outputs block supports the following:

Reference Data Sources

Add a Reference Data Source to the Kinesis Analytics Application. You can only have 1 Reference Data Source.

The reference_data_sources block supports the following:

Kinesis Firehose

Configuration for a Kinesis Firehose delivery stream.

The kinesis_firehose block supports the following:

Kinesis Stream

Configuration for a Kinesis Stream.

The kinesis_stream block supports the following:

Destination Schema

The Schema format of the data in the destination.

The schema block supports the following:

Source Schema

The Schema format of the data in the streaming source.

The schema block supports the following:

Parallelism

Configures the number of Parallel in-application streams to create.

The parallelism block supports the following:

Processing Configuration

The Processing Configuration to transform records as they are received from the stream.

The processing_configuration block supports the following:

Lambda

The Lambda function that pre-processes records in the stream.

The lambda block supports the following:

Starting Position Configuration

The point at which the application reads from the streaming source.

The starting_position_configuration block supports the following:

Record Columns

The Column mapping of each data element in the streaming source to the corresponding column in the in-application stream.

The record_columns block supports the following:

Record Format

The Record Format and relevant mapping information that should be applied to schematize the records on the stream.

The record_format block supports the following:

Mapping Parameters

Provides Mapping information specific to the record format on the streaming source.

The mapping_parameters block supports the following:

CSV Mapping Parameters

Mapping information when the record format uses delimiters.

The csv block supports the following:

JSON Mapping Parameters

Mapping information when JSON is the record format on the streaming source.

The json block supports the following:

S3 Reference

Identifies the S3 bucket and object that contains the reference data.

The s3 blcok supports the following:

Attribute Reference

This resource exports the following attributes in addition to the arguments above:

Import

In Terraform v1.5.0 and later, use an import block to import Kinesis Analytics Application using ARN. For example:

import {
  to = aws_kinesis_analytics_application.example
  id = "arn:aws:kinesisanalytics:us-west-2:1234567890:application/example"
}

Using terraform import, import Kinesis Analytics Application using ARN. For example:

% terraform import aws_kinesis_analytics_application.example arn:aws:kinesisanalytics:us-west-2:1234567890:application/example