Kinesis

Amazon Kinesis is a real-time data streaming service designed to process large scale data streams at any scale.

Terraform Name

aws_kinesis_stream

Kinesis

attributes:

‍name - (Required) A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
shard_count – (Optional) The number of shards that the stream will use. If the stream_mode is PROVISIONED, this field is required. Amazon has guidelines for specifying the Stream size that should be referenced when creating a Kinesis stream. See Amazon Kinesis Streams for more.
retention_period - (Optional) Length of time data records are accessible after they are added to the stream. The maximum value of a stream's retention period is 8760 hours. Minimum value is 24. Default is 24.
shard_level_metrics - (Optional) A list of shard-level CloudWatch metrics which can be enabled for the stream. See Monitoring with CloudWatch for more. Note that the value ALL should not be used; instead you should provide an explicit list of metrics you wish to enable.
enforce_consumer_deletion - (Optional) A boolean that indicates all registered consumers should be deregistered from the stream so that the stream can be destroyed without error. The default value is false.
encryption_type - (Optional) The encryption type to use. The only acceptable values are NONE or KMS. The default value is NONE.
kms_key_id - (Optional) The GUID for the customer-managed KMS key to use for encryption. You can also use a Kinesis-owned master key by specifying the alias alias/aws/kinesis.
stream_mode_details - (Optional) Indicates the capacity mode of the data stream. Detailed below.
tags - (Optional) A map of tags to assign to the resource. If configured with a provider default_tags configuration block present, tags with matching keys will overwrite those defined at the provider-level.

stream_mode_details Configuration Block

stream_mode - (Required) Specifies the capacity mode of the stream. Must be either PROVISIONED or ON_DEMAND.

‍

Associating resources with a

Kinesis

Resources do not "belong" to a

Kinesis

Rather, one or more Security Groups are associated to a resource.

Create

Kinesis

via Terraform:

The following HCL creates a Kinesis stream resource

Syntax:

resource "aws_kinesis_stream" "test_stream" {
name = "terraform-kinesis-test"
shard_count = 1
retention_period = 48

shard_level_metrics = [
"IncomingBytes",
"OutgoingBytes",
]

stream_mode_details {
stream_mode = "PROVISIONED"
}

tags = {
Environment = "test"
}
}

Create

Kinesis

via CLI:

Parameters:

create-stream
--stream-name <value>
[--shard-count <value>]
[--stream-mode-details <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]

Example:

aws kinesis create-stream \
--stream-name samplestream \
--shard-count 3

Costs

The cost of using Kinesis depends on the number of reads, writes, and data transfer for the Kinesis data streams, as well as the number of Kinesis Video Streams and Kinesis Firehose delivery streams. Kinesis charges a fee for each shard hour and for each MB of data transferred, as well as for any additional data retention.

Direct Cost

OnDemand-StreamHour

Indirect Cost

No items found.

Best Practices for

Kinesis

Categorized by Availability, Security & Compliance and Cost

Medium

AMI (Amazon Machine Images) not in use

AWS Cost Optimization