Amazon Kinesis

From NovaOrdis Knowledge Base
Revision as of 20:17, 24 October 2018 by Ovidiu (talk | contribs) (→‎Shard)
Jump to navigation Jump to search

External

Internal

Overview

Kinesis acts as a highly available conduit to stream messages between data producers and data consumers.

Concepts

Stream

A Kinesis data stream is a named set of shards. Streams can be created from the AWS Management Console, with AWS CLI and via the Kinesis Data Stream API. The name space if defined by the AWS Account and AWS region: for the same account, streams with the same name can exist in different regions.

Shard

Shards are identified in a stream by their partition key.

Record

Units of data stored in a stream. Records are made up of a sequence number, partition key and data blob. After the data blob is stored in a record, Kinesis does not inspect, interpret or change it in any way.

Data Blob

The data blob is the payload of data contained within a record.

Partition Key

The partition key is used to identify different shards in a stream, and allow a data producer to distribute data across shards.

Sequence Number

Unique identifiers for records inserted into a shard. They increase monotonically, and are specific to individual shards.

Services

Amazon Kinesis Streams

Amazon Kinesis Streams

Amazon Kinesis Firehose

Amazon Kinesis Firehose

Amazon Kinesis Analytics

Amazon Kinesis Analytics

Kinesis Data Streams API