Amazon Kinesis: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
Line 52: Line 52:
* https://aws.amazon.com/kinesis/data-firehose/
* https://aws.amazon.com/kinesis/data-firehose/
* https://docs.aws.amazon.com/firehose/latest/dev/writing-with-sdk.html
* https://docs.aws.amazon.com/firehose/latest/dev/writing-with-sdk.html
* https://www.sumologic.com/blog/devops/kinesis-streams-vs-firehose/


Kinesis Firehose is a solution for loading streaming data from all kinds of sources (web applications, mobile application, IoT, telemetry) directly into AWS storage. There is no need to write applications or manage resources.
Kinesis Firehose is a solution for loading streaming data from all kinds of sources (web applications, mobile application, IoT, telemetry) directly into AWS storage. There is no need to write applications or manage resources.

Revision as of 16:16, 24 October 2018

External

Internal

Overview

Kinesis acts as a highly available conduit to stream messages between data producers and data consumers.

Concepts

Stream

Streams are made out of shards.

Shard

Record

Units of data stored in a stream. Records are made up of a sequence number, partition key and data blob. After the data blob is stored in a record, Kinesis does not inspect, interpret or change it in any way.

Data Blob

The data blob is the payload of data contained within a record.

Partition Key

The partition key is used to identify different shards in a stream, and allow a data producer to distribute data across shards.

Sequence Number

Unique identifiers for records inserted into a shard. They increase monotonically, and are specific to individual shards.

Services

Amazon Kinesis Streams

Kinesis Streams is aimed at users who want to build custom applications to process or analyze streaming data.

Amazon Kinesis Firehose

Kinesis Firehose is a solution for loading streaming data from all kinds of sources (web applications, mobile application, IoT, telemetry) directly into AWS storage. There is no need to write applications or manage resources.

Amazon Kinesis Analytics

https://aws.amazon.com/kinesis/data-analytics/