Amazon Kinesis Streams: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
Line 86: Line 86:
  SHARD_ITERATOR=$(aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name bizEventIn --query 'ShardIterator')
  SHARD_ITERATOR=$(aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name bizEventIn --query 'ShardIterator')
  aws kinesis get-records --shard-iterator $SHARD_ITERATOR
  aws kinesis get-records --shard-iterator $SHARD_ITERATOR
=Delete a Stream=
aws kinesis delete-stream --stream-name <''stream-name''>

Revision as of 17:13, 25 October 2018

External

Internal

Overview

Kinesis Streams is aimed at users who want to build custom applications to process or analyze streaming data.

Amazon Kinesis Stream Access with Spring Data Cloud

Spring Cloud Stream AWS Kinesis Binder

AWS CLI Operations

Prerequisites

Set up the AWS credentials and region first.

List Streams

aws kinesis list-streams

Get Details about a Specific Stream

Returns informations about a shard, including the ID of its shards.

aws kinesis describe-stream --stream-name bizEventIn

Output sample:

{
    "StreamDescription": {
        "KeyId": null,
        "EncryptionType": "NONE",
        "StreamStatus": "ACTIVE",
        "StreamName": "bizEventIn",
        "Shards": [
            {
                "ShardId": "shardId-000000000000",
                "HashKeyRange": {
                    "EndingHashKey": "340282366920938463463374607431768211455",
                    "StartingHashKey": "0"
                },
                "SequenceNumberRange": {
                    "StartingSequenceNumber": "49589495463753313109232148314676658043283830396753543170"
                }
            }
        ],
        "StreamARN": "arn:aws:kinesis:us-west-2:673499572719:stream/bizEventIn",
        "EnhancedMonitoring": [
            {
                "ShardLevelMetrics": [
                    "IncomingBytes",
                    "OutgoingRecords",
                    "IteratorAgeMilliseconds",
                    "IncomingRecords",
                    "ReadProvisionedThroughputExceeded",
                    "WriteProvisionedThroughputExceeded",
                    "OutgoingBytes"
                ]
            }
        ],
        "StreamCreationTimestamp": 1540414211.0,
        "RetentionPeriodHours": 24
    }
}

Put a Record

aws kinesis put-record --stream-name my-stream --partition-key 1 --data testdata

Get a Record

First then get a shard iterator and then get the record:

SHARD_ITERATOR=$(aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name bizEventIn --query 'ShardIterator')
aws kinesis get-records --shard-iterator $SHARD_ITERATOR

Delete a Stream

aws kinesis delete-stream --stream-name <stream-name>