Amazon S3 Concepts: Difference between revisions
No edit summary |
|||
| (33 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
=External= | |||
* https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html | |||
=Internal= | =Internal= | ||
* [[Amazon S3#Concepts|Amazon S3]] | |||
=Overview= | |||
Amazon Simple Storage Service (S3) is an [[Storage_Concepts#Object_Storage|object storage]] service. It can be used to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices and big data analytics. Amazon S3 is designed for 99.999999999% (11 9's) of durability. The S3 service can be integrated and exposed externally via the [[Amazon_API_Gateway_Concepts#Amazon_API_Gateway|Amazon API Gateway]]. | |||
The fundamental storage entity in S3 is the [[#Object|object]]. Objects are contained in [[#Bucket|buckets]]. | |||
=<span id='Objects'></span>Object= | |||
{{External|[https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingObjects.html Working with Amazon S3 Objects]}} | |||
An object is the fundamental entity stored in Amazon S3. Each object consists of [[#Data|data]], a [[#Key|key]], and [[#Metadata|metadata]]. Every S3 object can be uniquely addressed through a combination of the web service endpoint, bucket name, [[#Key|key]] and optionally a [[#Version_ID|version]]. Assuming that "blue" is the bucket name, and "2021-01-01/picture.jpg" is the key, the object can be accessed as https://blue.s3.amazonaws.com/2021-01-01/picture.jpg | |||
==Data== | |||
The data portion of an object is opaque to Amazon S3. | |||
==Key== | |||
{{External|[https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html#object-keys Object Keys]}} | |||
The object key and its [[#Version_ID|version ID]] uniquely identify the object in a [[#Bucket|bucket]], and the combination of a bucket, key and version ID uniquely identify globally each object stored in Amazon S3. Every object in a bucket has exactly one key. | |||
==Version ID== | |||
{{External|https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html}} | |||
==Metadata== | |||
{{External|[https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html#object-metadata Object Metadata]}} | |||
The object metadata is a set of key/value pairs that describe the object. Metadata include some default pairs: date last modified, and standard HTTP metadata such as Content-Type. Custom metadata can be specified at the time the object is stored. After the object was stored, the object metadata cannot be modified. The only way to modify the metadata is to make a copy of the object and set the metadata. | |||
=<span id='S3_Bucket'></span>Bucket= | |||
The Amazon S3 data model is a flat structure: you create a bucket, and the bucket stores [[#Objects|objects]]. A bucket organizes the Amazon S3 namespace at the highest level. <font color=darkgray>A bucket seems to be associated with a specific AWS account - an unrelated AWS account cannot see the bucket in the console. However, I cannot seem to get the AWS account from the bucket.</font> | |||
Buckets are created in a [[#Regions_and_Buckets|specific region]]. | |||
There is no hierarchy of sub-buckets or subfolders; however, you can infer logical hierarchy using key name prefixes and delimiters. The Amazon S3 console supports a concept of folders. The console uses the key name prefixes and the delimiter ('/') to present a folder structure. | |||
==Bucket ARN== | |||
arn:aws:s3:::''bucket-name'' | |||
==Bucket Permissions== | |||
By default, all S3 buckets are private and can be accessed only by users that are explicitly granted access. | |||
===Bucket Public Access=== | |||
===Bucket Access Control List=== | |||
===Bucket Policy=== | |||
Operations: {{Internal|Amazon_S3_Operations#Grant_an_AWS_Account_Permissions_on_a_Bucket|Grant an AWS Account Permissions on a Bucket}} | |||
===Bucket CORS Configuration=== | |||
=Regions and Buckets= | |||
A [[#Bucket|bucket]] lives in a [[Amazon_AWS_Concepts#Region|region]]. The region may be chosen to optimize latency, minimize costs and address regulatory requirements. Objects stored in a region never leave the region unless they are explicitly transferred them to another region. | |||
=S3 | =Data Consistency Model= | ||
{{External|https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#CoreConcepts}} | |||
=Presigned URL= | |||
{{External|https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html}} | |||
{{External|https://medium.com/@aidan.hallett/securing-aws-s3-uploads-using-presigned-urls-aa821c13ae8d}} | |||
Example: https://novaordis.s3.us-west-2.amazonaws.com/testobject?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20211003T065621Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43&X-Amz-Credential=AKIA3999999999999999%2F20211003%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=523a48af48039871145651306016a937c988172578722c48c6f0b5b845bcd109 | |||
{{Internal|Generate an AWS S3 Presigned URL with Java JDK|Generate an AWS S3 Presigned URL with Java JDK}} | |||
==Uploading Objects with Presigned URL== | |||
{{External|https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html}} | |||
==Sharing an Object with Presigned URL== | |||
{{External|https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html}} | |||
=REST Authentication for S3 Requests= | |||
{{External|http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html}} | |||
Latest revision as of 21:03, 3 October 2021
External
Internal
Overview
Amazon Simple Storage Service (S3) is an object storage service. It can be used to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices and big data analytics. Amazon S3 is designed for 99.999999999% (11 9's) of durability. The S3 service can be integrated and exposed externally via the Amazon API Gateway.
The fundamental storage entity in S3 is the object. Objects are contained in buckets.
Object
An object is the fundamental entity stored in Amazon S3. Each object consists of data, a key, and metadata. Every S3 object can be uniquely addressed through a combination of the web service endpoint, bucket name, key and optionally a version. Assuming that "blue" is the bucket name, and "2021-01-01/picture.jpg" is the key, the object can be accessed as https://blue.s3.amazonaws.com/2021-01-01/picture.jpg
Data
The data portion of an object is opaque to Amazon S3.
Key
The object key and its version ID uniquely identify the object in a bucket, and the combination of a bucket, key and version ID uniquely identify globally each object stored in Amazon S3. Every object in a bucket has exactly one key.
Version ID
Metadata
The object metadata is a set of key/value pairs that describe the object. Metadata include some default pairs: date last modified, and standard HTTP metadata such as Content-Type. Custom metadata can be specified at the time the object is stored. After the object was stored, the object metadata cannot be modified. The only way to modify the metadata is to make a copy of the object and set the metadata.
Bucket
The Amazon S3 data model is a flat structure: you create a bucket, and the bucket stores objects. A bucket organizes the Amazon S3 namespace at the highest level. A bucket seems to be associated with a specific AWS account - an unrelated AWS account cannot see the bucket in the console. However, I cannot seem to get the AWS account from the bucket.
Buckets are created in a specific region.
There is no hierarchy of sub-buckets or subfolders; however, you can infer logical hierarchy using key name prefixes and delimiters. The Amazon S3 console supports a concept of folders. The console uses the key name prefixes and the delimiter ('/') to present a folder structure.
Bucket ARN
arn:aws:s3:::bucket-name
Bucket Permissions
By default, all S3 buckets are private and can be accessed only by users that are explicitly granted access.
Bucket Public Access
Bucket Access Control List
Bucket Policy
Operations:
Bucket CORS Configuration
Regions and Buckets
A bucket lives in a region. The region may be chosen to optimize latency, minimize costs and address regulatory requirements. Objects stored in a region never leave the region unless they are explicitly transferred them to another region.