Amazon S3 Command Line Operations: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(7 intermediate revisions by the same user not shown)
Line 2: Line 2:


* [[Amazon S3 Operations#Command_Line_Operations|Amazon S3 Operations]]
* [[Amazon S3 Operations#Command_Line_Operations|Amazon S3 Operations]]
=Commands=
==cp==
{{External|https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html}}
===Options===
====--quiet====


=Create a Bucket=
=Create a Bucket=
Line 15: Line 21:
==aws s3==
==aws s3==


aws s3 ls s3://some-bucket
===ls===
 
<syntaxhighlight lang='bash'>
aws s3 ls s3://some-bucket
</syntaxhighlight>


If a "some-prefix" exists in the bucket, to list all objects under that prefix it must end with a "/"
If a "some-prefix" exists in the bucket, to list all objects under that prefix it must end with a "/"


aws s3 ls s3://some-bucket/some-prefix/
<syntaxhighlight lang='bash'>
aws s3 ls s3://some-bucket/some-prefix/
</syntaxhighlight>


Listing multiple object names that match a name pattern:  
Listing multiple object names that match a name pattern:  


aws s3 ls s3://some-bucket/some-prefix/dir1/someth
<syntaxhighlight lang='bash'>
aws s3 ls s3://some-bucket/some-prefix/dir1/someth
</syntaxhighlight>


The following will match something-1, something-2, somethingelse, etc.
The following will match something-1, something-2, somethingelse, etc.


==aws s3api==
==aws s3api==
Line 79: Line 92:
==Copy a File into a Folder==
==Copy a File into a Folder==


  aws s3 cp file.txt s3://''bucket-name''/''folder-name''
  aws s3 cp file.txt s3://''bucket-name''/''folder-name/''
 
aws s3 cp file.txt s3://devops-bucket/jenkins/develop/
 
{{Warn|Trailing slash is required.}}
 
=Copy a File from a Bucket=
 
aws [--profile <some-profile>] s3 cp s3://<bucket-name>/path-element-1/path-element-2/<file-name> ./local-file.txt


=Copy Recursively the Content of a Folder=
=Copy Recursively the Content of a Folder=
Line 94: Line 115:


  aws s3 rm s3://''bucket-name'' --recursive
  aws s3 rm s3://''bucket-name'' --recursive
==Delete a Bucket==
aws s3 rb s3://bucket-name --force

Latest revision as of 00:26, 4 May 2023

Internal

Commands

cp

https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html

Options

--quiet

Create a Bucket

aws s3 mb s3://bucket-name

List all Buckets

aws s3api list-buckets --query "Buckets[].Name"

List Objects in a Bucket

aws s3

ls

aws s3 ls s3://some-bucket

If a "some-prefix" exists in the bucket, to list all objects under that prefix it must end with a "/"

aws s3 ls s3://some-bucket/some-prefix/

Listing multiple object names that match a name pattern:

aws s3 ls s3://some-bucket/some-prefix/dir1/someth

The following will match something-1, something-2, somethingelse, etc.

aws s3api

aws s3api list-objects --bucket bucket-name

Example:

aws s3api list-objects --bucket thalarion-release-pipeline

{
   "Contents": [
       {
           "LastModified": "2019-03-17T20:26:01.000Z",
           "ETag": "\"901056c645f90d6aa4edf0a98720a2a3\"",
           "StorageClass": "STANDARD",
           "Key": "thalarion/BuildspecP/Uew1P4u",
           "Owner": {
               "DisplayName": "aws-admin-dv1",
               "ID": "ca887c9c7e3f48cf499d19b960e5ee436cb4ce34e035463003b51c21ea6b4499"
           },
           "Size": 395
       },
       {
           "LastModified": "2019-03-17T20:25:40.000Z",
           "ETag": "\"bbaa592d88d0d5024c776eed46fb6370-1\"",
           "StorageClass": "STANDARD",
           "Key": "thalarion/Repository/5CfRwn8.zip",
           "Owner": {
               "DisplayName": "aws-admin-dv1",
               "ID": "1f887c9c7e3f48cf499d195960e5ee436cb4ce3be035463003b51c21ea6b4493"
           },
           "Size": 3156
       }
   ]
}

To return specific attributes:

 aws s3api list-objects --bucket bucket-name --query 'Contents[].{Key: Key}'

Equivalent:

aws s3 ls s3://bucket-name

Copy a File into a Bucket

aws s3 cp file.txt s3://bucket-name[/optional-different-key] [--grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers full=emailaddress=user@example.com]

"file.txt" will become the key of the S3 object corresponding to the file content, unless "optional-different-key" is used as part of the S3 URI.

Copy a File into a Folder

aws s3 cp file.txt s3://bucket-name/folder-name/
aws s3 cp file.txt s3://devops-bucket/jenkins/develop/

Trailing slash is required.

Copy a File from a Bucket

aws [--profile <some-profile>] s3 cp s3://<bucket-name>/path-element-1/path-element-2/<file-name> ./local-file.txt

Copy Recursively the Content of a Folder

aws s3 cp s3://<bucket-name>/<path>/<folder-name> ./local/dir --recursive

The command recursively downloads the content of the specified S3 folder into the specified local directory.

Deleting or Emptying a Bucket

Deleting or Emptying a Bucket

Empty a Bucket

aws s3 rm s3://bucket-name --recursive

Delete a Bucket

aws s3 rb s3://bucket-name --force