Amazon S3 Command Line Operations: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(18 intermediate revisions by the same user not shown)
Line 2: Line 2:


* [[Amazon S3 Operations#Command_Line_Operations|Amazon S3 Operations]]
* [[Amazon S3 Operations#Command_Line_Operations|Amazon S3 Operations]]
=Commands=
==cp==
{{External|https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html}}
===Options===
====--quiet====
=Create a Bucket=
aws s3 mb s3://''bucket-name''


=List all Buckets=
=List all Buckets=
Line 8: Line 18:


=List Objects in a Bucket=
=List Objects in a Bucket=
==aws s3==
===ls===
<syntaxhighlight lang='bash'>
aws s3 ls s3://some-bucket
</syntaxhighlight>
If a "some-prefix" exists in the bucket, to list all objects under that prefix it must end with a "/"
<syntaxhighlight lang='bash'>
aws s3 ls s3://some-bucket/some-prefix/
</syntaxhighlight>
Listing multiple object names that match a name pattern:
<syntaxhighlight lang='bash'>
aws s3 ls s3://some-bucket/some-prefix/dir1/someth
</syntaxhighlight>
The following will match something-1, something-2, somethingelse, etc.
==aws s3api==


  aws s3api list-objects --bucket ''bucket-name''
  aws s3api list-objects --bucket ''bucket-name''
Line 46: Line 80:
   aws s3api list-objects --bucket ''bucket-name'' --query 'Contents[].{Key: Key}'
   aws s3api list-objects --bucket ''bucket-name'' --query 'Contents[].{Key: Key}'


 
Equivalent:
Equivalent <font color=darkgray>TO TEST</font>:


  aws s3 ls s3://''bucket-name''
  aws s3 ls s3://''bucket-name''
Line 55: Line 88:
  aws s3 cp file.txt s3://''bucket-name''[/''optional-different-key''] [--grants read=uri=http&#58;//acs.amazonaws.com/groups/global/AllUsers full=emailaddress=user@example.com]
  aws s3 cp file.txt s3://''bucket-name''[/''optional-different-key''] [--grants read=uri=http&#58;//acs.amazonaws.com/groups/global/AllUsers full=emailaddress=user@example.com]


"file.txt" will become the key of the S3 object corresponding to the file content, unless "optional-different-key" is used.
"file.txt" will become the key of the S3 object corresponding to the file content, unless "optional-different-key" is used as part of the S3 URI.
 
==Copy a File into a Folder==
 
aws s3 cp file.txt s3://''bucket-name''/''folder-name/''
 
aws s3 cp file.txt s3://devops-bucket/jenkins/develop/
 
{{Warn|Trailing slash is required.}}
 
=Copy a File from a Bucket=
 
aws [--profile <some-profile>] s3 cp s3://<bucket-name>/path-element-1/path-element-2/<file-name> ./local-file.txt
 
=Copy Recursively the Content of a Folder=
 
aws s3 cp s3://''<bucket-name>''/''<path>''/''<folder-name>'' ./local/dir --recursive
 
The command recursively downloads the content of the specified S3 folder into the specified local directory.
 
=Deleting or Emptying a Bucket=
 
{{External|[https://docs.aws.amazon.com/AmazonS3/latest/dev/delete-or-empty-bucket.html Deleting or Emptying a Bucket]}}
 
==Empty a Bucket==
 
aws s3 rm s3://''bucket-name'' --recursive
==Delete a Bucket==
 
aws s3 rb s3://bucket-name --force

Latest revision as of 00:26, 4 May 2023

Internal

Commands

cp

https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html

Options

--quiet

Create a Bucket

aws s3 mb s3://bucket-name

List all Buckets

aws s3api list-buckets --query "Buckets[].Name"

List Objects in a Bucket

aws s3

ls

aws s3 ls s3://some-bucket

If a "some-prefix" exists in the bucket, to list all objects under that prefix it must end with a "/" 

aws s3 ls s3://some-bucket/some-prefix/

Listing multiple object names that match a name pattern: 

aws s3 ls s3://some-bucket/some-prefix/dir1/someth

The following will match something-1, something-2, somethingelse, etc.

aws s3api

aws s3api list-objects --bucket bucket-name

Example: 

aws s3api list-objects --bucket thalarion-release-pipeline

{
   "Contents": [
       {
           "LastModified": "2019-03-17T20:26:01.000Z",
           "ETag": "\"901056c645f90d6aa4edf0a98720a2a3\"",
           "StorageClass": "STANDARD",
           "Key": "thalarion/BuildspecP/Uew1P4u",
           "Owner": {
               "DisplayName": "aws-admin-dv1",
               "ID": "ca887c9c7e3f48cf499d19b960e5ee436cb4ce34e035463003b51c21ea6b4499"
           },
           "Size": 395
       },
       {
           "LastModified": "2019-03-17T20:25:40.000Z",
           "ETag": "\"bbaa592d88d0d5024c776eed46fb6370-1\"",
           "StorageClass": "STANDARD",
           "Key": "thalarion/Repository/5CfRwn8.zip",
           "Owner": {
               "DisplayName": "aws-admin-dv1",
               "ID": "1f887c9c7e3f48cf499d195960e5ee436cb4ce3be035463003b51c21ea6b4493"
           },
           "Size": 3156
       }
   ]
}

To return specific attributes: 

 aws s3api list-objects --bucket bucket-name --query 'Contents[].{Key: Key}'

Equivalent: 

aws s3 ls s3://bucket-name

Copy a File into a Bucket

aws s3 cp file.txt s3://bucket-name[/optional-different-key] [--grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers full=emailaddress=user@example.com]

"file.txt" will become the key of the S3 object corresponding to the file content, unless "optional-different-key" is used as part of the S3 URI.

Copy a File into a Folder

aws s3 cp file.txt s3://bucket-name/folder-name/

aws s3 cp file.txt s3://devops-bucket/jenkins/develop/

Trailing slash is required.

Copy a File from a Bucket

aws [--profile <some-profile>] s3 cp s3://<bucket-name>/path-element-1/path-element-2/<file-name> ./local-file.txt

Copy Recursively the Content of a Folder

aws s3 cp s3://<bucket-name>/<path>/<folder-name> ./local/dir --recursive

The command recursively downloads the content of the specified S3 folder into the specified local directory.

Deleting or Emptying a Bucket

Deleting or Emptying a Bucket

Empty a Bucket

aws s3 rm s3://bucket-name --recursive

Delete a Bucket

aws s3 rb s3://bucket-name --force
Retrieved from "https://kb.novaordis.com/index.php?title=Amazon_S3_Command_Line_Operations&oldid=96429"