Amazon EC2 Operations: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
(Created page with "=Internal= * Amazon EC2 =EC2 CLI= {{Internal|Amazon EC2 CLI|Amazon EC2 CLI}}")
 
 
(79 intermediate revisions by the same user not shown)
Line 6: Line 6:


{{Internal|Amazon EC2 CLI|Amazon EC2 CLI}}
{{Internal|Amazon EC2 CLI|Amazon EC2 CLI}}
=Instance Operations=
==Describe Instances==
aws [--region us-west-2] ec2 describe-instances [--instance-id ...]
<syntaxhighlight lang='json'>
{
    "Reservations": [
        {
            "Instances": [
                {
                    "Monitoring": {
                        "State": "disabled"
                    },
                    "PublicDnsName": "",
                    "State": {
                        "Code": 16,
                        "Name": "running"
                    },
                    "EbsOptimized": false,
                    "LaunchTime": "2019-04-10T16:56:01.000Z",
                    "PrivateIpAddress": "10.10.2.71",
                    "ProductCodes": [],
                    "VpcId": "vpc-1d3c8c4c1d99cd6ba",
                    "CpuOptions": {
                        "CoreCount": 1,
                        "ThreadsPerCore": 1
                    },
                    "StateTransitionReason": "",
                    "InstanceId": "i-01a053a09bf6d7b15",
                    "EnaSupport": true,
                    "ImageId": "ami-03338e1f67dae0168",
                    "PrivateDnsName": "ip-10-10-2-71.ca-central-1.compute.internal",
                    "KeyName": "outerworldly-key-pair",
                    "SecurityGroups": [
                        {
                            "GroupName": "some-sg",
                            "GroupId": "sg-04efb7c151570f2d0"
                        }
                    ],
                    "ClientToken": "",
                    "SubnetId": "subnet-00102cd730cc69e0a",
                    "InstanceType": "t2.micro",
                    "NetworkInterfaces": [
                        {
                            "Status": "in-use",
                            "MacAddress": "02:96:44:f3:af:08",
                            "SourceDestCheck": true,
                            "VpcId": "vpc-1d3c8c4c1d99cd6ba",
                            "Description": "Primary network interface",
                            "NetworkInterfaceId": "eni-07ebe1e63724f60f2",
                            "PrivateIpAddresses": [
                                {
                                    "Primary": true,
                                    "PrivateIpAddress": "10.10.2.71"
                                }
                            ],
                            "SubnetId": "subnet-00102cd730cc69e0a",
                            "Attachment": {
                                "Status": "attached",
                                "DeviceIndex": 0,
                                "DeleteOnTermination": true,
                                "AttachmentId": "eni-attach-003a2cd69ddac31a4",
                                "AttachTime": "2019-04-10T16:56:01.000Z"
                            },
                            "Groups": [
                                {
                                    "GroupName": "some-sg",
                                    "GroupId": "sg-04efb7c151570f2d0"
                                }
                            ],
                            "Ipv6Addresses": [],
                            "OwnerId": "777777777777",
                            "PrivateIpAddress": "10.10.2.71"
                        }
                    ],
                    "SourceDestCheck": true,
                    "Placement": {
                        "Tenancy": "default",
                        "GroupName": "",
                        "AvailabilityZone": "ca-central-1a"
                    },
                    "Hypervisor": "xen",
                    "BlockDeviceMappings": [
                        {
                            "DeviceName": "/dev/xvda",
                            "Ebs": {
                                "Status": "attached",
                                "DeleteOnTermination": true,
                                "VolumeId": "vol-0290718334f7ccdc0",
                                "AttachTime": "2019-04-10T16:56:02.000Z"
                            }
                        }
                    ],
                    "Architecture": "x86_64",
                    "RootDeviceType": "ebs",
                    "RootDeviceName": "/dev/xvda",
                    "VirtualizationType": "hvm",
                    "Tags": [
                        {
                            "Value": "01 instance",
                            "Key": "Name"
                        }
                    ],
                    "AmiLaunchIndex": 0
                }
            ],
            "ReservationId": "r-0c51f172352815e10",
            "Groups": [],
            "OwnerId": "777777777777"
        }
    ]
</syntaxhighlight>
==Instance Profile Operations==
===View Existing Instance Profiles===
View [[Amazon_AWS_Security_Concepts#Instance_Profile|instance profiles]].
aws iam list-instance-profiles
aws iam list-instance-profiles-for-role
===Create an Instance Profile===
{{External|[https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli Create an IAM Instance Profile for Your Amazon EC2 Instances]}}
This procedure describes how to create an EC2 [[Amazon_AWS_Security_Concepts#Instance_Profile|instance profile]].
====From Console====
If a role is created from the console, the corresponding instance profile is created automatically and it can be retrieved from the role's attribute view.
====With AWS CLI====
aws iam create-instance-profile
===Add a Role to an Instance Profile===
{{External|[https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli Create an IAM Instance Profile for Your Amazon EC2 Instances]}}
  aws iam add-role-to-instance-profile
===Associate an Instance Profile with an EC2 Instance===
{{External|[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html IAM Roles for Amazon EC2]}}
{{External|[https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-iam-instance-profile.html#getting-started-create-iam-instance-profile-cli Create an IAM Instance Profile for Your Amazon EC2 Instances]}}
This procedure describes how to associate an [[Amazon_AWS_Security_Concepts#Instance_Profile|instance profile]] with an EC2 instance.
<font color=darkgray>TODO</font>
If the instance is built with Vagrant, it is associated [[Vagrant_AWS_Provider#Configuring_the_Instance_Profile|as such]].
==Instance Metadata Operations==
[[Amazon_EC2_Concepts#Instance_Metadata_and_User_Data|Instance metadata]] can be accessed on an EC2 VM.
===Get Metadata Categories===
curl http&#58;//169.254.169.254/latest/meta-data
===Get the Instance's AMI===
curl http&#58;//169.254.169.254/latest/meta-data/ami-id
===Use Temporary Credentials Made Available via Instance Metadata===
{{External|[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials Retrieving Security Credentials from Instance Metadata]}}
Accessing [[Amazon_AWS_Security_Concepts#Temporary_Security_Credentials|temporary credentials]] implies that the EC2 instance was associated with an [[Amazon_AWS_Security_Concepts#Instance_Profile|instance profile]] that contains an [[Amazon_AWS_Security_Concepts#AWS_Service_Role_for_an_EC2_Instance|EC2 service role]] allowing appropriate access to resources we want to use the temporary credentials for. If the EC2 instance is configured correctly, the temporary credentials should be retrievable as such:
curl http&#58;//169.254.169.254/latest/meta-data/iam/security-credentials/<''ec2-service-role-name''>
The response should be similar to:
<syntaxhighlight lang='json'>
{
  "Code" : "Success",
  "LastUpdated" : "2019-11-19T22:11:04Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIA40000000000000000",
  "SecretAccessKey" : "ka000000000000000000000000000000000000000",
  "Token" : "IQo...w==",
  "Expiration" : "2019-11-20T04:45:32Z"
}
</syntaxhighlight>
==Block Device Mapping Operations==
For more details about block device mapping, see: {{Internal|Amazon_EC2_Concepts#Block_Device_Mapping|Amazon EC2 Concepts}}
===Specify the Root Device as Part of Creating the AMI===
* [[Packer_amazon-ebs_Configuration#Root_Device|Specify the Root Device as Part of Creating the AMI with Packer]]
===Specify Additional Block Devices as Part of Creating the AMI===
* [[Packer_amazon-ebs_Configuration#ami_block_device_mappings|Specify Additional Block Devices as Part of Creating the AMI with Packer]]
===Specify Additional Block Devices when the Instance is Started===
<font color=darkgray>TODO</font>
==Terminate an Instance==
<syntaxhighlight lang='bash'>
aws ec2 terminate-instances --instance-ids i-04444444444444444 --region us-west-2
</syntaxhighlight>
=Configure a Public DNS Name to an EC2 Instance=
<font color=darkgray>TODO</font>
=Load Balancing Operations=
{{Internal|AWS Elastic Load Balancing Operations|AWS Elastic Load Balancing Operations}}
=Key Pair Operations=
It seems there is no way to create a [[Amazon_EC2_Concepts#Key_Pair|Key pair]] as part of a CloudFormation template.
==Create a Key Pair with Amazon Console==
AWS -> EC2 -> Key Pairs -> Create Key Pair.
=Region Operations=
{{Internal|AWS Region Operations|AWS Region Operations}}
=Availability Zone Operations=
==Get Availability Zones in a Region==
aws ec2 describe-availability-zones --region ca-central-1
=Create an EC2 Instance=
==With AWS CLI==
aws ec2 run-instances --image-id ami-11aa22bb --iam-instance-profile Name="s3access-profile" --key-name my-key-pair --security-groups my-security-group --subnet-id subnet-1a2b3c4d
==<span id='Create_an_EC2_Instance_with_CloudFormation'></span>With CloudFormation==
{{External|[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html AWS::EC2::Instance]}}
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      [[Amazon_EC2_Concepts#AMI_.28Amazon_Machine_Image.29|ImageId]]: ami-03338e1f67dae0168
      [[Amazon_EC2_Concepts#Key_Pair|KeyName]]: !Sub infinity-${AWS::Region}
      InstanceType: t2.micro
      [[Amazon_EC2_Concepts#Networking|NetworkInterfaces]]:
        - AssociatePublicIpAddress: 'true'
          DeviceIndex: '0'
          [[Amazon_EC2_Concepts#Security_Groups|GroupSet]]:
            - !Ref SomeSecurityGroup
          SubnetId: !Ref PublicSubnet
      Tags:
        - Key: Name
          Value: !Sub ${MicroworldName}-bastion
===GetAtt Attributes===
{{External|[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html Fn::GetAtt Attributes]}}
====AvailabilityZone====
====PrivateDnsName====
====PublicDnsName====
====PrivateIp====
====PublicIp====
==<span id='Create_an_EC2_Instance_with_Terraform'></span>With Terraform==
{{Internal|Create an EC2 Instance with Terraform|Create an EC2 Instance with Terraform}}
=AMI Operations=
==Find an Image==
aws ec2 describe-images --filters "Name=image-id,Values=ami-0a85857bfc5345c38" "Name=architecture,Values=x86_64"
<syntaxhighlight lang='text'>
{
    "Images": [
        {
            "VirtualizationType": "hvm",
            "Description": "Amazon Linux 2 AMI 2.0.20191024.3 x86_64 HVM gp2",
            "Hypervisor": "xen",
            "ImageOwnerAlias": "amazon",
            "EnaSupport": true,
            "SriovNetSupport": "simple",
            "ImageId": "ami-0a85857bfc5345c38",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "SnapshotId": "snap-0f3ae06326a22c8b0",
                        "DeleteOnTermination": true,
                        "VolumeType": "gp2",
                        "VolumeSize": 8,
                        "Encrypted": false
                    }
                }
            ],
            "Architecture": "x86_64",
            "ImageLocation": "amazon/amzn2-ami-hvm-2.0.20191024.3-x86_64-gp2",
            "RootDeviceType": "ebs",
            "OwnerId": "137112412989",
            "RootDeviceName": "/dev/xvda",
            "CreationDate": "2019-10-31T06:01:08.000Z",
            "Public": true,
            "ImageType": "machine",
            "Name": "amzn2-ami-hvm-2.0.20191024.3-x86_64-gp2"
        }
    ]
}
</syntaxhighlight>
aws ec2 describe-images --filters 'Name=name,Values=amzn2-ami-hvm-2.0.*.*-x86_64-gp" "Name=architecture,Values=x86_64" "Name=virtualization-type,Values=hvm" "Name=root-device-type,Values=ebs"
===Query for Images with Specific Attributes and Sort by Creation Date===
From Amazon:
aws ec2 describe-images --owners amazon --filters "Name=name,Values=amzn*" --query 'sort_by(Images, &CreationDate)[].ImageId'
aws ec2 describe-images --owners amazon --filters "Name=name,Values=amzn*" --query 'sort_by(Images, &CreationDate)[].Name'
From a specific user, return all images, sorted:
aws ec2 describe-images --owners 000000000000 --filters "Name=name,Values=infra-worker-*" --query 'sort_by(Images, &CreationDate)[].ImageId'
The newest image:
aws ec2 describe-images --owners 000000000000 --filters "Name=name,Values=infra-worker-*" --query 'sort_by(Images, &CreationDate)[-1].ImageId' --output text
===With ec2-describe-images===
{{Internal|ec2-describe-images|Finding an AMI using EC2 API}}
==Deregister an AMI==
aws ec2 deregister-image --image-id <''ami_id''>
=Amazon Elastic Block Store (EBS) Operations=
{{Internal|Amazon_Elastic_Block_Store_Operations#Operations|Amazon Elastic Block Store Operations}}
=Auto-Scaling Operations=
{{External|https://docs.aws.amazon.com/cli/latest/reference/autoscaling/index.html}}
<syntaxhighlight lang='bash'>
aws autoscaling describe-auto-scaling-groups
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names something
aws autoscaling describe-auto-scaling-instances
</syntaxhighlight>
<syntaxhighlight lang='bash'>
aws autoscaling delete-auto-scaling-group --auto-scaling-group-name something --force-delete
</syntaxhighlight>

Latest revision as of 22:05, 21 July 2020

Internal

EC2 CLI

Amazon EC2 CLI

Instance Operations

Describe Instances

aws [--region us-west-2] ec2 describe-instances [--instance-id ...]
{
    "Reservations": [
        {
            "Instances": [
                {
                    "Monitoring": {
                        "State": "disabled"
                    },
                    "PublicDnsName": "",
                    "State": {
                        "Code": 16,
                        "Name": "running"
                    },
                    "EbsOptimized": false,
                    "LaunchTime": "2019-04-10T16:56:01.000Z",
                    "PrivateIpAddress": "10.10.2.71",
                    "ProductCodes": [],
                    "VpcId": "vpc-1d3c8c4c1d99cd6ba",
                    "CpuOptions": {
                        "CoreCount": 1,
                        "ThreadsPerCore": 1
                    },
                    "StateTransitionReason": "",
                    "InstanceId": "i-01a053a09bf6d7b15",
                    "EnaSupport": true,
                    "ImageId": "ami-03338e1f67dae0168",
                    "PrivateDnsName": "ip-10-10-2-71.ca-central-1.compute.internal",
                    "KeyName": "outerworldly-key-pair",
                    "SecurityGroups": [
                        {
                            "GroupName": "some-sg",
                            "GroupId": "sg-04efb7c151570f2d0"
                        }
                    ],
                    "ClientToken": "",
                    "SubnetId": "subnet-00102cd730cc69e0a",
                    "InstanceType": "t2.micro",
                    "NetworkInterfaces": [
                        {
                            "Status": "in-use",
                            "MacAddress": "02:96:44:f3:af:08",
                            "SourceDestCheck": true,
                            "VpcId": "vpc-1d3c8c4c1d99cd6ba",
                            "Description": "Primary network interface",
                            "NetworkInterfaceId": "eni-07ebe1e63724f60f2",
                            "PrivateIpAddresses": [
                                {
                                    "Primary": true,
                                    "PrivateIpAddress": "10.10.2.71"
                                }
                            ],
                            "SubnetId": "subnet-00102cd730cc69e0a",
                            "Attachment": {
                                "Status": "attached",
                                "DeviceIndex": 0,
                                "DeleteOnTermination": true,
                                "AttachmentId": "eni-attach-003a2cd69ddac31a4",
                                "AttachTime": "2019-04-10T16:56:01.000Z"
                            },
                            "Groups": [
                                {
                                    "GroupName": "some-sg",
                                    "GroupId": "sg-04efb7c151570f2d0"
                                }
                            ],
                            "Ipv6Addresses": [],
                            "OwnerId": "777777777777",
                            "PrivateIpAddress": "10.10.2.71"
                        }
                    ],
                    "SourceDestCheck": true,
                    "Placement": {
                        "Tenancy": "default",
                        "GroupName": "",
                        "AvailabilityZone": "ca-central-1a"
                    },
                    "Hypervisor": "xen",
                    "BlockDeviceMappings": [
                        {
                            "DeviceName": "/dev/xvda",
                            "Ebs": {
                                "Status": "attached",
                                "DeleteOnTermination": true,
                                "VolumeId": "vol-0290718334f7ccdc0",
                                "AttachTime": "2019-04-10T16:56:02.000Z"
                            }
                        }
                    ],
                    "Architecture": "x86_64",
                    "RootDeviceType": "ebs",
                    "RootDeviceName": "/dev/xvda",
                    "VirtualizationType": "hvm",
                    "Tags": [
                        {
                            "Value": "01 instance",
                            "Key": "Name"
                        }
                    ],
                    "AmiLaunchIndex": 0
                }
            ],
            "ReservationId": "r-0c51f172352815e10",
            "Groups": [],
            "OwnerId": "777777777777"
        }
    ]

Instance Profile Operations

View Existing Instance Profiles

View instance profiles.

aws iam list-instance-profiles
aws iam list-instance-profiles-for-role

Create an Instance Profile

Create an IAM Instance Profile for Your Amazon EC2 Instances

This procedure describes how to create an EC2 instance profile.

From Console

If a role is created from the console, the corresponding instance profile is created automatically and it can be retrieved from the role's attribute view.

With AWS CLI

aws iam create-instance-profile

Add a Role to an Instance Profile

Create an IAM Instance Profile for Your Amazon EC2 Instances
 aws iam add-role-to-instance-profile

Associate an Instance Profile with an EC2 Instance

IAM Roles for Amazon EC2
Create an IAM Instance Profile for Your Amazon EC2 Instances

This procedure describes how to associate an instance profile with an EC2 instance.

TODO

If the instance is built with Vagrant, it is associated as such.

Instance Metadata Operations

Instance metadata can be accessed on an EC2 VM.

Get Metadata Categories

curl http://169.254.169.254/latest/meta-data

Get the Instance's AMI

curl http://169.254.169.254/latest/meta-data/ami-id

Use Temporary Credentials Made Available via Instance Metadata

Retrieving Security Credentials from Instance Metadata

Accessing temporary credentials implies that the EC2 instance was associated with an instance profile that contains an EC2 service role allowing appropriate access to resources we want to use the temporary credentials for. If the EC2 instance is configured correctly, the temporary credentials should be retrievable as such:

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/<ec2-service-role-name>

The response should be similar to:

{
  "Code" : "Success",
  "LastUpdated" : "2019-11-19T22:11:04Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIA40000000000000000",
  "SecretAccessKey" : "ka000000000000000000000000000000000000000",
  "Token" : "IQo...w==",
  "Expiration" : "2019-11-20T04:45:32Z"
}

Block Device Mapping Operations

For more details about block device mapping, see:

Amazon EC2 Concepts

Specify the Root Device as Part of Creating the AMI

Specify Additional Block Devices as Part of Creating the AMI

Specify Additional Block Devices when the Instance is Started

TODO

Terminate an Instance

aws ec2 terminate-instances --instance-ids i-04444444444444444 --region us-west-2

Configure a Public DNS Name to an EC2 Instance

TODO

Load Balancing Operations

AWS Elastic Load Balancing Operations

Key Pair Operations

It seems there is no way to create a Key pair as part of a CloudFormation template.

Create a Key Pair with Amazon Console

AWS -> EC2 -> Key Pairs -> Create Key Pair.

Region Operations

AWS Region Operations

Availability Zone Operations

Get Availability Zones in a Region

aws ec2 describe-availability-zones --region ca-central-1

Create an EC2 Instance

With AWS CLI

aws ec2 run-instances --image-id ami-11aa22bb --iam-instance-profile Name="s3access-profile" --key-name my-key-pair --security-groups my-security-group --subnet-id subnet-1a2b3c4d

With CloudFormation

AWS::EC2::Instance
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-03338e1f67dae0168
      KeyName: !Sub infinity-${AWS::Region}
      InstanceType: t2.micro
      NetworkInterfaces:
        - AssociatePublicIpAddress: 'true'
          DeviceIndex: '0'
          GroupSet:
            - !Ref SomeSecurityGroup
          SubnetId: !Ref PublicSubnet
      Tags:
        - Key: Name
          Value: !Sub ${MicroworldName}-bastion

GetAtt Attributes

Fn::GetAtt Attributes

AvailabilityZone

PrivateDnsName

PublicDnsName

PrivateIp

PublicIp

With Terraform

Create an EC2 Instance with Terraform

AMI Operations

Find an Image

aws ec2 describe-images --filters "Name=image-id,Values=ami-0a85857bfc5345c38" "Name=architecture,Values=x86_64"
{
    "Images": [
        {
            "VirtualizationType": "hvm",
            "Description": "Amazon Linux 2 AMI 2.0.20191024.3 x86_64 HVM gp2",
            "Hypervisor": "xen",
            "ImageOwnerAlias": "amazon",
            "EnaSupport": true,
            "SriovNetSupport": "simple",
            "ImageId": "ami-0a85857bfc5345c38",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "SnapshotId": "snap-0f3ae06326a22c8b0",
                        "DeleteOnTermination": true,
                        "VolumeType": "gp2",
                        "VolumeSize": 8,
                        "Encrypted": false
                    }
                }
            ],
            "Architecture": "x86_64",
            "ImageLocation": "amazon/amzn2-ami-hvm-2.0.20191024.3-x86_64-gp2",
            "RootDeviceType": "ebs",
            "OwnerId": "137112412989",
            "RootDeviceName": "/dev/xvda",
            "CreationDate": "2019-10-31T06:01:08.000Z",
            "Public": true,
            "ImageType": "machine",
            "Name": "amzn2-ami-hvm-2.0.20191024.3-x86_64-gp2"
        }
    ]
}
aws ec2 describe-images --filters 'Name=name,Values=amzn2-ami-hvm-2.0.*.*-x86_64-gp" "Name=architecture,Values=x86_64" "Name=virtualization-type,Values=hvm" "Name=root-device-type,Values=ebs"

Query for Images with Specific Attributes and Sort by Creation Date

From Amazon:

aws ec2 describe-images --owners amazon --filters "Name=name,Values=amzn*" --query 'sort_by(Images, &CreationDate)[].ImageId'
aws ec2 describe-images --owners amazon --filters "Name=name,Values=amzn*" --query 'sort_by(Images, &CreationDate)[].Name'

From a specific user, return all images, sorted:

aws ec2 describe-images --owners 000000000000 --filters "Name=name,Values=infra-worker-*" --query 'sort_by(Images, &CreationDate)[].ImageId'

The newest image:

aws ec2 describe-images --owners 000000000000 --filters "Name=name,Values=infra-worker-*" --query 'sort_by(Images, &CreationDate)[-1].ImageId' --output text

With ec2-describe-images

Finding an AMI using EC2 API

Deregister an AMI

aws ec2 deregister-image --image-id <ami_id>

Amazon Elastic Block Store (EBS) Operations

Amazon Elastic Block Store Operations

Auto-Scaling Operations

https://docs.aws.amazon.com/cli/latest/reference/autoscaling/index.html
aws autoscaling describe-auto-scaling-groups
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names something
aws autoscaling describe-auto-scaling-instances
aws autoscaling delete-auto-scaling-group --auto-scaling-group-name something --force-delete