Amazon EC2 Concepts: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(101 intermediate revisions by the same user not shown)
Line 2: Line 2:


* [[Amazon EC2#Subjects|Amazon EC2]]
* [[Amazon EC2#Subjects|Amazon EC2]]
=Overview=
There are two Amazon Elastic Compute Cloud (EC2) platforms: EC2-VPC and EC2-Classic. EC2-VPC is aimed at launching AWS resources into a virtual private cloud ([[Amazon_VPC_Concepts#Virtual_Private_Cloud_.28VPC.29|VPC]]). Also see [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.FindDefaultVPC.html Determining Whether You Are Using the EC2-VPC or EC2-Classic Platform].


=Security Concepts=
=Security Concepts=
Line 8: Line 12:


=AMI (Amazon Machine Image)=
=AMI (Amazon Machine Image)=
{{External|http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html}}


An ''Amazon machine image'' is a template that contains a software configuration (operating system, application server, application).
An ''Amazon machine image'' is a template that contains a software configuration (operating system, application server, application).


External Documentation:
To browse through available AMIs go to the EC2 console -> Left Navigation Pane -> AMIs.
* http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html
 
AMIs can be chosen based on region, operating system, architecture (32-bit or 64-bit), launch permissions and [[#AMI_Storage_for_Root_Device|storage for root device]].
 
==AMI ID==
==AMI Name==
 
Must be unique <font color=darkgray>within account and region.</font>
 
==AMI Storage for Root Device==
 
Every time an EC2 instance is launched, a root storage device is created for that instance. The root storage device contains all the information necessary to boot the instance. An AMI can be either [[#Amazon_Elastic_Block_Store_.28EBS.29|backed by Amazon EBS]] or [[#AMI_Storage_for_Root_Device|backed by instance state]]. Backed by Amazon EBS mens that the root device for the instance launched from the AMI is an Amazon EBS volume, created from an Amazon EBS snapshot. Backed by instance state means that the root device is an [[#Instance_Store|instance store volume]] created from a template stored in Amazon S3. For more details: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html#storage-for-the-root-device.
 
==AMI Operations==
* [[ec2-describe-images|Finding an AMI using EC2 API]]
* [[Amazon_EC2_Operations#Deregister_an_AMI|Deregister an Image]]


=Instance=
=Instance=
Line 25: Line 45:


* http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html
* http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html
* Instance Type Description http://aws.amazon.com/ec2/instance-types/
* Instance Type Description (contains Instance Types Matrix) http://aws.amazon.com/ec2/instance-types/
* How many instances can I run? http://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2
* How many instances can I run? http://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2


==Instance Metadata and User Data==
==EC2 Service IAM Role==
{{Internal|Amazon_AWS_Security_Concepts#AWS_Service_Role_for_an_EC2_Instance|AWS Service Role for an EC2 Instance}}
==Instance Profile==
{{Internal|Amazon_AWS_Security_Concepts#Instance_Profile|Instance Profile}}
 
==<span id='Instance_Metadata'></span>Instance Metadata and User Data==
{{External|[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html Instance Metadata and User Data]}}
 
===Instance Metadata Operations===
* [[Amazon_EC2_Operations#Use_Temporary_Credentials_Made_Available_via_Instance_Metadata|Use Temporary Credentials Made Available via Instance Metadata]]
* [[Amazon_EC2_Operations#Get_Metadata_Categories|Get Metadata Categories]]
* [[Amazon_EC2_Operations#Get_the_Instance.27s_AMI|Get the Instance's AMI]]
 
==Temporary Security Credentials for EC2 Instances==
{{External|[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials Retrieving Security Credentials from Instance Metadata]}}
EC2 instance applications using an [[Amazon_AWS_Security_Concepts#Instance_Profile|instance profile]] retrieve their [[Amazon_AWS_Security_Concepts#Temporary_Security_Credentials_for_EC2_Instances|temporary security credentials]] from the [[#Instance_Metadata|instance metadata]].
 
Credentials and related values are available from <code>iam/security-credentials/<role-name></code> category. The temporary security credentials that are available on the instance are automatically rotated before they expire so that a valid set is always available. The application just needs to make sure that it gets a new set of credentials from the instance metadata before the current ones expire.
==Instance Boot==
{{External|https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html}}
EC2 instances are configured at boot using [[Cloud-init|cloud-init]].
 
<font color=darkgray>TODO</font>
 
=Networking=
 
{{External|[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-network-iface-embedded.html EC2 NetworkInterface]}}
 
An instance specifies its networking by declaring a list of network interfaces. An example of how to specify networking as a CloudFormation resource is available here:
 
{{Internal|Amazon_EC2_Operations#Create_an_EC2_Instance_with_CloudFormation|Creating an EC2 Instance with CloudFormation}}
 
==Subnet==
 
{{Internal|Amazon_VPC_Concepts#Subnet|AWS VPC Concepts - Subnet}}
 
==Elastic IP and EC2 Instances==
 
* An EC2 instance '''cannot be reached over ssh''', even if it has an Elastic IP (public) address configured on it, if the subnet has no [[Amazon_VPC_Concepts#Internet_Gateway|internet gateway]] associated with it (no 0.0.0.0/0 route to an internet gateway)
 
==Security Groups==
 
It seems that a security group must be associated with the instance, even if it allows 0.0.0.0/0. In the case no security group was deployed, the instance could not have been accessed - maybe it was a different type of transient failure, but it seems that it's a good idea to stand up a security group anyway.
 
=Storage=
{{External|https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Storage.html}}
 
An EC2 instance has access to several sources of storage ([[#Amazon_Elastic_Block_Store_.28EBS.29|block storage devices provided by EBS]], [[#EC2_Instance_Store|storage that is physically attached to the host computer]] running the instance, a [[#Amazon_Elastic_File_System_.28EFS.29|file system provided by EFS]]), each with particular performance and reliability characteristics. EC2 instances can also use storage provided by [[#Amazon_Simple_Storage_Service_.28S3.29|S3]] to store backup snapshots of their primary storage devices and instance store-backed AMIs. These storage options can be used independently or in combination.
 
[[File:EC2StorageConcepts.png]]
 
==Block Device Mapping==
{{External|http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html}}
 
Each EC2 instance has a [[#Root_Device|root device]] volume, and [[#Additional_Storage_Volumes|additional]] [[Storage_Concepts#Block_Device|block device]] storage can be configured. Block devices can be either [[#Amazon_Elastic_Block_Store_.28EBS.29|ESB volumes]] or [[#Instance_Store|instance store volumes]]. A '''block device mapping''' defines the block devices to attach to an instance. The block device mapping can be specified as part of creating the AMI, so that the mapping is used by all instances launched from that AMI, or when the instance is started - in which case the mapping overrides the one specified in the AMI.
 
Virtual storage devices, representing cloud storage, or paravirtualized storage in general, are typically exposed in through /dev/xvd nodes. /dev/xvda1 is the first partition of the first such device.
 
===Root Device===
Every time an EC2 instance is launched from an AMI, a root storage is created for that instance. The root storage device contains all the information necessary to boot the instance. It is common to inspect the AMI and see the block device mapping specifies a /dev/sda device:
 
/dev/sda1=snap-066b8c74f34223b6d:10:true:gp2
 
but then an instance is created from that AMI, the root device is available as /dev/xda:
 
Filesystem      Size  Used Avail Use% Mounted on
...
dev/xvda2      10G  2.9G  7.2G  29% /
 
===Additional Storage Volumes===
In addition to the [[#Root_Device|root device]], other storage volumes can be specified when an AMI is created, or when the instance is launched. An Amazon-provided AMI includes a root device only; to add more block devices to an AMI, a custom AMI must be created.
 
===Block Device Mapping Operations===
* [[Amazon_EC2_Operations#Specify_the_Root_Device_as_Part_of_Creating_the_AMI|Specify the Root Device as Part of Creating the AMI]]
* [[Amazon_EC2_Operations#Specify_Additional_Block_Devices_as_Part_of_Creating_the_AMI|Specify Additional Block Devices as Part of Creating the AMI]]
* [[Amazon_EC2_Operations#Specify_Additional_Block_Devices_when_the_Instance_is_Started|Specify Additional Block Devices when the Instance is Started]]
 
==Amazon Elastic Block Store (EBS)==
{{External|https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html}}
 
Amazon EBS provides durable, block-level storage volumes that can be attached to an EC2 instance. EBS can be used as primary storage for data that requires frequent and granular updates, such as database storage. An EBS volume behaves like a raw, unformatted external block device that can be attached to a single EC2 instance and used like a regular physical hard drive. The volume persists independently from the running life of the instance. Multiple EBS volumes can be attached to the instance. An EBS volume can be detached from an instance and attached to another. EBS volumes can be encrypted using the [[Amazon_Elastic_Block_Store_Concepts#Encryption|Amazon EBS encryption feature]]. EBS volumes can be backed up by taking a [[Amazon_Elastic_Block_Store_Concepts#Snapshot|snapshot]] that can be stored in [[#Amazon_Simple_Storage_Service_.28S3.29|Amazon S3]]. Conversely, EBS volumes can be created from snapshots and attached to another instance. Also see [[#AMI_Storage_for_Root_Device|AMI Storage for Root Device]] above.
 
For more details, see: {{Internal|Amazon_Elastic_Block_Store_Concepts#EC2_and_EBS|Amazon Elastic Block Store}}
 
==<span id='Instance_Store'></span>EC2 Instance Store==
{{External|https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html}}
An EC2 instance can access storage from disks that are physically attached to the host computer running the EC2 instance. This storage is referred to as '''instance store'''. Instance store provides temporary block storage for instance. The data on an instance store volume persists only during the life of the associated instance. If the instance is stopped or terminated, the data on the instance store volumes is lost. Also see [[#AMI_Storage_for_Root_Device|AMI Storage for Root Device]] above.
 
==Amazon Elastic File System (EFS)==
EC2 instances can be configured to mount file systems exposed by Amazon Elastic File System (EFS). EFS file system can be used as common data source for workloads and applications running on multiple instances.
For more details, see:
{{Internal|Amazon_Elastic_File_System_Concepts#EFS_and_EC2|Amazon Elastic File System}}
 
==Amazon Simple Storage Service (S3)==
 
S3 can be used to store backup copies of data and applications, such as ECS snapshots and instance store-backed AMIs.
For more details, see {{Internal|Amazon_S3|Amazon S3}}
 
=Auto-Scaling=
 
{{Internal|Amazon EC2 Auto-Scaling Concepts|Auto-Scaling Concepts}}
 
=Load Balancing=
 
{{Internal|AWS Elastic Load Balancing Concepts|Load Balancing Concepts}}
 
=Key Pair=


* http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
{{External|[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html Amazon EC2 Key Pairs]}}


=Subnet=
An EC2 instance needs the name of a pre-existing key pair to secure ssh access to itself. A key pair is region-based.


A ''subnet'' is a range of IP addresses in the Virtual Private Cloud (VPC) that can be used to isolate different EC2 resources from each other or from the Internet. Each subnet resides in one Availability Zone. Subnets enable you to group instances based on security and operational needs. To enable instances in a subnet to reach the Internet and AWS services, you must add an Internet ''gateway'' to the VPC and a route table with a route to the Internet to the subnet.
==Key Pair Operations==


Example:
It does. not seem to be a way to created key pairs with CloudFormation.


<pre>
* [[Amazon_EC2_Operations#Create_a_Key_Pair_with_Amazon Console|Create a Key Pair with Amazon Console]]
subnet-53993c24: 172.31.16.0/20
</pre>

Latest revision as of 18:56, 16 March 2021

Internal

Overview

There are two Amazon Elastic Compute Cloud (EC2) platforms: EC2-VPC and EC2-Classic. EC2-VPC is aimed at launching AWS resources into a virtual private cloud (VPC). Also see Determining Whether You Are Using the EC2-VPC or EC2-Classic Platform.

Security Concepts

AMI (Amazon Machine Image)

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html

An Amazon machine image is a template that contains a software configuration (operating system, application server, application).

To browse through available AMIs go to the EC2 console -> Left Navigation Pane -> AMIs.

AMIs can be chosen based on region, operating system, architecture (32-bit or 64-bit), launch permissions and storage for root device.

AMI ID

AMI Name

Must be unique within account and region.

AMI Storage for Root Device

Every time an EC2 instance is launched, a root storage device is created for that instance. The root storage device contains all the information necessary to boot the instance. An AMI can be either backed by Amazon EBS or backed by instance state. Backed by Amazon EBS mens that the root device for the instance launched from the AMI is an Amazon EBS volume, created from an Amazon EBS snapshot. Backed by instance state means that the root device is an instance store volume created from a template stored in Amazon S3. For more details: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html#storage-for-the-root-device.

AMI Operations

Instance

From an AMI, you can launch an instance, which is a copy of the AMI running as a virtual server on a host computer in Amazon's data center. You can launch multiple instances from an AMI. An instance can be accessed using its public DNS name or public IP address. The DNS name follows the following pattern: ec2-<public_ip>.<region_code>.compute.amazonaws.com. An instance keeps running until it is explicitly stopped or until they fail. You can start from an existing AMI, log onto the instance, customize it with additional settings and software, and then save this customized image as a new AMI.

Instance Type

Each instance type offers different compute, memory, and storage capabilities and are grouped in instance families based on these capabilities. An example of instance type is t2.micro.

External Documentation:

EC2 Service IAM Role

AWS Service Role for an EC2 Instance

Instance Profile

Instance Profile

Instance Metadata and User Data

Instance Metadata and User Data

Instance Metadata Operations

Temporary Security Credentials for EC2 Instances

Retrieving Security Credentials from Instance Metadata

EC2 instance applications using an instance profile retrieve their temporary security credentials from the instance metadata.

Credentials and related values are available from iam/security-credentials/<role-name> category. The temporary security credentials that are available on the instance are automatically rotated before they expire so that a valid set is always available. The application just needs to make sure that it gets a new set of credentials from the instance metadata before the current ones expire.

Instance Boot

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html

EC2 instances are configured at boot using cloud-init.

TODO

Networking

EC2 NetworkInterface

An instance specifies its networking by declaring a list of network interfaces. An example of how to specify networking as a CloudFormation resource is available here:

Creating an EC2 Instance with CloudFormation

Subnet

AWS VPC Concepts - Subnet

Elastic IP and EC2 Instances

  • An EC2 instance cannot be reached over ssh, even if it has an Elastic IP (public) address configured on it, if the subnet has no internet gateway associated with it (no 0.0.0.0/0 route to an internet gateway)

Security Groups

It seems that a security group must be associated with the instance, even if it allows 0.0.0.0/0. In the case no security group was deployed, the instance could not have been accessed - maybe it was a different type of transient failure, but it seems that it's a good idea to stand up a security group anyway.

Storage

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Storage.html

An EC2 instance has access to several sources of storage (block storage devices provided by EBS, storage that is physically attached to the host computer running the instance, a file system provided by EFS), each with particular performance and reliability characteristics. EC2 instances can also use storage provided by S3 to store backup snapshots of their primary storage devices and instance store-backed AMIs. These storage options can be used independently or in combination.

EC2StorageConcepts.png

Block Device Mapping

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html

Each EC2 instance has a root device volume, and additional block device storage can be configured. Block devices can be either ESB volumes or instance store volumes. A block device mapping defines the block devices to attach to an instance. The block device mapping can be specified as part of creating the AMI, so that the mapping is used by all instances launched from that AMI, or when the instance is started - in which case the mapping overrides the one specified in the AMI.

Virtual storage devices, representing cloud storage, or paravirtualized storage in general, are typically exposed in through /dev/xvd nodes. /dev/xvda1 is the first partition of the first such device.

Root Device

Every time an EC2 instance is launched from an AMI, a root storage is created for that instance. The root storage device contains all the information necessary to boot the instance. It is common to inspect the AMI and see the block device mapping specifies a /dev/sda device:

/dev/sda1=snap-066b8c74f34223b6d:10:true:gp2

but then an instance is created from that AMI, the root device is available as /dev/xda:

Filesystem      Size  Used Avail Use% Mounted on
...
dev/xvda2       10G  2.9G  7.2G  29% /

Additional Storage Volumes

In addition to the root device, other storage volumes can be specified when an AMI is created, or when the instance is launched. An Amazon-provided AMI includes a root device only; to add more block devices to an AMI, a custom AMI must be created.

Block Device Mapping Operations

Amazon Elastic Block Store (EBS)

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html

Amazon EBS provides durable, block-level storage volumes that can be attached to an EC2 instance. EBS can be used as primary storage for data that requires frequent and granular updates, such as database storage. An EBS volume behaves like a raw, unformatted external block device that can be attached to a single EC2 instance and used like a regular physical hard drive. The volume persists independently from the running life of the instance. Multiple EBS volumes can be attached to the instance. An EBS volume can be detached from an instance and attached to another. EBS volumes can be encrypted using the Amazon EBS encryption feature. EBS volumes can be backed up by taking a snapshot that can be stored in Amazon S3. Conversely, EBS volumes can be created from snapshots and attached to another instance. Also see AMI Storage for Root Device above.

For more details, see:

Amazon Elastic Block Store

EC2 Instance Store

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html

An EC2 instance can access storage from disks that are physically attached to the host computer running the EC2 instance. This storage is referred to as instance store. Instance store provides temporary block storage for instance. The data on an instance store volume persists only during the life of the associated instance. If the instance is stopped or terminated, the data on the instance store volumes is lost. Also see AMI Storage for Root Device above.

Amazon Elastic File System (EFS)

EC2 instances can be configured to mount file systems exposed by Amazon Elastic File System (EFS). EFS file system can be used as common data source for workloads and applications running on multiple instances. For more details, see:

Amazon Elastic File System

Amazon Simple Storage Service (S3)

S3 can be used to store backup copies of data and applications, such as ECS snapshots and instance store-backed AMIs.

For more details, see

Amazon S3

Auto-Scaling

Auto-Scaling Concepts

Load Balancing

Load Balancing Concepts

Key Pair

Amazon EC2 Key Pairs

An EC2 instance needs the name of a pre-existing key pair to secure ssh access to itself. A key pair is region-based.

Key Pair Operations

It does. not seem to be a way to created key pairs with CloudFormation.