⬡ Hub
Skip to content

AWS EC2 (Elastic Compute Cloud)

Detailed Content

Amazon EC2 (Elastic Compute Cloud) is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.

Core Concepts

  • Instances: A virtual server in the AWS cloud. You can choose from a variety of instance types with different CPU, memory, storage, and networking capacities. Each instance runs an operating system of your choice (Linux, Windows, etc.) and can be configured with various software.
  • Amazon Machine Images (AMIs): A pre-configured template for your instances that includes the operating system, application server, and applications. AMIs are regional resources. You can use AMIs provided by AWS (e.g., Amazon Linux 2, Ubuntu), the AWS Marketplace (third-party AMIs), or create your own custom AMIs from existing instances.
  • Instance Types: EC2 provides a wide variety of instance types optimized for different use cases, offering a balance of compute, memory, and networking capabilities. These are categorized into families:
    • General Purpose (e.g., T-series, M-series): Provide a balance of compute, memory, and networking resources. Ideal for web servers, small to medium databases, and development environments. (e.g., t3.medium, m5.large)
    • Compute Optimized (e.g., C-series): Offer high-performance processors. Best for compute-intensive applications like batch processing, high-performance computing (HPC), scientific modeling, and gaming servers. (e.g., c5.xlarge)
    • Memory Optimized (e.g., R-series, X-series): Designed for workloads that process large datasets in memory. Suitable for high-performance databases, distributed web scale in-memory caches, and big data analytics. (e.g., r5.2xlarge, x1e.xlarge)
    • Storage Optimized (e.g., I-series, D-series, H-series): Feature high-performance local storage. Ideal for I/O-intensive applications like NoSQL databases (Cassandra, MongoDB), data warehousing, and distributed file systems. (e.g., i3.large, d2.xlarge)
    • Accelerated Computing (e.g., P-series, G-series, F-series): Utilize hardware accelerators (GPUs, FPGAs) to perform functions, such as floating-point number calculations, graphics processing, or data pattern matching, more efficiently than software running on CPUs. Ideal for machine learning, high-performance computing, and video encoding. (e.g., p3.2xlarge, g4dn.xlarge)
  • Pricing Models: Understanding EC2 pricing models is crucial for cost optimization.
    • On-Demand Instances:
      • Description: Pay for compute capacity by the hour or second (minimum 60 seconds) with no long-term commitments.
      • Use Cases: Ideal for applications with short-term, spiky, or unpredictable workloads that cannot be interrupted. Good for development and testing environments.
    • Spot Instances:
      • Description: Request spare EC2 computing capacity for up to 90% off the On-Demand price. Instances can be interrupted by AWS with a two-minute notification if AWS needs the capacity back.
      • Use Cases: Ideal for fault-tolerant, flexible applications, batch jobs, data analysis, and other workloads that can tolerate interruptions.
    • Reserved Instances (RIs):
      • Description: Provide a significant discount (up to 75%) compared to On-Demand pricing in exchange for a commitment to a one- or three-year term. RIs are not actual instances but a billing discount applied to On-Demand instances.
      • Types:
        • Standard RIs: Offer the most significant discount but are less flexible (instance type, OS, tenancy are fixed).
        • Convertible RIs: Offer a lower discount than Standard RIs but allow you to change instance family, type, OS, or tenancy over the commitment term.
      • Use Cases: For applications with steady-state or predictable usage.
    • Savings Plans:
      • Description: A flexible pricing model that offers lower prices on EC2 and Fargate usage (up to 72% off On-Demand) in exchange for a commitment to a consistent amount of usage (measured in $/hour) for a 1 or 3 year term. More flexible than RIs as they automatically apply to any instance usage across a region, regardless of instance family, size, OS, or tenancy.
      • Types:
        • Compute Savings Plans: Apply to EC2 instance usage regardless of instance family, size, OS, or region, and also to Fargate or Lambda usage.
        • EC2 Instance Savings Plans: Apply to specific instance families in a given region.
      • Use Cases: For applications with predictable usage, but with the flexibility to change instance families and types over time.
  • Storage:
    • Elastic Block Store (EBS): Persistent block storage volumes for use with EC2 instances. EBS volumes are network-attached and persist independently from the life of an instance. They can be backed up using snapshots.
      • Types: General Purpose SSD (gp2/gp3), Provisioned IOPS SSD (io1/io2), Throughput Optimized HDD (st1), Cold HDD (sc1), Magnetic (standard).
    • Instance Store: Temporary block-level storage for your instances. Data on an Instance Store is lost when you stop, terminate, or hibernate the instance. It's physically attached to the host machine.
  • Security:
    • Security Groups: Act as a virtual firewall for your instances to control inbound and outbound traffic. They operate at the instance level. You define rules to allow or deny traffic based on protocol, port range, and source/destination IP addresses.
    • Network Access Control Lists (NACLs): Operate at the subnet level and act as a stateless firewall, controlling traffic in and out of subnets.
    • Key Pairs: Used to securely connect to your instances (e.g., SSH for Linux, RDP for Windows). AWS stores the public key, and you store the private key.
    • IAM Roles: Best practice for granting permissions to applications running on EC2 instances. Instead of embedding AWS credentials, you assign an IAM role to an instance, and applications on that instance can assume the role to get temporary credentials.
  • Networking:
    • Virtual Private Cloud (VPC): A logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define.
    • Subnets: A range of IP addresses in your VPC. You can launch instances into public or private subnets.
    • Elastic IP Addresses (EIPs): Static, public IPv4 addresses designed for dynamic cloud computing. An EIP is associated with your AWS account, not a specific instance, and you can re-map it to another instance in your account.
    • Network Interfaces (ENIs): A logical networking component in a VPC that represents a virtual network card. You can attach multiple ENIs to an instance.

Use Cases

  • Web Application Hosting: Host dynamic websites and web applications, using Auto Scaling to handle variable traffic and Elastic Load Balancing to distribute requests.
  • Backend Processing: Run backend services for mobile applications, APIs, or microservices architectures.
  • Batch Processing and Big Data: Use fleets of Spot Instances for cost-effective batch processing, data analysis, and big data workloads with frameworks like Hadoop and Spark.
  • High-Performance Computing (HPC): Leverage Compute Optimized and Accelerated Computing instances for scientific simulations, financial modeling, and other computationally intensive tasks.
  • Machine Learning: Train and deploy machine learning models using GPU-powered instances for accelerated performance.
  • Development and Testing: Quickly provision and de-provision isolated environments for development, testing, and continuous integration/continuous deployment (CI/CD) pipelines.
  • Disaster Recovery: Maintain backup instances in different AWS regions for disaster recovery, ready to be launched in case of a primary site failure.
  • Enterprise Applications: Host enterprise software like SAP, Oracle, and Microsoft applications, often using Dedicated Hosts to meet licensing requirements.

Auto Scaling

EC2 Auto Scaling helps you ensure that you have the correct number of Amazon EC2 instances available to handle the load for your application. You create collections of EC2 instances, called Auto Scaling groups. You can specify the minimum number of instances in each Auto Scaling group, and Auto Scaling ensures that your group never goes below this size. You can specify the maximum number of instances in each Auto Scaling group, and Auto Scaling ensures that your group never goes above this size.

Interview Questions

Conceptual Questions

  1. What is the difference between an AMI and an instance?
    • An AMI is a pre-configured template for an EC2 instance, containing the operating system, application server, and applications. An instance is a running virtual server launched from an AMI. Think of an AMI as a cookie cutter and an instance as the cookie.
  2. Explain the different EC2 pricing models and when you would use each.
    • On-Demand: For applications with short-term, spiky, or unpredictable workloads that cannot be interrupted. No upfront commitment.
    • Spot Instances: For applications that have flexible start and end times, or that are tolerant of interruptions. Offers significant cost savings (up to 90% off On-Demand).
    • Reserved Instances (RIs): For applications with steady-state or predictable usage over a 1 or 3-year term. Provides substantial discounts.
    • Savings Plans: A more flexible pricing model than RIs, offering lower prices on EC2 and Fargate usage in exchange for a consistent usage commitment (measured in $/hour) for 1 or 3 years. Allows for changes in instance family, size, OS, and region.
  3. How do you secure an EC2 instance?
    • Security Groups: Act as a virtual firewall to control inbound and outbound traffic at the instance level.
    • Network Access Control Lists (NACLs): Stateless firewalls that control traffic at the subnet level.
    • IAM Roles: Assign roles to EC2 instances to grant temporary permissions to AWS services, avoiding the need to store credentials on the instance.
    • Key Pairs: Used for secure SSH access to Linux instances and RDP access to Windows instances.
    • Patch Management: Regularly update the operating system and applications on the instance.
    • VPC: Launch instances in a private VPC with appropriate network configurations.
  4. What is the difference between EBS and Instance Store?
    • EBS (Elastic Block Store): Persistent, network-attached block storage. Data persists independently of the instance's lifecycle. Ideal for primary storage for databases, boot volumes, etc.
    • Instance Store: Ephemeral, physically attached block storage. Data is lost when the instance is stopped, terminated, or hibernated. Suitable for temporary storage, caches, or scratch data.
  5. Explain the concept of an Elastic IP address and its use cases.
    • An Elastic IP (EIP) is a static, public IPv4 address designed for dynamic cloud computing. It's associated with your AWS account, not a specific instance. You can remap an EIP to another instance in your account. Use cases include masking instance failures (by remapping the EIP to a healthy instance) or providing a consistent public IP for services that require it.
  6. What are the different types of EC2 instance tenancy?
    • Shared (Default): Multiple AWS accounts may share the same physical hardware.
    • Dedicated Instance: Your instance runs on hardware that's dedicated to a single AWS account. Other instances in your account might share the same hardware.
    • Dedicated Host: A physical server with EC2 instance capacity fully dedicated to your use. Gives you more control over instance placement and visibility into the underlying hardware. Useful for licensing requirements.

Scenario-Based Questions

  1. You have a web application that experiences traffic spikes at certain times of the day. How would you use EC2 to handle this efficiently and cost-effectively?
    • I would use an Auto Scaling Group to automatically scale the number of EC2 instances up or down based on demand. I would configure scaling policies based on metrics like CPU utilization, network I/O, or custom metrics from application load balancers. To handle the traffic distribution, I would place the Auto Scaling Group behind an Elastic Load Balancer (ELB). For cost-effectiveness, I might consider using a mix of On-Demand and Spot Instances within the Auto Scaling Group for fault-tolerant components.
  2. You need to launch a fleet of EC2 instances for a batch processing job that can be interrupted without data loss. What is the most cost-effective way to do this, and what considerations would you have?
    • I would primarily use Spot Instances due to their significant cost savings. Since the job can be interrupted, Spot Instances are ideal. I would use a Spot Fleet or an Auto Scaling Group with Spot Instances to manage the fleet. Considerations would include:
      • Checkpointing: Ensure the batch job can save its progress periodically to persistent storage (e.g., S3, EBS) so it can resume from the last checkpoint if an instance is interrupted.
      • Diversification: Request different instance types and Availability Zones to increase the likelihood of obtaining and maintaining Spot capacity.
      • Spot Interruption Handling: Implement logic to gracefully handle Spot instance interruptions (e.g., by listening for the two-minute warning).
  3. You are designing a highly available and fault-tolerant architecture for a critical application. How would you use EC2 and related services to achieve this?
    • I would deploy EC2 instances across multiple Availability Zones (AZs) within a region to protect against single points of failure. An Elastic Load Balancer (ELB) (Application Load Balancer or Network Load Balancer) would distribute incoming traffic across instances in different AZs. Auto Scaling Groups would ensure the desired number of healthy instances are running and automatically replace unhealthy ones. Data would be stored on EBS volumes (for persistent block storage) or in highly available databases (like RDS Multi-AZ). Route 53 would be used for DNS failover if needed across regions.
  4. Your application requires consistent, high-performance storage for a database. Which EC2 storage option would you choose and why?
    • I would choose Elastic Block Store (EBS), specifically a Provisioned IOPS SSD (io1/io2) volume. EBS provides persistent block storage that can be attached to an EC2 instance. Provisioned IOPS SSD volumes are designed for I/O-intensive workloads like databases that require consistent and high performance. I would also consider using EBS Multi-Attach if multiple instances need concurrent read/write access to a single volume (though this is less common for primary database storage and more for shared file systems or cluster applications).
  5. You need to migrate an on-premises application to AWS EC2. The application has strict licensing requirements that tie software licenses to physical CPU cores. How would you address this in AWS?
    • I would use Dedicated Hosts. Dedicated Hosts provide physical EC2 servers fully dedicated to your use, giving you visibility into the underlying sockets and physical cores. This allows you to meet licensing requirements that are tied to physical hardware, as you can bring your existing per-socket or per-core software licenses to AWS.

Coding/CLI Examples

Here are some common EC2 operations using the AWS CLI and Python (Boto3).

AWS CLI Examples

  1. Launch an EC2 instance: bash aws ec2 run-instances \ --image-id ami-0abcdef1234567890 # Replace with a valid AMI ID for your region --instance-type t2.micro \ --key-name MyKeyPair \ --security-group-ids sg-0abcdef1234567890 \ --subnet-id subnet-0abcdef1234567890 \ --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=MyWebServer}]'

    • --image-id: Specifies the Amazon Machine Image (AMI) to use.
    • --instance-type: Defines the hardware configuration of the instance.
    • --key-name: The name of the key pair to use for SSH access.
    • --security-group-ids: One or more security group IDs to associate with the instance.
    • --subnet-id: The ID of the subnet in which to launch the instance.
    • --tag-specifications: Adds tags to the instance for identification and cost allocation.

  2. Stop an EC2 instance: bash aws ec2 stop-instances --instance-ids i-0abcdef1234567890

    • --instance-ids: The ID of the instance to stop.

  3. Start an EC2 instance: bash aws ec2 start-instances --instance-ids i-0abcdef1234567890

  4. Terminate an EC2 instance: bash aws ec2 terminate-instances --instance-ids i-0abcdef1234567890

  5. Describe EC2 instances (list instances): bash aws ec2 describe-instances \ --filters "Name=instance-state-name,Values=running" \ --query 'Reservations[*].Instances[*].{InstanceId:InstanceId,InstanceType:InstanceType,State:State.Name,LaunchTime:LaunchTime,PublicIpAddress:PublicIpAddress}' \ --output table

    • --filters: Filters the results (e.g., only show running instances).
    • --query: Uses JMESPath to format the output and select specific fields.
    • --output table: Displays the output in a human-readable table format.

  6. Create a custom AMI from a running EC2 instance: bash aws ec2 create-image --instance-id i-0abcdef1234567890 --name "MyWebServerAMI" --description "Custom AMI for web server"

Python (Boto3) Examples

First, ensure you have Boto3 installed (pip install boto3) and your AWS credentials configured.

  1. Launch an EC2 instance: ```python import boto3

    ec2 = boto3.client('ec2')

    try: response = ec2.run_instances( ImageId='ami-0abcdef1234567890', # Replace with a valid AMI ID InstanceType='t2.micro', MinCount=1, MaxCount=1, KeyName='MyKeyPair', SecurityGroupIds=[ 'sg-0abcdef1234567890', # Replace with your security group ID ], SubnetId='subnet-0abcdef1234567890', # Replace with your subnet ID TagSpecifications=[ { 'ResourceType': 'instance', 'Tags': [ { 'Key': 'Name', 'Value': 'MyWebServerBoto3' }, ] }, ] ) print(f"Launched instance: {response['Instances'][0]['InstanceId']}") except Exception as e: print(f"Error launching instance: {e}") ```

  2. List all running EC2 instances: ```python import boto3

    ec2 = boto3.client('ec2')

    try: response = ec2.describe_instances( Filters=[ { 'Name': 'instance-state-name', 'Values': ['running'] }, ] )

    for reservation in response['Reservations']:
    for instance in reservation['Instances']:
        instance_id = instance['InstanceId']
        instance_type = instance['InstanceType']
        state = instance['State']['Name']
        public_ip = instance.get('PublicIpAddress', 'N/A')
        name_tag = 'N/A'
        for tag in instance.get('Tags', []):
            if tag['Key'] == 'Name':
                name_tag = tag['Value']
                break
        print(f"Name: {name_tag}, ID: {instance_id}, Type: {instance_type}, State: {state}, Public IP: {public_ip}")

    except Exception as e: print(f"Error describing instances: {e}") ```

  3. Stop an EC2 instance: ```python import boto3

    ec2 = boto3.client('ec2') instance_id = 'i-0abcdef1234567890' # Replace with your instance ID

    try: response = ec2.stop_instances(InstanceIds=[instance_id]) print(f"Stopping instance: {instance_id}") except Exception as e: print(f"Error stopping instance: {e}") ```

  4. Create an EBS volume and attach it to an instance: ```python import boto3

    ec2 = boto3.client('ec2') instance_id = 'i-0abcdef1234567890' # Replace with your instance ID availability_zone = 'us-east-1a' # Replace with the AZ of your instance

    try: # Create volume volume_response = ec2.create_volume( AvailabilityZone=availability_zone, Size=10, # 10 GiB VolumeType='gp3', TagSpecifications=[ { 'ResourceType': 'volume', 'Tags': [ { 'Key': 'Name', 'Value': 'MyNewEBSVolume' }, ] }, ] ) volume_id = volume_response['VolumeId'] print(f"Created volume: {volume_id}")

    # Wait for volume to be available
waiter = ec2.get_waiter('volume_available')
waiter.wait(VolumeIds=[volume_id])
print(f"Volume {volume_id} is available.")

# Attach volume to instance
attach_response = ec2.attach_volume(
    Device='/dev/sdf',  # Or /dev/xvdf, /dev/sdg, etc. (Linux)
    InstanceId=instance_id,
    VolumeId=volume_id
)
print(f"Attached volume {volume_id} to instance {instance_id}")

    except Exception as e: print(f"Error with EBS volume operations: {e}") ```