Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python. It enables developers to build software that uses Amazon services like EC2, S3, RDS, etc.

I will build a portable Python3.9 + Boto3 Docker environment to test some AWS automation tasks.

Build and run a docker with Python3.9 + Boto3

As I do not want to install Python, Boto3, and AWScli on my local PC, creating a Docker image with all software ready as a portable env is the best way to start.

root@ubt-server:~# vim Dockerfile
# Build from python:3.9.19-alpine3.19
From python:3.9.19-alpine3.19
# install boto3 and awscli
RUN pip install --upgrade pip && \
    pip install --upgrade awscli && \
    pip install --upgrade boto3
# set work dir
WORKDIR /work
# run Python
CMD "python"
# build a docker image from the above Dockerfile
root@ubt-server:~# docker image build -t zack_aws_boto3 .
# ls docker images
root@ubt-server:~# docker image ls
REPOSITORY               TAG             IMAGE ID       CREATED         SIZE
zack_aws_boto3           v1              07a13f7801ed   1 days ago     998MB
zackpy                   latest          287ba6873741   4 days ago     48.2MB
zackz001/gitops-jekyll   latest          d92894f7be6d   6 days ago     70.9MB
postgres                 15.0            027eba2e8939   19 months ago  377MB

# run docker and mount local python work dir
root@ubt-server:~/pythonwork# docker run -ti -v ${PWD}:/work zack_aws_boto3:v1 bash
root@c04670a43564:/#
root@c04670a43564:/# cd work && ls

# configure aws in the container
root@c04670a43564:/work# aws configure
AWS Access Key ID [****************GFNW]:
AWS Secret Access Key [****************Db7O]:
Default region name [ap-southeast-2]:
Default output format [None]:

# validate aws cred by listing ec2 instance id
root@c04670a43564:/work# aws ec2 describe-instances --query "Reservations[*].Instances[*].InstanceId" --output json
[
    [
        "i-076226daa5aaf7cf2"
    ]
]

Manage AWS resource with Python Boto3 script

Here we have Python and boto3 env ready; I will list some AWS tasks that I want to be achieved by Python scripts.

  • List EC2 instance name, instanceID, and state
root@ubt-server:~/pythonwork# vim app.py
# import boto3 library
import boto3

def list_ec2_instances():
    # Create a session using default AWS profile
    session = boto3.Session()
    # Create an EC2 client
    ec2_client = session.client('ec2')

    # Describe EC2 instances
    response = ec2_client.describe_instances()

    # Iterate over the instances
    for reservation in response['Reservations']:
        for instance in reservation['Instances']:
            # Get the instance ID
            instance_id = instance['InstanceId']

            # Get the instance state
            instance_state = instance['State']['Name']

            # Get the instance Name tag if exists
            instance_name = 'No Name'
            if 'Tags' in instance:
                for tag in instance['Tags']:
                    if tag['Key'] == 'Name':
                        instance_name = tag['Value']
                        break

            # Print instance ID, Name, and State
            print(f"Instance ID: {instance_id}, Name: {instance_name}, State: {instance_state}")

if __name__ == "__main__":
    list_ec2_instances()

root@c04670a43564:/work# python app.py
Instance ID: i-076226daa5aaf7cf2, Name: zack-blog, State: stopped
  • Filter EC2 instance without tag “owner”
# create app-untagged.py
root@ubt-server:~/pythonwork# vim app-untagged.py
import boto3

def get_untagged_ec2_instances():
    ec2_client = boto3.client('ec2')
    response = ec2_client.describe_instances()

    untagged_instances = []

    for reservation in response['Reservations']:
        for instance in reservation['Instances']:
            has_owner_tag = False
            if 'Tags' in instance:
                for tag in instance['Tags']:
                    if tag['Key'].lower() == 'owner':
                        has_owner_tag = True
                        break

            if not has_owner_tag:
                instance_id = instance['InstanceId']
                instance_state = instance['State']['Name']
                untagged_instances.append({'InstanceId': instance_id, 'State': instance_state})

    return untagged_instances

untagged_instances = get_untagged_ec2_instances()
print("Untagged Instances:", untagged_instances)

# run script to filter untagged "owner" ec2
root@c04670a43564:/work# python app-untagged.py
Untagged Instances: [{'InstanceId': 'i-076226daa5aaf7cf2', 'State': 'stopped'}]
  • Create lambda function to list EBS volume snapshots older than 30 days and delete them

To achieve this we need:

  1. Create lambda IAM role for lambda to manage EBS volume snapshot
  2. Create below Python lambda function
  3. Zip and upload zip function
  4. Create CloudWatch Event to Trigger and run it every 30 days
# create lambda function to delete snapshots older than 30 days
root@ubt-server:~/pythonwork# vim app-snapshot-older-30days.py
import boto3
from datetime import datetime, timezone, timedelta

def lambda_handler(event, context):
    ec2_client = boto3.client('ec2')

    # Get the current time
    now = datetime.now(timezone.utc)

    # Define the time threshold
    time_threshold = now - timedelta(days=30)

    # Describe snapshots
    snapshots = ec2_client.describe_snapshots(OwnerIds=['self'])['Snapshots']

    # Filter snapshots older than 30 days
    old_snapshots = [snap for snap in snapshots if snap['StartTime'] < time_threshold]

    # Delete old snapshots
    for snapshot in old_snapshots:
        snapshot_id = snapshot['SnapshotId']
        ec2_client.delete_snapshot(SnapshotId=snapshot_id)
        print(f"Deleted snapshot: {snapshot_id}")

    return {
        'statusCode': 200,
        'body': f"Deleted {len(old_snapshots)} snapshots."
    }

# zip Package for the Lambda Function
root@ubt-server:~/pythonwork# zip function.zip app-snapshot-older-30days.py
  • Email me when a security group allows inbound SSH (port 22) from everywhere (0.0.0.0/0)

To achieve this, we need:

  1. AWS CloudTrail enable
  2. Create CloudWatch Event Rule to capture AWS CloudTrail logs for security group changes
  3. Create below Lambda Function if inbound allows port 22 from everywhere are met
  4. Allow CloudWatch Events to Invoke the Lambda Function
  5. Add the Lambda Function as a Target for the CloudWatch Event Rule

Conclusion

There are many ways to automate AWS tasks using Python Boto3 script. Together with Lambda and trigger, many resource tasks can be scheduled and managed in a scripted way.