EC2 Runtime Alerts to Reduce AWS Costs

Managing compute resources in an academic lab or small organization can be challenging, especially when multiple people are running jobs and responsibility for cost management is dispersed. When using AWS EC2 instances, it’s surprisingly easy to forget to shut off an instance after a job has completed, leading to unnecessary expenses.

I learned this lesson the hard way when I accidentally left an EC2 instance running over winter break, during which it accumulated a couple thousand dollars in compute costs. Ouch. To prevent such scenarios from recurring, I got busy building a runtime alerts solution.

The result of my efforts was an automated EC2 runtime alert system. This system sends email notifications to users when their instances have been running beyond a specified time threshold, helping to prevent forgotten instances from silently accumulating charges.

This system is now being used successfully at my academic lab, the Sculpting Evolution Group, and at SecureBio, the nonprofit where I previously worked.

The system uses an AWS Lambda function triggered daily by Amazon EventBridge. Here’s how it works:

1. The Lambda function queries all running EC2 instances using boto3.
2. It calculates the runtime for each instance by comparing the current time to the launch time.
3. If an instance exceeds the threshold (e.g., 48 hours), the system sends an alert.
4. Alerts are sent via Amazon Simple Email Service (SES) to instance owners (identified by an “Email” tag on the EC2 instance) and designated admin emails.
5. Instances tagged with “AlwaysOn: True” are excluded from alerts.

Code and additional implementation details can found in this Github repo.

Possible extensions of this system include:

• CPU usage alerts where alerts are triggered when when CPU usage falls below a threshold for a specified duration, indicating potential idle instances.
• Auto-shutdown options, although these should be handled with much care and communication.
• Integration with Slack or other messaging platforms for more immediate notifications.