Query AWS CloudWatch logs and trigger SNS Alarms for error handling

Catch errors and send email alert to instantly get notified on an error in AWS lambda.

What is CloudWatch?

AWS Cloudwatch provides us with the ease of monitoring and logging to get insights for all our AWS resources. It is an ultimate place to configure and observe the behavior of AWS services in terms of performance, resource utilization and even catching undesirable output, which is useful in our case.

I worked on a feature which was event driven and triggered via API call which was using AWS Lambda with AWS CloudWatch as our primary logging and analysis tool. We wanted to get notified as soon as a particular error is thrown during the testing phase. That’s where I used AWS CloudWatch and the Metric filters.

The solution was to query CloudWatch logs for the particular lambda’s log group and set an alarm which fires an email through Simple Notification Service.

Architecture to the explained solution with AWS Lambda, Cloudwatch & SNS service.

Prerequisites

1. AWS Subscription
2. AWS Lambda with a suitable test template. (You can copy it from here if you want)

I am assuming that you are familiar with AWS services and console.

Once you are done with your lambda function (configured at standard settings and deployed), try testing your function (with empty JSON event: {}), it should throw an error. Now, we can start by adding a suitable alarm to catch this error.

Implementation and Configuration

1. Go to View CloudWatch logs button under Monitor tab, which redirects you to the cloudwatch log group for your particular lambda.

First button on the left redirects to function logs

This will show all the log streams that are created for each time the function is triggered. You can find your test trigger at the top and open it to find the error logged with whole description.

2. Click on Actions dropdown to find Create metric filter option.

Click to create a metric filter here.

A metric filter will help you define type of log to filter out and work with, ERROR being the type in our case.

This will take you a Define Pattern screen where you configure the metric.
a) Select Filter pattern as ERROR to match the term from the logs

b) Check if the caught log is same as the error we are catching (this can be used for all the errors) by selecting the log from dropdown under Test Pattern heading.

c) Click on next and write a name for your metric filter. For our simplicity, we will keep the following configuration with whole values and no units.

The namespace and name will help you find and observe the anomaly later in cloudwatch.

d) Review and create.

Once you do that, it will bring you back to Metric filters tab in CloudWatch logs for your function.

From here you can click on TestError (metric filter name).

Metric filter creation takes time and may not appear on CloudWatch metrics list when you click on the metric filter name (in our case that is TestError).

Create an alarm and SNS topic

In the previous image, there is a small tick box on the top right corner of the card. Check that, and the “Create Alarm” button should get active.

This will take you to step wise alarm configuration window.

1. Choose Statistics as Maximum. This allows you to catch the very first error by extracting the maximum value of the metric. You can select the period of observation in which it is supposed to poll the value. (Try setting it to include time when it threw error in testing and observe the graph)
2. Configure the conditions for error detection. You can select a whole value condition: static or use a range that you would want to flag as error: anomaly detection.

As soon as the value reaches 1 or more (count of errors), alarm is triggered.

3. In the next step, AWS allows you to create SNS alarm in this configuration. Select create new topic, add name and email Ids that subscribe to this topic for notification.

The receivers need to confirm to join the topic, they receive an email from AWS for this confirmation, once the alarm is created.

4. Next you just simply add the body in markdown and you are good to go!

Now that you have created the alarm, let’s catch an error in “In alarm” category.

Go ahead and test your function which will throw an error. Observe the spike in output of metric filters by going to CloudWatch logs -> In alarm.

This might take a while to update. If you do not find it “in alarm” category, go to Alarms, find your alarm and observe the behavior of the metric once to find if you missed something out.

There you have it, your own SNS topic connected alarm, to notify you on every and any error in your AWS Lambda!

Hope you liked it, connect with me here: Aishani Pachauri | LinkedIn

More bite-sized content: Aishanii — DEV Community