Celery Autoscale

Push celery queue metrics on AWS cloudwatch for monitoring and autoscaling

Celery Autoscale is a service powered by AWS Lambda that runs every minute and collect total pending task from broker (Redis or RabbitMQ) put the metric (pendingTask) on Cloudwatch. These celery inspect support redis and rabbitMQ broker.

This metric can be used for following purpose:

- To trigger scale up and scale down activity for celery worker autoscaling group.

- To trigger alarm mails if pending task count goes too high.

By Pratik patel and Fahim Sakri