New AWS Custom Integrations for Cloud-Native Service Ops

By |2018-04-17T17:49:58+00:00November 16th, 2017|Blog|

One of the best parts of working in the Technology industry is attending conferences and feeling the collective excitement of the attendees. The crowd shares a united sense of hope and confidence that the breakthrough technology for whatever problems we’ve been struggling with is just a keynote away. Sure, today you can watch live streams or recordings of conference sessions from the comfort of your home or office; it’s just not the same as being there. That’s why so many of us still hop on planes.

I used to get this kind of positive energy from VMworld a decade ago, back when the magic of seeing VMotion for the first time was still a recent memory. Virtualization is just not as compelling as it used to be. OpenStack was fun for a few years… until legacy vendors ruined the party. People realized the futility of trying to keep pace with the public cloud providers. Fortunately there is a fresh new source of techie enthusiasm at Amazon Web Service’s annual conference – AWS re:Invent – which is taking over Las Vegas again this year.

The BigPanda team is excited to be attending re:Invent, so we wanted to do our part to crank up the energy level just a little more…

BigPanda is announcing a new set of AWS Custom Integrations

These new AWS custom integrations will allow our Autonomous Service Operations platform to connect even more deeply with AWS-based infrastructure and applications. They complement our existing Integrations catalog, which now counts native connections to 60+ best-of-breed products. Included are integrations with popular enterprise IT monitoring tools, application performance monitoring solutions, on-premise network monitoring tools, and legacy workload monitoring systems.

Here are our three new AWS custom integrations:

Custom Analytics with ETL Export to S3, Athena or Redshift + Reporting with QuickSight

Enterprise IT organizations that are already working with AWS for data analytics and operational intelligence, pay attention. You can now directly integrate BigPanda’s weekly ETL data feeds of alert, incident and activity data to the most popular AWS analytics data stores: S3 buckets, Athena and Redshift. From here, our customers combine BigPanda’s monitoring analytics data with other critical operational data warehouse sources to produce more actionable insights. They monitor things like service health, SLA compliance, major incident workflow, NOC team productivity, and mean-time-to-resolution (MTTR) trends. To make it even easier to visualize and communicate these insights, BigPanda charts and report templates for AWS Quicksight can be accessed and used directly from your AWS account.

Alert routing to BigPanda via Your SNS Topics, CloudWatch Event Bus or Log Subscriptions

We already have easy integration with CloudWatch Alarms. Now we’re introducing three new ways to send alerts to BigPanda: Custom SNS Topics, CloudWatch Event Bus (cross-account) and CloudWatch Log Subscriptions (cross-account). These options make it even easier to integrate BigPanda into the fabric of your existing AWS applications, services and infrastructure. Customers can forward alerts to BigPanda from messages sent to an existing SNS topic, from events routed by rules in CloudWatch Event Bus, or from log lines processed by a CloudWatch Logs Lambda subscription. All without configuring any new alarms or metrics.

Ability to Respond to Correlated Incident Notifications from BigPanda with Your Lambda Functions or SNS Topics

BigPanda first correlates multiple related alerts or events into a single incident. A notification with all the details can then be shared with ticketing systems like ServiceNow, on-call notification tools like PagerDuty, chat apps like Slack, or even just basic channels like email or text messaging. This week we’re adding two new custom outbound integrations to the list: Lambda Functions and Custom SNS Topics. These give AWS users some amazing new capabilities to automate their incident response processes. Benefits include improved detection, assisted triage, problem identification and even auto-remediation.

If you’re not clear on what possibilities that opens up, let’s look at a real-world example…

It’s 3:26 AM. A Java Virtual Machine heap issue on your application server instances triggers 12 different alerts in Splunk, CloudWatch and New Relic. BigPanda correlates those alerts into a single incident and sends the details to a customer-managed Lambda function. The Lambda function inspects the payload and then kick off a Step Function to run a diagnostic command on the affected instances using EC2 System Manager. The output of the command confirms that it’s a known issue, so the Step Function updates the incident in BigPanda and runs another command to do a rolling restart of the JVM processes. As things return to normal, the Splunk, CloudWatch and New Relic alerts clear in BigPanda and the incident is automatically resolved. Meanwhile, the Ops engineer on-call gets to continue sleeping!

Interested in hearing more? Visit us at AWS re:Invent!

We’ll be in exhibit booth 2220 at the re:Invent Expo at the Venetian. Come by and hear how we can bring our advanced correlation, workflow automation and event management capabilities to your cloud operations service stack.

About the Author:

Chris is SE Director at BigPanda.