As virtualization, cloud, and mobility have moved into the mainstream, Ops teams have been forced into a reactive, firefighting role. Adding to the burden are other developments that look positive at first glance, such as the rise of open-source software and the emergence of DevOps, but which carry with them serious risks that threaten to undermine operational stability.
Ops teams still follow incident response workflows designed for dotcom-era production environments, and they typically have the personnel that would map to that circa-2000 reality. Ops budgets have also remained flat (or have decreased), which further erodes the Ops team’s ability to monitor, maintain, secure, and manage all of the moving parts that make up today’s complex IT infrastructures.
How outdated incident response workflows put your organization at risk
The major risk that computing and app sprawl introduces, downtime, is more costly than most would guess.
Research from IDC research conducted last fall found that unplanned application downtime costs the Fortune 1000 between $1.25 billion and $2.5 billion every year.
If the operations team can’t distinguish signal from noise, since they must cope with numerous monitoring tools, unplanned application downtime is an inevitability. A seemingly small alert could be warning you that some new configuration has made an app unstable, but if you don’t realize that, it could be a very short timebefore it becomes entirely unresponsive.
But the problem is even bigger than that. IDC is just pinpointing the downtime from known apps.
Apps are the new normal for how people and businesses consume information. Whether we’re talking about smartphone apps or cloud-based enterprise apps, one thing remains constant: they’re spiraling out of control.
Think about the last time you upgraded your mobile phone. How many apps did you decide not to move over because you used them a few times and then abandoned them? According to a recent study, twenty percent of mobile apps are used only once before being abandoned. Over time, nearly 98 percent of mobile apps are eventually abandoned.
If you think this is a separate issue confined to consumers, think again. Your marketing department can deploy cloud-based, single-purpose marketing apps in an instant. Your sales force may rely on lead-generation tools you never even heard of, and your developers could have various testing workloads in AWS or Rackspace or who knows where.
The risks from all of these poorly monitored, unmanaged, unsecured apps is huge. Even for mobile users, app abandonment poses risks from location tracking to identity theft to the mining of your personal data for nefarious uses. For the enterprise, apps that exist outside of the vision of IT could expose you to a data breach, open you up to IP theft, or put you out of compliance with industry regulations.
To cope with all of this rapid change, Ops teams need to map out new workflows based on cloud-era tools that separate the signal from the noise. Instead, the status quo has Ops teams suffering from alert fatigue, which means they’re pretty much flying blind.
We’ll discuss how Ops can start moving forward in our next post, but, in the meantime, a good first step is to find solutions, such as BigPanda, which will help you separate the high-level incidents from the constant flow of noisy alerts, delivering the instant insights necessary to start triage efforts.