Why IT Ops Has Become Such a Rich Target for Big Data Analytics

(Gorodenkoff/Shutterstock)

In the business world, we measure what we hope to impact. This leads companies to devise all sorts of metrics from a wide assortment of data. Increasingly, one of the most impactful pieces of data we’re collecting — and thus hoping to impact through big data analytics — involves IT operations itself.

It’s difficult to fathom how extraordinarily big the IT sector has become. According to Gartner, nearly $3.7 trillion will be spent this year on information technology products and services, a 4.5% increase from 2017. Three thousand, seven hundred billion dollars is quite a bit of money, and in fact is equivalent to about 4.5% of all the money spent on planet Earth in 2018 (the new Mars rover notwithstanding).

By all accounts, we’re getting quite a bit of value out of this massive investment in technology. One only has to look at one’s smart phone and the array of Internet-connected services that it uses to get an inkling of the impact of technology. When one considers all servers, storage arrays, and networking gear powering applications in massive data centers sprinkled around the world, the extent of digitization boggles the mind.

It’s really no wonder, then, that we turn to technology to help tame this technological beast that we’ve created. This becomes especially true as the pace of IT development accelerates, particularly with today’s agile, machine learning fueled-development cycles that emphasize continuous improvement.

Several big data analytic suites are focused on getting the upper hand on the monitoring and management challenges that today’s fast-paced IT environment creates. The Elastic Stack is a flexible group of big data software that can be used for a range of things, but it’s often brought into an organization by the IT team to monitor logs generated by servers and networking gear. The story is similar over at Splunk, which has parlayed its success in simplifying the collection and analysis of machine data for operations and security teams into a wider artificial intelligence (AI) play. Sumo Logic, LucidWorks, and others are also plying these fields.

(Mr.B-king/Shutterstock)

Further up the stack, New Relic uses big data technology to give enterprises insights into the performance of their application stacks from the comfort of a shrink-wrapped cloud environment. SolarWinds, Datadog, Dynatrace, and others offer similar application performance management (APM) services with the promise of helping to simplify notoriously difficult IT tasks, such as tracking down the root causes of application failures.

Cloud vendors have also incorporated big data analytics into their monitoring schemes. Amazon Web Services offers services like X-Ray to help simplify the identification and resolution of performance problems for applications running on AWS. Other public cloud providers offer similar services.

Recently, Anodot announced that the ad tech firm AppNexus is using its machine learning software to detect and resolve events that could impact business in real-time. AppNexus manages more than 7 million ad impressions per second on behalf of its clients, so being able to spot problems quickly is both a priority and a challenge.

Automation is required to spot anomalies for AppNexus, which generates 250 TB of new data daily, according to David Drai, CEO and co-founder of Anodot. “Ad tech companies today are generating millions of key metrics across business and IT operations that must be tracked in real time,” Drai says in a press release. “Without machine learning to do this in an automated way, such mission critical monitoring and root cause analysis is impossible.”

Python is a popular language — not just for data science, but for general Web development too. Companies that have been struggling to instrument their Python applications for improved monitoring and management may be interested in utilizing Instana’s AI-powered APM offering to bring them into the fold.

With the new release, Instana is now able to monitor applications developed in Python, Java, Scala, Kotlin, Clojure, .NET and PHP without making any modifications. With just one line of code added, Instana customers can add Ruby, Crystal, Go and Node.js to the list of languages supported with its APM offering.

Another Web firm leveraging big data tech to improve hosted apps is Zoho. Companies relying on the plucky Dutch company for core applications like CRM will find analytics and chat-bot capabilities, like Zoho’s Zia, are being integrated into the software.

“Companies now recognise the importance of AI in creating positive experiences for customers,” said Brent Leary, co-founder and partner, CRM Essentials. “The focus now has shifted to bringing AI across an integrated platform, creating a consistently positive experience across and throughout the customer journey.”

As the amount of machine data generated by our IT inventions increases and the complexity goes up, so too does the necessity of using leverage in the form of advanced analytics and machine learning to make it usable and useful. That’s not likely to change in the near future.

Related Items:

Sumo Logic Broadens Reach Into Modern Machine Data

New Relic Unleashes the NERD to Analyze Raw Performance Data