Elastic Eyes Security and APM for Expansion Effort

Shay Banon at Elastic{ON} 2018

IT administrators have long relied on the Elastic Stack to gain insight into the behavior of their servers and networks. Now the company behind the popular suite of tools is headed up the stack to take on big data challenges related to application performance management (APM) and security.

Elastic has ridden a wave of popularity in a way that’s unique in software the industry. Buoyed by 350 million downloads of the open source Elastic Stack, the company raised $250 million in an IPO last October. Last week, Elastic reported its financial results for the second time: $71 million of revenue, with a GAAP net loss of nearly $24 million.

Clearly, the company – not out of its first decade — is still in growth mode. While Elastic no longer shares the number of downloads of its open source stack, it’s clearly growing quickly. It grew from 5,500 paying customers to 7,200 as of last week, and now it’s looking for ways to expand its reach into other areas.

One of the ways Elastic wants to expand its footprint is by widening to do that is by getting into the APM and security businesses.  Elastic already provides some APM capabilities in its Elastic Stack, which is composed of Elastic Search, Logstash, Kibana, and Beats. But according to Elastic founder and CEO Shay Banon, the APM functionality is going to get much better in future releases.

“We’re in the process of building the best APM product out there,” Banon told Datanami last week in an interview. “We’re making progress in our APM implementation that I’m very happy with. Our vision is that APM is just another pillar of what is called the observability movement of logs and metrics.”

Elastic started out as a full text search engine, but it quickly morphed into something more. It built a solid reputation for its capability to quickly ingest massive amounts of log data from servers, storage, and network devices, and allowing customers to visualize and query that data.

APM is a natural offshoot to logging cases, Banon said. The company’s APM offering currently supports agents in most major programming languages, with the exception of .NET, which he said is coming soon.

“We have support for low-level features to high level features, like support for distributed tracing and supporting various standards like OpenTracing and W3C’s Trace Context and everything around that,” he said. “We’re in it for APM and we’re in it because we think APM and log metrics are all going to be part of the same story in the future. That’s an area were investing at and it’s an area we think we can make a difference in users’ lives.”

Similarly, Elastic is looking to branch out into security use cases. The company already supports some security capabilities, and many Elastic Stack customers have used to tools in support of security use cases. But Banon says Elastic wants to be move more definitively into building a business around APM and security.

Elastic has already entered into a partnership with SEIM vendor ArcSight, a subsidiary of Micro Focus, to bolster its credentials in the security information and event management (SIEM) space. The two companies built an integration whereby security data from ArcSight is processed using Logstash, ingested into Elasticsearch with a pre-defined index pattern, and then visualized in Kibana via pre-built dashboards.

“A lot of times the difference between logs and SIEM is very small and we already do logging well,” Banon sad. “The fact that we’re fast is so instrumental, as you can imagine in the security space.  If somebody is hacking into your system, the quicker you can find them, the better.”

Enterprises need a fast system to process all of the security data, Banon said. “As security keeps growing, you need a system that can handle these large amounts of data, and existing SIEM solutions were not built to hand large volumes of data, especially in the context of still being able to provide fast results. We’re in the process of trying to understand what we want to do there and try to productionize the various integrations and tools that have been built in that context.”

In comments with financial analysts last week, Banon said the move into security will be tangential to its current focus.

(Lagarto-Film/Shutterstock)

“When I look at that space, the way that I think about it is it feels very similar to how we got adopted in the logging space about three or [four] years ago, he said, according to a SeekingAlpha transcript of the call. “What people sometimes forget is that we actually started and people compared us … to enterprise search solutions and we actually went through an evolution of us being used in the context of logging, and I see that playing out now in the security space.

During his call with Datnami, Banon touched on several other initiatives underway at Elastic, which is co-headquartered in Amsterdam, The Netherlands, and Mountain View, California. These include:

• Site Search – Elastic acquired the San Francisco-based company Swiftype in 2017 for its site search and app search capabilities. Banon said it’s very easy for users to add an Elasticsearch-powered search box to their websites or apps.

“We’ll go and crawl it for you, provide you with a snippet of code, a simple UI to control which results you want to show and how you want to show them,” he said.

• Third-Party Visualization – Elastic already provides a visualization layer through Kibana, which is an original member of the Elastic Stack. But the company is making it easier for customers to bring other visualizations tools to data stored in Elasticsearch, which is a type of NoSQL database.

“We wrote a SQL layer on top of Elasticsearch and we’re working on integrating Elasticsearch as a data source for things like Tableau and Microsoft PowerBI,” Banon said. “And the reason is, if data is in Elasticsearch, we’re happy. Then how you visualize it, how you interact with it…those are things that we want you to have full flexibility.”

• Machine Learning Integration – Elastic gained machine learning capabilities with its acquisition of Prelert, the functionality of which is available in a subscription-only version of X-Pack. But the company has big plans to further its use of machine learning.

“Down the road, in the future, areas that we’re looking at are things like natural language processing and natural language understanding,” Banon said. “We see the community…has built integration with tools like Tensorflow and Pandas or others. We want to make sure that these integrations are working well and supported. Also at the same time we also want to help these types of workflows being done internally in Elastic, so we’re thinking about supporting things like DataFrames and other concepts natively in Elasticsearch itself.”

The company is currently developing version 7 of the Elastic Stack, which Banon said he hopes is delivered by the end of 2019.

Related Items:

Elastic IPO Expected to Raise $250M

Elastic Plots Its Own Course to Big Data Success

Elastic to Release Source Code for X-Pack

Editor’s note: This story has been corrected. Elastic isn’t Swiftype’s partner. It acquired Swiftype in 2017. Elastic has shared quarterly results twice, not once. Datanami regrets the errors.