Welcome to the Open Analytics Era
The explosion in enterprise data is paving the way for expanding analytics strategies in business applications. The embedded analytics market is estimated to nearly double to 46 billion between 2016 and 2021, according to Markets and Markets. The same research firm also estimates the advanced analytics market will be worth close to 30 billion by 2019. Demand for ‘open analytics’ strategies for business applications is fueled by the growth in embedded and advanced analytics markets. These combined strategies and technologies work together to help organizations tame the wide range of deployments of analytics across the enterprise for organizational intelligence and operational efficiency.
What is Open Analytics?
‘Open analytics’ is the integration of an open data access layer into business applications (such as CRM, ERP, Finance or HR) to be directly consumed by external analytics tools and popular programming languages. In contrast, embedded analytics is the integration of analytics into the business application itself. Open analytics has a somewhat symbiotic relationship with embedded analytics as many business application vendors consider both as part of their complete analytics strategy; they often share common data architectures, especially around data stores for analytics.
To illustrate the distinction, consider a marketing automation application running in the cloud that ships a popular embedded analytics solution built with Apache Zeppelin, an open source analytics tool, that provides a variety of charts around campaign performance. The organization has two new strategic initiatives: deliver detailed campaign reports to the CMO every Monday at 9:00 ET and enhance lead scoring based on statistical analysis of detailed activity data sets. With an open analytics strategy, the line of business or IT can then leverage their own analytics capabilities rather than go hunting for a new marketing automation tool.
Demand for Open Analytics
There is significant demand today for open analytics from enterprise business intelligence (BI) and data management platforms, and demand will continue to increase with the growth of the advanced analytics market. Beyond that, there are an increasing number of data engineers and data scientists using a variety of platforms and advanced analytics languages such as SAS, R, Python and Java, as well as frameworks such as Hadoop and Spark. All of these combined trends create significant demand for a “bring-your-own-analytics” (BYOA) capability to supplement embedded analytics in business applications. SAS is an example of an embedded and advanced analytics vendor who is moving toward supporting open analytics in their Viya platform that expands support to open models and popular programming languages mentioned above.
With the rapid pace of innovation, it’s impossible for any application vendor to meet all the analytics requirements of their customers with a fully embedded solution. (This was illustrated above with the Apache Zeppelin example.) Providing open access to data with an extensible architecture will make the applications core to the entire organization, which in turn reduces churn for the application vendor.
Wayne Eckerson, from the Eckerson Group, recently shared on a webinar contrasting open versus embedded analytics, “Business users are becoming data developers. That’s a very new phenomenon used to be solely the realm of IT to craft, curate, munge data. But now business users have been pulled into that realm.”
Drawing from a personal experience, the product marketing team at Progress loves the embedded analytics tools for each business application we work with – but we also needed a 360-degree view of our customer interactions. We embarked on a journey to build a customer data lake across CRM, service, marketing automation and web analytics to provide new insights, not available in the data warehouse managed by IT, into the digital experiences and content we deliver. Not of all of our business applications support open analytics strategies, but we uniquely have the benefit of using our own commercial data connectors. This concept, however, is becoming more common as the tools and workforce become more sophisticated with data analytics; cloud computing takes care of common infrastructure requirements.
Open Analytics in the Cloud
In traditional application deployments, data access for analytics was available with access to the physical database. However, two areas that require new innovation in open analytics strategies are disrupting this access:
- The cloud. The rise of application deployments in the cloud has created barriers to data access for external analytics. The database now resides behind cloud firewalls and opening a port for direct ODBC or JDBC database access is a major security concern to cloud vendors. What’s more, data is commonly exposed via application integration APIs which are not designed for analytics workloads.
- Shapes of data. Analytics and data management tools are widely developed around standard SQL interfaces which expect highly structured data. With the rise of big data and cloud, often times data is exposed as non-standard web services APIs with semi-structure or unstructured payloads or standard metadata.
Cloud Vendors are Delivering Open Analytics
The primary deliverables in an open analytics strategy are programming frameworks and a robust data access layer that includes secure and frictionless access to data from popular analytics tools. This is commonly achieved through open interfaces popular with analytics such as standard SQL, REST and Flat Files (CSV, TSV, XLSX, etc). OData is the popular industry standard; REST API and ODBC/JDBC are popular standards for SQL access. Virtually every business analytics and data management tool will support one or more of these interfaces to improve the data access experience.
A survey conducted by NetSuite, a cloud vendor that delivers an open analytics strategy with rich embedded analytics, found that more than 60 percent of its customers using ODBC capabilities in its applications gave a 4 or 5 out of 5 value rating. Other popular cloud applications delivering an open analytics strategy include Plex Systems, ServiceNow, Microsoft Dynamics, SAP Cloud for Customer, SuccessFactors and Acumatica via industry standard SQL and REST.
The Future of Open Analytics
With many analytics strategies becoming available to business application vendors, the market trends point to an expansion in open analytics capabilities. Cloud ISVs are improving their APIs with big data technologies and adding support for bulk interfaces and support for industry data access standards. Others are looking to third party services to warehouse their data into a popular cloud data warehouse. And business applications that target enterprise customers are adopting commercial hybrid data pipelines that securely expose analytics databases over standard SQL (ODBC/JDBC) and REST (OData) beyond the cloud firewall. Data scientists and business analysts will continue to drive innovation here as they demand frictionless access to data so they can maximize their time on analysis, rather than data acquisition.
With the advancement of machine learning and cognitive systems, embedded analytics will become more integrated and really drive business processes and streamline application functionality. Open analytics strategies will follow to expose that very same data for cognitive applications in other areas of the business.
About the author: As chief data evangelist at Progress, Sumit Sarkar works with leading analytics and data management vendors on data connectivity and ingestion for analytics using a wide variety of data, with a focus on standards such as ODBC, JDBC, ADO.NET, GraphQL and OData. Sumit is a frequent speaker at industry events, including Hadoop+Strata World, MongoDB World, Oracle OpenWorld and Dreamforce.