MapR Converged Data Platform Utilized by RiskIQ

July 26 — MapR Technologies, Inc., provider of the industry’s only Converged Data Platform, today announced that RiskIQ, the leader in external threat management, is using the MapR Converged Data Platform to cost-effectively provide faster and more comprehensive threat detection on extremely large datasets. 

RiskIQ’s external threat management platform scans, analyzes and stores entire websites, mobile app stores, and social media outlets across the entire internet. As the amount of data and sophistication of security threats grew, RiskIQ wanted to be able to provide their clients with faster and more comprehensive threat detection analysis in an economical way.  RiskIQ selected the MapR Platform and has been using it as a distributed storage system for several years. More recently the company is using MapR with Apache Spark, Hive, Parquet, and Oozie and has successfully introduced new threat detections product offerings for their customers.

“The MapR file system architecture is something we appreciated from the start,” said Chris Kiernan, RiskIQ CTO and co-founder. “We knew we could leverage MapR for almost anything we wanted to do. With the node management and the way clustering is done, we always knew it was built in the right way if we needed to do analysis.”  

RiskIQ’s 100 web crawlers collect about 10-20 TBs data each day from across the Internet. This number continues to grow as RiskIQ adds depth and new datasets to their crawl data. In order to reduce the size of the data, the company developed a technique to create Parquet files from raw crawl data that are 10x smaller and sent via NFS into their warehouse for analysis. “We can query an entire day’s worth of files in minutes instead of hours,” said Kiernan. “There are all kinds of things we can do, now that we have the data in a compact format. It’s a central part of our architecture.”

MapR architecture increases efficiency and reduces costs

RiskIQ was able to keep costs down by building the new data analysis use case on top of their existing MapR cluster. “We continue to use the cluster as a production file system while, at the same time, we’ve built an entire warehouse using the same infrastructure for a very small price point,” said Kiernan. “We have been able to cut Capex and Opex in half. We would have had to pay twice as much to build a vanilla Hadoop cluster. If we had built this in Cloudera, we would have needed separate clusters for production and analytics. It wouldn’t be a dual-purpose system.”

Comprehensive and reliable platform 

“The fact that MapR makes sure that everything is compatible has worked really well. If we want to try a new technology, we can install it and it’s ready to use,” said Adam Hunt, RiskIQ’s chief data scientist. “In my previous job, I worked with vanilla Hadoop, and the cluster did go down. That’s just not acceptable. We’ve never had that issue with MapR. It’s rock solid. We don’t see performance degradation no matter what we do to the cluster and upgrades are seamless.”

New capabilities provide competitive advantage

With the MapR Platform, RiskIQ has developed new product offerings that has allowed them to push new types of data into their application that helps them understand things about websites they may not have understood before. “We have built all new parts of our products based on this new analysis, so it has been absolutely instrumental to our host reputation service. We can now answer all of the ad hoc questions we could never answer before to provide even more advanced detection for our clients. It’s improving the way we run the business,” said Kiernan.

“RiskIQ has cost-effectively added extremely large datasets, developed new products and powerful new analytics for threat detection capabilities utilizing their existing, reliable MapR cluster,” said Jack Norris, senior vice president, data and applications, MapR Technologies.  “The MapR Platform has enabled customers like RiskIQ to achieve results never before attainable, proving our system is designed for mission-critical performance that can yield a measurable competitive advantage.”

About RiskIQ

RiskIQ is a cybersecurity company that helps organizations discover and protect their external facing known, unknown and 3rd party web, mobile and social digital assets. The company’s External Threat Management platform combines a worldwide proxy network with synthetic clients that emulate users to monitor, detect and take down malicious and copycat apps, drive by malware and malvertisements. RiskIQ is being used by leading financial institutions and other companies to protect their web assets and users from external security threats and fraud. We are headquartered in San Francisco, backed by growth equity firms Summit Partners and Battery Ventures.

About MapR Technologies

MapR enables organizations to create disruptive advantage and long-term value from their data with the industry’s only Converged Data Platform, which delivers distributed processing, real-time analytics, and enterprise grade requirements across cloud and on-premise environments, while leveraging the significant on-going development in open source technologies including Spark and Hadoop. Organizations with the most demanding production needs, including sub-second response for fraud prevention, secure and highly available data-driven insights for better healthcare, petabyte analysis for threat detection, and integrated operational and analytic processing for improved customer experiences, run on MapR. A majority of customers achieves payback in fewer than 12 months and realizes greater than 5X ROI. MapR ensures customer success through world-class professional services and with free on-demand training that over 50,000 developers, data analysts and administrators have used to close the big data skills gap. Amazon, Cisco, Google, HPE, Microsoft, SAP, and Teradata are part of the worldwide MapR partner ecosystem. Investors include Google Capital, Lightspeed Venture Partners, Mayfield Fund, NEA, Qualcomm Ventures and Redpoint Ventures. Connect with MapR onLinkedIn, and Twitter.

Source: MapR