A Spark-based analytics engine released last week runs on a low-latency, software-defined bare metal fabric and can scale Spark clusters and the IT infrastructure that supports them.
Bigstep, which has dual headquarters in London and Chicago, said its real-time Spark service is designed to speed deployment of real-time data streaming applications as more Spark implementations shift to the cloud. Among the emerging uses for such implementations are the Internet of Things and algorithm decision-making, Bigstep Founder and CEO Lucas Roh noted in a statement.
Roh added that Bigstep’s bare metal cloud platform would help reduce development requirements for its real-time Spark service. The company also said it would offer a “pay-per-use” container-based Spark cluster optimized for real-time streaming applications that use multiple concurrent Spark contexts.
A rapid prototyping feature includes a built-in Jupyter interface for Scala, Python or R programming languages. Jupyter Notebook is designed to allow data scientists to combine code, graphs, dashboards and descriptive texts within the same document while performing operations interactively.
Meanwhile, real-time data streaming applications run in parallel with other container application clusters.
The company added that its bare-metal fabric could run multiple Spark versions, including Spark 2.0, using the same pool of resources.
The introduction of the Spark platform follows the company’s release in early October of a real-time application container service designed specifically for streaming applications. The company said it container service also targets emerging infrastructure based on micro-services and memory-intensive workloads requiring low latency and higher performance.
The container service is based on Docker and can run distributed streaming applications on the company’s bare metal cloud. Those applications can be built on Spark Streaming, Apache Flink or the Heron streaming replacement for Apache Storm, the company said.
Persistent storage requirements have recently overtaken security as the top barrier to adoption of application containers in production. Hence, Bigstep stressed its real-time container service offers high-end persistent support, including storage volumes that follow containers as they move across clusters.
Meanwhile, the company said customers could deploy its new Spark service alongside either Zoomdata and Bigstep data lakes, or use it with applications deployed on-premises or on its managed container service.
Recent items:
Unraveling Hadoop and Spark Performance Mysteries
Overcoming Spark Performance Challenges in Enterprise Hadoop Environments
May 8, 2024
- Alteryx: Businesses Embrace Generative AI, but General Public Remains Wary
- Atlan’s Control Plane to Power Data & AI Governance Attracts $105M
- Cisco Unlocks AI-Powered Intelligence for Self-Hosted Observability
- Boomi Accelerates Enterprise Data Connectivity with Advanced API Management Acquisitions
- Elastic Security Labs Releases Guidance to Avoid LLM Risks and Abuses
- Neo4j Empowers Syracuse University with $250K Grant to Tackle Misinformation in 2024 Elections
- Cloudera Unveils New Flow Management and Streaming Operators at Red Hat Summit
- DataForge Open-Sources First Functional Data Programming Framework
May 7, 2024
- Datadog Launches IT Event Management to Enhance AIOps Capabilities
- Panasas Rebrands as VDURA: A New Chapter in Software-Defined Storage for AI Applications
- Niobium Secures $5.5M in Venture Financing to Commercialize FHE Accelerator Chip
- Oracle Code Assist Can Help Developers Build Applications Faster with AI
- Redpanda 24.1 Launches with New Write Caching Capabilities for 98% Improved Latency
- Persistent Transforms Enterprise Data Management with iAURA, a Portfolio of AI-Powered Data Solutions
- BigID Launches Industry-First Hybrid Scanning for Cloud Native Workloads
May 6, 2024
- Cisco and Splunk Integrate XDR and Enterprise Security for Advanced Threat Protection
- Normalyze Innovates to Secure LLMs and Data Lakes Against Emerging Threats
- Couchbase Survey Finds Enterprises Plan Massive Spend on AI, with Over $21M Allocated in 2023-24
- Elastic Enhances Security Operations with AI-Assisted Attack Discovery and Analysis
- UCSD Researchers Develop New Framework to Curb Undesirable AI Outputs
Most Read Features
Sorry. No data so far.
Most Read News In Brief
Sorry. No data so far.
Most Read This Just In
Sorry. No data so far.
Sponsored Partner Content
-
Get your Data AI Ready – Celebrate One Year of Deep Dish Data Virtual Series!
-
Supercharge Your Data Lake with Spark 3.3
-
Learn How to Build a Custom Chatbot Using a RAG Workflow in Minutes [Hands-on Demo]
-
Overcome ETL Bottlenecks with Metadata-driven Integration for the AI Era [Free Guide]
-
Gartner® Hype Cycle™ for Analytics and Business Intelligence 2023
-
The Art of Mastering Data Quality for AI and Analytics
Sponsored Whitepapers
Contributors
Featured Events
-
AI & Big Data Expo North America 2024
June 5 - June 6Santa Clara CA United States -
CDAO Canada Public Sector 2024
June 18 - June 19 -
AI Hardware & Edge AI Summit Europe
June 18 - June 19London United Kingdom -
AI Hardware & Edge AI Summit 2024
September 10 - September 12San Jose CA United States -
CDAO Government 2024
September 18 - September 19Washington DC United States