Features

Is Big Data the Next Big Civil Rights Issue?

Sep 19, 2014 |

There’s no doubt that the big data phenomenon is having a major impact on our lives. Algorithms are affecting what we buy, who we date, and how we work and play. But big data also takes a toll, including a loss of personal privacy and an increased risk of becoming victims of discrimination or unscrupulous vendors. Do these concerns make big data a civil rights issue? One group argues that it most definitely does. In a new report entitled “Civil Read more…

Under New CEO, Lucidworks Aims to Redefine Search and Itself

Sep 18, 2014 |

Lucidworks today unveiled an ambitious new enterprise search application called Fusion that uses advanced signal processing and analytics to drive a new level of personalization and push-delivery of information to users. The product is the culmination of a year-long development initiative under new CEO Will Hayes, who aims to carve a new big data niche for the Solr backer. Search is a ubiquitous component of the Internet, the original “killer app.” Type in your query, hit enter, and voila: a Read more…

Self-Provision Hadoop in Five Clicks, BlueData Says

Sep 17, 2014 |

Forget the data science–in some organizations, just getting access to a Hadoop cluster is a major obstacle. With today’s launch of EPIC, the software virtualization company BlueData says analysts and data scientists can self-provision a virtual Hadoop cluster in a matter of seconds, enabling them to iterate in a faster and more agile fashion. If things go as planned, BlueData‘s new EPIC product will usher in a new level of failure for Hadoop users around the world. “If you want Read more…

Tape Gets Second Wind as Big Data Mounts

Sep 16, 2014 |

Think tape is dead in our big data world? Think again. This week, the National Center for Supercomputing Applications (NCSA) announced that it bought 20PB of tape capacity to expand the world’s biggest data archive. Meanwhile, the LTO Program has plotted out a roadmap for the next decade that will eventually see a single LTO tape cartridge storing 120 TB of data. While tape is looked down upon in our speed-obsessed culture, the old standby has proven its relevance in Read more…

MapR Puts Apache Drill into Hadoop Distro

Sep 16, 2014 |

Organizations today demand tools that provide familiar SQL-based access to data stored on HDFS. Today, MapR Technologies gave its customers yet another SQL interface when it announced support for Apache Drill 0.5 in the new release of its commercial Hadoop distribution, MapR 4.0.1. Apache Drill is an open source framework that delivers a full SQL-compliant interface that allows users to query unstructured data stored on HDFS for data discovery purposes. The project, which is backed by MapR and is an Read more…

Three Things Apache Spark Needs to Out-Hadoop Hadoop

Sep 15, 2014 |

It’s only September, but it’s clear that 2014 will go down as the Year of Apache Spark. While the open source processing framework has gathered an enormous amount of momentum within the Hadoop ecosystem, there are three areas where the Spark community should focus on if it’s going to shine brighter in 2015. Apache Spark stormed the big data scene early in the year, becoming the Hot New Thing in an industry that generates Hot New Things at increasingly breakneck Read more…

Comcast Develops Advanced Advertising Platform to Handle Real Time Big Data

Sep 15, 2014 |

Comcast is working with national, regional and local advertisers to use data in meaningful and privacy-compliant ways to inform their advertising strategies and maximize their advertising spend. For Nathaniel Auvil, a Distinguished Engineer with the company’s Engineering and Platform Services Group, it’s applying the latest in high performance computing (HPC) capabilities to Comcast’s advertising offerings. Specifically, he has been designing and developing systems that enable Comcast to analyze data.  For example, he has developed systems that enables advertising on Comcast’s Read more…

What’s Driving the Explosion of Government Data?

Sep 11, 2014 |

Most people already know that the world produces massive amounts of data every day, and government organizations are a huge source of that data. For instance, it is estimated that next year the Department of Energy will create 300 terabytes of data per day from analyzing light sources, and by 2020, the department will create 15 petabytes of data per year analyzing high-energy physics. That’s a lot of data. But where does all this data come from? The amount of Read more…

How a Web Analytics Firm Turbo-Charged Its Hadoop ETL

Sep 10, 2014 |

The Web analytics firm comScore knows a thing or two about managing big data. With tens of billions of data points added to its 400-node Hadoop cluster every day, the company is no stranger to scalability challenges. But there’s one ETL optimization trick in particular that helped comScore save petabytes of disk and improve data processing times in the process. ComScore is one of the biggest providers of Web analytics used by publishers, advertising firms, and their clients. If you Read more…

Big Data Challenges in Social Sciences & Humanities Research

Sep 8, 2014 |

One commonly mentioned benefit of the study of history is that we may gain insight into the thoughts and behavior of our ancestors. Learning where we came from can help us better prepare for where we are going. At the launch of this age of Big Data, we are only now well positioned to accomplish that ambition. Armed with Big Data Analytics, we can improve the well-being of ourselves and our offspring, particularly in Social Sciences & Humanities Research. In Read more…