Follow Datanami:

Tag: LinkedIn

LinkedIn Unleashes ‘Nearline’ Data Streaming

Jul 22, 2019 |

LinkedIn is releasing its Brooklin data ingestion service to the open source community.

Brooklin has been running in production on the social media platform since 2016. The stateless and distributed service is used primarily for streaming data in near real time—also known as “nearline”—at scale. Read more…

Dr. Elephant Leads the Performance Parade

Jan 12, 2018 |

I started working on big data infrastructure in 2009 when I joined Cloudera, which at the time was a small startup with about 10 engineers. It was a fun place to work. Read more…

How Kafka Redefined Data Processing for the Streaming Age

Mar 7, 2017 |

The Apache Kafka phenomenon reached a new high today when Confluent announced a $50 million investment from the venture capital firm Sequoia. The investment signals renewed confidence that Kafka is fast becoming a new and must-have platform for real-time data processing, says Kafka co-creator and Confluent CEO Jay Kreps. Read more…

Dr. Elephant Steps Up to Cure Hadoop Cluster Pains

Mar 7, 2017 |

Getting jobs to run on Hadoop is one thing, but getting them to run well is something else entirely. With a nod to the pain that parallelism and big data diversity brings, LinkedIn unveiled a new release of Dr. Read more…

LinkedIn Adds to Growing List of ML Tools

Jun 7, 2016 |

LinkedIn is releasing to the open source community its machine-learning tool used to train the ranking algorithm for its newsfeed, advertising and customer recommendations.

The world’s largest professional network (NYSE: Read more…

Kafka Creators Tackle Consistency Problem in Data Pipelines

May 24, 2016 |

One of the big questions surrounding the rise of real-time stream processing applications is consistency. When you have a distributed application involving thousands of data sources and data consumers, how can you be sure that the data going in one side comes out the other unchanged? Read more…

LinkedIn Diagnostics Help Tune Hadoop Jobs

Apr 12, 2016 |

An open source tool released last by LinkedIn developers is intended to help Hadoop and Spark users analyze, tune and improve the performance of their workflows.

The self-service performance-tuning tool for Hadoop dubbed “Dr. Read more…

Kafka Gets a Stream-Processing Makeover

Mar 14, 2016 |

A new library for building streaming applications seeks to shift the focus from analytics to developing core application used to process data streams.

Confluent Inc. announced a technical preview this week of a new technical feature in Apache Kafka called Kafka Streams. Read more…

What Data Science Skills Employers Want Now

Jan 7, 2016 |

There’s good news if you’re for a job in data science in 2016 — the number of job openings in the field appears to be rising as companies look to leverage big data for competitive advantage. Read more…

Kafka Tops 1 Trillion Messages Per Day at LinkedIn

Sep 2, 2015 |

There is data in motion, and then there is really big data in motion. The folks at LinkedIn gave us a compelling example of the latter today when it announced that it’s using the distributed messaging system Kafka to process more than 1.1 trillion messages per day. Read more…

Do NOT follow this link or you will be banned from the site!