Follow Datanami:

Tag: LinkedIn

LinkedIn Open Sources Dagli to Simplify ML Pipeline Building

Nov 11, 2020 |

LinkedIn yesterday announced that it has open sourced Dagli, a Java-based framework for building and deploying machine learning pipelines.

While the number and quality of tools for developing machine learning models has continued to increase, bringing everything together to deploy an ML model continues to be problematic, explains LinkedIn research scientist Jeff Pasternack in a blog post yesterday. Read more…

LinkedIn Unveils Open-Source Toolkit for Detecting AI Bias

Aug 28, 2020 |

As AI becomes increasingly integrated in our day-to-day lives, the implications of bias in AI grow more and more worrisome. Training data that appears impartial is often influenced by historical and socioeconomic factors that render it biased, sometimes to the detriment of marginalized groups, and especially in AI applications in sectors like healthcare and criminal justice. Read more…

Navigating the AI and Analytics Job Market During COVID-19

Aug 5, 2020 |

The market for AI and analytics jobs has not been spared from the wrath of COVID-19, which has directly led to the loss of more than 30 million American jobs over the past four months. Read more…

LinkedIn Unleashes ‘Nearline’ Data Streaming

Jul 22, 2019 |

LinkedIn is releasing its Brooklin data ingestion service to the open source community.

Brooklin has been running in production on the social media platform since 2016. The stateless and distributed service is used primarily for streaming data in near real time—also known as “nearline”—at scale. Read more…

Dr. Elephant Leads the Performance Parade

Jan 12, 2018 |

I started working on big data infrastructure in 2009 when I joined Cloudera, which at the time was a small startup with about 10 engineers. It was a fun place to work. Read more…

How Kafka Redefined Data Processing for the Streaming Age

Mar 7, 2017 |

The Apache Kafka phenomenon reached a new high today when Confluent announced a $50 million investment from the venture capital firm Sequoia. The investment signals renewed confidence that Kafka is fast becoming a new and must-have platform for real-time data processing, says Kafka co-creator and Confluent CEO Jay Kreps. Read more…

Dr. Elephant Steps Up to Cure Hadoop Cluster Pains

Mar 7, 2017 |

Getting jobs to run on Hadoop is one thing, but getting them to run well is something else entirely. With a nod to the pain that parallelism and big data diversity brings, LinkedIn unveiled a new release of Dr. Read more…

LinkedIn Adds to Growing List of ML Tools

Jun 7, 2016 |

LinkedIn is releasing to the open source community its machine-learning tool used to train the ranking algorithm for its newsfeed, advertising and customer recommendations.

The world’s largest professional network (NYSE: Read more…

Kafka Creators Tackle Consistency Problem in Data Pipelines

May 24, 2016 |

One of the big questions surrounding the rise of real-time stream processing applications is consistency. When you have a distributed application involving thousands of data sources and data consumers, how can you be sure that the data going in one side comes out the other unchanged? Read more…

LinkedIn Diagnostics Help Tune Hadoop Jobs

Apr 12, 2016 |

An open source tool released last by LinkedIn developers is intended to help Hadoop and Spark users analyze, tune and improve the performance of their workflows.

The self-service performance-tuning tool for Hadoop dubbed “Dr. Read more…