New Open-Source Projects Emerge for Machine Learning
Two open-source projects contributed by Chinese tech giants Baidu and Tencent will focus on machine and deep learning advances with the long-term goal of making the AI technologies easier to use while advancing cloud services using deep learning frameworks.
The Linux Foundation said it would add the two projects to its deep learning community projects focused on boosting the ecosystem for AI, machine learning and deep learning. Tencent’s Angel Project consists of a distributed machine learning platform running on Apache Spark and YARN. Baidu’s Elastic Deep Learning (EDL) framework aims to allow cloud service providers to use deep learning tools to build clustered cloud offerings.
Baidu (NASDAQ: BIDU), which has followed Google and other U.S. tech companies in steadily releasing its machine learning tools to the open-source community, said the EDL project will use its PaddlePaddle tool along with TensorFlow to accelerate cluster cloud services deployments. EDL uses the Kubernetes container orchestrator as a cluster controller along with a PaddlePaddle auto-scaler. The combination “changes the number of processes of distributed jobs to the idle hardware resource in the cluster, and a new fault-tolerable architecture,” the Linux Foundation said.
The project claims nearly 1,000 commits under an Apache 2.0 license. “As an elastic deep learning framework for PaddlePaddle, we believe that EDL will substantially benefit the deployment of large-scale deep learning services,” said Yanjun Ma, who heads Baidu’s Deep Learning Technology Department.
Meanwhile, Tencent’s Angel Project is based on the Parameter server designed to handle large-scale machine learning problems. Baidu along with Google (NASDAQ: GOOGL) and researchers at Carnegie Mellon University first proposed the distributed machine learning framework. Running on Spark and YARN, the platform is geared to big data performance for handling advanced machine learning models, including those with billions of parameters.
The combination is designed “to make machine learning easier to use,” said Xiaolong Zhu, a senior AI researcher at Tencent (HKG: 0700) and a member of the Deep Learning Foundation.
With that goal in mind, Tencent said Angel Project uses “out-of-the-box” machine learning algorithms, thereby eliminating the need for analysts and data scientists to write code.
The Angel Project has so far attracted more than 1,000 developer commits.