Deep learning is a form of artificial intelligence that utilizes neural networks, which are computing systems inspired by the human brain and nervous system — essentially a multi-layered “mesh” architecture. Neural networks are not new, but their use in tackling machine learning problems has become so specialized and valuable, it has emerged as the discipline of deep learning. The magic of DL models is in how well they handle data with a huge number of input variables and/or very complex relationships between input variables.

Performance: Deep Learning vs. Machine Learning

When the number of input variables and the complexity of relationships between them are very great, deep learning techniques outperform traditional machine learning. This is often the case with image classification, natural language processing, and complex anomaly detection. For example, a relatively common DL model for image classification takes as input 150,000 values (per image!) and predicts one of 20,000 image categories. This would be extremely hard to handle with other ML techniques. DL models are also commonly used for natural language processing (NLP) and complex anomaly detection, such as the detection of fraud and manufacturing defects.

These applications are even more valuable to businesses when used in combination. For example, combining NLP and image recognition makes it possible for airlines to leverage photographs and even mechanics’ handwritten notes to improve maintenance performance.

How to Plan for GPU-accelerated Deep Learning

For deep learning projects, data scientists need access to a server or high-end workstation with a powerful CPU, plenty of memory, and a GPU co-processor. In some cases, they need access to more than one of these machines. However, no data science team runs exclusively on GPU computing. Buying one GPU or even an entire GPU workstation for every data scientist may be overkill, depending on your team and their needs.

Here are some guidelines for IT leaders looking to equip their teams for deep learning without overtaxing their budgets.

• Estimate your capacity needs
  Work with your data science team to arrive at a reasonable estimate of GPU usage. The best-practice recommendation is “only one user per GPU” — meaning that it’s best to allow only one application or data scientist to use the GPU at a time. This gives the user the most GPU memory for their training batches and ensures maximum responsiveness.
• Buy only what you need
  The most cost-efficient approach for supporting a deep learning practice is to implement a heterogeneous cluster with a mixture of GPU and non-GPU nodes. The sweet spot for most organizations is likely 2-6 GPUs per system, depending on the tasks and cost containment requirements. Each GPU includes 2-4 cores, and more cores are needed for projects that require extensive data processing before model training. CPU memory should be 2-3x GPU memory, or more if a training dataset is very large.
• Consider cloud GPU
  It’s a great way to get started, but know the trade-off point. If you know your expected utilization, you can compare the costs of on-premise versus the cloud and make a decision up front. If you can’t forecast utilization accurately, keep an eye on the numbers as your deep learning practice ramps up. Once the cost of cloud outstrips the cost of owning your own GPU systems, it’s time to pull things in house.

Automate Access to Shared GPU Resources

World-class deep learning requires petaflop-scale model training, made economically viable and more practical via GPUs and automated deployment into production environments. Anaconda Enterprise makes it easy for IT leaders to manage GPU resources and for data scientists to be more productive in deep learning projects. Users can simply check out a GPU when needed. When the job completes, the GPU is automatically returned to the cluster. This approach makes sharing GPUs across an organization cost-effective while also ensuring availability for users.

For a deeper dive into the democratization of deep learning and how IT leaders can help enable it in the enterprise, get the white paper: Equipping Enterprise Data Science for Deep Learning.