Introduction

Ray is a Python framework that allows the execution of Python applications in parallel and in a distributed manner. It is primarily designed for use in Machine Learning and AI applications, but it can also be used in any Python application that requires distributed computing.

A Ray cluster can be set up to run applications by deploying it on any infrastructure: in the clouds of AWS, GCP, Azure, on Kubernetes, On-Premise, etc.

We are going to show you how we have deployed a Ray cluster with containers on a single machine and how it has been used to serve a Hugging Face model that translates texts from English to French.

Machine Learning in Ray

Ray’s Machine Learning libraries simplify the ecosystem of machine learning frameworks, platforms, and tools by providing a unified, open-source experience for scalable machine learning.

They reduce the friction of moving from development to production environments as the same Python code scales seamlessly from a laptop to a large cluster.

These libraries allow you to use the most popular frameworks, such as XGBoost, PyTorch, Hugging Face, etc., and with minimal code changes, you can perform everything from model training to deployment.

Ray’s libraries for Machine Learning are:

• Ray Data: Ray Data abstracts the sharding of training data, the parallelization of inference over these shards, and the transfer of data from storage to CPU and GPU using a streaming paradigm that is better suited for GPU workloads.
• Ray Train: Ray Train is a high-level library for distributed training of models using PyTorch, Hugging Face, TensorFlow, Keras, XGBoost, etc.
• Ray Tune: For running experiments and hyperparameter tuning at any scale, using frameworks like PyTorch, XGBoost, TensorFlow, Keras, etc., and running algorithms like Population Based Training (PBT) and HyperBand/ASHA.
• Ray Serve: For deploying Machine Learning models. It allows serving models from PyTorch, TensorFlow, Scikit-learn, XGBoost, etc., on a Ray cluster.

Deploying a Ray Cluster

To deploy a Ray cluster on a single machine, we used Docker with Docker Compose.

We create a Docker image rayapps:1.0.0 with all the necessary Python libraries to deploy the Hugging Face model and the Ray cluster.