.png)
System Requirements
Windows
Both Client and GPU Agent are supported on Windows 10/11.
Linux
The Client and GPU Agent are supported on Debian based distributions on version 11 and greater, for example Ubuntu 20.04 and greater. Other distributions with a similar release date or newer should be supported as well.
For running within WSL on Windows, only the Client is currently supported (not the GPU Agent).
Preconditions for running a Client app
The following are necessary to use a remote GPU:
Windows
Windows 10/11
If you’re running on a machine without a GPU and drivers installed, e.g. a VM, then you must install the Vulkan Runtime. Download and install from https://sdk.lunarg.com/sdk/download/latest/windows/vulkan-runtime.exe
Linux
Ubuntu 20.04 or greater, Debian 11 or greater
The following packages must be installed:
libatomic1
libnuma1
These can be installed by running apt update & apt install libatomic1 libnuma1
Preconditions for running a GPU agent
The following are necessary to host a remote GPU:
On all platforms an NVIDIA GPU with an installed driver version 535 or greater, and a CUDA driver version 12.2 through to 12.4.
Windows
Windows 10/11
Linux
Ubuntu 20.04 or greater, Debian 11 or greater
The following packages installed:
libatomic1
libnuma1
libvulkan1
libgl1
libglib2.0-0
These can be installed by running apt update & apt install libvulkan1 libgl1 libglib2.0-0 libatomic1 libnuma1
Accessing the Trial
Contact will@juicelabs.co if you need an email invitation to the trial.
You should have received an email with the subject “Set up your Juice GPU account”. Follow the instructions in that email to set your password. Please create a new password for this trial.
Access this download screen:
Windows
Click to download the Windows release.
Double-click the downloaded file to install Juice GPU for Windows. Restart if instructed.
See below regarding using the desktop app.
Linux
Click to download the Linux release. Depending on your Linux setup, it might be necessary to download the file on a different system, then copy the file to your intended Linux client system.
Double-clickOnce the file is on your Linux client system, run tar -xf [name of file] in the directory you want the application to run from. the downloaded file to install Juice GPU for Windows. Restart if instructed.
See below regarding using the CLI application and sharing a GPU.
Using the Desktop App (Windows)
Open the “Juice Desktop” app (look in C:\Program Files\Juice GPU if you have trouble finding it).
Click the Login button, and log in with your email address and password.
At times during the Trial period, a new version of the desktop app may become available. If you see a message at the top of the app window prompting you to do so, please Update and Install the latest version.
The main navigation in the desktop app is via the vertical bar on the left. The rest of this section of the guide will cover the functions of each element in the nav.
Settings - “Active Paths”
Any application that is listed here - or that has a parent folder listed here - will be launched via remote GPU instead of the local GPU, if there is a remote GPU connected.
Add specific applications or folders by either typing/pasting a path:
Getting Started
This section allows for connection to a remote GPU, or displays any existing connection to a remote GPU.
Pools
A pool is a collection of remote GPUs that have been offered up for use, combined with a set of users who are permitted access to those remote GPUs.
You might have been invited to other pools by communication with operations@juicelabs.co or by other colleagues (see below regarding administration). If not, you will have only your own pool called “My Pool” at first, and it will not have any GPUs in it by default.
Connecting to a remote GPU
If there are remote GPUs available to you inside any of the pools you have access to:
Check the box next to the GPU pool you would like to connect to, like this:
Clicking the small refresh button in the upper right will ensure you have the most current view of available GPUs, Pools, and Host Machines.
Select one GPU-type panel (e.g. NVIDIA RTX A6000 shown with orange outline indicating it as selected).
Click the Start Session button.
4. You should now see a screen indicating that you are connected to a remote GPU, like this:
Running an application via remote GPU
While your system is connected to a remote GPU, any application which has its path listed in the Active Paths in the Settings tab will run via the remote GPU instead of any local GPU.
While running applications via remote GPU the “Getting Started” tab displays application telemetry in a section at bottom. Expanding an application row using the arrow at right will reveal real-time performance metrics for that application, e.g. "vkcube.exe" shown here:
“Run Agent” Nav
“Run Agent” tab
To make the GPU(s) from the system that is running this desktop app available to a pool as remote GPU(s):
Select the GPU(s) you want to share in GPU selection
Select a pool to add the GPU(s) to
Click the “Start” button
You will see a screen indicating that your GPU is shared in that pool, a button to stop sharing, and an option to automatically do the same sharing action whenever the app is started up. You will also see the console output for the GPU server.
“Settings” Nav
For the purposes of this Trial, please do not change any other settings here without discussing via operations@juicelabs.co.
“Feedback” Nav
Your mechanism for reporting issues, ideas, or any other feedback to the Trial team.
“Account” Nav
The account under which you’re logged in, and the version of the desktop app you are running.
“Logout” Nav
Logs you out of the desktop app.
Using the CLI app (Linux, Windows, WSL)
juice is a command-line interface (CLI) tool designed to facilitate various operations and configurations for users. It provides a wide range of commands and options to manage configurations, contexts, agents, pools, organizations, sessions, runs, users, and authentication.
NOTE: In this document every example command will be shown as ./juice as it works on Linux. For Windows, substitute juice.exe in place of ./juice.
This document will focus on the main capabilities and primary features of the CLI app. For
Help within the app
At any point you can get in-context help by appending –-help, or just -h. For example, ./juice –h gives you top-level help, while something like ./juice session –h gives you help with sessions.
Logging in
./juice login
Scan the QR code using your phone, and log in.
After a few moments, the CLI interface will show you as “Logged in”.
Running an application against a remote GPU
The most straightforward way to run an application is to use the run command, like this:
./juice run <application> [<arg>...]
The run command encapsulates several steps: it requests a session (connecting you to a remote GPU), runs the application using that remote GPU, then releases the session, freeing that remote GPU back into its pool.
Try run -h for more advanced capabilities, including the ability to narrow your remote GPU target(s) by pool(s), session(s), or even specific GPUs.
Managing sessions manually
Beyond the simple elegance of the run command, you can also manage sessions yourself. You might want to do this if you want to:
keep connected to the same GPU over a period of time and over multiple application runs
establish several sessions simultaneously to multiple different GPUs
./juice session request will connect you to a GPU in one of your pools, and you will remain connected until you session release (or of course until something else happens to sever the connection, like the GPU agent being disconnected for example).
Though you can maintain multiple sessions simultaneously, at any given time you have one and only one “active” session. You can change which session is active using session activate <session>.
The release command will release the active session by default. You can also release a specific session using session release <session> .
In order to specifically reference any session, you will need to populate <session> with the correct ID. The ID looks something like a1350d37-e84c-4bd1-882c-0fb41e844d81 and is returned when you request. You can also use session list to see current sessions, including their IDs.
Choosing a specific GPU
If there is a specific GPU that you want to connect to:
Use gpu list to see all GPUs available to you in any of your pools.
Look for a GPU that is not currently in use, by looking for the key-value pair "usedvram":0
Copy the uuid value of the GPU that you want to connect to.
Specify the GPU’s ID for the run command using run -–gpu <uuid> or by requesting a session using session request --gpu <uuid>
Multi-GPU Support
Juice allows you to take advantage of multiple GPUs on a single host. This enables you to run more demanding workloads or to run multiple applications concurrently on a single machine.
Using the CLI:
To request a session with multiple GPUs, use the --num-gpus flag when creating a session or running a workload. For example:
juice run --num-gpus 2 python
content_copyUse code with caution.Bash
This command requests an agent that has 2 available GPUs.
Important Considerations:
To use multiple GPUs, you must have a host device with multiple GPUs installed.
The Juice agent running on the host will expose all GPUs by default.
The effectiveness of utilizing multiple GPUs is highly workload dependent. For example, some workloads may be able to leverage multiple GPUs for faster processing, while others may not benefit from this.
Example:
Here is a trivial example of using PyTorch to print out the device properties for a multi-GPU system:
import torch
# Check the number of GPUs availableprint(torch.cuda.device_count())
# Print the properties of each GPU
print(torch.cuda.get_device_properties(0))
print(torch.cuda.get_device_properties(1))
content_copyUse code with caution.Python
General Best Practices for Multi-GPU Workloads:
LLMs: For LLMs, using frameworks like llama.cpp (https://github.com/ggerganov/llama.cpp) can help split the model layers across multiple GPUs, enabling larger models to be loaded and used. The oobabooga UI (https://github.com/oobabooga/oobabooga) can manage multiple backends, including llama.cpp, and utilize multiple GPUs effectively.
Image Generation: Explore frameworks like Stable Diffusion, which might offer multi-GPU support in more recent versions. Look for updates and documentation on their GitHub repository or official website.
Fine Tuning: A good starting point for fine tuning models is Stable Diffusion's documentation (https://huggingface.co/docs/diffusers/v0.9.0/en/training/text2image). You can use this training with tools like AUTOMATIC1111/stable-diffusion-webui for a user-friendly experience.
Training: Consider replicating training benchmarks like ResNet, YOLOv8, GPT2, or even the MLPerf training benchmark to test the performance of multiple GPUs for your specific training needs.
Remember, the best approach for utilizing multiple GPUs will depend on your specific workload and application requirements.