Skip to main content

Using DRA for GPU

DRA (Dynamic Resource Allocation) is a device resource management mechanism in Kubernetes, released as Generally Available (GA) starting from Kubernetes v1.34. DRA allows workloads to request and use hardware resources such as GPUs in a more flexible manner compared to the traditional Device Plugin mechanism.

Instead of requesting a fixed number of GPUs (for example: nvidia.com/gpu: 1), DRA enables workloads to request devices based on specific attributes such as GPU type, memory capacity, or other hardware characteristics. This helps optimize GPU utilization, supports resource sharing models, and lays the foundation for advanced GPU allocation mechanisms.

Prerequisites for Using DRA on Managed Kubernetes with GPU​

  1. Cluster using Kubernetes version v1.34 or later
  2. GPU Driver installed with type Managed-install, using NVIDIA Driver version 570 or higher. Currently, only H100 or H200 GPUs are supported with Managed-install type
  3. kubectl

Installing DRA​

Step 1: Create a cluster with a worker group using H100/H200 GPU​

Note: Select Driver Installation Type = Managed-install, driver version >= 570

On GPU worker groups, add/edit the following label:

nvidia.com/gpu.deploy.device-plugin: "false"

For the BASE worker group, add the following label:

node-role.kubernetes.io/control-plane: "secondary"

Step 2: Install DRA GPU Driver​

Install Helm CLI:

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Add the NVIDIA repository:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \
&& helm repo update

Install GPU DRA version 25.3.2:

helm install nvidia-dra-driver-gpu nvidia/nvidia-dra-driver-gpu --version="25.3.2" --create-namespace --namespace nvidia-dra-driver-gpu \
    --set gpuResourcesEnabledOverride=true \
    --set controller.affinity=null \
    --set nvidiaDriverRoot=/run/nvidia/driver

Step 3: Verify the Installation​

Check the DRA Driver pod:

kubectl get pod -n nvidia-dra-driver-gpu

Check DRA CRDs:

kubectl get deviceclass,resourceslice

Deploying GPU Workloads Using DRA Driver​

The following example demonstrates how to use DRA GPU.

Step 1: Create a ResourceClaimTemplate​

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: single-gpu
spec:
  spec:
    devices:
      requests:
        - exactly:
            allocationMode: ExactCount
            deviceClassName: gpu.nvidia.com
            count: 1
          name: gpu

Create a Pod Using ResourceClaim to Request GPU from DRA Driver​

apiVersion: v1
kind: Pod
metadata:
  name: pod1
  labels:
    app: pod
spec:
  containers:
    - name: ctr
      image: ubuntu:22.04
      command: ["bash", "-c"]
      args: ["nvidia-smi -L; trap 'exit 0' TERM; sleep 9999 & wait"]
      resources:
        claims:
          - name: gpu
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: single-gpu

Verify the ResourceClaim​

kubectl get resourceclaim <claim-name>

Verify the Pod​

kubectl exec -it <pod> -- nvidia-smi