GPU service on Kubernetes
FPT Cloud provides Kubernetes with NVIDIA GPU support, with these key capabilities:
- Flexible GPU configuration — choose GPU type and GPU memory per Worker Group.
- Automated GPU resource management and allocation in Kubernetes through the NVIDIA Operator.
- GPU visualization and monitoring with NVIDIA DCGM.
- Automatic Container/Node scaling with Autoscaler as application GPU demand rises or falls.
- GPU sharing through the Multi-Instance mechanism, to optimize GPU resource usage and cost.
FPT Cloud uses the NVIDIA GPU Operator, which automatically manages all the software components needed to use GPUs on Kubernetes — letting you use GPU resources the same way you use CPU resources in a cluster. Those components include:
- NVIDIA Drivers (CUDA, MIG, and so on)
- NVIDIA Device Plugin
- NVIDIA Container Toolkit
- NVIDIA GPU Feature Discovery
- NVIDIA Data Center GPU Manager (monitoring)
MIG configuration on FPT Cloud
The GPU Operator configures MIG on a Worker Node automatically — to apply a MIG config, label the Worker Node with a profile NVIDIA supports. The available MIG configurations are listed in the default-mig-parted-config ConfigMap, in the gpu-operator namespace.
MIG on Kubernetes works as a controller: it watches the nvidia.com/mig.config label on a Worker Node and applies the MIG configuration you request. When the label changes, MIG first stops all GPU Pods (including the device plugin, GPU Feature Discovery, and dcgm-exporter), then stops any systemd services running on the GPU Worker Node if drivers were pre-installed (listed in the default-gpu-clients ConfigMap). It then reapplies the MIG configuration and restarts the GPU Pods (and, if needed, the GPU systemd services on the Worker Node). Enabling MIG mode requires restarting the Worker Node.
FPT Cloud currently supports the NVIDIA A30 GPU, with these MIG profiles:
| No. | GPU A30 Profile — Label | Strategy | Number of instances | Instance resource |
|---|---|---|---|---|
| 1 | all-1g.6gb | single | 4 | 1g.6gb |
| 2 | all-2g.12gb | single | 2 | 2g.12gb |
| 3 | all-4g.24gb | single | 1 | 4g.24gb |
| 4 | all-balanced | mixed | 2 | 1g.6gb |
| 1 | 2g.12gb | |||
| 5 | none (no label) | none | 0 | 0 (entire A30-24GB) |
For example, with strategy single and label all-1g.6gb, the Operator divides the A30 GPU on that Worker Node into 4 MIG devices, each with 1 logical GPU (a quarter of the physical GPU) and 6 GB of GPU RAM. A MIG configuration applies to every GPU card on a Worker Node that carries the label.
GPU telemetry
FPT Cloud uses NVIDIA GPU Telemetry, integrated with the kube-prometheus-stack, to monitor GPU usage on Kubernetes. The toolset includes a collector, a time-series database for metrics, and a visualization layer, built on Prometheus and Grafana. Prometheus includes Alertmanager for creating and managing alerts, and is deployed alongside kube-state-metrics and node_exporter to expose cluster-level metrics for Kubernetes API objects and node-level metrics such as GPU usage.
To collect GPU telemetry on Kubernetes, NVIDIA recommends dcgm-exporter, which exposes GPU metrics to Prometheus in a format it can scrape, and visualizes them through Grafana. dcgm-exporter uses the Kubelet Pod Resources API and includes a ServiceMonitor to expose its endpoints.
Is GPU on Kubernetes a good fit for AI workloads?
Yes. A growing set of open-source extensions for Kubernetes (KNative, Istio, Kubeflow, and KFServing) make it easier to run AI and machine learning workloads on Kubernetes, removing complexity and simplifying deployment and management at scale.
Kubernetes has become a central technology for deploying AI: libraries like PyTorch, TensorFlow, and Keras streamline the path from data and model to prototype and production, while still supporting fine-grained customization when you need to build custom components or adapt existing models with transfer learning. Containers package the result into an image with all requirements and dependencies, portable to run almost anywhere without performance loss — and Kubernetes makes deploying, maintaining, and scaling that image straightforward.
Pairing GPU with Kubernetes improves performance and processing speed for AI applications. GPU on FPT Cloud's Kubernetes service delivers quality and time savings — especially valuable for finance and banking workloads that process large volumes of data.