GPU Service Overview on Kubernetes FPT CloudFPT Cloud provides Kubernetes with NVIDIA GPU support with the following key features:›Installing and initializing a GPU Kubernetes clusterFPT Cloud supports the following GPU cards:›Modifying a GPU worker groupPrerequisites:›Deploying GPU applications on KubernetesKubernetes manages and uses GPU resources similarly to CPU resources. Declare GPU resources for your application based on the GPU configuration selected for the worker group.›Using GPU telemetryFPT Cloud uses NVIDIA GPU Telemetry integrated with kube-prometheus-stack as the monitoring and observability toolset for GPU-based systems on Kubernetes. The monitoring stack includes a collector, a time-series database for storing metrics, and a visualization layer. It uses the popular open-source applications Prometheus and Grafana. Prometheus also includes Alertmanager for creating and managing alerts. Prometheus is deployed together with kube-state-metrics and node_exporter to display cluster-level metrics for Kubernetes API objects and node-level metrics such as GPU utilization.›Using Autoscaler with GPUContainer-level autoscaling›Using GPU sharing modesGPU sharing modes allow a physical GPU to be shared among multiple containers in order to optimize GPU utilization. The following GPU sharing strategies are supported:›Adding a GPU worker groupPrerequisites:›GPU driver installation guide on KubernetesUsers can self-install the desired GPU driver on an FPT Kubernetes Engine cluster with GPU integration.›Configuring auto scale using GPU custom metricsKubernetes supports automatic scaling based on custom metrics such as GPU metrics by integrating with Prometheus. This guide explains how to configure auto scaling for GPU-based applications running on the FPT Kubernetes Engine platform.›FPT Kubernetes Engine with GPUFPT Kubernetes Engine with GPU›Configuring auto scale using KEDA and PrometheusPrerequisites›Using DRA for GPUDRA (Dynamic Resource Allocation) is a device resource management mechanism in Kubernetes, released as Generally Available (GA) starting from Kubernetes v1.34. DRA allows workloads to request and use hardware resources such as GPUs in a more flexible manner compared to the traditional Device Plugin mechanism.›GPU Software on Kubernetes for NVIDIA H100/H200Overview›