Use of (NVIDIA) GPUs in k8s

Solution involves:

• device plugin, e.g. NVIDIA
• use of node labels and node selectors to pick the node with the right GPU
• Node Feature Discovery

NVIDIA stack includes (high to low abstraction level):

• NVIDIA device plugin with gpu-feature-discovery
• NVIDIA Container Toolkit which includes NVIDIA Container Runtime.
• CUDA toolkit
• NVIDIA GPU drivers

NVIDIA Container Runtime can use either of:

• containerd for use with k8s
• alternatively use CRI-O
• use podman with Container Device Interface

Architecture