Skip to main content
Version: Next

What is HAMi?

HAMi (Heterogeneous AI Computing Virtualization Middleware) is an open source platform for managing heterogeneous AI computing devices in Kubernetes clusters. Formerly known as k8s-vGPU-scheduler, HAMi enables device sharing across multiple containers and workloads.

HAMi is a Cloud Native Computing Foundation Incubating project and is listed in both the CNCF Landscape and CNAI Landscape.

The problem​

Kubernetes counts GPUs in whole numbers. A Pod that asks for nvidia.com/gpu: 1 gets an entire card, even when its model needs 2 GiB of a 24 GiB card. The rest of the card's memory and compute cannot go to other Pods. GPU utilization stays low, expensive hardware goes to waste, and other Pods wait for the next free card.

One GPU without HAMi and with HAMi One GPU without HAMi and with HAMi

Key Features​

Device Sharing​

  • Multi-device Support: Compatible with various heterogeneous AI computing devices (GPUs, NPUs, and other accelerators)
  • Shared Access: Multiple containers can simultaneously share devices for improved resource utilization

See Device sharing and Device supported by HAMi.

Memory Management​

  • Hard Limits: Enforce strict memory limits inside containers to prevent resource conflicts
  • Dynamic Allocation: Allocate device memory on-demand based on workload requirements
  • Flexible Units: Specify memory allocation in MB or as a percentage of total device memory

See Device sharing and Device resource isolation.

Device Specification​

  • Type Selection: Request specific types of heterogeneous AI computing devices
  • UUID Targeting: Specify exact devices using device UUID for precise allocation

See Scheduling.

Ease of Use​

  • Transparent to Workloads: No code changes required inside containers
  • Simple Deployment: Install and uninstall using Helm with minimal configuration

See Quick Start.

Open Governance​

  • Community Driven: Jointly initiated by organizations across internet, finance, manufacturing, and cloud services
  • Neutral Development: Managed as an open source project with CNCF

See Governance.

How it works​

When a Pod asks for HAMi resources, a webhook routes it to the HAMi scheduler, which picks a node and a card with enough room. On that node, the HAMi device plugin mounts the card and HAMi-Core into the container, and HAMi-Core enforces the container's memory and compute limits. See Architecture.

Next steps​

CNCFHAMi is a CNCF Incubating project