In Amazon Web Services, each instance type has a different upper limit on how many Pods it can run. For example, an m5.large instance can only run 29 Pods, but an m5.4xlarge can run up to 234. The reason is that each EC2 instance can only have a limited number of IP addresses assigned to it.

In Kubernetes clusters, GPUs are expensive. Really expensive. And when your data science team is fighting over GPU resources like it’s Black Friday at Best Buy, you know something’s broken.

That was the challenge I faced a few years ago. I implemented this solution back then, and though it’s been a while, I thought now would be a great time to share the experience.

We had a k8s cluster with 8 NVIDIA A100 GPUs, and our data scientists were… unhappy. Small model training jobs waited hours for GPU access while large distributed training jobs hogged entire GPUs at 30% utilization.

The culprit? GPU allocation granularity. Kubernetes treats GPUs as atomic resources, you either get a whole GPU or nothing. There’s no middle ground.

I needed to fix this. And the solution wasn’t what I expected.

The Kubernetes ecosystem is undergoing a significant transformation in how it handles networking, shifting from traditional Ingress API and annotations to the more advanced Gateway API.

Cluster API (CAPI) is an open-source Kubernetes sub-project. Its goal is to bring Kubernetes-style, declarative APIs and controllers to the problem of bootstrapping, configuring, upgrading and operating entire Kubernetes clusters, treating clusters themselves as first-class Kubernetes resources rather than as external, manually-provisioned infrastructure.

ORAS is the de facto tool for working with OCI Artifacts. It provides CLI and client libraries to distribute artifacts across OCI-compliant registries.

A free, open-source, local-first task manager.

Ingress2gateway helps translate Ingress and provider-specific resources (CRDs) to Gateway API resources. Ingress2gateway is managed by the Gateway API SIG-Network subproject.

Sveltos is a Kubernetes add-on controller. It deploys and manages add-ons and applications across many clusters using label-based matching. Sveltos does not compete with GitOps controllers like ArgoCD or Flux. Instead, it extends their capabilities.

A GitOps controller monitors a repository and syncs manifests. In contrast, Sveltos takes these manifests and applies them across the entire fleet. Its configurations are cluster-agnostic; they do not reference a specific cluster. Instead, they target clusters by labels, which means when a new cluster joins the fleet, it requires no configuration changes, only the right labels. One configuration can serve any number of clusters that meet the defined criteria.

A modular Ansible role and playbook that performs automated operating system patching and system maintenance on K3s cluster nodes with zero-downtime semantics.