Ever noticed that unused resources, such as Secrets and ServiceAccounts, might pile up in your Kubernetes cluster? There is a tool that removes them.

K8s cleaner is a controller that finds stale and unhealthy resources and removes or updates them. Here’s what it offers:

- Identifying various types of unused Kubernetes resources, including ready-to-use examples for ConfigMaps, Secrets, Roles/ClusterRoles, ServiceAccounts, PVs/PVCs, Deployments, and StatefulSets;
- Identifying resources based on annotations for maximum lifespan or expiration date;
- Using Lua scripts to define custom selection criteria;
- Scheduling the scans for finding and removing/updating unused resources;
- Notifications via emails, Slack, Discord, Teams, Telegram, etc.;
- Web UI showing existing issues, cleaners, and Lua scripts.

▶️ GitHub repo

Language: Go | License: Apache 2.0 | 755 ⭐️

#tools

Kubernetes updates its AI usage policy

Earlier this month, the Kubernetes Steering Committee announced an update to the project’s policy on the use of AI tools for contributors. While the policy allows AI-assisted contributions, it’s essential to follow a few rules when creating such PRs:

- There should always be a human who is responsible for understanding all the suggested changes. This author should reply to the reviewers’ comments personally, without engaging AI tools.
- AI tools shouldn’t be used for so-signing commits, specified as co-authors or within assisted-by/co-developed commit trailers. (Using this metadata leads to unwanted marketing behaviour.)

You can find the latest version of the updated Kubernetes AI policy at kubernetes.dev.

#news #genai

Kubernetes rewritten in Rust is not a joke anymore

There is a new project called Rusternetes, written by a single enthusiast (Chris Alfonso from Red Hat), with assistance from Claude. With 216k lines of code in Rust, implementing 31 controllers and 3.1k tests, and even its own Web dashboard, it’s quite an impressive effort. Moreover, the project claims to pass 90% of conformance tests from the official Kubernetes e2e test suite. However, these numbers are challenged by the community (see comments in LinkedIn here).

The project also opens a wide community debate over whether anyone might need this implementation for real-world workloads. Anyway, the author seemed to start Rusternetes out of his own research and curiosity.

#news #fun

Kubernetes 1.36 is released

The release was announced in the blog and nicknamed "Haru". It comes with 70 enhancements, 18 of which are graduated to stable (GA). They include fine-grained API authorisation, volume group snapshots, several DRA features, mutating admission policies, node log query, support for user namespaces, OCI volume source, and
PSI (Pressure Stall Information) metrics based on cgroup v2.

Find more details about the changes introduced in Kubernetes 1.36 by following the links from our previous post.

#news #releases

Here comes our newest digest of the prominent software updates in the Cloud Native ecosystem!

1. Headlamp, a Kubernetes web UI developed by the Kubernetes SIG, released 0.41.0, adding rollback for several resources, cluster deletion in browser, support for Traefik and other reverse proxies to handle auth, and MCP server support for plugins.

2. KubeVirt, implementing virtual machine management for Kubernetes (a CNCF Incubating project), reached its v1.8.0. New features include ContainerPath volumes (mapping container paths for VM storage), incremental backups with CBT (Changed Block Tracking), PCIe NUMA-aware topology placement for GPU, Hypervisor Abstraction Layer (enabling KubeVirt to integrate multiple hypervisor backends beyond KVM), and live updates to NAD references without VM restarts.

3. CloudNativePG, a platform designed to manage PostgreSQL in Kubernetes (a CNCF Sandbox project), released 1.29.0, which highlighted the integration of Image Catalogs with a new, dedicated ecosystem for PostgreSQL extensions via the postgres-extensions-containers project. It also introduced dynamic network access control via Pod selectors, shared ServiceAccount support, and granular TLS configuration for PgBouncer. Finally, the project started signing all its release artifacts and container images.

4. ExternalDNS, which synchronises exposed Kubernetes Services and Ingresses with DNS providers (a Kubernetes SIG project), released v0.21.0. This update added client flags for Kubernetes client rate limiting, a new unstructured source for DNS records, a new annotation to request a specific DNS record type for a source, improved Gateway API support, and more new features.

5. Kube-OVN, which integrates OVN-based network virtualisation with Kubernetes (a CNCF Sandbox project), was updated to v1.16.0. It brought provider-scoped policies for multi-network Pods in NetworkPolicy, IPv6 and dual-stack support in the MetalLB underlay integration, several improvements to VPC Egress Gateway (BGP and EVPN support, custom resources and bandwidth limits) and VPC NAT Gateway (user-defined annotations on NAT gateway, SNAT EIP to FIP EIP traffic).

6. kagent, a Kubernetes native framework for building AI agents (a CNCF Sandbox project), was updated to v0.9.0, which added agent sandbox support, UI for prompt templates, token exchange for model auth, and SAP AI Core as a new model provider.

7. Cozystack, a PaaS platform and framework for building clouds (a CNCF Sandbox project), released v1.3, featuring storage-aware scheduling via LINSTOR Extender, a GUI for LINSTOR, a new vm-default-images package for out-of-the-box VM provisioning, improved application observability, and a RestoreJob experience for backups in the dashboard.

#news #releases

Where do ingress-nginx users migrate?

According to a recent Reddit poll, Traefik and Envoy Gateway are the leading alternatives for those who previously relied on ingress-nginx in their production Kubernetes clusters. Other options (HAProxy, F5 NGINX, and Kong) got much fewer votes.

As for even more other solutions mentioned in the comments, Kgateway and specific Envoy implementations (namely, in Istio and Cilium) got the most upvotes.

#networking

OpenTelemetry graduates, Microcks incubates

Two days ago, the CNCF TOC voted to graduate the OpenTelemetry project. Today, OpenTelemetry is used "at hundreds (potentially thousands) of organizations worldwide", with dozens of them listed here. It's also supported by major observability vendors, including Dash0, Datadog, Dynatrace, Grafana, Honeycomb, New Relic, Splunk, and many more.

A week ago, another CNCF project, Microcks, was officially moved to the Incubating status. Microcks is a tool for API mocking and testing. It was created by an individual in 2015 and accepted to the CNCF Sandbox in 2023. Its public adopters include Amadeus, Amway, BNP PARIBAS, Michelin, and Nordic Semiconductor.

#news #cncfprojects

This project is a Kubernetes operator that implements a so-called health-check platform.

Canary checker is a monitoring tool that includes 35+ ready-to-use health checks for various Kubernetes workloads. It works as a K8s operator that has no dependencies and offers:

- checks for popular protocols (HTTP, DNS, ICMP, TCP), databases (SQL-based, MongoDB, Redis, Elasticsearch), infrastructure components (such as Kubernetes resources and EC2 instances), file systems (NFS, S3, SMB), and more;
- alert aggregation from Prometheus, AWS CloudWatch, and Dynatrace;
- integration testing support for JMeter, JUnit, K6, Newman and Playwright;
- Prometheus exporter replacement with custom metrics from the check’s results;
- built-in Web dashboard displaying existing checks and their stats.

▶️ GitHub repo

Language: Go | License: Apache 2.0 | 326 ⭐️

#tools #observability

Sharing our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "ingress-nginx to Envoy Gateway migration on CNCF internal services cluster" by Koray Oksay, CNCF/Kubermatic.

CNCF hosts a Kubernetes cluster to run some services for internal purposes [..] The migration from ingress-nginx to Envoy Gateway required careful attention to: certificate ownership and cross-namespace access; cloud load balancer integration (NodePort, health checks, externalTrafficPolicy); backend TLS configuration for services requiring HTTPS upstream connections.

2. "Duolingo's Kubernetes Leap" by Franka Passing, Duolingo.

Previously, the 500-plus backend services that I mentioned at the beginning are running on AWS ECS. We have some workloads that are running on different infrastructure, but for the mass majority, they're on ECS. That's what I'm going to be focusing on. We're going to be moving from ECS to EKS.

3. "How does the Kubernetes controller manager work?" by Daniele Polencic, LearnKube.

When you delete a Pod in a Deployment, Kubernetes creates a replacement. But who creates it? It's not the API server. The API server stores resources and notifies watchers, but it doesn't decide what should run. It's not the scheduler either. The scheduler assigns Pods to nodes, but it doesn't create them. The answer is a controller: a loop that watches for changes in the cluster and takes action to move the current state closer to the desired state.

4. "From Kubernetes Dev Setup to Production: What Actually Changes" by Georg Schwarz.

Moving towards a production deployment changed the operating model. Delivery moved to GitOps, secrets moved to SOPS, object storage moved outside the cluster, database backups became restore-tested, identity flows were customized, and observability gave us a baseline for operating the application. This article is not about how to deploy an app to Kubernetes. It walks through what had to change after it already ran there.

5. "Securing CI/CD for an open source project: lessons from Cilium" by André Martins and Feroz Salam, Isovalent at Cisco.

Cilium runs in the kernel-level networking path of millions of Kubernetes pods. If our supply chain were compromised, the blast radius would not be small. Hardening the project against that scenario is something we work on continuously, and we wanted to write down what we actually do, in detail. Most of what follows isn't Cilium-specific: any open source project running CI/CD on GitHub Actions can apply these patterns. We've also called out where we still fall short, in case any of it makes a useful starting point for someone else.

6. "Kubernetes is migrating from SPDY to WebSockets" by Henrique Cavarsan.

How Kubernetes is replacing SPDY with WebSockets for kubectl exec, attach, cp, and port-forward. I maintain an app that builds on top of Kubernetes port forwarding, so I track KEP-4006 because the streaming protocol underneath keeps changing.

7. "Back up and restore your Amazon EKS cluster resources using Velero" by Sapeksh Madan and Shalabh Srivastava, AWS.

In this post, you’ll learn to back up and restore Amazon EKS cluster resources and persistent volume data using Velero. You’ll deploy a sample stateful application, back it up, and restore it to a different namespace within the same cluster. Along the way, you’ll configure least-privilege AWS Identity and Access Management (AWS IAM) roles using Amazon EKS Pod Identity and scope Velero’s Kubernetes permissions with a custom ClusterRole.

#articles

etcd 3.7.0-beta.0 arrived with RangeStream

Last week, the first beta release of etcd 3.7.0 was announced. Its main feature is RangeStream, which significantly improves the use case of large result sets by allowing clients to retrieve results in chunks. Another highlight is that etcd v3.7 is the first release to be fully based on v3store, as the remaining parts of v2store (deprecated since v3.4) have been finally removed.

#releases #databases

Multigres v0.1 Alpha

Yesterday, Supabase announced the first public version of its new project called Multigres. It aims to make Vitess available for PostgreSQL. Basically, it means it simplifies horizontal scaling for Postgres and provides high availability, pooling, backups, and sharding. The project is Open Source and available on GitHub.

You can try the first alpha version of Multigres by deploying its Kubernetes operator on AWS EKS following these instructions.

#news #releases #databases

Announcing our latest digest of the prominent software updates in the Cloud Native ecosystem!

1. Kyverno, a Kubernetes-native policy engine (a CNCF Graduated project), released 1.18, which added secure HTTP calls with blocklist/allowlist, namespaced image registry credentials, extended policy support in kyverno apply and kyverno test, success event filtering, and many other improvements.

2. NATS, the cloud- and edge-native messaging system (a CNCF Incubating project), released its server v2.14, which added feature flags in the server configuration. It also introduced fast-ingest batch publishing, repeating and cron-based message schedules, scheduled subject sampling and scheduled subject rollups, and an asynchronous stream state snapshots for replicated streams.

3. Freelens, an IDE for Kubernetes originated as a community-driven fork of Lens, was updated to v1.9.0. It is now capable to resize the Pods by patching their resources, wrap logs for better readability, work with OpenShift Prometheus provider, use HTTP POST requests for Prometheus, and exit remote shells gracefully.

4. OpenYurt, a platform extending Kubernetes for edge use cases (a CNCF Incubating project), released v1.7.0, which added image preheating for OTA (Over-The-Air) upgrades, local K8s-on-K8s deployments with a Kubernetes cluster on top of an existing OpenYurt cluster, label-driven YurtHub deployment, and Kubernetes v1.34 support.

5. Traefik, a HTTP reverse proxy and load balancer for Cloud Native apps, released v3.7.0, featuring the support for dozens of new annotations for Kubernetes Ingress-Nginx: limits for connections/RPM/RPS, proxy timeouts, buffering, redirects, canary, session-cookie-expires, custom headers and HTTP errors, enable-access-log, allowlist-source-range, global-auth and auth-signin, and many more. Other new features include stdio access logs, priority ordering for providers routing, support for adding middlewares on services, and a new encodedCharacters middleware.

6. Istio (a CNCF Graduated project) announced 1.30, introducing experimental support for agentgateway as a Gateway API implementation, support for TLSRoute termination and mixed mode, several improvements for the ambient mode, namespace-level traffic distribution annotation, DNS upstream timeout, DNS failover priority support, configurable port overrides for the network gateway service, and many other updates.

7. Crossplane (a CNCF Graduated project) made another quarterly release, v2.3.0, adding the actual composite reconciler to crossplane render, provider deletion protection, per-resource reconciliation control, an “all resources of that kind” selector for composition functions, and a new scale subresource for XRs.

#news #releases