Spotlight

The AI Inference Stack in 2026: GPUs, KV Caches, Routing, and Kubernetes

Rishi Chhabra

This article maps the AI inference stack from gateways and GPUs to engines, KV caches, routing, distributed serving, Kubernetes, and the metrics that explain token latency.

More articles →

Tools and utilities

  • Cardamon: Find Unused Prometheus Metrics

    Cardamon finds Prometheus metrics that are stored but never queried by alerts, rules, or Grafana, then shows their cardinality and last use and creates safe drop rules.

  • KusionStack: Unified IaC Engine for Kubernetes and Cloud

    KusionStack is a declarative infrastructure as code platform that lets you define, preview, and apply changes to Kubernetes and cloud resources.

  • Spin Operator: WASM on K8s

    Spin Operator enables Wasm-based Spin apps to run on Kubernetes using custom CRDs and a controller built with Kubebuilder.

  • etcd-k8s-extract

    etcd-k8s-extract reads an etcd data directory or db file from a cluster and outputs all stored Kubernetes resources as YAML files, organized by kind and namespace.

  • Chaos Snake

    chaossnake is a Go terminal game that deletes real Kubernetes pods as you play Snake.

More projects →

Events starting soon

Discover more events onn Kube Events →

Subscribe to Learn Kubernetes Weekly

Trusted by 77K engineers. Delivered 150 issues and counting.

or subscribe via

Learn from production

More case studies →

Matching jobs

    • Data Engineer with GitLab

    • Salary: $99K to $220K a year

    • Location: fully remote

    • Tech stack: Kubernetes, GCP, Docker, Python, SQL, Snowflake, Airflow, Terraform, Gitlab

    • Data Engineer with Inter

    • Salary: $4.32K to $220K a year

    • Location: based in the office in Belo Horizonte, BR

    • Tech stack: Kubernetes, AWS, Python, SQL, PostgreSQL, MySQL, Kafka, Airflow, Spark, Terraform

    • Data Engineer with Protegrity

    • Salary: $54K to $4.13L a year

    • Location: based in the office in Navi Mumbai, IN

    • Tech stack: Kubernetes, AWS, Azure, GCP, Java, Python, Shell, Hadoop

    • Data Engineer with Smarkets

    • Salary: US$112.5k to US$291.5k a year

    • Location: based in the office (and remote from home) in London, GB

    • Tech stack: Kubernetes, AWS, GCP, Docker, C++, Python, Rust, Elastic Search, Kafka, Airflow

Discover more Kubernetes jobs on Kube Careers →

Thanks to our sponsors who make Kube Today possible

Find out more about being a sponsor →

Build something

More tutorials →

More articles

Even more articles →