Spotlight

Cost Optimization of Spark on Kubernetes Batch Workloads on Public Clouds

Ankit Prakash Gupta

This case study shows how Spark on Kubernetes batch workloads cut cloud costs by colocating executors with drivers, reducing inter-zone shuffle costs, and moving executors to Spot VMs safely.

More articles →

Tools and utilities

  • Migratowl

    Migratowl is an AI-powered dependency migration analyzer that upgrades dependencies in isolated sandboxes, runs tests, reads changelogs, explains breakages, and suggests fixes.

  • Kondense: automated resouce sizing

    Kondense is an automated resource sizing tool.

  • kubectl-mcp-server – Kubernetes Management via MCP

    kubectl-mcp-server lets AI assistants use natural language to inspect and manage Kubernetes clusters through kubectl operations, Docker support, kubeconfig mounting, and MCP-compatible tooling.

  • Distribution tooling for Helm

    dt is a set of utilities available as a standalone or Helm Plugin for making offline work with Helm charts easier.

  • Cardamon: Prometheus Metric Cleanup Tool

    Cardamon audits Prometheus metrics against query logs, rules, and Grafana dashboards, then finds unused series and generates drop relabeling rules to reduce TSDB storage and cardinality waste.

More projects →

Events starting soon

Discover more events onn Kube Events →

The Hidden Cost of Slow Autoscaling
The Hidden Cost of Slow Autoscaling

Forced platform migrations are usually treated as something to survive. At Scout24, a mandatory OS migration became an opportunity to rethink Kubernetes autoscaling, node provisioning, and infrastructure efficiency.

John Ford explains how Scout24 moved its EKS-based Infinity platform from a polling autoscaler and over-provisioned capacity to Karpenter and Bottlerocket. The result was faster node startup, a safer migration path, and about a 30% infrastructure reduction without major downtime.

In this interview:

  • Why two-minute node provisioning forced a 25% capacity buffer
  • How Karpenter made the Bottlerocket migration safer
  • What broke around EC2 metadata, AWS SDKs, and cgroups
  • How the new foundation enables Spot, ARM, and GPU workloads

Learn from production

More case studies →

Matching jobs

    • DevOps Engineer with North Point Technology

    • Salary: $70.74K to $440K a year

    • Location: based in the office in Herndon, VA, USA

    • Tech stack: Kubernetes, AWS, Azure, GCP, Java, Python, SQL, Cloudformation, Gitlab, Jenkins

    • DevSecOps Engineer with Nebius

    • Salary: $135K to $385K a year

    • Location: remote from

    • Tech stack: Kubernetes, SQL, Splunk

    • DevSecOps Engineer with NinjaTrader

    • Salary: $145K to $195K a year

    • Location: based in the office in Chicago, IL, USA

    • Tech stack: Kubernetes, GCP, Go, Python, Terraform

    • DevSecOps Engineer with North Point Technology

    • Salary: $77.76K to $485.65K a year

    • Location: based in the office in Springfield, VA, USA

    • Tech stack: Kubernetes, AWS, Docker, Typescript

Discover more Kubernetes jobs on Kube Careers →

Subscribe to Learn Kubernetes Weekly

Trusted by 77K engineers. Delivered 192 issues and counting.

or subscribe via

Build something

More tutorials →

Call for Papers closing soon

  1. 0

    days

    Devopsdays Cairo

    The Call For Paper is open until 17 July 2026 at GMT-4. More info →
    • Location: Cairo, EG

    • In-person conference organized by Devopsdays.

    • The conference starts on the 26 September 2026.

    • Apply here
  2. 1

    days

    Kubernetes Community Days Sofia 2026

    The Call For Paper is open until 18 July 2026 at GMT-4. More info →
    • Location: Sofia, BG

    • In-person conference organized by KCD Sofia.

    • The conference starts on the 29 September 2026.

    • Apply here
  3. 7

    days

    WeAreDevelopers Conference India

    The Call For Paper is open until 24 July 2026 at GMT-4. More info →
    • Location: Bengaluru, IN

    • In-person conference organized by WeAreDevelopers.

    • The conference starts on the 26 November 2026.

    • Apply here
  4. 14

    days

    ContainerDays & AI Context Singapore

    The Call For Paper is open until 31 July 2026 at GMT-4. More info →
    • Location: Singapore, SG

    • In-person conference organized by ContainerDays.

    • The conference starts on the 28 October 2026.

    • Apply here
  5. 14

    days

    Cloud Native Summit APAC 2026

    The Call For Paper is open until 31 July 2026 at GMT-4. More info →
    • Location: Melbourne, AU

    • In-person conference organized by Cloud Native Summit.

    • The conference starts on the 29 October 2026.

    • Apply here
  6. 14

    days

    ServerlessDays Cardiff 2026

    The Call For Paper is open until 31 July 2026 at GMT-4. More info →
    • Location: Cardiff, GB

    • In-person conference organized by ServerlessCWL.

    • The conference starts on the 3 October 2026.

    • Apply here
  7. 28

    days

    Cloud Native AI Summit Europe

    The Call For Paper is open until 14 August 2026 at GMT-4. More info →
    • Location: Paris, FR

    • In-person conference organized by mate.dev.

    • The conference starts on the 3 December 2026.

    • Apply here

Thanks to our sponsors who make Kube Today possible

Find out more about being a sponsor →

More articles

Even more articles →