Spotlight

Cost Optimization of Spark on Kubernetes Batch Workloads on Public Clouds

Ankit Prakash Gupta

This case study shows how Spark on Kubernetes batch workloads cut cloud costs by colocating executors with drivers, reducing inter-zone shuffle costs, and moving executors to Spot VMs safely.

More articles →

Tools and utilities

  • Migratowl

    Migratowl is an AI-powered dependency migration analyzer that upgrades dependencies in isolated sandboxes, runs tests, reads changelogs, explains breakages, and suggests fixes.

  • Kondense: automated resouce sizing

    Kondense is an automated resource sizing tool.

  • kubectl-mcp-server – Kubernetes Management via MCP

    kubectl-mcp-server lets AI assistants use natural language to inspect and manage Kubernetes clusters through kubectl operations, Docker support, kubeconfig mounting, and MCP-compatible tooling.

  • Distribution tooling for Helm

    dt is a set of utilities available as a standalone or Helm Plugin for making offline work with Helm charts easier.

  • Cardamon: Prometheus Metric Cleanup Tool

    Cardamon audits Prometheus metrics against query logs, rules, and Grafana dashboards, then finds unused series and generates drop relabeling rules to reduce TSDB storage and cardinality waste.

More projects →

Events starting soon

Discover more events onn Kube Events →

The Hidden Cost of Slow Autoscaling
The Hidden Cost of Slow Autoscaling

Forced platform migrations are usually treated as something to survive. At Scout24, a mandatory OS migration became an opportunity to rethink Kubernetes autoscaling, node provisioning, and infrastructure efficiency.

John Ford explains how Scout24 moved its EKS-based Infinity platform from a polling autoscaler and over-provisioned capacity to Karpenter and Bottlerocket. The result was faster node startup, a safer migration path, and about a 30% infrastructure reduction without major downtime.

In this interview:

  • Why two-minute node provisioning forced a 25% capacity buffer
  • How Karpenter made the Bottlerocket migration safer
  • What broke around EC2 metadata, AWS SDKs, and cgroups
  • How the new foundation enables Spot, ARM, and GPU workloads

Learn from production

More case studies →

Matching jobs

    • Data Engineer with Capco

    • Salary: $162 to $2.64L a year

    • Location: based in the office in Bangalore, IN; Pune, IN; Hyderabad, IN

    • Tech stack: Kubernetes, AWS, Docker, SQL, SQL Server, Kafka

    • Data Engineer with Capital Technology Group

    • Salary: $54K to $275K a year

    • Location: remote from

    • Tech stack: Kubernetes, AWS, Docker, Java, Python, SQL, PostgreSQL, Kafka, Terraform, Splunk

    • Machine Learning Engineer with Capital Technology Group

    • Salary: $150K to $230K a year

    • Location: remote from

    • Tech stack: Kubernetes, AWS, Docker, Python

    • Platform Engineer with Cake

    • Salary: $116.55K to $401.28K a year

    • Location: remote within UTC-5 and UTC+2

    • Tech stack: Kubernetes, AWS, Azure, GCP, Go, Typescript, Terraform, CDK, Istio

    • Platform Engineer with Calendly

    • Salary: $238.97K to $347.49K a year

    • Location: remote from

    • Tech stack: Kubernetes, GCP, Go, Datadog

Discover more Kubernetes jobs on Kube Careers →

Subscribe to Learn Kubernetes Weekly

Trusted by 77K engineers. Delivered 192 issues and counting.

or subscribe via

Build something

More tutorials →

Call for Papers closing soon

  1. 6

    days

    WeAreDevelopers Conference India

    The Call For Paper is open until 24 July 2026 at GMT-4. More info →
    • Location: Bengaluru, IN

    • In-person conference organized by WeAreDevelopers.

    • The conference starts on the 26 November 2026.

    • Apply here
  2. 13

    days

    ContainerDays & AI Context Singapore

    The Call For Paper is open until 31 July 2026 at GMT-4. More info →
    • Location: Singapore, SG

    • In-person conference organized by ContainerDays.

    • The conference starts on the 28 October 2026.

    • Apply here
  3. 13

    days

    Cloud Native Summit APAC 2026

    The Call For Paper is open until 31 July 2026 at GMT-4. More info →
    • Location: Melbourne, AU

    • In-person conference organized by Cloud Native Summit.

    • The conference starts on the 29 October 2026.

    • Apply here
  4. 13

    days

    ServerlessDays Cardiff 2026

    The Call For Paper is open until 31 July 2026 at GMT-4. More info →
    • Location: Cardiff, GB

    • In-person conference organized by ServerlessCWL.

    • The conference starts on the 3 October 2026.

    • Apply here
  5. 27

    days

    Cloud Native AI Summit Europe

    The Call For Paper is open until 14 August 2026 at GMT-4. More info →
    • Location: Paris, FR

    • In-person conference organized by mate.dev.

    • The conference starts on the 3 December 2026.

    • Apply here
  6. 28

    days

    Cloud Security Space 2026

    The Call For Paper is open until 15 August 2026 at GMT-4. More info →
    • Location: Buenos Aires, AR

    • In-person conference organized by Cloud Security Space.

    • The conference starts on the 9 October 2026.

    • Apply here
  7. 41

    days

    Monster Scale Summit 2027

    The Call For Paper is open until 28 August 2026 at GMT-4. More info →
    • This is a virtual event

    • Online conference organized by ScyllaDB.

    • The conference starts on the 11 March 2027.

    • Apply here

Thanks to our sponsors who make Kube Today possible

Find out more about being a sponsor →

More articles

Even more articles →