July 2nd, 2024

For the Love of God, Stop Using CPU Limits on Kubernetes

Using CPU limits on Kubernetes can lead to CPU throttling, causing more harm than good. Setting accurate CPU requests is crucial to avoid throttling. Memory management best practices are also discussed, along with a tool for resource recommendations.

Read original articleLink Icon
For the Love of God, Stop Using CPU Limits on Kubernetes

The article discusses the drawbacks of using CPU limits on Kubernetes, emphasizing that they often do more harm than good, leading to CPU throttling. The author provides analogies to explain the negative impact of CPU limits, highlighting the importance of defining CPU requests instead. By setting accurate CPU requests, pods can access the CPU they need without being throttled, even without limits. The article recommends using CPU requests for all pods, ensuring accuracy, and avoiding CPU limits. It also mentions that memory management differs from CPU and provides best practices for memory limits and requests. Additionally, a tool called Kubernetes Resource Recommender is introduced to help determine CPU and memory requests based on historical data. The article concludes with a reminder to always use memory limits and requests and to set memory requests equal to limits.

Related

Lessons Learned from Scaling to Multi-Terabyte Datasets

Lessons Learned from Scaling to Multi-Terabyte Datasets

Insights on scaling to multi-terabyte datasets, emphasizing algorithm evaluation before scaling. Tools like Joblib and GNU Parallel for single machine scaling, transitioning to multiple machines, and comparing performance/cost implications. Recommendations for parallel workloads and analytical tasks using AWS Batch, Dask, and Spark. Considerations for tool selection based on team size and workload.

Should you upgrade GPU or CPU for faster gaming? Many hardware combos tested

Should you upgrade GPU or CPU for faster gaming? Many hardware combos tested

Tom's Hardware study compares CPU and GPU upgrades for gaming. Pairing top GPUs with older CPUs can lead to 40% performance drop at 1080p. Balanced upgrades crucial for optimal performance across settings.

My First Kubernetes: k3s 'cluster' on 3 Orange Pi Zero 3's

My First Kubernetes: k3s 'cluster' on 3 Orange Pi Zero 3's

Setting up a Kubernetes cluster with Orange Pi Zero 3 boards involves control and worker nodes, deploying apps, and overcoming ARM64 image compatibility challenges. Detailed steps, including Dashboard setup and user management, are shared, offering practical insights for Kubernetes enthusiasts.

The weirdest QNX bug I've ever encountered

The weirdest QNX bug I've ever encountered

The author encountered a CPU usage bug in a QNX system's 'ps' utility due to a 15-year-old bug. Debugging revealed a race condition, leading to code modifications and a shift towards open-source solutions.

Cubernetes

Cubernetes

Justin Garrison built "Cubernetes," a visually appealing Kubernetes hardware lab for training and content creation. The $6310 setup included unique parts like Mac Cube cases and LP-179 computers with Intel AMT support. Creative solutions like 3D printing and magnetic connectors were used. Lights were controlled by attiny85 and Raspberry Pi Pico for visualizations. The project prioritized functionality and education.

Link Icon 6 comments
By @danjrslp - 5 months
I used to be on the same page as the author. But then I saw tons of application teams not setting CPU limits, and coming to rely on the bursting (in other words, their requests were too low). Thus when the system came under load their application started slowing in unexpected ways.

We've had success with CPU limits, and horizontal scaling.

By @jauntywundrkind - 5 months
It's sad to me that Kubernetes doesn't expose the excellent hierarchical system for rationing CPU that's built into the kernel: cgroups. It has its own separate constraint system, makes its own scheduler. And it just seems not as good, not as flexible, as the hierarchical system cgroups offers.

Being able to make a cgroup where essential services as a whole share a pool guaranteed 30%, then further refining & trading off that pool & other work pools feels like such a superpower. Compared to having to manage all services in flat, absolute terms.

By @rcarmo - 5 months
I’d say being able to set I/O limits can be much more useful than CPU limits regardless of platform. Less chance of bringing an entire host to a halt.
By @sadops - 5 months
Can't your operating system manage your CPU resources for you already? Why does Kubernetes need to be involved in process scheduling?
By @gladiatr72 - 5 months
Almost 2-yo post and still as wrong as it was when it was posted.