Talos: Secure, immutable, and minimal Linux OS for running Kubernetes

I considered deploying Talos a few weeks ago, and I ran into this:

https://github.com/siderolabs/talos/issues/8367

Unless I’ve missed something, this isn’t a big deal in an AWS-style cloud where extra storage volumes (EBS, etc) have essentially no incremental cost, and maybe it’s okay on bare metal if the bare metal is explicitly designed with a completely separate boot disk (this includes Raspberry Pi using SD for boot and some other device for actual storage), but it seemed like a mostly showstopping issue for an average server that was specced with the intent to boot off a partition.

I suppose one could fudge it with NVMe namespaces if the hardware cooperates. (I’ve never personally tried setting up a nontrivial namespace setup.)

Has anyone set up Talos in a useful way on a server with a single disk or a single RAID array?

Thanks for the interest in Talos Linux! I work at Sidero (creators of Talos) and there are lots of “secure, immutable, and minimal” Linux distos out there.

Something that Talos does differently is everything is an API. Machine configuration, upgrades, debugging…it’s all APIs. This helps with maintaining systems way beyond the usual cloud-init and systemd wrappers in other “minimal” distros.

The second big change is Talos Linux is only designed for Kubernetes. It’s not a generic Linux kernel+container runtime. The init system was designed to run the kubelet and publish an API that feels like a Kubernetes native component.

This drastically reduces the Linux knowledge required to run, scale, and maintain a complex system like Kubernetes.

I’ve been doing a set of live streams called Talos Linux install fest walking new users through setting up their first cluster on Talos. Each install is in a new environment so please check it out.

https://www.youtube.com/siderolabs/streams

We use Talos really extensively in production. It’s been an amazing solution for our Kubernetes clusters. Highly recommended for a really smart, really directed Linux distro.

Previously on Hacker News:

https://news.ycombinator.com/item?id=37846995

https://news.ycombinator.com/item?id=31798362

https://news.ycombinator.com/item?id=31486815

This team has a pretty active YouTube channel that is worth checking out too.

https://youtube.com/@siderolabs

Around 90MB when downloaded compress. Not Sure am I the only one who is interested in its size when I see the word minimal.

We've been using it for a while, and I'm absolutely happy with the project.

Before that, we had a Kubespray based setup. It's a bunch of Ansible script and it allows to make any custom setup, like absolutely anything as you in control of the machines. But the other side of this is that it's extremely easy to break everything. Which we did a couple of times. And so any upgrade is a risk of loosing the whole cluster, so we decided it must be run in VM with full backup before each upgrade. Another problem that it takes about an hour to apply a change, because Ansible has to apply all the scripts each time.

Then we migrated to Talos, and it's a day and night. The initial setup took like an hour, including reading the docs and a tutorial. Easy to setup, easy to maintain, easy to upgrade (and it takes minutes). Note that we run the nodes as VMs in Proxmox, so the disk and network setup are outside of Talos scope, as well as backups, and it's actually simplifies everything. So it "just works" and we can focus on your app not the cluster setup.

A related insightful read: https://www.siderolabs.com/blog/there-are-only-12-binaries-i...

I think a word is missing from the front page:

Talos improves security further by mounting the root filesystem as read-only and removing any host-level such as a shell and SSH.

After host-level, probably 'access'.

The documentation seems to be lacking. I am specifically interested in gvisor and kata support, but cannot find information on installing additional runtimes

Can you install a shell? Does it work with MS Secure Boot? If not, what small distro might do that?

Is this available and usable on a raspberry pi?

I'm comfortable with using alpine, is this another one of the many container solutions for tiny containers?

If you can't login to it then it is not good for development. If it is not good for development it is not good for production because ideally your dev and production environment should be the same.