Show HN: Kardinal – Building light-weight Kubernetes dev ephemeral environments

Not gonna lie. I designed a system that looks exactly like this. However, the plugin system was an operator instead of annotating a service which is kinda cleaner but it still felt dirty. A better approach is having a service registry to track the versions, values and clusters where this has been pushed but that means you needed a different API or extend the k8s API and deal with the whole auth/authz system.

I think the biggest hurdle to implement a solution like this is your application MUST be a 12 factor app which sadly is not the case from what I've seen. Many devs hardcode values when shipping their code and containers which make it really difficult to add nodes to a DAG. Had this change, SDLC testing would be waaay easier.

In the same space, labeling of all services is a thing. As you are annotating it can be tracked but if you had different gateways or destination routes or other things that need to be tweaked at the mesh level, it could be daunting to track it. In my case, I ensure the whole mesh representation of a version could be tracked by a label so if you had a app=vote1,version=release-4,feature=shiny you could track the shiny components across the whole cluster.

Another hurdle is you are tied to a service mesh implementation. Istio is ok but it can be a beast. It also constraints your org to implement other networking alternatives which is something we wanted to explore.

I do like the project uses Nix =).

Man this brilliant and goes straight to test it with my development workflow. Namespace isolation was always PITA,but full isolation would require a dedicated clusters so having a best of both solution really changes the game for lean and cost effective continues delivery.

As for value prop, maybe emphasize multi tenancy a bit more as this has the most cost saving potential at scale.

Reading the docs, I consistently get blank page with an error after less than 30s: "Application error: a client-side exception" it feels strange that a text document needs to be an Application. I feel so old.

How is this different from vcluster? In fact vcluster has no dependency on Istio which I think is a huge detriment to kardinal. We use cilium so the istio requirement becomes a non starter for us.

I'm kind of new to the k8 world, can I ask how this compares to the approach taken by Tilt?

Looks great. Unfortunately, I have no desire to manage Istio in my clusters. Would be interested one alternative methods are available.

This looks interesting, but is there a reason the kardinal-cli deploys kardinal-manager into the default namespace rather than taking a --namespace argument (for example)? Personally I never run anything in the default namespace but I am probably an outlier.

Congratulations on shipping! I like how opinionated Kardinal is, which means it should work nicely for anyone who shares the same kind of infras vision as you.

It's also such an interesting moment for you folks to show up on HN, I just shipped the first preview version of my Kubernetes operator(https://github.com/pier-oliviert/sequencer) that manages ephemeral environments. I can see some things that are similar with both our options as well as some things that are quite different.

Maybe if I had one question is: What made you go for Istio as the main network mesh?

Good luck with the launch!

Looks interesting. I currently deploy dev instances as a single pod with everything in it, basically exactly the same as what devs get if they run docker compose locally. It works but pretty wasteful for many cases that don't need their own db etc. Is this supposed to make that less wasteful? Is it any different in the "full app"/end to end mode?

There is an alternative and complementary approach worth considering - instead of duplicating infrastructure in a smaller scale, you can simplify it to the simplest possible requirements needed to make your service run. Consider an old shared hosting - a single server could run 100s of wordpress websites just as well.

unfortunately, this tool assumes full coverage with k8. but let's say you have an external pub/sub solution and your services are decoupled. that's a pretty low bar.

sidenote: I noticed Istio (envoy actually) has some weird non-deterministic behavior when you hit pod resource limits (504 bad gateway, 0DC)

Maybe I am misunderstanding the value prop but how does this compare to namespaces?

wow, kardinal sounds cool for consolidating pre-production clusters! let's stop duplicating and start simplifying our kubernetes environments. who else is all in for the hyper-lightweight multitenancy framework trend? let's discuss!