Server Setup Basics for Self Hosting

Especially when writing a tutorial for beginners - please use the long-form flags (e.g. sudo usermod --append --groups sudo newuser) instead of short-form flags (e.g. sudo usermod -aG sudo newuser). Short-form flags make commands look like arcane voodoo magic. They make sense only to help you save time entering commands if you know them by heart already. Tutorials are read by beginners who are not necessarily familiar with the commands in the first place - long-form flags help communicate what these commands are actually doing and thus make for a more effective tutorial.

I recommend checking out Caddy <https://caddyserver.com/>, which replaces both Nginx and Certbot in this setup.

Tailscale <https://tailscale.com/> can remove the need to open port 22 to the world, but I wouldn't rely on it unless your VPS provider has a way to access the server console in case of configuration mistakes.

I'd switch to Userify if you have a team to distribute keys for, because it's ultra-lightweight and also keeps you from messing up permissions on the ssh key/directory, which I've done too many times! (also it does sudo which is quite nice)

Also, restarting ssh will not boot you out of the session (your session has already been forked as a different process), so leave your terminal window open (to fix any screwups) and then log in on a separate window on the new port and just make sure you can get in.

For backups, don't set up logins from your main server(s) to your backup server; log in from your backup server to your main server. That way, if someone breaks into your main server, they can't get into your backup server.

You may want to update this post to disable password authentication, and thus you'll no longer need to install fail2ban. An important goal is to tighten your attack surface, not expand it. At this point you will still have an exposed SSHd server, so I'd recommend throwing the server under tailscale. You can setup the SSHd listener to use your tailscale IP or setup tailscale for SSH via ACLs (https://tailscale.com/tailscale-ssh).

Additionally you can further tighten controls of incoming logins with the use of AllowGroups to tighten your controls on which groups can log into the system. This would mitigate a scenario where an adversary is able to escalate enough privileges to write an .authorized_keys file to a non-privileged user which may have a shell still configured.

Finally, unless you're treating this server as a bastion host of sorts, you probably should disable forwarding for agents or X11 etc. We've seen a lot of adversaries move laterally due to this agent forwarding.

At the end of the article, there's a link to a script[1] that does the steps covered in the article.

That got me thinking: how do other self-hosters/homelabbers here go about automating their server setups? None/purely manual? One big shell script? Multiple scripts wrapped in a Makefile (or justfile, or other command runner)? More enterprisey provisioning/automation tools like Ansible, Puppet, etc.?

[1] https://git.sovbit.dev/Enki/sovran-scripts

Love seeing devops post on HN.

Wish it included server monitoring as a section.

> Differential backups back up all the changes since the last full backup (...) An incremental backup backs up data that was changed since the last backup

I'm not sure I understand the distinction?

So... about docker, did they backtrack on their licensing landgrab?

About a year ago I swear everyone was going to podman, but in the last few months I see nothing but docker references.

Podman is supposed to be drop-in. Well, it was advertised. I haven't touched anything in six months.

> You want to use SSH (Secure Shell) and make sure that SSH is the only way to log in.

Some distributions (like openSuSE) also enable KbdInteractiveAuthentication by default so just disabling PasswordAuthentication won't work.

Article author here. Glad some people found this useful and to those with suggestions, ill keep those in mind when updating the post.

Cheers.

And for those of you that don't have an external IP, you can use services that provide egress for you like IPv6.rs. [1]

[1] I'm DevOps there! ;)

Great post! I (relatively) recently switched my primary home server over to NixOS and am now a huge fan of it as a distribution for self-hosting.

Here is how setting this all up would like in NixOS (modulo some details & machine-specific configuration). It's <100 lines, can be executed/configured with a single CLI command (even from a different machine!), rolled back easily if things go wrong, and can be re-used on any NixOS machine :)

    {
      networking = {
        # Server hostname
        hostName = "myserver";
        # Firewall
        firewall = {
          enable = true;
          allowedTCPPorts = [ 80 443 2222 ];
        };
      };
    
      # Users
      users.users = {
        newuser = {
          isNormalUser = true;
          home = "/home/newuser";
          hashedPassword = "my-hashed-pwd";
          openssh.authorizedKeys.keys = [ "my-pub-key" ];
        };
      };
    
      # SSH
      services.openssh = {
        enable = true;
        ports = [ 2222 ];
        settings = {
          PermitRootLogin = "no";
          PasswordAuthentication = false;
          AllowUsers = [ "newuser" ];
        };
        extraConfig = ''
          Protocol 2                 # Use only SSH protocol version 2
          MaxAuthTries 3             # Limit authentication attempts
          ClientAliveInterval 300    # Client alive interval in seconds
          ClientAliveCountMax 2      # Maximum client alive count
        '';
      };
    
      services.fail2ban.enable = true;
      
      # Nginx + SSL via LetsEncrypt
      services.nginx = {
        enable = true;
        recommendedOptimisation = true;
        recommendedProxySettings = true;
        recommendedTlsSettings = true;
        virtualHosts = {
          "example.com" = {
            locations."/" = {
              proxyPass = "http://localhost:8080";
              proxyWebsockets = true;
            };
            forceSSL = true;
            enableACME = true;
          };
        };
      };
      security.acme = {
        acceptTerms = true;
        defaults.email = "myemail@gmail.com";
        certs."example.com" = {
          dnsProvider = "cloudflare";
          environmentFile = ./my-env-file;
        };
      };
    
      # Logrotate
      services.logrotate = {
        enable = true;
        configFile = pkgs.writeText "logrotate.conf" ''
          /var/log/nginx/*.log {
            weekly
            missingok
            rotate 52
            compress
            delaycompress
            notifempty
            create 0640 www-data adm
            sharedscripts
            postrotate
                [ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
            endscript
          }
        '';
      };
    
      # Bonus: auto-upgrade from GH repo
      system.autoUpgrade = {
        enable = true;
        flake = "github:myuser/nixos-config";
        flags = [
          "-L" # print build logs
          "--refresh" # do not use cached Flake
        ];
        dates = "00:00";
        allowReboot = true;
        randomizedDelaySec = "45min";
      };
    }

Here are some "First Things on a Server" Notes. https://gist.github.com/klaushardt/07f5e3068355aafc2dce660a5...

Ansible/Puppet or NixOS would be better, but this is what works in Self Hosting.

Hey I just want to say thank you for the write up. I just got into the hobby of self hosting my own applications and it's quite a bit. I appreciated your sections about logging and user management.

While these seems to be secure... tampering with default settings always cause PITA; especially during automated upgrades. In addition, ssh port changes are all security thru obscurity.

Why the `sudo ufw allow outgoing`? Wouldn't it be worth to deny all to prevent extrusion and only open ports for services that need to communicate externally?

A monitoring setup with for instance Prometheus+Grafana would be a great addition to this.

And then maybe automating all of it with something like Ansible.

With SSH Keys, do you have to just carry around your private key everywhere on a usb or something if you want to connect from multiple locations? Sometimes I find myself somewhere I've never been and I want to connect to my server. With a password that's easy, I just type it in, but I can't exactly create a new private key to access the server when I don't have access to the server in the first place

Qq, do people doing their own server setup like this use containerization at all?

When I looked at it, it was like “yeah you can run Docker or k3s,” and I think Hashicorp had their own version, but it seemed like folks didn't really bother? Also like setting up virtual networks among VPSes seemed like it required advanced wizardry.