Ha, I wish I could.

I’m not 100% satisfied, so I’m still searching for the “perfect distro for me”, if it even exists.

I have been using Arch Linux on my personal PC and company laptop for 4 years, but I couldn’t get some things to work. Things that, after installing Fedora, worked out of the box.

My current setup is:

• EndeavourOS (e.g. arch linux with a GUI-installer) for my PC at home
• Fedora Workstation 43 for my company laptop
• Servers are all running Debian, I’ll probably never change that
• Hypervisor for VMs is Proxmox VE, which is Debian too

Currently using nginx-proxy-manager for exactly this purpose. Nice and easy-to-use UI, including automatic LetsEncrypt ssl certificates :)

I’m using CheckMK to monitor my hypervisor, physical hardware like disks, CPU etc. and SNMP-capable hardware like my pfSense firewall via a CheckMK instance in docker. It either works in docker or on a few different linux based OS like ubuntu and debian (see CheckMK download page).

There’s a free and open source version (called raw edition, GitHub Link) which I am using. It comes with a lot of checks / plugins for monitoring stuff out of the box and if there’s something it doesn’t ship, you can easily create your own check in whatever language your server is capable of executing a binary of. Or you could look up if there’s a user-contributed plugin on the official CheckMK Exchange Platform.

The whole configuration of this is based on rules with a lot of predefined rules and sane defaults already set.

To have an example for your use-case: You can monitor docker-logfiles and let CheckMK warn you, if specific keywords are or are not in a logfile. You will then be able to view the offending lines in the monitoring UI.

Why do I use this?

• We use it at work
• FOSS
• docker makes updating this easy
• can send mails, teams notifications, …
• very customizable and expandable

my docker compose file

# docker-compose.yml

services:
  monitoring:
    image: checkmk/check-mk-raw:2.4.0-latest
    container_name: monitoring
    restart: unless-stopped
    environment:
      - CMK_PASSWORD=changeme
    ports:
      # WEB UI port
      - "5000:5000"
      # agent communication port
      - "8000:8000"
      # used for SNMP
      - "162:162/udp"
      - "514:514/tcp"
      - "514:514/udp"
    volumes:
      - "./monitoring:/omd/sites"
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env

Proxmox Virtual Environment (PVE, Hypervisor), my beloved. Especially in combination with Proxmox Backup Server (PBS).

My homelab would not exist without Proxmox VE, as I’m definitely not going to use Nutanix or VMWare. I love working with linux and Proxmox VE is literally debian with a modified kernel and a Management Webinterface on top.

I first learned about Proxmox VE in my company, while we still had VMWare for us and all of our customers. We gradually switched everyone over to Proxmox VE and now I’m using it at home too. Proxmox is an Austrian (my country) company, so I was double hyped about this software.

A few things I like most about Proxmox VE

• Ease of access to the correct part of the documentation you currently need (*)
• Open Source
• Company resides in my country (no US big tech walled garden)
• Linux / Debian based, so no learning new OS’s and toolchains
• Free version available
• Forum available and actually used

(*) What I mean by ease of access to the correct part of the documentation is: Whenever you’re in the WebUI and need to decide on some settings, there’s a button somewhere on the same page which is going to lead you directly to the portion of the documentation you need right now. I don’t know why this seems like such a great luxury, every software should have something like this.

Next steps

My “server” (some mini PC with spare parts I already had) is getting too weak for the workload I put it through, so I’m going to migrate to a better “server”. I already have a PC and most of the necessary parts, I just need some SSDs and an AMD CPU.

Even migrating from PVE (old) -> PVE (new) couldn’t be easier:

• PVE (old): create last backup to PBS, shut down PVE (old)
• PVE (new): add PBS, restore Backups
• ???
• profit

I think it’s great to have a series posting about personal achievements and troubles with selfhosting. There’s so much software out there, you always get to see someone doing something you didn’t even know could be done or using a software you didn’t realize even existed. Sharing is caring.

Of course you can always build a good PC or server.

I could have done that too, but I wanted my first real homelab-do-it-all-yourself setup to be a little more on the cautiously small side. I didn’t want to have too much noise in my apartment and also didn’t want to stress my electricity-bill and wallet too much, so I opted to build small and reuse what I had lying around.

I already had 2 Mini-PCs and a raspberry pi from earlier experiments with selfhosting. I just bought some disks and RAM. If you don’t have any mini-PCs, they’re relatively cheap in comparison with full PCs. Or you could use some older PC you still have but do not use.

My motto more or less was you can always spend more money and build bigger later

The final Hardware

• Mini-PC: Zotac ZBox CI665 nano
• RAM: 32 GB DDR4 RAM (according to specs, CI665 cannot go beyond 32GB sadly)
• SSD: 1x 2TB Samsung SATA SSD
• external USB HDD (6TB)

What I host on my Proxmox VE

• CheckMK Monitoring
• GitLab Runner
• pihole (network-wide, DNS-based adblocker)
• Nextcloud (production and staging instance for testing new updates before rolling out to production with my data)
• nginx proxy manager
• syncthing server

The 2nd Mini PC (some old intel NUC with 4 cores and 16 GB RAM) + a USB HDD is my Proxmox Backup Server for all this. And what’s really important (my data from nextcloud + some configs) gets backed up to my Hetzner Storage Box with restic.

The raspberry pi is now my WiFi Access Point :)

Conclusion

Homelab doesn’t need to be big or small, it can be whatever you want it to be or whatever you can afford or are willing to have and maintain. From my experience, if you’re not hosting anything CPU-intensive, older or smaller machines will do just fine.

For example, my nextcloud could easily use more resources than the whole Zotac ZBox could house, if there were more users. But as my services are only used by me, most of them are idle most of the time.

Tip at the end about your opsense-VM on Proxmox

I tried letting Proxmox host my pfSense too, but that got old pretty fast. Whenever Proxmox needed a Reboot, my internet was gone too for that time, as the pfSense VM on Proxmox was the gateway to my ISP-modem. In the end, I just bought a real Netgate pfSense appliance.