Every engineer ends up with a stack that’s part necessity, part indulgence.
Mine started as a way to keep remote work, backups and research experiments from falling over each other. It’s since become a quiet testbed — somewhere to watch how systems behave when they’re stitched together from real hardware rather than drawn on a slide.
The edge: symmetric, and that matters
At the boundary is a 1 Gbps symmetric fibre line terminating in an Optical Network Terminal.
Symmetric is the part worth dwelling on. Most domestic connections assume you are a consumer: generous down, miserly up. My work and hobbies push as much data out as in — VM snapshots, firmware repositories, remote lab traffic, the occasional backup of something ill-advised but interesting. An asymmetric line quietly shapes what you are willing to attempt.
The ONT feeds a compact x86 board: four 2.5 GbE copper ports, two 10 Gb SFP+ cages, fanless and low-power. Considerably over-specced, in the best way.
Virtualising the gateway
That board runs Proxmox VE. Each network port is passed directly through to a virtual machine running OPNsense, my firewall and router of choice.
Why do it this way? Because configuration drift and experimentation are constant. Proxmox lets me snapshot before an upgrade, clone a test setup, and run a temporary routing experiment without taking the household offline — which is the difference between an environment you learn from and one you are frightened of.
There is an honest cost, and it took me a while to name it: the hypervisor becomes part of the network’s control plane. Lose that box and you lose the router with it. Everything about how this gets backed up follows from that one sentence.
The 10 Gb fabric
One SFP+ port runs a Direct-Attach Copper line into a managed 10 Gb switch. The other carries fibre to a second switch elsewhere on the property, where the NAS and a rotating cast of prototypes live, depending on what I’m playing with that week.
The case for it is not the internet connection — that’s a tenth of the speed. It’s everything that never leaves the building. A full VM backup crosses the fabric in about two minutes. When moving a dataset costs nothing, you move it; when it costs twenty minutes, you don’t, and you quietly stop doing the experiment.
Storage and the service edge
A NAS holds the data and runs local model inference. A separate container host runs Traefik as the internal service edge, handling TLS termination and reverse-proxying for the handful of self-hosted applications I actually use — this blog among them. Certificates are issued over DNS-01, so nothing needs an inbound HTTP path merely to prove a domain.
The division of labour stays clear: OPNsense owns ingress, Traefik owns internal presentation and encryption, the NAS owns persistence.
Then I audited what I had just described
Everything above is what I would have told you the network was. Writing it down is a claim; an audit is the test of the claim. So before publishing any of it I went through the thing properly — the hypervisor, the firewall, the switch, the containers, the public DNS and the certificate transparency logs — and the two versions disagreed in five places.
What follows is uncomfortable to publish and is the most useful part of the exercise. Four of the five are fixed, and none of them were when I started writing. The fifth is the flat network itself — not a setting to correct but a migration to sequence. It is still open, and it is the next piece of work.
The segmentation existed in the diagram, not the cabling
I had been telling myself the network was segmented. It wasn’t. Every port sat in a single bridge, so cameras, IoT devices, management interfaces and workstations shared one broadcast domain — a flat /24 that I had been mentally drawing with VLANs the cabling had never been given. I had even written “completing VLAN segmentation” in an earlier draft of this post, which implies a partial progress that did not exist. It is still one bridge as I publish this.
The router had no backup. Not a stale one — none
The single component the entire household depends on, running as a virtual machine on a box with one NVMe drive, had never been backed up in any form. Neither had anything else on that hypervisor. I had built the argument for virtualising the gateway around being able to snapshot before an upgrade, and had never once taken a snapshot.
Two fixes that would have done nothing
This is the pattern I keep coming back to, because it is the one that actually costs you time: not a missing fix, but a fix that looks correct, applies cleanly, and changes nothing.
Internal name resolution was broken — not one internal hostname resolved. The obvious remedy is a setting in the resolver that registers DHCP leases in DNS, and it is right there in the interface. It would have done nothing, because leases here are handed out by a different daemon from the one that setting configures. I would have ticked the box, seen no change, and blamed propagation.
The second was in the home automation stack, where the option I reached for had moved out of the configuration file and into the interface a release earlier. The file still parses. It is imported once and then ignored. You get no warning, no error, and a config file that reads exactly as though it is in charge.
A dashboard I was confident was internal
The reverse proxy’s admin dashboard, which enumerates every route, service and middleware on the network, was reachable from the public internet. No credentials, no allowlist.
The path to it needed no guesswork on an attacker’s part. Certificate Transparency publishes every hostname you ever request a certificate for, so the name was already a matter of public record; the apex resolves to a public address; a request to that address carrying the right Host header arrived. My first test returned a 200 — from inside my own network, which proves nothing at all, because NAT reflection would produce exactly that. The only test worth anything came from a machine in Ohio with no relationship to my house.
It answers 403 now, while the blog returns 200 from the same vantage point, which is how you know you are looking at a rule and not a broken route. That distinction — between a test that confirms what you hoped and a test that could have proved you wrong — is most of what separates an audit from a look around.
And two quieter ones
The container management interface was listening on every address on its host while holding a read-write socket to the container daemon, which is as close to a root shell as makes no difference. Restricting it meant going around the NAS appliance’s own application template, which insists on host networking and silently discards the address restriction you hand it. The abstraction did not refuse the setting. It accepted it and dropped it.
And the home automation instance had its IP banning switched off, sitting behind a proxy it had not been told to trust — so every login attempt in the world appeared to originate from the proxy. The ban feature would have had nothing meaningful to ban even had it been enabled: block the abusive address and you block your own front door.
The circle you don’t want to discover at 2am
Fixing the backup gap turned out to be more interesting than expected, because the obvious design has a bootstrap problem hiding in it.
Back the router VM up to the NAS nightly and you have a perfectly good backup. Now lose the drive in the router box. The recovery is: install the hypervisor, then fetch a several-gigabyte archive from the NAS across the network — except the router that makes that network work is inside the archive you cannot reach yet. The network you need in order to reach the backup is the thing being backed up.
The answer is not a better backup, it is a second one with a different failure mode. The VM image restores the whole machine as it was and depends on the network to arrive. The firewall’s configuration export is 58 KB of XML that rebuilds the entire configuration onto a bare install, and fits on a USB stick, in a password manager, on a phone. With it the network comes back first, and then you can reach everything else.
58 KB is the difference between a bad evening and a very bad week.
What is actually worth keeping
The same reasoning applies to what goes off-site, and it produced the most counter-intuitive number of the whole exercise.
The storage pool is just under 10 TB, which sounds like a hard rationing problem against the cloud storage I already pay for. It isn’t, because of how the data is distributed: 9.57 TB of it is workstation images and installer media, and everything else combined — configuration, container state, mail archives, documents, credential exports — comes to about 115 GB.
Machine images are the least valuable thing per gigabyte you can put in off-site storage. They restore a machine you could rebuild anyway, and they go stale the week after you take them. Everything genuinely irreplaceable is a rounding error next to them. So the question was never “which 5 TB do we pick”; it was “send the 46 GB that matters, verify it, and stop”.
The first tier of that is under 100 MB — the firewall config, the container definitions, the certificate store, the NAS configuration, a password-manager export. It fits on a phone, and it converts a total-loss scenario from impossible into a tedious weekend.
The thing that could not be recovered
None of this is hypothetical. In August I lost the contents of a share, and the investigation only finished while I was writing this.
The pool’s own history rules out the dramatic explanations: no dataset was destroyed or renamed. The loss happened at the file level, and it falls inside the window in which the NAS applied a system update and rebooted. Which process actually removed the files is still unidentified — I have narrowed it to that window rather than explained it, and I would rather say so than invent a tidy ending.
The part that is fully explained is why none of it came back. The datasets holding live application state had never had a single snapshot taken. Not a lapsed schedule — none had ever existed. That is not why the data was deleted. It is precisely why none of it was recoverable, and that is the failure I actually own.
Knowing the fundamentals
One habit I’ve never managed to break is decomposing tools down to their simplest operations.
Wireshark? Packet capture and protocol dissection. Dragos? Asset discovery and correlation wrapped around network telemetry.
Once you understand the primitives — what is being copied, parsed or filtered — everything else becomes navigable. The same logic applies to the network itself: know what each part really does, and the rest is orchestration.
Building this stack was partly an exercise in refusing dependency. If a component fails, I can rebuild it. If a vendor vanishes, the logic remains. The audit was a reminder that the same scepticism has to point inward: a setting you applied is not a change you made, and a diagram you drew is not a network you have.
Infrastructure as thought experiment
This network isn’t a cyber-physical miniature or a statement piece. It’s a working environment that mirrors how I approach assurance and engineering more broadly: make reasoning visible, keep options open, design systems you can learn from.
It reminds me that even at home, theory only matters once it meets a cable.
Looking ahead
Segmentation is the next real piece of work rather than a finished one, followed by getting the last of the backups off-site and snapshots onto the datasets that still have none.
A home network is never finished — just iterated. That’s the fun part.