This is my dream as well, but for security I feel like you need multiple independent systems. I’m doing mine with power-hungry recycled 2012-vintage server hardware (Xeon E5-1620s and 2620s and Opteron 6276s, bought for $100 each several years ago, plus a few hundred more to their maximum amounts of DDR3 ECC) but this hypothetical box could easily have raspberry pis or something similar. Public services can become compromised and you’ll only want certain hardware to be trusted to do certain things.
My plan is a terrible one and I’m taking way too long to do it. I really want someone else to build this better and faster, but if my crappy plan ends up being the first usable version of this, that will suck but at least it’s available.
I had a dumb personal domain from June of 2000, tried to make it a public internet site, offered services to people on IRC for internet social points, but after a few years it got ahead of me and I let it die. (I’ve been paying for the same business internet ever since, though, and I still have the same static IPs as from back then.) Time passed, got married, got a computer science degree and a development job with a billion dollar SAAS company.
I can see how they do big public internet hosting. I want everyone to be able to do this, too. Been trying to build the same kinds of architecture with open source tools at home. Struggling, I keep over designing it and getting stuck and frustrated. It takes me a month to do what a competent ops person from work does in a couple days.
OnceI have this working for me, I can share it, because it’s my own work product. It’ll be a guide, a recipe to follow, for creating the kind of secure and isolated web application and general VM hosting environment I see us use at work. This stuff is the difference between “I’m hosting one thing and if it gets hacked, everything is owned” and “I’m hosting a hundred things, all different, and if one gets hacked that will suck - but the other 99 things will stay safe.”
Biggest problem I think with creating this with open-source is just picking a direction for everything and getting the internet to not pitch a fit. “Why did you use postfix?” “I hate Greenbone / GSA and refuse to use it.” “Hardware is expensive, you say I need a jump box for this AND for this, and dedicated hardware for a firewall here AND here? Each of those could clearly be a VM. Your project wastes hardware and I’m not doing it this way.”
Sure, once this is done these decisions are pretty much baked in and I won’t have the energy to redo them yet again. But getting the architecture perfectly designed for your exact scenario … that takes a ton of work. Big companies pay a ton of money in just payroll hours to build this kind of thing bespoke for their needs. I’ll be giving away my version, and I’m afraid the internet won’t care.
But I think we need to keep this ability alive, that private citizens can set up their own DIY hosting that can stand up to hostile internet actors decently well. They can pay (I’ll grant) exploitative rates for business internet connections so they can have static IPs at home as well. If we all stop, we all just decide all hosting should be done by big cloud service companies or big enterprises, we lose a crucial bit of internet freedom. Someone needs to say “yeah this is kinda dumb but I’m doing it anyway.”
And if they could do it with a box you just plug in, instead of my (likely) month-long two hundred step recipe, and still have it stand up to attacks and “Internet background radiation” and stuff, that would be epic. I kind of don’t want my thing to be the way that self-hosting-public-web-services is done.
To add to Onomatopoeia’s excellent post, separate devices also limit the blast radius of any compromise. Attackers pivot when they compromise a system. They use one system to talk to others and attack them from inside your network. So you don’t want everything on the same OS kernel.
Unfortunately I don’t feel like I’m qualified to say what works well yet, not until I have the pieces of my site put together and working, and vetted by whatever security professionals I can get to look at it and tell me what I did wrong.
But right now I think that looks like every service VM on its own VLAN on a /30 net, and ideally the service VM and firewall/router VM serving it on different physical hardware joined by a managed switch. That managed switch shouldn’t let either VM host touch its management VLAN, and (I think, I don’t do this yet) should send monitor traffic to yet another physical host for analysis.
(“I can see why you’re not done yet” - yeah I know.)