I am technologically skilled to some extent, but not nearly enough to justify the financial investment I'm about to make In a home server without checking that I'm not way off the mark. Reddit tech answers are usually useless, all jerking each other off about being the most vague and useless. So I'm here, just humbly asking if my plan will work before I waste a ton of money if it doesn't.
So I want to run immich, jellyfin, vaultwarden, and a couple other small programs (including backup automation) on a home server. The photos and videos are very important and I want off of large corporate cloud services for all the reasons you can imagine. There are about 2TB of important photos and data right now to backup.
So my plan: a mini pc with a low-level i3 or i5 chip but about 256gb SSD and 8gb ram. I will buy a UPS, a hub/NAS for 2 4TB HDDs, and a separate hub for backups elsewhere on another 4TB HDD. I already have a pc with an ok graphics card (only like 8 years old, but it was above average then) and good processor. My plan is to run the server with Linux, docker, and portainer on the mini-pc, and do the most intensive work (things like local Machine Learning on the photo library in immich) on the gaming pc. To save on energy, the mini pc is the only thing on most of the time, and I will turn on the old gaming pc to do the hard tasks.
I am hoping a friend will allow me to (somehow? I don't know how yet) keep a hub+HDD backup at their place which is updated weekly automatically whenever their computer turns on.
Is this all achievable? Am I missing something small or huge? Any tech people here know something that I'll likely miss on my first attempt?
This will all work but you may want to leave yourself with a few paths for growth that you may want to take now or later, so you'll want to plan your base components accordingly.
Very good information! I had no idea that 3 was the minimum for drives for RAID error correction. I thought a RAID6 configuration with 2 drives was already doing error correction?
You can't have a RAID6 with 2 drives. RAID6 uses 2 drives just for parity.
You can have a RAID1 (mirroring) with 2 drives.
Yeah I definitely confused those. I do not have money for higher RAID than 1 lol. Also don't need it, I think. But I would like to be as little dependent on 1 HDD as possible, because I care about the data. So does it make sense to do RAID 1 so that 1 can break and the other still works? Then I still have back while redundancy is lost?
It's fine if you are willing to use half of your disk space on redundancy.
ZFS can do that.
Bever heard of ZFS, dont really understand it. Does a software achieve this for my backups?
It's a filesystem with support for software RAIDs among other things. There is also Linux device mapper.
A 3-way mirror is basically software level RAID 1 but you can lose 2 discs without losing data. This is the same redundancy level as RAID 6 / RAIDZ2, but it's less storage efficient. On the plus side, it's very low overhead, simple to think about, and will be faster at reading / writing.
So let's say you're doing a 4 TB disc buy and can choose between mirror and erasure coding, same level of redundancy at somewhat optimal setups for this scale.
For 3 way mirror: 3 discs, you have 4 TB useable storage and should really try to keep it under 80% utilization so more like 3.2 TB. Cost is 3 x drive price. You can add one disc at a time and restripe if using a software RAID that supports it or get 3 more discs to double storage. You can use ZFS or btrfs for this, maybe some others.
For erasure coding: 6 discs, you have 16 TB useable storage, 12.8 TB if staying under 80% utilization. Cost is 6 x drive price. This is harder to expand but it's sometimes possible, usually with a lengthy redistribution process. You basically need to use ZFS for this. You pay more for power due to the number of discs always spinning.
IMO you should figure out a reasonable guess at how much storage you need and then double it, then compare prices for the drives you'd need to get that with either level.
Is a 3 way mirror possible with one at a distance/on a different network or processor? Or is there some way I could make the 3rd of the mirror at a distance at a friend's house?
It is possible but performance is so bad and setup so complex that your definitely don't want to do it.
Ok so reliable saving of my data in a safe way is, instead of 3-2-1, actually 4-2-1 if I want mirroring for corruption check... damn
Kinda yeah the redundancy is less of a backup and more of a resiliency for your main data repository. 3-2-1 could be 1 off-site backup and a local backup within that NAS, e.g., so only 1 off-site drive needed.
Ok back again after doing a week of research and am very thankful for your help, now have a question about it:
For this encryption, I need the HDD to only be decryptable with a password that only I have, right? Like thats the end goal? So even if someone gets it or ssh into the server on my friend's side, its an encrypted system? Does thay mean each update also has to be a complete wipe and re-backup??? Or are there systems which can just add the new data by knowing the encryption key?
The main thing is to put up some roadblocks to getting your data.
Your main defense is to ensure your data is encrypted and that you're using ssh to send it, specifically using ssh public + private keys and not a password. Then even if someone manages to access your system, they can't get the data without being a pretty high level attacker (like a government). If you set up just this it'll probably be enough.
Regarding full disc encryption, it is possible but a little complex to set up in this instance because it requires you to put in a password before the kernel starts (kernel is loaded from disc) so you would need to be present or buy another thing (a kvm box). Or have server level hardware that provides a kvm already, which requires $$$ or a lot of research to find a consumer machine with it. A kvm box might actually be handy for you (gives you a remote keyboard, screen, mouse at the hardware level) and there are less expensive ones on AliExpress but you don't need it. Example: imagine you do a system update to keep things secure and something goes wrong before the ssh server starts. Without a kvm you'll be unable to access the system remotely to fix it - you'll need to go to your friend's house. Probably not a huge deal but just something to think about.
But you can give yourself a lot of the benefits by just encrypting one partition. You can then either use something like a mini ssh server (dropbear) to enter a password to unlock it remotely or have the tpm auto unlock it. The former is better security-wise but then you have to have internet access at that early stage (more setup). The auto-unlock with the tpm means if someone takes only the drive and not the whole computer, then they can't decrypt it. So it's quite a bit less protective but also very easy to set up and you can try to add the drop bear functionality later once you have tpm auto unlock. Please note that you'll 100% want a password as your main fallback option for unlocking a partition, as then you can still access the data even if something else fails (machine breaks, bios update goes wrong, etc).
I think a good compromise is the use of ssh + encrypted backups, a single encrypted partition that is unlocked by the tpm (and has a backup password), and requiring a password for accessing your bios + boot menu. This eliminates all plausible threats:
If you just do my very first suggestion (encrypt the data itself, use ssh + keys) you'll probably be just fine though. It could also be just fine to just do that and keep the other stuff in mind for the future. Just getting remote access to your friend's network will be a decent enough project!