Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
Yeah from some other comments I think my initial plan (that I'll research some more) will be:
This way I should get a chance to update storage medium as technology changes as well.
That is not gonna work - read up on byte rot.
Yeah since then I've been convinced I need two drives mirrored under zfs, which should handle that scenario.
Hard drives loose their data fast if not powered (within a few years),so do SSD based media. Furthermore the former are very suspectable to mechanical destruction, electromagnetic interference,etc. And even if for some reason your drives last that long there will be nothing to connect them to - you know how we connected hard drives 25 years ago? Via SCSI/IDE. Good luck finding a converter to these now. If you go back further you need ISA controllers for the drives.
This is a really bad idea. Really really bad, especially with the goal you want to achieve. Your data will be gone within 5 to 10 years.
The idea is that I'd swap out drives every 5 years or so. If USB A is no longer in use I'd swap out at that point for something newer. Plus the drives would be powered on every year for the update, it's just the point that I stop doing it (too old/hit by bus/etc) that the clock would start ticking.
I do like the M-Disc idea though. Probably a similar price, and more in line with the shelf-stable solution I was looking for.
There are still problems with the hard drive solutions:
Powering up the drives for a short period does not help with error correction when sectors get compromised
As said before it is relatively risky as mechanical parts of HDs do not like to be moved only occasionally. While this problem has become less severe over the last years it still exists.
The updating will include copying from one drive to another - this process is highly suspectable to errors that might be correct with the right file systems - but it's not a guarantee.
And the main problem: You want to achieve a long shelf life - which means you must consider periods of time when you might not be able to maintain the data. What happens when you are not able to do so? And your next of kin are not quite ready to go through your things? To give you an example: You copy your data on the HDs today, maintain the disk's for four years and want to change disk's in 5, which means in 2030. Sadly a weeks before you are able to do so, John,your neighbourhood's stupid school bus driver hits you and you suffer a major traumatic brain injury. Even worse,you don't die right away but suffer for another 5 years in a nursing home before a infection gets you. Your family meanwhile is not quite ready to get through your things as you are still alive, aren't you? (For real,this is the case a lot) After your funeral it takes them another year to finally get through all your things. Now your drives haven't been used for 7 years. Even worse,one of them slips through your next of kind hand and hits the ground hard. How big do you think the chances are the data is still available? I think we both know the answer. While M-Disks are also suspectable to damage there are hardened multi-disk cases that make them pretty much indestructible - nothing any HD case can ever achieve.
you need to use fat32 if you want normal people to access the files
Otherwise, they will get the "You need to format the disk in drive D: before using it. Do you want to format it?" dialog, they blindly click "yes", then they will mumble to themselves "weird, he left behind a massive collection of blank drives..."
This is why I can’t/don’t have a lot of the “best practices” in my family archive. I’m not encrypting local drives, I’m not using BTRFS, or a ZFS pool. If I did I’d have to ensure my Will provided for the lawyer to hire a tech shop to help recover them. No, exFAT and NTFS, in the clear so those left behind can just plug them in and get to making their own copies. Otherwise the archive would die with me.
Does that mean someone could steal my drives and go through my family photos? Sure. I hope it brings them much guilt, something a garbled encrypted drive could never do.
Oh shit you're right. Argh ok I'm going to have to rethink that. Two drives and something to compare against each other to check for errors. I'm not sure about FAT32 as there are some multi-GB video files. Shit.
exFAT is a newer and viable alternative to FAT32, with better size limits and some pretty good cross-platform capabilities. That said, if your primary access is through Windows, NTFS may have some better features and is at least read-only on other platforms.
I don't use Windows, I'm just thinking of someone needing to be able to pick up and use a drive, and for most people it's going to be Windows.
Maybe I just need to leave instructions that specify it needs to be my laptop they use to get the photos off.