193
submitted 1 year ago* (last edited 1 year ago) by Kalcifer@lemmy.world to c/linux@lemmy.ml

I'm trying to find a good method of making periodic, incremental backups. I assume that the most minimal approach would be to have a Cronjob run rsync periodically, but I'm curious what other solutions may exist.

I'm interested in both command-line, and GUI solutions.

(page 2) 48 comments
sorted by: hot top controversial new old
[-] HumanPerson@sh.itjust.works 2 points 1 year ago

When I do something really dumb I typically just use dd to create an iso. I should probably find something better.

[-] JoMiran@lemmy.ml 2 points 1 year ago

At the core it has always been rsync and Cron. Sure I add a NAS and things like rclone+cryptomator to have extra copies of synchronized data (mostly documents and media files) spread around, but it's always rsync+Cron at the core.

[-] avidamoeba@lemmy.ca 2 points 1 year ago* (last edited 1 year ago)

Setup

Machine A:

  • RAIDz1 takes care of single-disk failure
  • ZFS doing regular snapshots
  • Syncthing replicates the data off-site to Machine B

Machine B:

  • RAIDz1 takes care of single-disk failure
  • ZFS doing regular snapshots
  • Syncthing receiving data from Machine A

Implications

  • Any single-disk hardware failure on machine A or B results in no data loss
  • Physical destruction of A won't affect B and the other way around
  • Any accidentally deleted or changed file can be recovered from a previous snapshot
  • Any ZFS corruption at A doesn't affect B because send/recv isn't used. The two filesystems are completely independent
  • Any malicious data destruction on A can be recovered from B even if it's replicated via snapshot at B. The reverse is also true. A malicious actor would have to have root access on both A and B in order to destroy the data and the snapshots on both machines to prevent recovery
  • Any data destruction caused by Syncthing can be recovered from snapshot at A or B
[-] anarchyreloaded@lemmy.world 2 points 1 year ago

I use timeshift. It really is the best. For servers I go with restic.

[-] lord_ryvan@ttrpg.network 1 points 1 year ago* (last edited 1 year ago)

I use timeshift because it was pre-installed. But I can vouch for it; it works really well, and let's you choose and tweak every single thing in a legible user interface!

[-] BRINGit34@lemmygrad.ml 2 points 1 year ago* (last edited 1 year ago)

I've got a smb server setup with a 12tb server drive. Anything important gets put on there

Edit: fixed spelling

[-] A1kmm@lemmy.amxl.com 2 points 1 year ago* (last edited 1 year ago)

I use Restic, called from cron, with a password file containing a long randomly generated key.

I back up with Restic to a repository on a different local hard drive (not part of my main RAID array), with --exclude-caches as well as excluding lots of files that can easily be re-generated / re-installed/ re-downloaded (so my backups are focused on important data). I make sure to include all important data including /etc (and also backup the output of dpkg --get-selections as part of my backup). I auto-prune my repository to apply a policy on how far back I keep (de-duplicated) Restic snapshots.

Once the backup completes, my script runs du -s on the backup and emails me if it is unexpectedly too big (e.g. I forgot to exclude some new massive file), otherwise it uses rclone sync to sync the archive from the local disk to Backblaze B2.

I backup my password for B2 (in an encrypted password database) separately, along with the Restic decryption key. Restore procedure is: if the local hard drive is intact, restore with Restic from the last good snapshot on the local repository. If it is also destroyed, rclone sync the archive from Backblaze B2 to local, and then restore from that with Restic.

Postgres databases I do something different (they aren't included in my Restic backups, except for config files): I back them up with pgbackrest to Backblaze B2, with archive_mode on and an archive_command to archive WALs to Backblaze. This allows me to do PITR recovery (back to a point in accordance with my pgbackrest retention policy).

For Docker containers, I create them with docker-compose, and keep the docker-compose.yml so I can easily re-create them. I avoid keeping state in volumes, and instead use volume mounts to a location on the host, and back up the contents for important state (or use PostgreSQL for state instead where the service supports it).

[-] Goldmaster@lemmy.ml 2 points 1 year ago

I use lucky backup to mirror to external drive. And I also use Duplicacy to back up 2 other separate drives at the same time. Have a read on the data hoarder wiki on backups.

[-] conorab@lemmy.conorab.com 2 points 1 year ago

I run ZFS on my servers and then replicate to other ZFS servers with Syncoid.

load more comments (2 replies)
[-] rikudou@lemmings.world 2 points 1 year ago

Github for projects, Syncthing to my NAS for some config files and that's pretty much it, don't care for the rest.

[-] useless@lemmy.ml 1 points 1 year ago* (last edited 1 year ago)

I use btrbk to send btrfs snapshots to a local NAS. Consistent backups with no downtime. The only annoyance (for me at least) is that both send and receive ends must use the same SELinux policy or labels won't match.

[-] ryannathans@lemmy.world 1 points 1 year ago

Restic with deja dupe gui

[-] danielfgom@lemmy.world 1 points 1 year ago

Periodic backup to external drive via Deja Dup. Plus, I keep all important docs in Google Drive. All photos are in Google Photos. So it's only my music really which isn't in the cloud. But I might try upload it to Drive as well one day.

[-] ebits21@lemmy.ca 1 points 1 year ago

Restic to Synology nas, Synology software for cloud backup.

[-] sol@lemm.ee 1 points 1 year ago

Most of my data is backed up to (or just stored on) a VPS in the first instance, and then I backup the VPS to a local NAS daily using rsnapshot (the NAS is just a few old hard drives attached to a Raspberry Pi until I can get something more robust). Very occasionally I'll back the NAS up to a separate drive. I also occasionally backup my laptop directly to a separate hard drive.

Not a particularly robust solution but it gives me some piece of mind. I would like to build a better NAS that can support RAID as I was never able to get it working with the Pi.

[-] nyan@lemmy.cafe 1 points 1 year ago

I use duplicity to a drive mounted off a Pi for local, tarsnap for remote. Both are command-line tools; tarsnap charges for their servers based on exact usage. (And thanks for the reminder; I'm due for another review of exactly what parts of which drives I'm backing up.)

[-] Trail@lemmy.world 1 points 1 year ago

A separate NAS on an atom cpu with btrfs of raid 10 exposed over NFS.

[-] ShellSurf@kbin.social 1 points 1 year ago

Anything important I keep in my Dropbox folder, so then I have a copy on my desktop, laptop, and in the cloud.

When I turn off my desktop, I use restic to backup my Dropbox folder to a local external hard drive, and then restic runs again to back up to Wasabi which is a storage service like amazon's S3.

Same exact process for when I turn off my laptop.. except sometimes I don't have my laptop external hd plugged in so that gets skipped.

So that's three local copies, two local backups, and two remote backup storage locations. Not bad.

Changes I might make:

  • add another remote location
  • rotate local physical backup device somewhere (that seems like a lot of work)
  • move to next cloud or seafile instead of Dropbox

I used seafile for a long time but I couldn't keep it up so I switched to Dropbox.

Advice, thoughts welcome.

[-] RoboRay@kbin.social 1 points 1 year ago

I actually move my Documents, Pictures and other important folders inside my Dropbox folder and symlink them back to their original locations

This gives me the same Docs, Pics, etc. folders synced on every computer.

[-] GustavoM@lemmy.world 1 points 1 year ago

Either an external hard drive or a pendrive. Just put one of those in a keychain and voila, a perfect backup solution that does not need of internet access.

...it's not dumb if it (still) works. :^)

[-] happyhippo@feddit.it 1 points 1 year ago

Vorta + borgbase

The yearly subscription is cheap and fits my storage needs by quite some margin. Gives me peace of mind to have an off-site back up.

I also store my documents on Google Drive.

[-] fmstrat@lemmy.nowsci.com 1 points 1 year ago

zfs snap and zfs send to an external or another server.

[-] hydration9806@lemmy.ml 1 points 1 year ago

I use Duplicacy to encrypt and backup my data to OneDrive on a schedule. If Proton ever creates a Linux client for Drive, then I'll switch to that, but I'm not holding my breath.

[-] SethranKada@lemmy.ca 1 points 1 year ago

I use Pika backup, which uses borg backup under the hood. It's pretty good, with amazing documentation. Main issue I have with it is its really finicky and is kind of a pain to setup, even if it "just works" after that.

load more comments (2 replies)
[-] HR_Pufnstuf@lemmy.world 1 points 1 year ago

ZFS send / recieve and snapshots.

[-] pound_heap@lemm.ee 2 points 1 year ago

Does this method allow to pick what you need to backup or it's the entire filesystem?

[-] HR_Pufnstuf@lemmy.world 2 points 1 year ago

It allows me to copy select datasets inside the pool.

So I can choose rpool/USERDATA/so-n-so_123xu4 for user so-n-so. I can also choose copy copy some or all of the rpool/ROOT/ubuntu_abcdef, and it's nested datasets.

I settle for backing up users and rpool/ROOT/ubuntu_abcdef, ignoring the stuff in var datasets. This gets me my users home, roots home, /opt. Tis all I need. I have snapshots and mirrored m2 ssd's for handling most other problems (which I've not yet had).

The only bugger is /boot (on bpool). Kernel updates grown in there and fill it up, even if you remove them via apt... because snapshots. So I have to be careful to clean it's snapshots.

[-] jimmy90@lemmy.world 2 points 1 year ago

me too. ZFS is amazing

[-] PhilBro@lemmy.world 1 points 1 year ago

I run Openmediavault and I backup using BorgBackup. Super easy to setup, use, and modify

[-] redcalcium@lemmy.institute 0 points 1 year ago

Good ol' fashioned rsync once a day to a remote server with zfs with daily zfs snapshot (rsync.net). Very fast because it only need to send changed/new files, and saved my hide several times when I need to access deleted files or old version of some files from the zfs snapshots.

load more comments
view more: ‹ prev next ›
this post was submitted on 25 Jul 2023
193 points (98.0% liked)

Linux

48329 readers
665 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS