Example

1. sometimes foobar2000 randomly changes some settings and things break, I don't want to create duplicates (to save space) instead I want to track changes in it.
2. Same for chromium.

I say select because I don't want the imaging solution

These software have option to config directory using .ignore file

1. Kopia - .kopiaignore
2. Restic (hard to use)

• --exclude-file
• Ignore File: create a custom text file e.g., excludes.txt
• .exclude_me supports --exclude-if-present, which lets you skip any directory containing a specific file, such as .exclude_me.
• Uses gitignore-style syntax
• wrapper like Restatic - crawls directories before Restic starts and looks for .restaticignore files. It merges all those local files into one big exclude list and passes it to Restic automatically.

3. Duplicacy .duplicacy

https://github.com/gilbertchen/duplicacy/wiki/set

• ~While the official binary uses .duplicacy/filters, many users use scripts to generate that filter file by scanning for .cvsignore or custom ignore files in subdirectories.~

• ~https://forum.duplicacy.com/t/filters-include-exclude-patterns/1089/23~

• ~https://github.com/TheBestPessimist/duplicacy-utils~

• ~https://github.com/markfeit/duplicacy-scripts~

• ~https://github.com/gilbertchen/duplicacy/issues/337~

• ~https://forum.duplicacy.com/tag/filters~

~~Rclone~~ ~~(Only ignores no option to config)~~

1. --exclude-if-present .rcloneignore (any name)
2. Global --exclude-from list.txt

Why is it only recommended for "Large-scale cloud copies" and "Google Drive/Dropbox/S3"?

~~Borg~~ ~~(Only ignores no option to config) .borgignore~~

~~FreeFileSync~~ ~~(Maybe global only).ffs_gui or .ffs_batch~~

How to set up like this with features -

1. Forever full backups/DeDuplication - Option Delete changes older than x. Where full-backup is taken once then incremental-s are merged. I have heard terms like, snapshot and CBT but don't understand it.
2. Save a "Ghost" for excluded data ie only Filename, Metadata and Folder-Structure.
3. "File Change Tracker" to see summary of what files are moved/deleted/renamed between 2 backups. Like Kopia's https://kopia.io/docs/reference/command-line/common/diff/
4. "File History" where I see previous version of files on the main disk. Like Kopia's kopia snapshot list <filename>

To Backup

• External and Internal Disk (files) to backup,
• all separately backup-ed to the same backup disk, .

~~Old Post:~~ ~~https://www.reddit.com/r/DataHoarder/comments/1rzmg67/discussion/_backup/_solution/_comparison/~~

Duplicacy has a -nobackup-file

publication croisée depuis : https://lemmy.dbzer0.com/post/67080379

Hello, since it's complicated to index DHTs, I figured it'd be more efficient to build an index of fingerprints from real data once.

So I've been collecting releases hashes for this index. It can be used for various purposes:

• check the integrity of your own files (bit rot is a real thing)
• identify BTv2 torrent files that contain specific files (a database of torrent files is required)
• locate alive IPFS swarms to join more easily (no need to read all your data multiple times to recompute various CIDs yourself)

The collection contains around 1K releases and weights 40MB. I've prioritized scene Bluray rips of movies (1080p / 2160p). No infohash will be included, as these are not reproducible enough.

I'm using a basic script to add a new release (filename must match the official release name). I'm using others to discover scene releases in a filesystem; retrieve official release names from files using the srrdb api (crc32 search); collect torrents from Prowlarr and H&R them (although I'd prefer crowd-source directly from the community!).

The index is stored on git to allow collaboration. It is hosted using Radicale to avoid centralization and reduce hosting pressures.

If you are interested, join and add your own hashes to the collection in Radicle patches! (see instructions in the README)

Let me know what you think, suggest improvements or discuss similar projects you know about!

cross-posted from: https://lemmy.ml/post/46121046

so i've been searching for a long time for an image tagging software, there are some tools out there, some are electron based so no thanks, then there's Hydrus which is actually very similar to what this does, but the GUI is horrible and the installation is way more complicated. I created a docker compose file and I was running Blombooru in a few minutes. So if anyone was looking for something like this to organize their system, please give it a try because it is really good!

Crossposted from https://thebrainbin.org/m/datahoarder@lemmy.world/t/1576555

After the original post, I ordered the generic reader I mentioned and it arrived today.

Testing around, and it can generate encrypted ISO files just fine with Linux Mint's default Disks program. No ddrescue needed!

Also apparently one of the discs I have had the key shipped with the reader revoked. But using the issue as an opportunity to test MakeMKV, I created a decrypted backup with it and tried opening the backup with VLC.

This decrypted backup seems to behave just like the encrypted ISOs I had been creating, thankfully, though further tests may be needed as I am still very unfamiliar with this revocation situation.

But all in all, I'm happy I can finally do my backups properly. In retrospect, the PS3 is too moody and slow even when the hardware is healthy, making backing stuff up with it a rather miserable process. Better something fast, more predictable, and that isn't tied to FAT32 with gambiarras applied to work with NTFS.

And on a side note, I also tested the reader with a DVD for the same purpose and it worked fine, so I guess it'll be working overtime. e.e"

1. Kopia .kopiaignore
2. Duplicacy .duplicacy
3. Borg .borgignore
4. FreeFileSync .ffs_ignore

How to set up like this with features -

1. Forever full backups/DeDuplication
2. Option Delete changes older than x.
3. How to Only backup Select Data, like only personal data,
4. Save a "Ghost" for other data (internet data not personal), which is only Filename, Metadata and Folder-Structure.
5. "File Change Tracker" to see summary of what files are moved/deleted/renamed.
6. "File History" where I see previous version of files.
7. Config from inside folders for disks (not OS) with e.g..backupconfigfile containing e.g. backup=1, or have select file backuped/not-backed-up.

To Backup

• External and Internal Disk (files) and OSes to backup,
• Backup select data from Disks,
• all separately backup-ed to the same backup disk, .

Old Post: https://lemmy.ml/post/44707979

Is it a viable option?

I've gotten a few discs, and up to some years prior, I'd use the PS3 to back them up. However, as its HDD is starting to fail (took it long enough e.e''), I'd been considering getting a bluray reader for dumping the ISOs, possibly a Wabcom 5-in-1 but still evaluating that part.

But as I am on Linux Mint, and I don't mind the ISOs being encrypted (iirc I'd use FOSS keys and they'd work fine), would ddrescue be enough for that, or would I need to use some other program, perhaps even some dedicated/paid one?

Also bonus question: I also got CDs and DVDs to backup. Would I be able to do it for them too?

Thanks in advance!

https://archive.org/details/sms_mods_and_romhacks_collection_20260409_patched

My personal collection of Master System Romhacks, in an already patched and ready to play ROM format. Most games are patched by myself, but not all are tested. Each .sms file comes with a text description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

• 110 Romhacks across 53 different games (or across 52 games, depending on how you process data and count).
• Download one package size: 9.7 MB
• Unpacked size: 44 MB

flat structure: mastersystem_mods_and_romhacks_collection_20260409_patched_flat.7z

    mastersystem_mods_and_romhacks_collection_20260409/
        Alex Kidd in Miracle World_Snappy Snorg and the Seven Silver Stones v1.4.sms
        Alex Kidd in Miracle World_Snappy Snorg and the Seven Silver Stones v1.4.txt

or sub structure: mastersystem_mods_and_romhacks_collection_20260409_patched_sub.7z

    Master System Mods and Romhacks Collection 2026-04-09/
        Documents/
            Alex Kidd in Miracle World/
                Snappy Snorg and the Seven Silver Stones v1.4.txt
        Games/
            Alex Kidd in Miracle World/
                Snappy Snorg and the Seven Silver Stones v1.4.sms

Both contain same files, just different file structure.

If you archive emails to S3 (via AWS SES inbound or other pipelines), QuickMailBites lets you browse them with a proper native email client.

No need to write scripts to pull emails from S3 anymore — just configure the bucket and folder prefix.

It's redirecting to a new site completely, does anyone have a copy of the old source?

https://archive.org/details/n64_mods_and_romhacks_collection_20260404_patched

My personal collection of Nintendo 64 Romhacks, in an already patched and ready to play ROM format. Most games are patched by myself, but not all are tested. Each .z64 file comes with a text description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

• 207 Romhacks across 38 different games (or across 31 games, depending on how you process data and count).
• Download one package size: 2.3 GB
• Unpacked size: 5.7 GB

flat structure: nintendo64_mods_and_romhacks_collection_20260404_patched_flat.7z

         nintendo64_mods_and_romhacks_collection_20260404/
            Super Mario 64_Mario Builder 64 v1.1.txt
            Super Mario 64_Mario Builder 64 v1.1.z64

or sub structure: nintendo64_mods_and_romhacks_collection_20260404_patched_sub.7z

            Nintendo 64 Mods and Romhacks Collection 2026-04-04/
                Documents/
                    Super Mario 64/
                        Mario Builder 64 v1.1.txt
                Games/
                    Super Mario 64/
                        Mario Builder 64 v1.1.z64

Both contain same files, just different file structure.

Nintendo 64 emulator compatibility with Romhacks is a bit wonky. Sometimes they work out of the box and sometimes they require specific settings, and other times they just don't work on my setup.

I play Nintendo 64 games with Mupen64Plus-Next core on RetroArch. Two distinct configuration files are included. These Romhacks are quickly tested for compatibility and categorized into one of these setups. The configuration is not required and are included as a reference. There is no guarantee that the included Romhacks will work.

Gaming Historian didn't post in 3 years and this is his goodbye video today: https://youtu.be/nV_Aww8_6wQ And his last gift about an unfinished video of him, he releases said documents: https://archive.org/details/universal-v-nintendo-court-documents

Gaming Historian has some of the highest quality gaming related documentations. I highly recommend watching past episodes.

Windows/Linux to Android - on PC i will create a folder of data to be synced created via SymbLinks

Android to Windows/LAN - Only sync select data preferably via WiFi

https://archive.org/details/md_mods_and_romhacks_collection_20260320_patched

My personal collection of Mega Drive / Genesis Romhacks, in an already patched and ready to play ROM format. Most games are patched by myself, but not all are tested. Each .md file comes with a text description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

• 421 Romhacks across 166 different games (or across 163 games, depending on how you process data and count).
• Download one package size: 182 MB
• Unpacked size: 815 MB

flat structure: megadrive_mods_and_romhacks_collection_20260320_patched_flat.7z

     megadrive_mods_and_romhacks_collection_20260320/
        Sonic_Character Pak v1.0.md
        Sonic_Character Pak v1.0.txt

or sub structure: megadrive_mods_and_romhacks_collection_20260320_patched_sub.7z

        Mega Drive Mods and Romhacks Collection 2026-03-20/
            Documents/
                Sonic/
                    Character Pak v1.0.txt
            Games/
                Sonic/
                    Character Pak v1.0.md

Both contain same files, just different file structure.

How to set up like this with features -

1. Forever full backups/DeDuplication
2. Option Delete changes older than x.
3. How to Only backup Select Data, like only personal data,
4. Save a "Ghost" for other data (internet data not personal) which is only Filename, Metadata and Folder-Structure.
5. "File Change Tracker" to see summary of what files are moved/deleted/renamed.
6. "File History" where I see previous version of files.
7. Config from inside folders for disks (not OS) with e.g..backupconfigfile containing e.g. backup=1, or have select file backuped/not-backed-up.

To Backup

• External and Internal Disk (files) and OSes to backup,
• Backup select data from Disks,
• all separately backup-ed to the same backup disk, .

Dead but FOSS

https://github.com/zmanda/amanda

https://en.wikipedia.org/wiki/List/_of/_backup/_software

Freemium

Backup and Imaging

Disk Imaging

Sync Software

Cloud

https://www.reddit.com/r/Backup/wiki/index/cloud/_backup/_services/backblaze/

Info

~~Incremental backup method, where I only make full backups once ,~~

~~Question - as i understand from "grandfather,father,son" method - that Full backups are still necessary when using snapshots/incremental backups, why is that ?~~

"deduplication" is of 2 types

Data/Block de-duplication - tech to reduce amount storage required. This breaks files upon into chunks and creates a DB. Data de-duplication is influenced heavily by data type. Data de-duplication is a waste of time with compressed multimedia and encrypted data. Data de-duplication is often incorporated into backup products and can exist in independently, its built into some file system types (Windows server and some Linux).

File deduplication - Removing identical files. by comparing the whole file. Homelabers and home users are more worried about this than commercial environments. In commercial environments users and projects are usually allocated a quota storage space and its up to them how they want to manage it.

Incrementals Forever and Synthetic Full Backup - incremental may/not have de-duplication. They can also use Change Block Tracking (CBT) to save a lot of backup time.

"Forever Full" are simply a variation on synthetic full backups with data deduplication and CBT being optional.

Data deduplication occurs when you:

• Have multiple copies of the same data across multiple machines e.g. the operating system files of the computers you are backing up.
• Data that has not changed since you last successive backup. This includes files that have partially changed, only new unique data is added to the dedupe database/repository. Old Post about methodology - https://lemmy.ml/post/44433232

1. I like Incremental backup methodology, but it needs frequent full backups (as i understand from "grandfather,father,son" method), How to have version control where i only create full-backup only once ?

2. and I can choose to Delete changes older than 1 month.

3. How to Only backup Select Data, like only personal data,

4. and a "Ghost" for other data. Ghost is only filename and it's metadata (also folder structure). Data selected for Ghost is from internet which can be downloaded.

Related

5. "file change tracker" to see summary of what files are moved/deleted/renamed.

6. "File History" where I see previous version of files.

Software ?

1. Seems like https://restic.net/ is best as it's enterprise trusted works on all OS,

How do I set it up like I described in the Original Post.

I have a external disk and internal HDD, I want to backup select data, to a 3rd disk, both backup to the same disk, both separately backup-ed.

There is also -

2. Pika (https://gitlab.gnome.org/World/pika-backup , https://www.youtube.com/watch?v=W30wzKVwCHo) ,
3. https://www.borgbackup.org/ ,
4. Rsync - https://en.wikipedia.org/wiki/Rsync
5. RClone - https://github.com/rclone/rclone , https://www.youtube.com/watch?v=QKCIi-NxJEo
6. https://www.urbackup.org/download.html (https://www.youtube.com/watch?v=tXGVzMUsuE4 , https://christitus.com/urbackup/)

I want to have windows image but have it saved incrementally Is there a way only backup data created by me so the backup is small, and I pull the windows OS data from ISO.

Nice to only backup "user data"

UPDATES

2026-030-20: Recreated and uploaded the "_sub.7z" archive, as the file structure was not build as intended. Not sure what happened there, but now its correct.

https://archive.org/details/snes_mods_and_romhacks_collection_20260312_patched

My personal collection of Super Nintendo Romhacks, in an already patched and ready to play ROM format. Most (if not all) games are patched by myself, but not all are tested. Each .sfc and .smc file comes with a description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

• 1009 Romhacks across 174 different games (or across 169 games, depending on how you process data and count).
• Download one package size: 406 MB
• Unpacked size: 2.7 GB

flat structure: snes_mods_and_romhacks_collection_20260312_patched_flat.7z

         snes_mods_and_romhacks_collection_20260312/
            Super Metroid_Nature v1.03.smc
            Super Metroid_Nature v1.03.txt

or sub structure: snes_mods_and_romhacks_collection_20260312_patched_sub.7z

            Super Nintendo Mods and Romhacks Collection 2026-03-12/
                Documents/
                    Super Metroid/
                        Nature v1.03.txt
                Games/
                    Super Metroid/
                        Nature v1.03.smc

Both contain same files, just different file structure.

spayee/graphy course

Webpage has a sidebar with category and sub-category and each opens just a PDF.

PDF files are stored here - https://randomlettersandnumbers.cloudfront.net/w/o/randomLettersAndNumbers/v/randomLettersAndNumbers/u/randomLettersAndNumbers/p/assets/pdfs/2021/01/13/randomLettersAndNumbers/file.pdf

https://myrient.erista.me/ - main site

This is arguably the best site ever made for this kind of preservation. And they shutdown because of insufficient funding and increased prices for hardware. They have full sets for NoIntro, Redump, TOSEC, MAME, RetroAchivements supported games, exo sets and lots of important coverage from good Internet Archive sources. All of this with direct downloads, no ads, super fast. Everything neatly organized and always available.

Either people start donating fast, or its gone. I recommend to download as fast as possible what you need. Its closing in about a month from now on March 31st, 2026.

it's time. this is not a test. download or let be destroyed.

The drive has been only been powered on and used for read over the last 3+ years. CrystalDiskInfo reports it's bad but CrystalDiskMark shows decent read/write speeds. Only wrote to it in the very beginning when I dumped a lot of archives into it. Otherwise, very few actual write cycles which is making me think it's still ok to use. However, this isn't a NAS drive and is consumer-grade bought many years ago.