On the road to fully automated luxury gay space communism.
Spreading Linux propaganda since 2020
Rules:
Finally ready to switch my final windows device over to linux, but I'm running into a problem. NTFS drives aren't a good idea under Linux, but my 4 NTFS drives are this full. I own zero extra hard drives, and do not want to spend $300 on another one. 
How can I bring these files into Bazzite WITHOUT just wiping everything and re-downloading 10+ terabytes of movies, shows, music, games, and other data I refuse to delete?
Anyone who owns this much storage could probably afford to buy a local IT company to fix it for them.
I'm a disabled person who bought like $300 worth of HDDs several years ago.
If you send me your files I can hold on to them for a bit :3 they're safe with me :3
Wow, thanks! Here's 10 TB of 3d printer files of firearms.
First you transfer the rabbit, then the lettuce, transfer the rabbit back again, then the fox and finally the rabbit.
I think you'd need to use awk for that
consume it all in one sitting so you don’t feel bad about deleting it
NTFS drives aren't a good idea under Linux, but my 4 NTFS drives are this full.
NTFS works fine on Linux. You can't install Linux on an NTFS filesystem because NTFS doesn't support UNIX permissions, but you can mount these filesystems read-write to any arbitrary sub-directory and use them as you please.
The tricky thing is, if you have no more drives, you will need to shrink one of these NTFS filesystems to make room for a Linux-compatible filesystem (ext4, btrfs, xfs, etc) to actually install a Linux distro. Microsoft has a minimal amount of info here, but I don't have a Windows machine at my disposal to test this. Shrinking an NTFS partition should also be possible via gparted (which is available in LiveCD/LiveUSB form). Either way, it is a potentially dangerous operation. On a nearly-full filesystem it will be very slow, and if the power goes out or something in the middle of the process, 
.
I usually allocate 400GB for a Linux rootfs, but 200GB should be comfortable if you aren't installing a million packages.
I can attest that I shrank an active W10 partition and dual booted it alongside an arch based distro and did not have problems. Doesn't mean I'd recommend dual booting tho.
No no no no no no no no no no no no no no dont suggest using ntfs under linux no no no no no no no no.
At least caveat it for christs sake
yeah ime they will just bork after a while and need chkdsk or defragging which you can't do on linux so it's a big pita
??? As a Linux power user for over a decade I genuinely don't know what possible caveat there is. I've used NTFS drives under Linux every day and the only problems I've ever had were when I mounted the drives in Windows because it loves to fuck with NTFS settings and set dirty bits and stuff.
I dual-boot Linux and Windows, on their own drives, and then have two big 4TB shared NTFS drives on which EVERYTHING - Media, games, programs, docs, etc goes. I dinnae overexaggerate when I say I have had a total of zero problems that weren't directly caused by Windows.
So zero problems except for the problems caused by dual booting, something first time linux users might do. Sounds like a caveat to me.
Apparently the 7.1 kernel has a full rewrite i may try but fuse and ntfs3 wasn’t decades of no issues in my experience. But ofc thats trumped by your experience
I apologise - I'm not saying one's experience is invalid, just that I really honestly ain't seen problems.
True I suppose that dual booting has the caveat - disable fast startup and always shut down cleanly if you don't want Windows to fuck up your NTFS.
Sorry I was having a series of bad migraines and health scares I overreacted. So things I would list would be
Sorry for being aggressive having a bad life rn
RE: partioning.
I strongly recommend that users set up separate partitions for home and root. I use LVM and it's awesome for snapshot reasons, but I think btrfs does everything and is easier to set up for a new user?
My setup atm (8 year old install, many partitions inherited as far back as 2004):
root: 80G (LaTeX lol) homes: 216G (user files, stuff installed only for me etc) media: 364G games: 541G mail: 20G (I host my own email and download to my computer) archive: 11T (long term storage on rust)
That way if I want to e.g. backup just my user files and not all my legitimately obtained and licensed movies I can just backup homes and not chase down files. Or if I want to install a different OS or fix a broken one I only need to look at root.
Start them small and extend them as needed. You don't have to plan in advance!
I strongly recommend that users set up separate partitions for home and root.
This is a very good idea (and my personal practice). It allows you to replace the OS without touching your personal files. Only reason I didn't suggest it is it looked like available space was already very limited.
but I think btrfs does everything and is easier to set up for a new user?
It is certainly possible to use Btrfs subvolumes instead of a separate partition, but many Linux distro GUI installers do not have robust support for this and will prevent you from using the same partition for different mountpoints (they typically don't understand or provide a way to set the btrfs-specific subvol= mount option). On a manual distribution like Arch or Gentoo this isn't a problem at all, but I've had to get creative when using GUI installers for Debian and Solus. Fedora's GUI installer "supports" it, but last time I did it, it triggered another bug in the process later on based on a similar assumption.
It is easy enough to do post-install (turning /home into a subvolume and adding it to fstab), but you will still have a hard time later on replacing only the OS, which is the main point of setting things up like this.
If people are willing to read a bit the archwiki install section on setting up LVM is very good and I used to use it to teach newbies when I worked as a sysadmin.
Wow this sounds like a great setup
the mount points are
I dual boot and actually have a second NVME drive with Bazzite on it, but it's not visible under Windows.
It seems like it's just not possible to have games (on the F drive) accessible to both without problems? Bazzite says something about it being a bad idea to run games from an NTFS drive every time I mount it.
If you're dual-booting, there are some potential problems. The biggest danger comes from hibernation. Hibernating one OS and then booting the other OS and modifying the files (even incidentally just by mounting the filesystem), and then loading the hibernated OS will cause very bad things to happen. To the best of my knowledge, the Linux NTFS drivers refuses to operate on a filesystem which appears to be hibernated, but I do not know if the inverse is true.
Without knowing the exact text of the warning you're getting, I'm assuming it is a "this is not officially supported, don't blame us and don't send us bug reports" sort of deal. From what I gather, the most likely cause of problems is the potential for creating files with illegal names on the NTFS filesystem (filenames containing a ':' for instance). There is an unofficial guide on the Proton wiki about running Windows games in Proton which are stored on a shared NTFS partition which covers the relevant potential issues.
If the main risk is games not running correctly, I wouldn't lose any sleep.
This setup is not ideal, but neither is having 10TB of games turned into 20TB of games so you can duplicate them on separate filesystems. A compromise of some sort is required. In the longer run, I would aim to replace one of these NTFS filesystems with a Linux filesystem and install games there that you don't play on Windows any more.
Would an overlay filesystem work? Its been a long time since I looked at it (and it was for very different purposes) but if you can get NTFS game drive mounted as a read only layer of an overlay fs and then idk some ext4 as the writeable layer, you can prevent writing data to the NTFS drive and making windows sad with illegal filenames, while also getting to like, yk, save your savegames and whatnot.
Might make loading times pretty long tho. Idk what the overhead is for an overlay fs.
Fortunately the D and E drives are Movies and TV, respectively. F is games. My plan is to run Linux 99% of the time, and I can probably keep the few games or random pieces of software I can only run under Windows on the windows C drive.
Basically I just want to have a Windows SSD as a backup.
Would it be better to reformat the D and E drives as exFAT if I can cycle the content through another drive or something? I'd like to be able to access those two drives under both Bazzite and Windows.
I think exFAT would be a step down from NTFS. NTFS is a journaling filesystem which makes it a lot more robust against data corruption than exFAT. It also has a number of more advanced features (for a 30 year old filesystem) like symlinks, transparent compression, and sparse files. NTFS is also less prone to fragmentation than the FAT family of filesystems. On the other hand, exFAT doesn't eliminate any of the potential compatibility problems (like the potential to create a file named "C:"). The only thing exFAT has going for it is that Microsoft has actually published a specification for it, but in practice NTFS has been reverse-engineered enough to have multiple reliable third-party implementations.
If you need Windows and Linux to access the same filesystem, NTFS is the least bad option (the best option is probably setting up a NAS for this amount of data). Another option is to install third-party ext2/3/4 or btrfs drivers on Windows, but last time I looked into this, they weren't very robust.
the Linux NTFS drivers refuses to operate on a filesystem which appears to be hibernated, but I do not know if the inverse is true
This is true, writing to a hibernated system is not a thing you can unknowingly do.
I've never hibernated on Linux but Windows is basically incapable of mounting any UNIX filesystem, so I can't see that ever happening.
Windows is basically incapable of mounting any UNIX filesystem, so I can't see that ever happening.
Yes, but if the NFTS was mounted by Linux when it hibernated, suddenly this becomes a possibility.
You can use NTFS on Linux. Yes it can encounter problems eventually, yes the tools to fix it properly are only on Windows for the most part. But it works fine in my experience especially if you're mostly reading and not writing.
If that's not an option then you do need to move your files. You can either buy another drive or buy a temporary cloud storage plan (pcloud, google, many others offer 2TB for $9.99/month and 5TB for maybe twice that) and upload your stuff there on the monthly plan then cancel when done. To minimize danger even if you go the cloud route I'd upload one drive's contents at a time, wipe, reformat, download the contents, move to next drive. You can use something like Cryptomator (https://cryptomator.org/) to encrypt your files as you upload them if you wish and there's also 7zip with AES password encryption and encrypt file names though it requires duplicating the files in question which takes up space.
Another option would be freeing up some space, using that free space to create a new partition under Linux using a Linux file system, moving files from your existing NTFS partition there gradually in batches, shrinking your NTFS partition using Gparted or similar gradually while expanding your Linux ext4, zfs, etc partition as you transfer them in batches. Do this for each drive. It'll be a pain in the ass for the fuller drives and there are no guarantees doing this kind of resizing and copying couldn't cause something to break. But it is free. I would suggest another drive or going the cloud route at least as back-up if you're trying this.
The games in particular you may want to reinstall through Steam, Lutris, Heroic launcher, etc though I guess you can deal with that when you get there. The only reason I'd keep the games themselves as opposed to just their save files and any mods I have would be if I'd done modding that altered the game files.
How about pirated games? I am ethically opposed to paying for copied digital content.
How does that even work? Can I just run the fitgirl (or whoever) installer in wine? Last time I tried this (c. 2009) you were just up shit creek without a paddle when it came to all game piracy on Linux.
Can I just run the fitgirl (or whoever) installer in wine?
That worked for me last time I tried.
You will have to reinstall your pirated games from the repack and let Lutris (or Heroic or Faugus or whatever other launcher gui) make a wine prefix (sort of a fake little windows filesystem for each game to live in) for each game. Trying to just take the already installed games and make proton run them will result in all sorts of weird hard to fix problems.
For audio and video content NTFS will work alright.
For games you should use a linux native filesystem like ext4 or btrfs. Bazzite itself will warn you about this if you plug in an NTFS drive.
The last tried I installing fitgirl was years ago and it did not work. I have seen people on reddit say that her installers work under proton but I have never been able to replicate it. I have taken to downloading pre-installed repacks or clean steam files and cracking them myself. On cs.rin, repackers like rexagames or astralgames will post suck repacks. You are supposed to just unzip and run these repacks. There is a significant downside to this that these repacks are not usually distributed via torrent so you need to download them from file hosting websites. But it works at least. I have been pirating the shit out my steam deck this way.
So I can't just download ISOs from my private tracker anymore?
uncompressed shit like that is ideal actually and should work fine, I've had issues with fit girl on Linux i think due to her uncompressing tool
You can have it work too, you just run the installer through Proton, either through Steam or Heroic Games Launcher, Faugus, Vermouth, Bottles or Lutris (I find Heroic the easiest).
I prefer to use Hydra Launcher to download the games then install it through Heroic, but you can download directly from fitgirl or dodi or whatever. Hydra recently got Proton support but I haven't tested it yet to see if it's any good.
Also, if you're planning to use Bazzite, in their Bazzite Portal app you can find a setting to enable deduplication, if you enable that on your OS drive it should help with saving a bit of space, it's very useful with gaming since proton prefixes might have a lot of duplicated data.
Do some exchange fraud on a portable hard drive is the most realistic answer tbh.
When I last rebuilt my raid I uploaded like 20TB to backblaze setup the raid and downloaded the data again. Took a while and costed 120€ but not sure what else I could have done.
not really a super easy way, you seem to have a ton stuff, i just uninstalled everything on windows in steam and then installed it in linux. each game has different ways of backing up a save, so be careful about deleting any save files
"just re-download 10 TB" is just a wild thing to suggest users do. Like, I get that this is a really niche case and most users aren't like me but it's wild that I will have to shuffle data this way just to keep my Plex library and stuff.
What you’re asking for isn’t simple from a technical perspective. Tools to rewrite a filesystem do exist but they’re risky as fuck so its not like a standard experience. The best option is to copy your data to another device then copy it back but also like do you like not back this shit up at all? Are you not worried if a drive dies and losing everything?
The only real option with your existing setup without buying anything is tiny ext4 partition copy resize ntfs and ext4 copy but you’re gonna be there for a while. If there is a single drive you can realistically wipe for this procedure to give you a bigger copy to space in ext4 then that will speed it up.
If I was doing this then I’d opt for robocopy to a drive big enough to store all the stuff i wanted to keep in a filesystem thats cross compatible then rclone it back.
Naturally windows linux and mac dont like each other and moving from any os to any other is quite the task if you arent willing to delete everything and start again.
Are you not worried if a drive dies and losing everything?
It's copies of movies and tv shows for Plex. Yes, I'm worried about it but I can't afford the amount of storage needed to have backups of media available on my private tracker.
At this point your only options are the filesystem shuffle for a billion hours or using ntfs on linux where it might just corrupt your data one day silently or break in a way that needs windows to fix it. Or continue using windows.
I haven’t used ntfs on linux in a long time. Hopefully its better now then when i last tried but it can be afaik a bit nefarious where it works great until it doesn't
They are describing a scenario where enough of the data is steam games that deleting them would create the swap space needed to get started shuffling stuff around in which case it wouldn't necessarily be that wild... you would be avoiding a lot of the headaches that redownloading other stuff would have; high reliability of the files - nothing would be lost, fast downloads, wouldn't be much labor to que them up, you probably won't need all your games right away... your mileage will vary but it's not the worst suggestion for some people who might be reading the thread in a similar situation.
I ended up deleting all the games, and I'll reinstall those as I need under bazzite yeah.
Leaving the 10 TB of video alone ecause it seems NTFS will be Fine™ for just video files at least for now.
This might be a good guide to get you started and give you ideas:
https://linuxbasecamp.com/learn/switch-windows-to-linux-keep-files
Carrier pigeon. 