this post was submitted on 09 Feb 2025
79 points (96.5% liked)

Selfhosted

60507 readers
441 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world
79
Homelab upgrade - "Modern" alternatives to NFS, SSHFS? (sh.itjust.works)
submitted 1 year ago* (last edited 1 year ago) by MajorSauce@sh.itjust.works to c/selfhosted@lemmy.world
64 comments fedilink hide all child comments
 

Hi all!

I will soon acquire a pretty beefy unit compared to my current setup (3 node server with each 16C, 512G RAM and 32T Storage).

Currently I run TrueNAS and Proxmox on bare metal and most of my storage is made available to apps via SSHFS or NFS.

I recently started looking for "modern" distributed filesystems and found some interesting S3-like/compatible projects.

To name a few:

  • MinIO
  • SeaweedFS
  • Garage
  • GlusterFS

I like the idea of abstracting the filesystem to allow me to move data around, play with redundancy and balancing, etc.

My most important services are:

  • Plex (Media management/sharing)
  • Stash (Like Plex ๐Ÿ™ƒ)
  • Nextcloud
  • Caddy with Adguard Home and Unbound DNS
  • Most of the Arr suite
  • Git, Wiki, File/Link sharing services

As you can see, a lot of download/streaming/torrenting of files accross services. Smaller services are on a Docker VM on Proxmox.

Currently the setup is messy due to the organic evolution of my setup, but since I will upgrade on brand new metal, I was looking for suggestions on the pillars.

So far, I am considering installing a Proxmox cluster with the 3 nodes and host VMs for the heavy stuff and a Docker VM.

How do you see the file storage portion? Should I try a full/partial plunge info S3-compatible object storage? What architecture/tech would be interesting to experiment with?

Or should I stick with tried-and-true, boring solutions like NFS Shares?

Thank you for your suggestions!

you are viewing a single comment's thread
view the rest of the comments
[โ€“] catloaf@lemm.ee 3 points 1 year ago (1 children)

What are you hosting the storage on? Are you providing this storage to apps, containers, VMs, proxmox, your desktop/laptop/phone?

[โ€“] MajorSauce@sh.itjust.works 1 points 1 year ago (1 children)

Currently, most of the data in on a bare-metal TrueNAS.

Since the nodes will come with each 32TB of storage, this would be plenty for the foreseeable future (currently only using 20TB across everything).

The data should be available to Proxmox VMs (for their disk images) and selfhosted apps (mainly Nextcloud and Arr apps).

A bonus would be to have a quick/easy way to "mount" some volume to a Linux Desktop to do some file management.

[โ€“] catloaf@lemm.ee 2 points 1 year ago (1 children)

Proxmox supports ceph natively, and you can mount it from a workstation too, I think. I assume it operates in a shared mode, unlike iscsi.

If the apps are running on a VM in proxmox, then the underlying storage doesn't matter to them.

NFS is probably the most mature option, but I don't know if proxmox officially supports it.

[โ€“] MajorSauce@sh.itjust.works 1 points 1 year ago (1 children)

Proxmox does support NFS

But let's say that I would like to decommission my TrueNAS and thus having the storage exclusively on the 3-node server, how would I interlay Proxmox+Storage?

(Much appreciated btw)

[โ€“] catloaf@lemm.ee 3 points 1 year ago (1 children)

I think the best option for distributed storage is ceph.

[โ€“] Appoxo@lemmy.dbzer0.com 1 points 1 year ago

At least something that's distributed and fail safe (assuming OP targets this goal).
And if proxmox doesnt support it natively, someone could probably still config it local on the underlying debian OS.