Selfhosted

60507 readers

424 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Homelab upgrade - "Modern" alternatives to NFS, SSHFS? (sh.itjust.works)

submitted 1 year ago* (last edited 1 year ago) by MajorSauce@sh.itjust.works to c/selfhosted@lemmy.world

64 comments fedilink hide all child comments

Hi all!

I will soon acquire a pretty beefy unit compared to my current setup (3 node server with each 16C, 512G RAM and 32T Storage).

Currently I run TrueNAS and Proxmox on bare metal and most of my storage is made available to apps via SSHFS or NFS.

I recently started looking for "modern" distributed filesystems and found some interesting S3-like/compatible projects.

To name a few:

MinIO
SeaweedFS
Garage
GlusterFS

I like the idea of abstracting the filesystem to allow me to move data around, play with redundancy and balancing, etc.

My most important services are:

Plex (Media management/sharing)
Stash (Like Plex 🙃)
Nextcloud
Caddy with Adguard Home and Unbound DNS
Most of the Arr suite
Git, Wiki, File/Link sharing services

As you can see, a lot of download/streaming/torrenting of files accross services. Smaller services are on a Docker VM on Proxmox.

Currently the setup is messy due to the organic evolution of my setup, but since I will upgrade on brand new metal, I was looking for suggestions on the pillars.

So far, I am considering installing a Proxmox cluster with the 3 nodes and host VMs for the heavy stuff and a Docker VM.

How do you see the file storage portion? Should I try a full/partial plunge info S3-compatible object storage? What architecture/tech would be interesting to experiment with?

Or should I stick with tried-and-true, boring solutions like NFS Shares?

Thank you for your suggestions!

you are viewing a single comment's thread
view the rest of the comments

[–] nesc@lemmy.cafe 11 points 1 year ago (3 children)

Gluster is ~~shit~~ really bad, garage and minio are great. If you want something tested and insanely powerful go with ceph, it has everything. Garage is fine for smaller installations, and it's very new and not that stable yet.

[–] possiblylinux127@lemmy.zip 9 points 1 year ago

Ceph isn't something you want to jump into without research

[–] MajorSauce@sh.itjust.works 3 points 1 year ago (1 children)

Darn, Garage is the only one that I successfully deployed a test cluster.

I will dive more carefully into Ceph, the documentation is a bit heavy, but if the effort is worth it..

Thanks.

[–] nesc@lemmy.cafe 5 points 1 year ago (1 children)

I had great experience with garage at first, but it crapped itself after a month, it was like half a year ago and the problem was fixed, still left me with a bit of anxiety.

[–] possiblylinux127@lemmy.zip 1 points 1 year ago

You need to know what you are doing with Ceph. It can scale to Exobyte levels but you need to do it right.

[–] corsicanguppy@lemmy.ca 3 points 1 year ago (1 children)

go with ceph[:] it has everything

I heard running an object store as a filesystem was considered risky, but that's not why it sometimes hoses your storage.

[–] nesc@lemmy.cafe 1 points 1 year ago

Last time I had a problem with ceph losing data was during 0.10, does it still happen?