266
submitted 8 months ago by padook@feddit.nl to c/selfhosted@lemmy.world

I woke up this morning to a text from my ISP, "There is an outage in your area, we are working to resolve the issue"

I laugh, this is what I live for! Almost all of my services are self hosted, I'm barely going to notice the difference!

Wrong.

When the internet went out, the power also went out for a few seconds. Four small computers host all of my services. Of those, one shutdown, and three rebooted. Of the three that ugly rebooted some services came back online, some didn't.

30 minutes later, ISP sends out the text that service is back online.

2 hours later I'm still finding down services on my network.

Moral of the story: A UPS has moved to the top of the shopping list! Any suggestions??

(page 2) 48 comments
sorted by: hot top controversial new old
[-] bin_bash@lemmy.world 3 points 8 months ago

when you say some services on your network are you talking about machines or softwares?

for machines yes ups makes sense for softwares writing some scripts to run on start up should be enough another alternative can be setting up wake on Lan that way you can bring all up again wherever you are

[-] mhredox@lemmy.world 3 points 8 months ago

Exact same thing happened to me the other day. Like exactly. Maybe we live in the same area.

[-] oldfart@lemm.ee 3 points 8 months ago* (last edited 8 months ago)

Two pitfalls I had that you can avoid:

  • look at efficiency. It's not always neglible, was like 40% of my energy usage because I oversized the UPS. The efficiency is calculated from top power the UPS can supply. 96% efficient 3kW UPS eats 4% of 3kW, 120 watts, even if the load you connected is much smaller than 3kW
  • look at noise level. Mine was loud almost like a rack server, because of all the fans.

I replaced that noisy, power hungry beast with a small quiet 900W APC and I couldn't be happier

[-] notannpc@lemmy.world 2 points 8 months ago

Could also be a good opportunity to add a service monitor like Uptime Kuma. That way you know what services are still down once things come back online with less manual discovery on your part.

[-] Decronym@lemmy.decronym.xyz 2 points 8 months ago* (last edited 8 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
CSAM Child Sexual Abuse Material
DNS Domain Name Service/System
NAS Network-Attached Storage
PiHole Network-wide ad-blocker (DNS sinkhole)
Plex Brand of media server package
RAID Redundant Array of Independent Disks for mass storage
SATA Serial AT Attachment interface for mass storage
VPS Virtual Private Server (opposed to shared hosting)
ZFS Solaris/Linux filesystem focusing on data integrity

8 acronyms in this thread; the most compressed thread commented on today has 11 acronyms.

[Thread #567 for this sub, first seen 3rd Mar 2024, 16:05] [FAQ] [Full list] [Contact] [Source code]

[-] knobbysideup@sh.itjust.works 2 points 8 months ago

In addition to ups, an LTE failover. I've had my Comcast crap be offline for hours.

[-] ripcord@lemmy.world 2 points 8 months ago

I'd like that, but also a really long-running UPS. multi-hour power outages are surprisingly common in my area.

[-] towerful@programming.dev 1 points 8 months ago

Thats no longer a UPS.
You could get something like a powerwall, something designed to power things from batteries for a long time.
Or get a generator with an automatic failover. The UPS then covers the downtime between powerfailure and generator taking load

[-] ripcord@lemmy.world 1 points 8 months ago

Why is that no longer a UPS?

[-] towerful@programming.dev 1 points 8 months ago

Generally, UPS (lead acid) batteries are not designed for long-cycle deep discharge.
They are designed to hold their rated load for a minute or so until the power is restored (generators start, power-uncuts) or the servers have a chance to shut down.
But maybe thats dated information, and modern UPSs are designed to run from batteries for a few hours.

[-] ripcord@lemmy.world 1 points 8 months ago

That seems like a weirdly and artificially narrow definition of UPS.

[-] bitwolf@lemmy.one 1 points 8 months ago

Does this require a lot of gear? Or does it simply act as another gateway?

[-] knobbysideup@sh.itjust.works 3 points 8 months ago

There are devices like the Netgear lm1200 that can do it inline by themselves.

I have that device, but configured as a second gateway. My firewall manages the failover based on primary packet loss and latency.

[-] themoonisacheese@sh.itjust.works 1 points 8 months ago

It requires an LTE capable gateway and a data plan. As for the rest you can simply write your routing tables so that if the main gateway doesn't work, use the secondary gateway with lower prio.

[-] JovialSodium@lemmy.sdf.org 1 points 8 months ago

My suggestion just changes your threat model, so may not be a good one based on your wants.

Perhaps consolidate systems? Managing less devices = less points of failure. But adds the risk of any given failure being more severe.

[-] padook@feddit.nl 1 points 8 months ago

This thought came to me this morning. I have 4 machines both because the BEAST grows organically, and because we're always trying to avoid that single point of failure. Then a scenario comes along that makes you question your whole way of thinking, diversifying may actually create more problems

[-] redcalcium@lemmy.institute 1 points 8 months ago

I feel your pain. Just the other day the disk on my home assistant machine died after a power outage and I had to replace it with another disk and restore from backup.

load more comments
view more: ‹ prev next ›
this post was submitted on 03 Mar 2024
266 points (96.8% liked)

Selfhosted

40113 readers
819 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS