10
Archiveteam Veoh grab (tracker.archiveteam.org)
submitted 2 weeks ago* (last edited 2 weeks ago) by kabi@lemm.ee to c/datahoarder@lemmy.ml
7 comments fedilink hide all child comments

Just looking at the numbers, it doesn't seem to me like archival will complete before the shutdown date (nov. 11). There are 2million+ elements left, likely 100TB+ of videos.

If you care to help them out, see instructions at the top of the page. Be sure you have a "clean connection", though.

edit: They're saying that the current rate seems to be plenty enough to finish by the deadline. Workers are often left idling at the moment.

top 7 comments
sorted by: hot top controversial new old
[-] clb92@feddit.dk 3 points 2 weeks ago* (last edited 2 weeks ago)

I've started a few warriors, but it's not helping because they've activated rate limiting. I just get:

Tracker rate limiting is active. We don't want to overload the site we're archiving, so we've limited the number of downloads per minute. Retrying after 240 seconds...

So the bottleneck is that they don't want to overload Veoh.

[-] kabi@lemm.ee 1 points 2 weeks ago

They are about to double the rate lmit, so it should be a little better...

[-] kabi@lemm.ee 1 points 2 weeks ago

Yeah, that's what I'm getting now, too. Hopefully they take the deadline into account for rate limiting... Still good to have warriors ready to retrieve, I guess.

[-] ReversalHatchery@beehaw.org 3 points 2 weeks ago* (last edited 2 weeks ago)

thanks for the reminder! recently I keep the warrior down because my amount of ram started to be a bottleneck to me, but certainly manageable when there's urgent need.

why don't they switch the "current project" selection to it, though? It's on telegram now. it would receive more help because that's the automatic choice

[-] Kissaki@beehaw.org 1 points 2 weeks ago

I tried getting the warrior to work with Hyper-V. But that doesn't seem possible or feasible.

[-] Kissaki@beehaw.org 1 points 2 weeks ago

Why does warriorhq link to warrior version 3 when there's a successor version 4 available?

[-] kabi@lemm.ee 1 points 2 weeks ago

Not sure. They don't mention 4 being in beta or anything. Though they don't list much advantage of using it over 3 either.

this post was submitted on 01 Nov 2024
10 points (100.0% liked)

datahoarder

6766 readers
13 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago
MODERATORS
archivist@lemmy.ml