Sorry OP that you're getting downvote bombed. This is actually really neat. People go nuts when they hear AI but this is fully local so I think this reaction is unjust. This has nothing to do with ram prices since that stems from data centers or corpos pushing AI on you. Thank you for sharing
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
-
No low-effort posts. This is subjective and will largely be determined by the community member reports.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
Anti-AI evangelism is at its peak rn.
@pfr @nikodindon That assumes it won't get worse, which I hope it does. AI companies have forced me to take down web stuff that I had running for almost 2 decades, because their scrapers are so aggressive.
Like what and what have you tried to block it?
@meldrik They're impossible to block based on IP ranges alone. It's why all the FOSS git forges and bug trackers have started using stuff like anubis. But yes, I initially tried to block them (this was before anubis existed).
It was a few things that I had to take down; a gitweb instance with some of my own repos, for example. And a personal photo gallery. The scrapers would do pathological things like running repeated search queries for random email addresses or strings.
I’m hosting several things, including Lemmy and PeerTube. I haven’t really been aware of any scrapers, but do you know of any software that can help block it?
This is a cool tool. Thanks for sharing. Don’t worry about the downvotes. The Fediverse has a few anti-AI zealots who love to brigade.
The more local inference, the better. Nice work!
ai;dr
Since no one is leaving critical comments that might explain all downvotes, I'm going to assume they're reflexively anti-AI, which frankly, is a position that I'm sympathetic to.
But one of the benign useful things I already use AI for, is giving it criterias for shows and asking it to generate lists.
So I think your project is pretty neat and well within the scope of actually useful things that AI models, especially local ones, can provide the users.
Seriously; local AI use is what everyone should strive for not only for privacy but because it's better than using a large data centre and the power use for Ollama is negligible.
I remember building something vaguely related in a university course on AI before ChatGPT was released and the whole LLM thing hadn't taken off.
The user had the option to enter a couple movies (so long as they were present in the weird semantic database thing our professor told us to use) and we calculated a similarity matrix between them and all other movies in the database based on their tags and by putting the description through a natural language processing pipeline.
The result was the user getting a couple surprisingly accurate recommendations.
Considering we had to calculate this similarity score for every movie in the database it was obviously not very efficient but I wonder how it would scale up against current LLM models, both in terms of accuracy and energy efficiency.
One issue, if you want to call it that, is that our approach was deterministic. Enter the same movies, get the same results. I don't think an LLM is as predictable for that
I'm not an expert, but LLMs should still be deterministic. If you run the model with 0 creativity (or whatever the randomness setting is called) and provide exactly the same input, it should provide the same output. That's not how it's usually configured, but it should be possible. Now, if you change the input at all (change order of movies, misspell a title, etc) then the output can change in an unpredictable way
One issue, if you want to call it that, is that our approach was deterministic. Enter the same movies, get the same results. I don't think an LLM is as predictable for that
Maybe lowering the temperature will help with this?
Besides, a tinge of randomness could even be considered a fun feature.
that's pretty cool, this is just the wrong crowd, don't worry about the downvotes
A recommendation for Moonrise Kingdom based on Mickey 17? The genres might match, but those are totally different movies.