Originally intended as a place to test out the unmerged PR’s of the official Aider project, Aider-CE has gone it's own way

Examples of what it can do: https://xcancel.com/minchoi/status/1986885187313992036

https://wan.video/

Qwen offers similar UI to openai - free max, vision, image generation, has android app, seemingly uncensored.

And are there any that https://jan.ai/ supports? That'd be great!

EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) released Apertus today, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity.

So my relevant hardware is:
GPU - 9070XT
CPU - 9950X3D
RAM - 64GB of DDR5

My problem is that I can't figure out how to get a local LLM to actually use my GPU, I tried Ollama with Deepseek R1 8b and it kind of vaguely ran while maxing out my CPU and completely ignoring the GPU.

While I'm here model suggestions would be good too, I'm currently looking for 2 use cases.

• Something I can feed a document too and ask questions about that document (Nvidia used to offer this) To work as a kind of co-GM to quickly reference more obscure rules without having to hunt through the PDF.
• Something more storytelling oriented that I can use to generate background for throwaway side NPCs when the players innevitably demand their life story after expertly dodging all the NPCs I actually wrote lore for.

Also just an unrelated asside, Deepseek R1 8b seems to just go into an infinite thought loop when you ask it the strawberry question which was kind of funny.

Recent DeepSeek, Qwen, GLM models have impressive results in benchmarks. Do you use them through their own chatbots? Do you have any concerns about what happens to the data you put in there? If so, what do you do about it?

I am not trying to start a flame war around the China subject. It just so happens that these models are developed in China. My concerns with using the frontends also developed in China stem from:

• A pattern that many Chinese apps in the past have been found to have minimal security
• I don't think any of the 3 listed above let you opt out of using your prompts for model training

I am also not claiming that non-China-based chatbots don't have privacy concerns, or that simply opting out of training gets you much on the privacy front.

I'm trying to find a way to translate audio in a FOSS, ideally offline way. I currently use Jan.AI for everything but I realized that I've never tried to upload files to it before, and my current configuration doesn't seem to allow uploading.

after making this post a while ago, i tried out these three techniques for providing tool-result data to the LLM

• append to assistant message
• send as user response
• send model-specific tool-response type

Findings

turns out - the assistant-message appending works great for larger LLMs, but not so well for smol ones.

meanwhile the user-side method works better than expected!

i didnt spend too much time with the model-specific tool role stuff, since i want my tooling to remain model-agnostic.

i will probably switch to the user-side method now for gopilot, leaving behind the assistant-only approach

Tool call formatting improvements

Turns out - my initial tool calling formatting was SUPER token-inefficient - who knew...

So I went from this formatting

okay lemme look that up online
{"tool_name": "web_search", "args": {"query": "how to make milk rice"}}
just put milk and rice in a bowl and mix em

to this, MUCH simpler format

okay lemme look that up online
Tool: web_search("how to make milk rice")
Result: just put milk and rice in a bowl and mix em

which is like - just.... WAY better!!!!

• tokens reduced from 43 down to 24 (cost savings)
• way easier to read
• relies on models code-writing ability
• allows for specific assignment like in json: Tool: web_search(query="my query here")

i hope this is useful to someone out there.

if so, maybe share where you are applying it and tell us about your experience! <3

Hugging Face: https://huggingface.co/PlayHT/PlayDiffusion

Background: This Nomic blog article from September 2023 promises better performance in GPT4All for AMD graphics card owners.

Run LLMs on Any GPU: GPT4All Universal GPU Support

Likewise on GPT4All's GitHub page.

September 18th, 2023: Nomic Vulkan launches supporting local LLM inference on NVIDIA and AMD GPUs.

Problem: In GPT4All, under Settings > Application Settings > Device, I've selected my AMD graphics card, but I'm seeing no improvement over CPU performance. In both cases (AMD graphics card or CPU), it crawls along at about 4-5 tokens per second. The interaction in the screenshot below took 174 seconds to generate the response.

Question: Do I have to use a specific model to benefit from this advancement? Do I need to install a different AMD driver? What steps can I take to troubleshoot this?

Sorry if this is an obvious question. Sometimes I feel like the answer is right in front of me, but I'm unsure of which key words from the documentation should jump out at me.

My system info:

• GPU: Radeon RX 6750 XT
• CPU: Ryzen 7 5800X3D processor
• RAM: 32 GB @ 3200 MHz
• OS: Linux Bazzite
• I've installed GPT4All as a flatpak

I don't have many specific requirements, and GPT4All is working mostly well for me so far. That said, my latest use case for GPT4All is to help me plan a new Python-based project with examples as code snippets, and it lacks a specific quality of life feature, that is the "Copy Code" button.

There is an open issue on GPT4All's GitHub, but as there is no guarantee that feature will ever be implemented, I thought I'd take this opportunity to explore if there are any other tools out there like GPT4All that offer a ChatGPT-like experience in the local environment. I'm neither a professional developer nor a sysadmin, so a lot of self hosting guides go over my head, which is what drew me to GPT4All in the first place, as it's very accessible to non-developers like myself. That said, I'm open to suggestions and willing to learn new skills if that's what it takes.

I'm running on Linux w/ AMD hardware: Ryzen 7 5800X3D processor + Radeon RX 6750 XT.

Any suggestions? Thanks in advance!

the goal is to have an agent that can:

• Understand a complex problem description.
• Generate initial algorithmic solutions.
• Rigorously test its own code.
• Learn from failures and successes.
• Evolve increasingly sophisticated and efficient algorithms over time.

https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/AlphaEvolve.pdf

Model: https://huggingface.co/fancyfeast/llama-joycaption-beta-one-hf-llava

Demo: https://huggingface.co/spaces/fancyfeast/joy-caption-beta-one

Release Post: https://civitai.com/articles/14672

Github: https://github.com/ace-step/ACE-Step

Checkpoints: https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B

Demo: https://huggingface.co/spaces/ACE-Step/ACE-Step

https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2