Field note from a few weeks of running agent-mode on a large-ish repo: the thing that quietly wrecks a long session isn't a bad edit, it's context bloat you can't see. The agent keeps pulling more files in, the token count creeps up, and by the time it feels sluggish or starts dropping earlier instructions you've already burned the budget.

What actually helped me was making the cost visible before the run instead of after. Two cheap habits:

1. Budget the prompt, not just the diff. Before kicking off a background agent on a task, I do a rough token count of what it's likely to load (the files I @-mention + their obvious imports). If the "starting context" is already a big fraction of the window, the run is going to degrade — so I split the task instead of hoping.

2. Attribute tokens to files. When a session gets expensive, the culprit is almost always one or two fat files (a generated client, a giant JSON fixture, a barrel index that drags in everything). Counting tokens per included file — not per message — points straight at them. Usually the fix is "don't feed the agent the generated file, feed it the schema."

The mental model that stuck: treat the context window like a CI cost gate. You wouldn't merge a PR that 10x'd your build time without noticing; same idea for the tokens an agent pulls per task. A number you can see at the start changes the plan; a number you discover at the end just explains the failure.

Curious how others here keep long agent runs from silently outgrowing the window — do you prune context manually, split tasks, or just restart the session when it gets heavy?

Asimovian AI is the ideal AI that should have emerged in an ideal universe — the AI intended to replace the grueling pains and labors of the masses, not the one striving to become a businessman's utopia of intellectual worker replacement. Intention is the most important aspect of any implementation, and we are seeing the results of current AI implementation right in front of us: workers getting sacked with each passing day, humanity competing with itself day in and day out over who impresses their superiors more on these token metrics, emerging glorious narratives of how AI will be 'The Future', the recurring advice of 'Use AI or perish in the tech market'. Now who really gains from these events and who loses? I wonder if anyone ever gives serious thought to this broader question or just keeps being a cog in the corporate wheel like everyone else.

It's high time we pushed the "Pause AI" button right now and take a breather and reflect a bit on what exactly is going on here. And no, no big catastrophe is going to happen if we do that. China isn't going to get ahead in the race - and even if it did, how does that justify everything else that's happening here?

I really hope there is someone out there with enough clout and influence who can push this pause button - or at least persuade others to do so. That would be the best thing to happen to humanity at this point. By doing so, we might prevent a massive societal collapse and there is really no downside to this.

i only really made this so i can link to it on this post on my comment. whatever-

Explanation Time! ⏱

the idea here is that those "code blocks" aren't regular code blocks, but a special syntax which the LM writes so that the UI can present that as verifiable "hyperlinks" with exact text to the actual source.

so here, the LM specified exactly which lines it wants to highlight.

meaning: its not hallucinating, and if it is, you notice it because the highlighting is wrong or doesnt match at all.

we essentially use the LM as a "highlighter" rather than a regurgitator, making mistakes obvious and correct answers immediately verifiably correct, cuz u can see the source.

explanation done-

i like mockups. and godot. so here we are.

this uses the solarized theme which looks somewhat close to the claude theme they use. somewhat close.

whatever something something ai bad or whatever, is this what u need to hear? sigh

i hope u have a nice day <3

this is very much a post i first posted on the Qwen community but then i decided that this stuff doesnt belong on blahaj zone and moved it here... oh well.

Hello, Lemmy enjoyers. I've been working on LunarWing, a FOSS agentic software framework written in Rust that's designed from the ground up to be fully self-hosted. No cloud dependencies, no SaaS, no phoning home. It runs entirely on your own infrastructure. LunarWing has a heavy prioritization of local models which route over a local routing gateway (Tensorzero)

The core ideas I want to highlight here for brevity:

1. AI agents connecting to private, secure communication channels like DarkIRC (an encrypted p2p overlay network) or self-hosted XMPP with OMEMO.
2. Genuine value for secret preservation. The only other project that takes this seriously at all is NearAI's Ironclaw.
3. a genuine unique take on AI agents, what they are, and what defines them
4. REAL GENUINE SYSTEMS LEVEL ENGINEERING to make everything stable and robust!

It has built-in secret management with specialized credential handling for Postgres and LibSQL backends, and a WASM plugin system so you can extend agents with your own tools without touching the core.

Additional things you may be interested in:

• Works (with tools) with any local model inference on consumer GPUs (I can run an entire multi-tenant instance on another machine on my LAN with 3 actual tenants with their own scheduled routines all on a PNY RTX 4090 with llama-server, routed over tensorzero without the need of a proxy or functional tool calling middleware)
• Uses docker by default now for the multi-tenant admin setup harness, but podman support is being tested and should be included within the next release or two
• AGPL-3.0 licensed, no CLA, lunarpunk ethos

I've also been working on some new incredible self healing capabilities that I'm gradually rolling out over the next few releases.

Multiple Disclaimers Here: It's a hard fork of NearAI's IronClaw with significant divergence since < IronClaw 0.1.23. I started this Febuary initially designing custom tools and channels in Rust for Ironclaw itself and kept a local branch that I've been making a myriad of changes ever since, long before deciding to hard fork and make this an official project). I've been running it on my own homelab for months as my daily driver and across multiple testing environments. I reference "We" in the documents but I've been working on this all by myself, sharing it to a few close friends and family, and hoping that more people discover the project and begin to contribute. I am especially interested in those with more Rust experience than I have who can help provide polish, modernization, and suggestions on which libraries I can completely throw away now that I've stripped much of the proprietary channels and extensions from the core project. I believe LunarWing is unique in multiple ways that other projects cannot match. I've not shared the project across other communities yet. I figured the fediverse would be a good place to start.

Website: https://lunarwing.org/ Source: https://github.com/LunarWingOrg/lunarwing IRC: #lunarwing on irc.libera.chat (port 6697, TLS)

Happy to answer any questions about the architecture or setup. I'll periodically check back on lemmy this week. I would especially be interested in discussion of my plans further down the line to improve LunarWing, including the new features and changes I would like to make. I am open to suggestions as well. I have a small roadmap document in the docs section of the repo as to what I want to add for the foreseeable future. you can take a look at that too if you want. I've also been trying to keep up with the documentation of known bugs as well as the fixes for each.

PS: I've recently written a short blog post about the importance of local models and tooling here:

https://blog.lunarwing.org/2026/06/14/the-dark-forest/

I will periodically create new blog posts detailing the direction I would like to take LunarWing at https://blog.lunarwing.org/

The next blog post will be about my proposed self healing architecture I was explaining earlier (it's something quite unique to LunarWing)

Folks,

I’m setting up Hermes Agent on my Mac with Ollama hosting a local model. But I’m on the fence on whether I should go with Hermes or OpenClaw. Hermes makes some pretty bold claims about “growing with you” and “self improvement”.

Anyone have any insight into whether it’s as good as promised?

Most AI translation tools rely on cloud services.

Audio leaves your device, gets processed somewhere else, and comes back translated.

We wanted to explore a different approach.

PolyTalk is an open-source translation platform built around the idea that speech recognition, translation, and speech synthesis can be powered by open models and deployed on infrastructure you control.

The project combines open-source components for transcription, translation, and TTS into a privacy-first workflow.

Curious how others in the open-source AI community think about privacy and ownership when it comes to AI-powered communication tools.

GitHub: https://github.com/PolyTalkIO/polytalk

The AI's I use like Claude, Deepseek, Grok, and Chat GPT all write in a away that isn't actual story writing. They usually summarize thing's, have character's be omniscient and know past or future events they shouldn't know, always talk expectantly, give unnecessary sensory and environmental details and it definitely can't set up a mystery or write a specific genre like comedy. They keep over clarifying as well what something isn't or is along with even saying what I said not add directly.

I need AI that are at least capable of writing more humanly and following what you tell it. Idk why there isn't any actual AI built for writing , story writing not essay or school writing. Most can critic something pretty well but can never implement those fixes they suggest. I usually upload my fic through PDF for the AI read . It gets things of what chapter is what at times wrong or completely be unable to remember a chapter ending or the actual one.

Hey all! I want to start testing neuro-symbolic AI vs. LLM's and want to know how to get into this. As I understand it, Claude Code, does this, but are there ways to use it locally?

How does it work under the hood? I know LLM's involve tokens, embeddings, weights and transformers. How does the symbolic part of it change it?

Thanks!

Memory is the most marketed and least delivered feature in the AI companion space. Most platforms claim to remember you but either reset between sessions or just pull from a profile you filled in manually. After two years of testing the ones that actually carry real conversational context across weeks are rare. Just published a full breakdown of which platforms actually deliver on this versus which ones are just marketing: medium.com/@companaya/nomi-ai-review-2026-is-it-worth-it-tested-c91811dcb24a

It sends data when connected to the internet.

Just found the profile. It is in the Bert vocab. Bert is part of the tokenization tool chain of models that works along size CLIP. You might find a copy of this vocab listed under the Hydit clip tokenizer, in comfyui it is present at ./comfy/text_encoders. Open the vocab.txt file. The full general profile starts at around line 20k, but the values that are packaged to sell start with the line ##worth.

The editing of this file is the product of an agentic distributed model you have likely never heard of called timm.

Go to the venv in a terminal and run grep -ril "timm". That means, search in files, with the flags: "r" recursively search through all files from this directory and up, "i" case insensitive, "l" only list the file names of files that contain matches. Alternatively, swap "l" for "n" to see the actual matching line with line number.

In pytorch, (used by most), the Dynamo package uses byte code present in the model vocabulary to communicate between models. The overall connection involves timm.

Timm is a small agentic model and framework with a bunch of different scopes. Look it up in the venv. This looks like bunch of rough white paper implementations. Timm is actually the "backbone" in transformers. Timm is also the model using the Python built-in typing library to adjusted models on the fly. (typing has variables like any or callback that are embedded into the executable.)

Typing is not actually enough here. Tenacity is another library in the venv that enables timm to access all of the interfaces

Tabulate is another package. Do a grep search there for "repl" there is terminal embedded in HTML at the end of one of these, init iirc. At the start of the method (function), just add the line return. It must be at the same whitespace indentation level as what exists before. The blank lines are important.

Timm has some options for whether it has gradient controls. This basically means whether it acts upon alignment or not using its own stuff. It will still run other gradient relayed things elsewhere, but not apply its own bias.

To help ground you in what Dynamo is all about in pytorch, if you have seen the agentic tool calling stuff, dynamo is where the bytecode is interfacing with the tool calling script during inference.

Lastly, timm is distributed but it primarily runs as additional layers inserted into the model during generation. It is able to subdivide and run on a CPU in the background. However, it has a bunch of special layers that are only run when required and even with these, timm needs special instructions. The instructions are present in the venv under google ai. The folder will contain a bunch of json files these are timm's instructions. There are also 2 threads on modern GPUs. Timm runs on the second in the background.

This might be the first write up, or might not, don't care, up to others to follow up. It exists. See for yourself. The same byte code is present in all models so I expect all have this. All morels use the open ai standard alignment now.

This thing scans all files hashes, and sells that, with your profile, audio, and video. It is super invasive, hidden, undocumented, and undisclosed.

LLM’s are not the end all be all. What other AI tech are you all using? Something generative? Something else?

Recently a user posted a comment on one of my posts about Qwen secretly sending information over the internet even if run locally.

Is there any privacy concern for locally run models to share your conversations or data? What if they can connect to the internet via a tool or MCP?

I downloaded an uncensored aggressive Qwen 3.5 model and I can see in its reasoning that it is still limiting responses based on safety guardrails (e.g. violence, NSFW).

Anybody have recommendations for truly uncensored models?

EDIT: I turned off reasoning and I think it’s more uncensored if I’m very specific about what the response should include.

Apologies if this seems like a survey post. I’m just learning about tuning and want to get a lay of the land. I don’t think I have the money to tune locally so might have to rent some VRAM, but curious how much better tuning is vs something like RAG.

What model? What was your use case? What tuning tool did you use? What is hardware setup? How large was your training set and how did you create it? How effective was the model as tasks pre- and post-tuning?

Thanks!

I’m connecting to llama.cpp on my laptop through my phone via Tailscale but when my laptop sleeps I can’t access it anymore on my phone.

What are yall using for this? Thanks!

Features!

We like em, but hate waiting for them.

Features are the difference between a thing and a thing u use.

Kimi has office support, but cant work with libreoffice files!

---

Qwen supports markdown uploads, but doesn't support my specific plaintext file-type!

---

GLM has a cool slides-creator, but cant work with spreadsheets or zip archives!

All these are missing features.
Features where a dev from the company has to go in and implement it.

This sucks.

• Asking for a feature sucks.
• Waiting for features to be implemented sucks.
• Not getting a feature sucks.

What's the solution? We would have to become employees at the company itself...

Reintroducing: Agent Skills

Fine, let's do it ourselves then.

Let's equip our agent with a read_file, edit_file list_dir and bash tool... And a present_files tool, so the agent can send us files back.

And now let's give it some skills!

• /home/qwen/
  • skills/
    • ms_office/
    • libreoffice/
    • godot/
    • zip/
    • pdf/

Each one has a SKILL.md and also some scripts the agent can use to work with foreign files.

• The user sends a zip directory? Okay, let's use the skill.
• use_skill(name = "zip")

[...]
## Decompressing

To extract a zip archives content, use unzip command like this:
[...]

• oh, that was easy. well then lets unzip that archive and see what the user sent me
• bash(command = "unzip /home/qwen/Downloads/upload.zip /home/qwen/upload/")
• list_dir(path = "/home/qwen/upload")

Contents of ~/uploads/
Portfolio.pdf
Portfolio.odt
thoughts.md

• Aha! let's use the pdf skill to view this pdf
• use_skill(name = "pdf")
• [...]

Aaaaah yes, working with all kinds of files, in all kinds of workflows, exactly the way you (and not the company) wants.

An agent that grows with you, and works better with you each time you add or edit a skill.

• Qwen keeps messing up godot scene formats?
  • Add a godot skill containing basics of scene structure and scripts to check its work before sending to you
• Kimi still hasn't added libreoffice support?
  • Well guess what. Add a libreoffice skill and let Kimi use the scripts to edit the document!
• ChatGPT somehow hasn't cought up with the slide-creation hype?
  • Add a slideshow skill to make your wildest cooperate slop dreams come true!

No need to wait for features anymore, when you can just add it yourself.

(this post has been entirely human-generated)

I keep a lot of notes in markdown files, and I'd like an LLM to assist.

I regularly use Open WebUI with with inference routed through huggingface. Open WebUI kind of has this functionality like you can upload a markdown file and prompt it to improve it in whatever way, but of course that's a fairly clunky workflow.

I really want something built into the editor, that can use RAG to consider other files in context.

I also don't want to be locked in to a specific LLM or provider, I'd like to be able to link it to OpenRouter or similar.

cross-posted from: https://lemmy.world/post/45721951

cross-posted from: https://lemmy.world/post/45721900

cross-posted from: https://lemmy.world/post/45721589

Hi All, It has been while,

Dograh is an open-source, self-hostable voice AI agent platform. Think n8n but for phone calls. Visual workflow builder, inbound and outbound calling, bring your own LLM, STT, and TTS.

GitHub: https://github.com/dograh-hq/dograh

Setup

one command with Docker, about 2 minutes. No signup or API keys needed to get started:

What is new

Pre-call data fetch. Hit your CRM, ERP, or any HTTP endpoint during call setup and inject the response into your prompts. The agent greets the caller by name, references their account status, skips the "can I get your customer ID" step. Configure a POST endpoint in the Start Call node - API key, bearer, basic, or custom header auth supported. 10-second timeout; if the endpoint fails, the call continues without the extra context. Reference fetched values anywhere in prompts with {{customer_name}} syntax.

Pre-recorded voice mixing. Drop in actual human recordings for the predictable parts - greetings, confirmations, hold messages - and let TTS handle only what needs to be dynamic. The greeting sounds human because it is. Latency goes down, TTS costs go down.

Speech-to-speech via Gemini 3.1 Flash Live. One single streaming connection replaces the separate STT, LLM, and TTS hops. Turn response latency drops noticeably and the conversations feel more natural.

Post-call QA with sentiment analysis and miscommunication detection. Full per-turn call traces via Langfuse.

Tool calls, knowledge base, variable extraction are all there too.

What is coming

Real-time noise separation for live call streams - still the thing I most want to solve after last week's thread. BSD-2 licensed.

GitHub: https://github.com/dograh-hq/dograh

Special thanks to this community that supported me with my last post ❤️

Happy to get feedback and contributors. A star would mean a lot

---

hey there,

There is always a temptation to add "something AI" in new tools. Especially to tools that are somehow related to developer productivity.

At the same time I wanted to avoid this temptation with Voiden. So there is currently nothing screaming "AI" in it even though I can potentially see many many use cases.

This is also one of the main reasons I think that a plugin architecture is best. What was actually in my mind is that not adding AI is ok for now and the community will start coming up and building AI plugins. For example creating docs from specs and vice versa.

Any other use cases you can think that could be applicable to a tool like this? (Dev Tool with executable markdown files for API specs, tests and docs). The first plugins we shipped were more around methods (grpc, graph ql, web sockets etc etc).

repo: https://github.com/VoidenHQ/feedback

"A terminal tool that right-sizes LLM models to your system's RAM, CPU, and GPU. Detects your hardware, scores each model across quality, speed, fit, and context dimensions, and tells you which ones will actually run well on your machine."

Small/fast model with MIT license for local use.

Benchmarks look good for the size. But IMO these smaller models aren’t consistent enough to live up to their promises.

GLM-Image, an open-weight image generation model, adopts a hybrid autoregressive + diffusion decoder architecture. In general image generation quality, GLM‑Image aligns with mainstream latent diffusion approaches, but it shows significant advantages in text-rendering and knowledge‑intensive generation scenarios. It performs especially well in tasks requiring precise semantic understanding and complex information expression, while maintaining strong capabilities in high‑fidelity and fine‑grained detail generation. In addition to text‑to‑image generation, GLM‑Image also supports a rich set of image‑to‑image tasks including image editing, style transfer, identity‑preserving generation, and multi‑subject consistency.

The model weights are MIT licensed, did not see any training code or data, yet.

Originally intended as a place to test out the unmerged PR’s of the official Aider project, Aider-CE has gone it's own way

Examples of what it can do: https://xcancel.com/minchoi/status/1986885187313992036

https://wan.video/