LocalLLaMA

4916 readers

11 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 3 years ago

MODERATORS

pax@sh.itjust.works

noneabove1182@sh.itjust.works

Smokeydope@lemmy.world

MonsterBug@sh.itjust.works

-6

I gave my local LLM agent a face — an avatar frontend with an "emotion-beat" output contract (open-source) (sh.itjust.works)

submitted 1 week ago* (last edited 1 week ago) by catch88@sh.itjust.works to c/localllama@sh.itjust.works

16 comments fedilink hide all child comments

I got tired of keeping a Telegram tab open all day just to talk to my local agent, so I built a monitor-resident avatar that fronts it instead — a video-call-style face that performs the replies instead of me reading a wall of text.

A few things that might interest this crowd:

No runtime GPU for the avatar. Instead of live-animating a face, the agent's reply carries inline emotion-beat tags — [working], [confirm] deploy?, [happy] — and a player selects and blends between ~30 pre-rendered clips. The GPU stays 100% on the model. Code and logs render as chat cards instead of being read aloud, and [confirm] pops a human-in-the-loop approve/cancel that blocks the agent until you answer.

It hooks into a real agent as a connector. The agent's gateway dials out to a local WebSocket the app hosts, so from the agent's side the avatar is just "another channel," indistinguishable from a messenger. Adapters for Hermes and OpenClaw are included, plus a demo mode that runs with zero setup.

Local voice both ways. Edge TTS out (swappable to Qwen3-TTS / MeloTTS / Piper), Silero VAD + faster-whisper in.

Open-core, MIT. The engine is fully usable on its own: drop in a folder of clips named by emotion, point it at your agent. Avatars are pure-data bundles — no code runs when you install one — so you can build your own. The repo ships the engine only; there's no bundled character.

The engine just plays video clips — it doesn't care what produced the pixels. So the avatar can be photoreal, 2D anime, a 3D render, pixel art, anything. You can even use a Live2D or VRM model: pre-render its expressions into clips and it becomes a Ghost Vessel preset. A Live2D shell can't do the reverse — it can't take live-action footage. Clips are a superset. The trade-off is that you give up real-time rig interactivity (no dragging the head around) and you pay a one-time render step.

Repo + 45s demo: https://github.com/ghdtjrtka/ghost-vessel

The part I'd most like feedback on is the emotion-beat output format — the contract that turns model text into a UI performance. Has anyone here tried similar output contracts to drive an interface from LLM output? Curious what broke on your setup, and whether smaller local models keep the tags straight.

you are viewing a single comment's thread
view the rest of the comments

[–] catch88@sh.itjust.works 2 points 1 week ago

Ha — not intentional, but that's a hell of a catch, and honestly the best description of the project anyone's given me. Pre-rendered FMV people who pop up to tell you what's happening… yeah, that's exactly what this is, just with an LLM deciding which advisor shows up. I'll take it as a compliment.