Home Assistant

This is an automated archive made by the Lemmit Bot.

The original was posted on /r/homeassistant by /u/rdehuyss on 2026-07-20 21:45:46+00:00.

Disclaimer: written by hand, without any AI 😄

I was about to give up on a local LLM using Home Assistant and Voice Assistant on my Mac Mini M4 Pro with 24GB ram.

Since Home LLM had the most stars on GitHub, I was trying to use that together with Qwen 3.6 or the recent Gemma 4 update. I switched from Ollama to LMStudio, to Llama.cpp. I used different versions of Gemma 12B, Gemma 26bA4b, different quants, ...

Nothing worked... it either was way too slow to respond (due to Home LLM adding a lot of tools which polluted the context to > 6000 tokens) for a simple hello or if I used legacy tool calling, it was fast enough but tool calling failed. I had copilot with claude try to fix it but nothing worked.

My final attempt was to switch integrations. The other actively developed local LLM integration I tried was Local OpenAI LLM by skye-harris - and bam, everything worked out of the box. It was fast, responsive, and felt much closer to using ChatGPT or Gemini (except for the first call).

We often focus on finding the best model, but perhaps we should also be asking which integration provides the best experience.

That is not to say Home LLM is a bad project. I’m sure it works well for many users and has its own strengths. In my case, though, I simply couldn’t get it working reliably.

My setup:

• HomeAssistant and MusicAssistant (+ all the seeers) running via Colima (docker)

• Local OpenAI LLM by skye-harris

• Llama.cpp

• Gemma 4

Yes, all on this Mac Mini M4 with only 24GB ram.

This is how I start llama.cpp:

llama serve \
   --hf-repo ggml-org/gemma-4-12B-it-GGUF \
   --hf-file gemma-4-12B-it-Q4_0.gguf \
   --alias gemma-4-12b-ha \
   --host 0.0.0.0 \
   --port 6543 \
   --ctx-size 16384 \
   --parallel 1 \
   --n-gpu-layers 999 \
   --flash-attn auto \
   --reasoning off \
   --cache-type-k q8_0 \
   --cache-type-v q8_0 \
   --jinja \
   --api-key "redacted"  

If you think the llama.cpp startup / config can be improved, feel free to comment. I'm new to this scene.

And to prove it, here a screenshot (the light is on/off after about 1 sec after the question). I added the asjeblieft so it would get routed to the LLM.

voice assistant

And it even works with Music Assistant!

Music Assistant integration

I normally don't post much but I thought this was worth sharing...