Recent DeepSeek, Qwen, GLM models have impressive results in benchmarks. Do you use them through their own chatbots? Do you have any concerns about what happens to the data you put in there? If so, what do you do about it?

I am not trying to start a flame war around the China subject. It just so happens that these models are developed in China. My concerns with using the frontends also developed in China stem from:

A pattern that many Chinese apps in the past have been found to have minimal security
I don't think any of the 3 listed above let you opt out of using your prompts for model training

I am also not claiming that non-China-based chatbots don't have privacy concerns, or that simply opting out of training gets you much on the privacy front.

all 19 comments

sorted by: hot top controversial new old

[+] xodoh74984@lemmy.world 11 points 6 months ago* (last edited 1 day ago) (1 children)

[deleted]

[–] Valmond@lemmy.world 3 points 6 months ago (5 children)

Any idea about the minimum specs to run them locally? Especially VRAM.

[–] MTK@lemmy.world 7 points 6 months ago (1 children)

Generally, the file size of the model is slightly larger than the VRAM needed. That's an easy way to estimate VRAM requirements.

[–] Valmond@lemmy.world 3 points 6 months ago (1 children)

Thank you! This is valuable information.

[–] TheLeadenSea@sh.itjust.works 6 points 6 months ago

I ran some 7B models fine on my old laptop with 6GB VRAM. My new laptop has 16GB VRAM and can run 14B models fast. My phone with 8GB normal RAM can even run many 2 or 3B models, albeit slowly.

[–] greplinux@programming.dev 4 points 5 months ago* (last edited 5 months ago)

VRAM vs RAM:

VRAM (Video RAM): Dedicated memory on your graphics card/GPU - Used specifically for graphics processing and AI model computations - Much faster for GPU operations - Critical for running LLMs locally

RAM (System Memory): Main system memory used by CPU and general operations - Slower access for GPU computations - Can be used as fallback but with performance penalty

So - For basic 7B parameter LLMs locally, you typically need:

Minimum: 8-12 GB VRAM - Can run basic inference/tasks - May require quantization (4-bit/8-bit)

Recommended: 16+ GB VRAM - Smoother performance - Handle larger context windows - Run without heavy quantization

Quantization means reducing the precision of the model's weights and calculations to use less memory. For example, instead of storing numbers with full 32-bit precision, they're compressed to 4-bit or 8-bit representations. This significantly reduces VRAM requirements but can slightly reduce model quality and accuracy.

Options if you have less VRAM: CPU-only inference (very slow) - Model offloading to system RAM - Use smaller models (3B, 4B parameters)

[–] sith@lemmy.zip 1 points 5 months ago (1 children)

I believe the full size DeepSeek-R1 require about 1200 GB of VRAM. But there are many configurations that require much less. Quantization, MoE and other hacks. I don't have much experience with MoE, however I find that quantization tend to decrease performance significantly. At least with models from Mistral.

[–] Valmond@lemmy.world 2 points 5 months ago

That's lots of vram

[–] MTK@lemmy.world 10 points 6 months ago

Everyone is stealing your data, the US is doing so in the most intrusive and harmful way by far. If you don't mind using chatgpt, you shouldn't mind deepseek or qwen.

But really, you should avoid all of them as much as possible.

[–] Sabata11792@ani.social 9 points 6 months ago

I been running Deepseek and Gemma locally recently but like to hop to whatever tops the charts and I can manage to run at home. If I need to do anything with a big model I got a few API credits for Mistral but try to avoid it.

I don't like using anything online, especially out of China or the US. If I got to use one online I treat it as publicly posted because we have no privacy.

[–] fitgse@sh.itjust.works 5 points 6 months ago

I use them heavily but through deepinfra.com

They work great.

I personally would not use them through a Chinese provider, but I also wouldn’t use Gemini through Google either.

[–] Jozav@lemmy.world 4 points 6 months ago

I run AI locally for privacy reasons and because I sometimes work without Internet connections. The main purpose is to correct my English for formal letters and emails. I found the Gemma3 model to be working best on the limited resources of my laptop. Deepseek-R1 does not work at all, 'Thinking...' for a long before giving an answer (if at all). The text that Qwen produces is insufficient. So I choose a non-chinese model for practical reasons.

[–] greplinux@programming.dev 4 points 5 months ago

Absolutely. China's models are very advanced - especially Qwen. I don't code professionally - just for personal projects. If I used these tools as an employee in a company, I'd defer to that company's wishes. Regardless of whether it's U. S. or China, these models are using and probably storing my data for further training. As they should! I don't care - they're providing me with powerful tools. I use free tiers and jump between them all. ChatGPT, Perplexity (general search), DeepSeek, Qwen (advanced programming), Google AI Studio and others. I'm grateful, not fearful.

[–] Mvp_rx1212@lemmy.world 1 points 4 months ago

Is Qwen good for coding its coding ai ?

[–] Mvp_rx1212@lemmy.world 1 points 4 months ago

Never used qwen

[–] hendrik@palaver.p3x.de 1 points 6 months ago

I guess the average unpaid AIStudio, ChatGPT or Grok records all your interactions and private data as well and might use it for future purposes. The Chinese might be way more relaxed with people's data, though. I tried various AI services. I try not to put in personal data in general. And I'm more careful with Chinese apps and services.

[–] keepthepace@slrpnk.net 1 points 6 months ago* (last edited 6 months ago)

I support companies that open source their models. I am more worried about proprietary AIs.