alot of really small SLMs seem to be too small to even function properly.
however, there may be some unconventional uses for some of them.
one usecase keyword generation. heres what gemma-3-270m-it-qat-Q4_0 gave me
prompt: keywords similar to "construction"
Construction activities
* Construction design
* Construction equipment
* Construction management
* Construction materials
* Construction methodology
* Construction organization
* Construction planning
* Construction process
* Construction project
* Construction project details
* Construction project management
* Construction project management activities
* Construction project management assessment
* Construction project management checklist
* Construction project management document
* Construction project management evaluation
* Construction project management framework
* Construction project management methodology
* Construction project management plan
* Construction project management project
* Construction project management report
* Construction project management review
* Construction project management schedule
* Construction project management software
* Construction project management strategy
* Construction project management system
* Construction project management tool
* Construction project planning
* Construction project scope
* Construction schedule
* Construction strategy
* Construction work
```
90 TPS (but got stuck in a repeating loop)
i think theres alot of other uses. but they arent really LLM answer that people usually reach for an LLM.
im not aware of any nlp tools that generate new keywords(without attaching to an existing database).
i'm not advertising that all LLMs do what mainstream users expect it to do.
there are many applications, just for this single example. for other uses or the exact application.. you need to figure that out yourself.
it's true that larger modern LLMs are cool. but this post isnt about that.
I think we have several text extraction methods... You can look up the distance between embeddings... We have models which can assign keywords to images, topic modeling (and keyword extraction)... It's not entirely clear to me where we're going with this... But I'd say if you're not using a text corpus or "database" or working on text, you'd use word embeddings and just look up the distance between them. Very close ones will be synonyms. Next will be similar ones. That's way less effort than doing inference. And it'll generate the new keywords just from the embedding space itself. There's of course also traditional lexical databases. Something like a thesaurus or WordNet comes with least effort.
Coincidentally, I think the BERT model is still being used for tasks like this. It's an encoder model, but with a similar amount of parameters like what you mentioned in the post. But it's not being used for text generation in this way.
I mean I'm not really disagreeing here. We have a lot of small machine learning models and they're very useful. I just can't come up with generative-AI-like tasks for LLMs at that size category.