wow! agents sure can do stuff.

Agent Basics

so like - claude code is getting a lot of attention lately. its some coding agent thing using their models.

lemmy just define what an agent is real quick:

agents dont just yap but they actually cook, as in: they can do stuff.
this works by having a secret back and fourth with the computer, its essentially doing function calls, but we call them "tool calls" because... why not.

Here an example of what this might look like

(every "quote" is the "user" role and every bullet point is the assistant response)

---

(here are some hidden instructions on how to make tool calls) hey, plz put the stock prices of fairphone over last 2 years in some txt plzplz <3 also gimme a kiss when ur done >////<

• Sure! First off all all, imma look up the stock prices.
  get_stock_prices(company="fairphone", start="01-01-2024", end="01-01-2026", increment="month")

<tool_response>
blablabla some stock prices here...
</tool_response>

• Ah, okay, I now have the stock prices, not imma make that txt
  create_file(path="/fairphone_stock_prices_2025.txt", content="the actual file content here")

<tool_response>file /fairphone_stock_prices_2025.txt has been created sucessfully. You can send the file to the user using the send_file(path: str, subtitle: str) command</tool_response>

• Now I will send you the file
  send_file(path="/fairphone_stock_prices_2025.txt", subtitle="heres ur file")

<tool_response>"/fairphone_stock_prices_2025.txt" has been prepared to send and it will be attached to the last message you send</tool_response>

• Hey user, i made that txt u asked for! The file is attached below for download. Oh yea, also kisses u <3

---

Agents scaling Problem

this agentic loop works pretty well. but it doesn't scale well to hundreds - or even thousands of tools.

agents must be fed with tool-definitions, which have:

• name
• description
• arguments (usually with descriptions and types)
• example tool uses (optional)

for each individual tool.

now, all this stuff stuffs the LMs context, resulting in

• higher costs (because of more input tokens)
• it just confuses the model. could you remember hundreds of functions all at once?... i couldn't.

aaaaa How do we solve this?

introducing!!!!!!!!!!!!

docs!!! but for language models.

yes! that's what anthropic is trying to propose with SKILL.md. here's what that looks like:

• u just give the agent a filesystem with some docs and some useful files.
• a "skill" is just a directory with a SKILL.md file in it. that file must include a yaml header with name and description
• every single SKILL.md file header in the filesystem is automatically loaded into the agents context along with the files location and an instruction "Read any skill file if it appears useful" or something similar

your filesystem may look like this

• /
  • skills/
    • stock_prices/
      • SKILL.md
      • stock_stools.py
      • report_format.md
    • stack_overflow/
      • SKILL.md
      • stack_overflow_tools.py
    • creative_writing/
      • SKILL.md
      • themes.md
      • preferred_format.md

the /skills/stock_prices/SKILL.md may look like this:

***
name: stock_prices
description: Useful for detailed stock analysis
***
## Getting stock Prices
Use ./stock_prices_tools.py to run code which gets you the stock prices of registered companies:

` ` `python
import "/skills/stock_prices/stock_tools.py" as stonks
stocks_for_a_year = stonks.get_prices(company="nvidia", from="01-01-2001", to="01-01-2020", increment="1 month")

stocks_today = stonks.get_price(company="nvidia")

print("Stock prices from 2001 til 2020:\n" + str(stocks_for_a_year) + "\n\ntodays price: " + str(stocks_today))
` ` `
no need to read the file itself.

## Creating reports
Read ./report_format.md for the users preferred structure of a stock prices report

[...]

so... we simply decentralize the knowledge and the tools, by just putting them into files, so that things can be read once they become useful.

but Isn't this super unsafe?

this approach assumes that the given agent can write and run python code.

this works great for such agents... but well... its also rather unsafe, isn't it?

so you would have to

• either trust the agent with whatever it's doin (like editing and removing files and such)
• or supervise its every step

both options are bad. so! let's build our own better solution

Let's make this better

how about this? we add a .tools/ dir into every skill dir. and each file in that .tools/ dir will be a safe python file, which defines a single tool, like this:

• /
  • skills/
    • stock_prices/
      • SKILL.md
      • report_format.md
      • .tools/
        • get_prices.py
        • get_price.py
        • [...]

then, once a SKILL.md file is read, all the tools in its .tools/ dir are loaded into the agents memory, so it kinda explores to find new tools.

So what does this look like in practice?

this SKILL.md setup gives us:

• essentially infinite tools
• dynamic context retrieval (only reads stuff when necessary)
• usually lower token prices
• the ability to define standard operating procedures without forcing it into the agents memory

so like - gimme an example

okay okay, heres an example of what this might look like, without anything hidden. again, each quote is a "user" message and each bullet point is an "assistant" message:

---

You are an agent and you have access to these tools:

• read(path: str) # returns file content or dir content
• create_file(path: str, content: str)
  You can use tools using this format:
  name_of_the_tool(arg1=your value here, ag2=another value here, ...) Simply include that format at the end of your message to use it.

Skills

Read these skill files when they become useful:

• /skills/stock_prices/SKILL.md (Useful for detailed stock analysis)
• /skills/stack_overflow/SKILL.md (Read this to search for help on programming problems)
• /skills/creative_writing/SKILL.md (For writing stories, poems and whatever creative task the user feels you should perform) Today is 16-01-2026 hey, plz put the stock prices of fairphone over last 2 years in some txt plzplz <3 also gimme a kiss when ur done >////<

• Sure! I will read the stocks skill
  read(path="/skills/stock_prices/SKILL.md")

content of /skills/stock_prices/SKILL.md

blablabla here the entire content of that file

two tools have been added:

• get_stock_prices(company: str, from: str, to: str, increment: str) # retrieves stock prices over time. "from" and "to" must have this format "DD-MM-YYYY" and increment can be one of "day", "week" "month", "year" with optional number prefix like "3 week"
• get_stock_price(company: str) # returns current stock price of given company

• Right, I will look at the report format.
  read("/skills/stock_prices/report_format.md")

content of /skills/stock_prices/report_format.md

some format stuff here, eh - whatever

• okay, now lemmy see those stock prices!
  get_stock_prices(company="fairphone", start="01-01-2024", end="01-01-2026", increment="month")

<tool_response>
blablabla some stock prices here...
</tool_response>

• (the rest of this continues like the initial example i gave)

now, the important thing to note here is that we didn't include the context of all those other skills. here, the agent doesn't need to know about creative writing or whatever - just about stocks stuff.

we essentially lead the agent around the filesystem, by prompting it in the files to look at other files if they appear useful.

this - is - huge!

the tool amount was rather limited previously, and - best of all: it doesn't even have to be about tools! it can be SOPs, style directions, or just some info about yourself which doesn't need to be loaded all the time.

butbutbut!!! *even bester-of-all!?!!?!

You can share Skills online!

now this - THIS!!! changes things.

sure, being able to make a skill to solve some edge case is cool. BUT!

now you can go share your skill online, and nobody has to deal with that edge case anymore.

this way, you slowly accumulate more skills, improving your agent with more and more capabilities.

But what if I have too many Skills?

oh no! the same problem from earlier came back! aaaaaaaa >o<

worry not ~ for there is a super simple solution:

just move some folders and edit some files.

yes, just cluster some skills together! maybe put stock_prices along with web_search into a new skill online_stuff with a description "For searching the web and getting stock prices". then rename those old SKILL.md files to something else... like maybe general.md or whatever and remove their header.

Verdict iguess

this stuff is cool.
i always liked the idea of structuring stuff as files and dirs instead of json or whatever sql uses. it makes editing the data SO much quicker and easier for everyone.

if u wanted to edit an agents behaviour earlier, you would have to

• know prompting stuff
• know programming basics
• go and find the files somewhere in your agents directory
• hope that they expose their tools
• somewhere in there, adjust a system prompt

but now! u can just - add a directory with stuff in it you want the agent to have.

its - so simple and obvious - but soooooooooooooooo useful.

i feel the urge to.... write some random skill and see the agent explore the dir and say "ah, this seems useful" and get all giddy bout it ----- but that's probably just me....

* the background was taken from the Sky: Children of the Light Wiki

alt image link from blahaj zone

so anyway - I didnt post in quite a long while, so here I go yappin bout that one game i keep playin. there really is no reason to do good besides the good feels, and som peeps seem to not have that very much -..... so here we r.

imagine how fun and cool the world were if there would be a reason to play along nicely - like -... gosh, now that sounds pretty nice, hm? a world where slip-ups r oki - but bein systematically evil.,.. well - its actually bad for that peep.

but no! now instead, im considering stealing from supermarkets for my own benefit because there really is nothing holding me back from doing that. like - I dun wana! but if that somehow hurts the larger chain, im all here for it-.... iguess it doesnt hurt the larger chain so i should leave it,... but in larger - clearly profitable markets it feels fine-

feels so wrong.... or rather -.... doesnt feel so wrong anymore when u notice that all the big peeps r also bein evil. if that means that all the cool and nice peeps stay at the bottom which the cheaters and not-cool evils peeps get to rise, well like - that feels like the opposite of karma (or whatever that is called-)! theres clearly an inverted reward system happening here!

piracy and the like doesnt feel evil if the prices for watching the art legally is rising so sharply, but not to offer a better service, but to just.... make more möni.

CW: really agregiously rage-bait kinda stuff

this might sound really unlike a Lemmy user right here, but im actually not that opposed to a capitalist-like system, where exploitation and monopolies r somhow mitigated.... to be very clear: im from som country employing social democracy where most of the basic living this r cared for to some degree kinda for free, so i have more hope in that than a fellow 'merican 🇺🇸🍔🛢️🏚️🤑🔴🔵🪦🧀👨‍💼📈🔫 might think about it, cuz im actually experiencing som of those benefits.

but like -,... i mean.,. of course id prefer fully automated luxary gay space communism, but for that revolution to take place, there r too many peeps to still convince. maybe its a bit too early for that- maybe a basic "heyo, we kill more CEOs and hope good comes of it" is more applicable ~

oh yea ohgosh I just opened a whole can of worms I probably shouldnt have. plz,.... be kind in the comments....

som post i wanted to "refine" - but didnt get around to it....

alt image link (blahaj zone)

---

soooo have this version with a line through some l makin it look like a t... oh well...

oh yea - also dont forget to put water on face beforehand.

been posting very little on the 196 lately.... have stuff to do... like poating dum stuff bout lms for the 100th time-

so - how do u feel about a journey-game like experience? should it be accessible

• only on pc
• only on mobile
• only on web browser
• mobile and pc only
• pc and browser only
• mobile and web only

?

i feel the mobile form factor makes games waaaay more approachable, installing an app feels like less of an act compared to pulling an AppImage for desktop use-

language models like chatgpt are specifically trained to accel at these tasks:

• summarization
• extraction
• and classification

all these can usually be done by small language models, even ones running locally on your laptop at reasonable speeds, no gpu required, especially with todays efficient "mixture of experts" (MoE) models.

now HOLD UP. these are already some incredible capabilities! with classification alone, we can do incredible amounts of big data analysis. when combining this classification ability with text extraction (the ability to cite specific parts of a given text corpus), we can already create all sorts of cool things!

let's imagine an LM powered search engine feature.

in googles case, they added their "AI overview" at the top of the search results, which

• displays a lengthy LM summary of the top search results
• pushes actual results off the screen
• sometimes contains info which wasn't in the source text

now this is what i call a boring and annoying application of LMs.
let's design a search engine feature which people might intentionally use:

• a Bubble Relevant 🫧 button.

what does the bubble relevant button do? well, it:

1. scans through the first page of search results
2. extracts sentences and quotes which fit the search query
3. check quotes against source text (so no quotes are made up)
4. rerank these extracted quotes by how relevant they are to the search query
5. present the quotes to the user with their original sources

here, LM outputs become pointers to actual real sources. how about that?

• worst case: the suggested quotes are useless to you
• best case: the LM finds a quote which fits the exact thing you wanted to look up

i might use that feature sometimes maybe. especially since the environmental burden isn't as insanely high, due to the use of drastically smaller LMs in use here (about 50 to 200 times smaller).

this isn't a feature worth investing billions into, but maybe a one which has me pick one search engine over another.

just in case anyone is wondering: hellooo world, this text was written by a real life human. you might want want to believe it, but markdown highlighting can even be used by people-

some stuff i maybe should mention

the "ai" hype has made LMs very polarizing.

training todays largest LMs (also called "LLMs") costs a lot of money and CO2 emissions, and features like googles "ai overview", especially when citing hilariously bad sources, the existance of elons mechahitler and the fact that all LMs are trained unethically makes LMs appear useless and dangerous.

others see potential in LMs, pointing to things like AlphaEvolve system finding more efficient ways to multiply matricies, and their Cell 2 Sentences Scale Gemma 27B model finding genuine potential ways to cure certain types of cancers (both of which also stem from alphabet). they hope for more such things down the line.

both sides have genuine arguments here, there really is no clear winner, at least from my point of view.

it makes me not want to ignore that entire discussion, ignore what the "ai bros" are yelling and test things for myself with local models, to see if these models are useful to me. im not sure about that yet.

are language models useful to you?

---

wanna test local models for yourself?

try installing this free and open source server ollama. it's available for mac, gnu/linux and windows.

it lets you download and run open-weight language models on your own computer, essentially letting you host your own "OpenAI API" endpoint, alongside a CLI.

after installation, you can download models with ollama pull model-name and test it in the CLI with ollama run model-name. for CPU-only computers, i recommend trying the model granite4:tiny-h and for people with GPUs with over 6GB of VRAM, try qwen3:4b-instruct.

you can search through the huge log of models on their model library

---

EDIT: added image - cuz for some reason setting a "thumbnail url" doesnt actually display that image as a thumbnail

EDIT 2: you know what? - no. i wont. im removing that image again. it was just some chatgpt image. how boring. this is a mediocre rambling about LMs in search engines, it's already spicy enough. i put a google logo instead.

i am a moni expert. contact me today and we fogure out ur business plans with moneymaking strategies like smart investments, work-money balance and betting-bingo.

i suggest u read up on money beforehand in my book "how to make money", by maria friedrich (thats me btw, im maria friedrich, the person who wrote the book "how to make money").

i hope my revenue plans r of som use to u peeps.

alt image (blahaj zone)

EDIT: the reason i made this post is cuz - whenever i use this laptop here, i feel like some wana-be business slobbbby kinda person- cuz its a business kinda thinkpad---

so thats why i made this, to kinda put those.... feelings into words. i kinda feel like a macbook user in a starbucks with a scarf and piercings, wearing some weird fancy hat while havin som weirdly styled book on the side and a pencil in my hand, even tho i got no paper to write onto- but i feel about two times cooler, cuz its not a macbook >v<

EDIT2: i upgraded the resolution from 800 pixlz to 1300 px. so now u can read most stuff bettr - while still staying under 0.2MB

Device: Thinkpad X1 Yoga Gen 8

i noticed that my thinkpad runs at a noticably lower but largely stable framerate aber waking up from sleep (after opening the lid and logging in).

Playing some audio also reveald crackeling noises about twice a second, which werent there before.

cpu usage remains low at 3%

actual cpu performance seems to be as good as before sleep.

ive had some issues with sleep on my desktop before (same config) but they were never so severe...

does someone here have similar issues or found a solution for this yet?

Device: Thinkpad X1 Yoga Gen 8

i noticed that my thinkpad runs at a noticably lower but largely stable framerate aber waking up from sleep (after opening the lid and logging in).

Playing some audio also reveald crackeling noises about twice a second, which werent there before.

cpu usage remains low at 3%

actual cpu performance seems to be as good as before sleep.

ive had some issues with sleep on my desktop before (same config) but they were never so severe...

does someone here have similar issues or found a solution for this yet?

soooo - i got a... thinkpad x1 ... yoga gen 8 - and its great!

works flawlessly, i love the trackpoint - great navigation.

however! i have found that this middle trackpad button isnt really a middle mouse button at all! its great for scrolling with the trackpoint and it works great for a middle-mouse-button click when just clicked once, but when held, the laptop seems to expect a scroll from the trackpoint.

i need the middle-mouse-button-dragging for navigation within the godot game engine and blender.

sooo - i tried finding a way to emulate middle click using both left - and right trackpad buttons, but to no avail.

the closest i have gotten is by cloning and running this repo here, which made it so that, when left and right buttons are pressed, they emulate the physical middle trackpad button pres... which lets me scroll using the trackpoint, but doesnt actually emulate the real middle mouse button press (so still no middle-mouse-buton drag ;( )

(yes, i tried the gsettings command they provide in the - that didnt work either, seemingly nothing changed)

does someone have a solution for this?

apparently, this is part of the emoji set now. its,.. super cutie,,

mygosh i..,,.. i really find this moji very cutie..,.. luv it,.. a lot.,. i..., yea umm.... i luv it, a lot.,..

its juuuuust meant to be a funi, oki? dont get too worked up about it....

for anyone who doesnt get the funi here, look at the alt text, i kinda explained it there.

alternative image link (blahaj zone)

i have some annoying neighbours and kinda thought bout how in anime, the characters yell haaiii and hooooiiii a lot when home - so heres my funi on that. (no, my neighbours dont do that >v< but instead they look into our windows and r uncomfy)

so like - how do u do today? i highly recommend drinking some water - its pretty refreshing. huge fan.

something something creative expression and self realization - who cares---

also - mygosh umm... ive been doing some stuff in blender and... see - its about time i actually make a thing which kinda.. goes beyond what i usually do, u kno? not anything comercially, but something which is out there - which exists and such...
something i can point to and say

yea, i made that - but who cares

u kno?-- just some stuff i ... make and then it.. exists. and its not just some temporary funi which gets looked at by some hundreds of peeps - but instead more like.. a genuine experience which maybe about 5 people will have (including myself...) and then these people say "yea - that was something" and then move on with the knowledge that this experience exists at all.

and - BAM! i have left something - in the world - like - a REAL LIFE thing which peeps can look at, interact with, plan within, explore and find some kind of story and meaning for themselves in it... yea - thats what i wana make.

ive not been posting much cuz i have a lot of.,. difficult stuffs goin on rn...

for anyone who cares

• i am not studying bioinformatics
• i am STILL too scared to get my blood drawn for those hormones - so im still... without... ;( and feel worse everyday.,.,
• thats kinda it honestly, yea very smol things if im honest - but it feels like a lot when i also wana make cool experiences and be somewhat interesting...

something i thought a lot about the recent weeks is the idea of "artists".

here my yapping about artists (which is bad btw)

for some reason, i had this idea in my head that artists are pretentious evils who go "oh well - you have a job which RUINS the environment, makes capitalists RICHER and you get mediocre moni- meanwhile i - they Artist (yesyes, capital A that is-) am earning money from the fact that my art is SOOOO GOOOD that people pay me REAL money for it - and im not ruining anything for anyone! im doing a purely positive thing for humanity -" and then they look down upon kinda everyone else while saying that.

this is obviously not true. like - totally false. but somehow i have a hard time getting rid of this mindset. its some brainworm presumably. its probably just envy - cuz they get to make a living off of something fun, creative and design-oriented, meanwhile everyone else works their ass off on some hard physical or mental labour which - in most cases - is not interpreted as being fun...

iguess its kinda like the imposter syndrom, but im not an imposter, i just imagine that if i were - id feel shiddy, cuz id feel like an evil pretentious peep... but now im really just askin for affirmations and whatever -. which is kindsa not what im here for... i got a lot of stuff to tell to my thearpist tomorrow >v<

have a nice day u peeps! <3

im leaving for now - so imma post-

these schnitzels r nothin special-

alt image on blahaj zone

---

high quality original schnitzel pic

---

EDIT:

INFO: image above "slop" text is generated by diffusion model (i found it on duckduckgo, did not generate it myself)

alternative image link on blahaj zone

i looked at dis "stop doin math" funi, n thought "eh - why not" n now i made dis.

generally tried to keep it as close to the original as possibl, while also makin it maria-themed - so here we go. i think it turned out preddi well.

in case dis post blows up n reaches the lemmy world peeps who dun kno me, here a little note to all u who r about to write a comment bout my writin style:

• i dun hav a stroke, n so do u.

n for anyone curious, here the matrix multiplication thingy i mentioned in smol text. tldr: google put their gemini-2.0 into a loop of tryin stff out, n at som point it came up with the currently fastest matrix multiplication method to date.

EDIT:

for anyone even more curious (only read this if u care bout my stance)

yes, dis post is kindsa meant to be sepf-deprecating humor.

u might wondr buut mariaaaaa - why??? whads self-deprecating bout postin out the obvious downsides of current ml systems?

well let me tell u a story ~ ~ ~ ~ ~ 🌳🌸💖🍩🍙🥧

(which i wrote btw, no language model was used)

once upon a time, there was a maria 🌸 in a magical world of gud 🟢 and evil 🟣.

maria 🌸 hated how the evil magic 🟣 was destroying the environment: evil powerful magic-wielders 😈 would create huge evil-magic-producers 🟪 with the hope of replacing the human!

she also hated how suddenly, these dark-magic creatures 💜 wud write boring useless text and create uncanny scary imagery. typical of the evil-magics 😈, she 🌸 thought.

the black-magic-advancements 🟣 at the time scared her so much, she believed all of human work could be replaced with this new dark magic 🟣! she imagined a future where no peep worked, with many takin their lifes becuz they felt it was meaningless without goals to achieve.

---

despite all dis, maria 🌸 imagined they cud be used for gud, so she 🌸 lead one of the little creatures 💜 home. not to profit from, but to see what it 💜 cud do ~

at home, she 🌸 put the dark creature 💜 into a bunni-cage 🐇, wrote a page of instructions and pushed it thru the cage to the creature 💜

it 💜 licked the page, and started pretending to do the task, by writing down the actions it 💜 would take... but not actually doing them...

turns out, she 🌸 had to let the written text be turned into actual actions using another well-known ancient spell: text-parsing 🟢 :o

after bits of tinkerin, maria 🌸 ... didn hav anythin useful, but she had fun trynna make the lil one 💜 do increasingly complex tasks. at som point, she felt accelerated by her own doings, n put it 💜 into a larger, bear-sized cage, n gave it 💜 much longer, mor complex instructions

she 🌸 wud change its 💜 personality, look at its 💜 own writing, refine it, revise it, test its 💜 ideas, see its 💜 own failure n keep on tryin.

...

unknowingly, she 🌸 had becom a black magic wielder 🟣. she 🌸 had becom, whad she 🌸 hated so much 🟣 -

though maria 🌸 didn giv brith to the monster 💜 (which wud increase environmental problms), she only fed it 💜 vrri little magical energy (much less than others wud use to play magic games) - n didn share the creatures 💜 scribbles with others, cuz she didn wana be like those other peeps 🟣 who pretend the output is worth sharin.

she 🌸 imagined her frens frowning upon her, once findin out bout her 🌸 "hobby" 🟣.

...

one question bugged her tho:

aftr all dis time, am i still maria 🌸? or hav i becom... maria 😈?

she couldnt answer dis question.

---

the end

oh yea - u prolli need som context:

i am developin an lm-based helper thingy for godot game engine for about a year by now. dis is why the post is kindsa self-deprecating, its cuz im litrlli buildin a thing which "does tasks". vrri funi.

anway, plz dun call be an ai-sis, i did make dis funi pictur aftr all, so i must be sane, right?

(plz be cool bout dis >o< )

---

^ ^ ^ dis includes a little story too btw - its got mojis! ^ ^ ^