Mozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizations (www.phoronix.com)

submitted 6 months ago by ylai@lemmy.ml to c/localllama@sh.itjust.works

3 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[-] xcjs@programming.dev 1 points 4 months ago* (last edited 4 months ago)

I just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/

permalink
fedilink
source
parent

this post was submitted on 10 May 2024

30 points (96.9% liked)

LocalLLaMA

2249 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup@sh.itjust.works

pax@sh.itjust.works

noneabove1182@sh.itjust.works