this post was submitted on 24 Aug 2023
14 points (100.0% liked)

LocalLLaMA

3458 readers
42 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 2 years ago
MODERATORS
SkySyrup@sh.itjust.works
pax@sh.itjust.works
noneabove1182@sh.itjust.works
Smokeydope@lemmy.world
MonsterBug@sh.itjust.works
14
Code Llama: Open Foundation Models for Code | Meta AI Research (ai.meta.com)
submitted 2 years ago by noneabove1182@sh.itjust.works to c/localllama@sh.itjust.works
2 comments fedilink hide all child comments
 

We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B and 34B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B and 13B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 53% and 55% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.

top 2 comments
sorted by: hot top controversial new old
[โ€“] PixelProf@lemmy.ca 4 points 2 years ago (1 children)

This should be interesting to play with. Does anyone know of any Copilot-like VS Code extensions that provide similar UX but hook into a custom local/remote server? I would love to write my own pipeline and context builder for it, but I haven't written a VS Code extension before and a starting point would be good.

I haven't worked with any of the current code-based open models so I don't really know how it compares, excited to hear thoughts from others on this.