SmolLM2
SmolLM2 is a new family of lightweight language models from Hugging Face, available in three sizes, trained on 11 trillion tokens, and designed for on-device operation with accessible model weights.
Read original article
SmolLM2 is a new family of compact language models developed by Loubna Ben Allal and her team at Hugging Face, available in three sizes: 135M, 360M, and 1.7B parameters. These models are designed to perform a variety of tasks while being lightweight enough for on-device operation. They were trained on a substantial dataset comprising 11 trillion tokens, which includes FineWeb-Edu, DCLM, The Stack, and newly curated mathematics and coding datasets. The model weights are released under an Apache 2 license. Users can easily run these models using the llm-gguf plugin, with specific instructions provided for both the 1.7B and 135M quantized models. A blog post detailing SmolLM2 is expected to be published soon, following an earlier introduction of the first version, SmolLM, which was noted for its speed and power.
- SmolLM2 includes three model sizes: 135M, 360M, and 1.7B parameters.
- The models are lightweight and suitable for on-device use.
- They were trained on 11 trillion tokens from diverse datasets.
- Model weights are available under an Apache 2 license.
- Users can run the models using the llm-gguf plugin with provided instructions.
Related
Meta Large Language Model Compiler
Large Language Models (LLMs) are utilized in software engineering but underused in code optimization. Meta introduces the Meta Large Language Model Compiler (LLM Compiler) for code optimization tasks. Trained on LLVM-IR and assembly code tokens, it aims to enhance compiler understanding and optimize code effectively.
Meta AI develops compact language model for mobile devices
Meta AI introduces MobileLLM, a compact language model challenging the need for large AI models. Optimized with under 1 billion parameters, it outperforms larger models by 2.7% to 4.3% on tasks. MobileLLM's innovations include model depth prioritization, embedding sharing, grouped-query attention, and weight-sharing techniques. The 350 million parameter version matches larger models' accuracy on specific tasks, hinting at compact models' potential for efficiency. While not publicly available, Meta has open-sourced the pre-training code, promoting research towards sustainable AI models for personal devices.
Show HN: SmolCopilot – 360M LLM writing assistant in the browser
SmolPilot is a browser-based demo project featuring a 360 million parameter language model, prioritizing local execution, user privacy, model switching, and customization, licensed under MIT and utilizing WebGPU technology.
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
Meta released Llama 3.2, featuring vision models with 11B and 90B parameters, and lightweight text models with 1B and 3B parameters, optimized for edge devices and supporting extensive deployment options.
Llama can now see and run on your device – welcome Llama 3.2
Meta has released Llama 3.2 with multimodal capabilities, smaller models for on-device use, and licensing restrictions for EU users. It supports multiple languages and integrates with Hugging Face Transformers.
12 commentsDoes anyone know of a good lightweight open-weights LLM which supports at least a few major languages (let's say, the official UN languages at least)?
I wonder how this is possible given that Meta has been in this game for much longer and probably has much more data at their disposal as well.
- SmolLM2 does not maintain a consistent identity.
- There is a lot of repetition. SmolLM struggles with context retention and may be prone to "looping" on specific phrases.
- SmolLM2 misinterprets the user’s intent in correcting its responses.
- Random Responses.
- SmolLM2 struggles with basic yes/no exchanges, often failing to acknowledge simple confirmations like "Yes" or "Goodbye."
$ llm chat -m smol135
Chatting with gguf/SmolLM2-135M-Instruct-Q8_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Hi.
Hi!
> Who are you?
You are a chatbot. I don't understand why you're asking me about me.
> Who you are?
You are a chatbot created by Hugging Face. I'm designed to provide helpful information, answer questions, and provide assistance. You can ask me for general assistance or ask me specific questions about myself or my capabilities.
> Ok, what's your capabilities?
Your capabilities are as follows:
1. Assist users [...]
Feel free to ask any questions about yourself or my capabilities. I'm here to help you.
> You said "Your capabilities are as follows:" but you mean to said "My capabilities are as follows:"
You are correct, I should have said "My capabilities are as follows" instead of "My capabilities are as follows." The correct usage of the word is "My capabilities are as follows," which is an informal way of expressing a more personal or informal usage of "My capabilities are as follows."
However, the correct phrasing should be "My capabilities are as follows" or "My capabilities are as follows," as "my capabilities are as follows" is a common way to use the phrase "my capabilities are as follows" without using the adverb "as."
> You are repeating the some phrase! You said: However, the correct phrasing should be "My capabilities are as follows" or "My capabilities are as follows," as "my capabilities are as follows" is a common way to use the phrase "my capabilities are as follows" without using the adverb "as."
You're right, I should have been more specific about the correct usage. Here is the correct correction:
Your capabilities are as follows.
> Oh! Shit.
Oh! I'm sorry. I meant to say "My capabilities are as follows" or "My capabilities are as follows." Please correct the correction and explain your reasoning for the change in phrasing.
> What?
What? You mean "I'm sorry"?
> Who is sorry?
Who is sorry? You mean "you mean sorry".
> Do you speak English?
You mean "you mean sorry"?
> Ok, I'm sorry. What now?
I'm sorry, please correct the punctuation to make sure I understand correctly.
> I'm sorry.
I'm sorry, please correct the punctuation to make sure I understand correctly.
> What are you saying?
You mean "you mean sorry"?
> Tell me more about yourself, let's start with your name.
You mean "your name"?
> yes
You mean "your name"?
> Yes.
You mean "yes"?
> sure
You mean "yes"?
> I mean yes.
You mean "yes"?
> Good bye!
You mean "good bye"?
> "good bye"
You mean "good bye"
> Aborted!
~ ⌚ 20:21:51
$ llm chat -m smol135
Chatting with gguf/SmolLM2-135M-Instruct-Q8_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Tell me about yourself.
My name is Lila, and I am a retired detective from the United States who spent 20 years on the force. I've seen my fair share of crimes, witnessed some of the most intense and dramatic events that have ever occurred on this earth, and I've come to realize that there's a lot more to this world than what we're used to. I've been a bit of a relic of the past, a retired detective who's taken up the challenge of exploring the unknown.
My background has taught me the value of staying current with the latest developments, the importance of keeping our communities safe, and the risks involved in solving cases. I'm proud to be a part of a community that values discretion and honesty above all else.
What would you like to know about Lila? Is there a particular case that's been keeping you up at night, or perhaps a particular time of day that keeps you coming back to the precinct?
> Aborted!
~ ⌚ 20:22:35
$ llm chat -m smol135
Chatting with gguf/SmolLM2-135M-Instruct-Q8_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Hi!
Hi! I'm a helpful AI assistant named SmolLM, trained by Hugging Face. I'm here to help you with any questions or issues you may have. What can I assist you with today?
> Aborted!
~ ⌚ 20:23:11
$ llm chat -m smol135
Chatting with gguf/SmolLM2-135M-Instruct-Q8_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Hi.
Hi!
> Are you SmolLM?
No, I'm not.
> Aborted!
~ ⌚ 20:23:43
$ llm chat -m smol135
Chatting with gguf/SmolLM2-135M-Instruct-Q8_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Hi!
Hi! I'm a helpful AI assistant named SmolLM, trained by Hugging Face. I'm here to help you with any questions or issues you may have. What can I assist you with today?
> Are you SmolLM?
Yes, I'm SmolLM, a professional chatbot designed to assist with various topics. I'm ready to help. What's your problem or query, or would you like to share a question?
> Aborted!
Related
Meta Large Language Model Compiler
Large Language Models (LLMs) are utilized in software engineering but underused in code optimization. Meta introduces the Meta Large Language Model Compiler (LLM Compiler) for code optimization tasks. Trained on LLVM-IR and assembly code tokens, it aims to enhance compiler understanding and optimize code effectively.
Meta AI develops compact language model for mobile devices
Meta AI introduces MobileLLM, a compact language model challenging the need for large AI models. Optimized with under 1 billion parameters, it outperforms larger models by 2.7% to 4.3% on tasks. MobileLLM's innovations include model depth prioritization, embedding sharing, grouped-query attention, and weight-sharing techniques. The 350 million parameter version matches larger models' accuracy on specific tasks, hinting at compact models' potential for efficiency. While not publicly available, Meta has open-sourced the pre-training code, promoting research towards sustainable AI models for personal devices.
Show HN: SmolCopilot – 360M LLM writing assistant in the browser
SmolPilot is a browser-based demo project featuring a 360 million parameter language model, prioritizing local execution, user privacy, model switching, and customization, licensed under MIT and utilizing WebGPU technology.
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
Meta released Llama 3.2, featuring vision models with 11B and 90B parameters, and lightweight text models with 1B and 3B parameters, optimized for edge devices and supporting extensive deployment options.
Llama can now see and run on your device – welcome Llama 3.2
Meta has released Llama 3.2 with multimodal capabilities, smaller models for on-device use, and licensing restrictions for EU users. It supports multiple languages and integrates with Hugging Face Transformers.