Show HN: SmolCopilot – 360M LLM writing assistant in the browser
SmolPilot is a browser-based demo project featuring a 360 million parameter language model, prioritizing local execution, user privacy, model switching, and customization, licensed under MIT and utilizing WebGPU technology.
Read original article
SmolPilot is a demo project hosted on GitHub that features a small language model with 360 million parameters, designed to operate directly in web browsers. It is licensed under the MIT license and runs on port 3000. The model is powered by SmolLM-360M and utilizes WebLLM technology to leverage WebGPU for execution. Key features of SmolPilot include local execution, which reduces latency and costs by eliminating the need for external servers, and enhanced privacy, as user data remains within the browser. Additionally, the platform offers flexibility for users to switch between different models or providers and allows for customization to meet specific use cases. The repository also contains visual aids, including demo images and a GIF that illustrate the functionality of SmolPilot.
- SmolPilot is a browser-based demo of a small language model with 360M parameters.
- It runs locally, ensuring low latency and cost-effectiveness.
- User privacy is prioritized, with data remaining in the browser.
- The platform allows for model switching and customization.
- The project is licensed under MIT and utilizes WebGPU technology.
Related
Gemma 2 on AWS Lambda with Llamafile
Google released Gemma 2 9B, a compact language model rivaling GPT-3.5. Mozilla's llamafile simplifies deploying models like LLaVA 1.5 and Mistral 7B Instruct, enhancing accessibility to powerful AI models across various systems.
Meta AI develops compact language model for mobile devices
Meta AI introduces MobileLLM, a compact language model challenging the need for large AI models. Optimized with under 1 billion parameters, it outperforms larger models by 2.7% to 4.3% on tasks. MobileLLM's innovations include model depth prioritization, embedding sharing, grouped-query attention, and weight-sharing techniques. The 350 million parameter version matches larger models' accuracy on specific tasks, hinting at compact models' potential for efficiency. While not publicly available, Meta has open-sourced the pre-training code, promoting research towards sustainable AI models for personal devices.
Self hosting a Copilot replacement: my personal experience
The author shares their experience self-hosting a GitHub Copilot replacement using local Large Language Models (LLMs). Results varied, with none matching Copilot's speed and accuracy. Despite challenges, the author plans to continue using Copilot.
Show HN: We made glhf.chat – run almost any open-source LLM, including 405B
The platform allows running various large language models via Hugging Face repo links using vLLM and GPU scheduler. Offers free beta access with plans for competitive pricing post-beta using multi-tenant model running.
Show HN: Demo App for PLaMo-100B – A New Japan's 100B Parameter Language Model
PLaMo-100B, a large language model with 100 billion parameters, outperforms GPT-4 on Japanese benchmarks. Development spanned from February to August 2024, with a demo and trial API available online.
1 commentsRelated
Gemma 2 on AWS Lambda with Llamafile
Google released Gemma 2 9B, a compact language model rivaling GPT-3.5. Mozilla's llamafile simplifies deploying models like LLaVA 1.5 and Mistral 7B Instruct, enhancing accessibility to powerful AI models across various systems.
Meta AI develops compact language model for mobile devices
Meta AI introduces MobileLLM, a compact language model challenging the need for large AI models. Optimized with under 1 billion parameters, it outperforms larger models by 2.7% to 4.3% on tasks. MobileLLM's innovations include model depth prioritization, embedding sharing, grouped-query attention, and weight-sharing techniques. The 350 million parameter version matches larger models' accuracy on specific tasks, hinting at compact models' potential for efficiency. While not publicly available, Meta has open-sourced the pre-training code, promoting research towards sustainable AI models for personal devices.
Self hosting a Copilot replacement: my personal experience
The author shares their experience self-hosting a GitHub Copilot replacement using local Large Language Models (LLMs). Results varied, with none matching Copilot's speed and accuracy. Despite challenges, the author plans to continue using Copilot.
Show HN: We made glhf.chat – run almost any open-source LLM, including 405B
The platform allows running various large language models via Hugging Face repo links using vLLM and GPU scheduler. Offers free beta access with plans for competitive pricing post-beta using multi-tenant model running.
Show HN: Demo App for PLaMo-100B – A New Japan's 100B Parameter Language Model
PLaMo-100B, a large language model with 100 billion parameters, outperforms GPT-4 on Japanese benchmarks. Development spanned from February to August 2024, with a demo and trial API available online.