LM Studio 0.3.0
LM Studio version 0.3.0 enhances its desktop application with document chat, Retrieval Augmented Generation, a Structured Output API, multiple UI themes, improved regeneration, and simplified migration of previous chats.
Read original article
LM Studio has released version 0.3.0, enhancing its desktop application designed for running local large language models (LLMs) offline. This update introduces several new features, including the ability to chat with documents, where users can input documents for the LLM to reference. For lengthy documents, the application employs Retrieval Augmented Generation (RAG) to extract relevant sections. The update also supports an OpenAI-like Structured Output API, allowing for reliable JSON outputs. Users can now choose from multiple UI themes, including Dark, Light, and Sepia, and the application automatically configures load parameters based on the user's hardware. A new "Serve on Network" feature enables access to the LM Studio server from other devices, and users can organize chats into folders. The regeneration feature has been improved to allow multiple generations for each chat. Additionally, the update includes a refreshed UI, enhanced model loading capabilities, support for embedding models, and initial translations for several languages. Users can migrate their previous chats to the new version easily. Overall, LM Studio 0.3.0 aims to provide a more user-friendly and versatile experience for managing local LLMs.
- LM Studio 0.3.0 introduces document chat functionality and Retrieval Augmented Generation (RAG).
- The update supports an OpenAI-like Structured Output API for reliable JSON outputs.
- Users can now choose from multiple UI themes and organize chats into folders.
- The application automatically configures load parameters based on hardware capabilities.
- Migration from previous versions is simplified, allowing users to retain their chat history.
Related
LLMs on the Command Line
Simon Willison presented a Python command-line utility for accessing Large Language Models (LLMs) efficiently, supporting OpenAI models and plugins for various providers. The tool enables running prompts, managing conversations, accessing specific models like Claude 3, and logging interactions to a SQLite database. Willison highlighted using LLM for tasks like summarizing discussions and emphasized the importance of embeddings for semantic search, showcasing LLM's support for content similarity queries and extensibility through plugins and OpenAI API compatibility.
LLMs can solve hard problems
LLMs, like Claude 3.5 'Sonnet', excel in tasks such as generating podcast transcripts, identifying speakers, and creating episode synopses efficiently. Their successful application demonstrates practicality and versatility in problem-solving.
Llama 3.1 Official Launch
Llama introduces Llama 3.1, an open-source AI model available in 8B, 70B, and 405B versions. The 405B model is highlighted for its versatility in supporting various use cases, including multi-lingual agents and analyzing large documents. Users can leverage coding assistants, real-time or batch inference, and fine-tuning capabilities. Llama emphasizes open-source AI and offers subscribers updates via a newsletter.
Llama 3.1: Our most capable models to date
Meta has launched Llama 3.1 405B, an advanced open-source AI model supporting diverse languages and extended context length. It introduces new features like Llama Guard 3 and aims to enhance AI applications with improved models and partnerships.
An Open Course on LLMs, Led by Practitioners
A new free course, "Mastering LLMs," offers over 40 hours of content on large language models, featuring workshops by 25 experts, aimed at enhancing AI product development for technical individuals.
- Users appreciate the enhancements in version 0.3.0, particularly the new features like document chat and Retrieval Augmented Generation.
- Some users express frustration over the lack of open-source availability and the limitations of the licensing model.
- Comparisons are made between LM Studio and other tools like Ollama, with some users preferring the latter for its integration capabilities.
- Several comments highlight the ease of use and accessibility of LM Studio for local AI experimentation.
- Users seek more information on system requirements, changelogs, and performance benchmarks across different setups.
22 commentsWhat LM Studio is today is a an IDE / explorer for local LLMs, with a focus on format universality (e.g. GGUF) and data portability (you can go to file explorer and edit everything). The main aim is to give you an accessible way to work with LLMs and make them useful for your purposes.
Folks point out that the product is not open source. However I think we facilitate distribution and usage of openly available AI and empower many people to partake in it, while protecting (in my mind) the business viability of the company. LM Studio is free for personal experimentation and we ask businesses to get in touch to buy a business license.
At the end of the day LM Studio is intended to be an easy yet powerful tool for doing things with AI without giving up personal sovereignty over your data. Our computers are super capable machines, and everything that can happen locally w/o the internet, should. The app has no telemetry whatsoever (you’re welcome to monitor network connections yourself) and it can operate offline after you download or sideload some models.
0.3.0 is a huge release for us. We added (naïve) RAG, internationalization, UI themes, and set up foundations for major releases to come. Everything underneath the UI layer is now built using our SDK which is open source (Apache 2.0): https://github.com/lmstudio-ai/lmstudio.js. Check out specifics under packages/.
Cheers!
-Yagil
Has anyone found the same thing, or was that a fluke and I should try LM Studio again?
Similarly with images, LLMs and ML in general feel like DOS and config.sys and autoexec.bat and qemm days.
How is it possible that there's still no way to search through your conversations?
Does anyone have a recommendation?
For context: I have almost ten years experience with deep learning, but I want something easy to set up in my home M2 Mac, or Google Colab would be OK.
It allows both local and cloud models.
* Not associated with them in any way. Am a happy user.
Related
LLMs on the Command Line
Simon Willison presented a Python command-line utility for accessing Large Language Models (LLMs) efficiently, supporting OpenAI models and plugins for various providers. The tool enables running prompts, managing conversations, accessing specific models like Claude 3, and logging interactions to a SQLite database. Willison highlighted using LLM for tasks like summarizing discussions and emphasized the importance of embeddings for semantic search, showcasing LLM's support for content similarity queries and extensibility through plugins and OpenAI API compatibility.
LLMs can solve hard problems
LLMs, like Claude 3.5 'Sonnet', excel in tasks such as generating podcast transcripts, identifying speakers, and creating episode synopses efficiently. Their successful application demonstrates practicality and versatility in problem-solving.
Llama 3.1 Official Launch
Llama introduces Llama 3.1, an open-source AI model available in 8B, 70B, and 405B versions. The 405B model is highlighted for its versatility in supporting various use cases, including multi-lingual agents and analyzing large documents. Users can leverage coding assistants, real-time or batch inference, and fine-tuning capabilities. Llama emphasizes open-source AI and offers subscribers updates via a newsletter.
Llama 3.1: Our most capable models to date
Meta has launched Llama 3.1 405B, an advanced open-source AI model supporting diverse languages and extended context length. It introduces new features like Llama Guard 3 and aims to enhance AI applications with improved models and partnerships.
An Open Course on LLMs, Led by Practitioners
A new free course, "Mastering LLMs," offers over 40 hours of content on large language models, featuring workshops by 25 experts, aimed at enhancing AI product development for technical individuals.