I have been testing LM Studio, an application for Windows, Linux and Mac that allows you to use artificial intelligence models on your PC without sharing the data with third parties.
|No te pierdas nada y ¡Síguenos en Google News!
In Spain, we are still celebrating the Constitution Day holiday, and this short break has given me enough time to try out some new technological toys. I love researching and trying new things, and the lack of time and work are two limitations that affect my ability to play around with these “inventions.”
Thanks to a post from XDA Developers, I discovered LM Studio, an application for running Language Model-based Artificial Intelligence (LLM) on a home PC or laptop. There is also a version for Mac, and you can request another for Linux on the company’s official Discord. In addition to the flexibility and cost savings offered by this application, the main advantage is that certain language models can read links if you have an internet connection.
Before getting started, minimum requirements and basic tips
This is a completely free application that allows you to choose from a vast catalog of advanced language models. The idea is quite basic, and it’s surprising that something like this hasn’t been done before, or at least, I haven’t tried anything with such good results and functionality. So let me briefly share my experience.
Before you start using it, keep in mind that you’ll need at least a good processor and, especially, a lot of RAM. Because these natural language models consume a considerable amount of memory with each query, not just any computer will do. If you have a good graphics card, it will also help reduce the load. In no case expect the processing speed of ChatGPT. In my case, it was straightforward because I have a well-equipped laptop with 32GB, an Nvidia 3070Ti, and a Ryzen 5000 Series. But if you want to know the minimum requirements to get it up and running, they are as follows:
- Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or higher.
- For Windows or Linux, a processor that supports AVX2.
- More than 16GB of RAM is ideal.
- More than 6GB of VRAM.
- NVIDIA or AMD GPU.
My first steps with LM Studio
Using this tool is simple; you just need to download the LLM you want to use and start experimenting. You have a vast range available, and the tool itself tells you the RAM requirements your computer needs to handle a specific language. The interface is very straightforward. On one side, you have the cover, which gives you a list of the latest added LLMs, a breakdown of what each one does, and the required RAM to run it. If you don’t want to get lost among so many options, you can use the search bar to find one to play with.
I recommend trying any variant of Zephir, which only requires 8GB of RAM and understands Spanish well. Although you have many more languages to experiment with. Once you’ve downloaded one or more, you just need to go to the message icon, load the language you want to use, and start typing the prompt you want.
Options and presets
One of the things I liked the most is that LM Studio allows you to generate a predefined prompt where you give it instructions or tell it to behave in a specific role, for example, a technology expert. Among the available options, here are some of the most notable ones. Keep in mind that these settings are applied in the chat section:
- Choose the role you want for the prompt.
- Preconfigure an initial instruction, for example, “Always respond in Spanish,” and it will be injected into the prompt.
- You can choose the number of words you want it to output, limit the tokens, and more.
- Save multiple presets and load or export them in JSON format.
- Choose whether you want it to respond in plain text or markdown; the latter is ideal if you work with code.
- Add notes in a chat.
Last but not least, you can create a “Local Inference Server” that mimics the behavior of the OpenAI Chat Completion API but runs locally on your machine. This is a server running on your machine and performs inference operations locally. Inference in this context refers to the process of predicting or completing results based on a trained model, in this case, the OpenAI Chat Completion model.
My final experience with LM Studio
Currently, as I write these lines, I am using version beta v0.2.8 of the application, and there are some languages that solve many tasks for me. If you have a capable computer, it is perhaps the best option for working with code, texts, and other more specific tasks without exposing your data to a third party like OpenAI.
Keep in mind that, for now, you cannot train models; all the ones you download will come pre-trained. In any case, you have LLMs of all kinds, for example, xDAN, Microsoft Phi2-2, or AdaptLLM variants for medicine, finance, or biomedicine.
In general, it seems to me something very necessary for both professionals and individuals who do not want to automate applications, simply using an interface and giving orders to a specialized artificial intelligence system.
Note: This content has been translated with an artificial intelligence tool, so the translation may be slightly inaccurate. The original version written by our editor is the the Spanish version