Hands-on with Unsloth Desktop, for running and training LLMs locally

reviews Aug 31, 20267 mins Like LM studio, Unsloth Desktop lets you run generative AI models on local hardware. But you can also create multi-step workflows and train new models using a convenient web-based UI. LM Studio makes it easy to run LLMs on one’s own hardware, whether as a desktop app or a server for others in one’s organization. But LM Studio isn’t the only project of its kind. Unsloth, a team of two brothers (Daniel and Michael Han), have created their own local-first application for running and training models on local hardware, called Unsloth Desktop. Unsloth Desktop, an open-source Apache-licensed application, intends to do more than just host models locally. It can also be used as a model-training workbench, can be used for image generation as well as standard chat, and runs cross-platform (Microsoft Windows, macOS, and Linux). Unlike LM Studio, Unsloth Desktop does not run as a windowed desktop application. When launched, it opens a console (handy for debugging, to be sure), then kicks open a web browser tab to provide a GUI. Although internal updates (for instance, for llama.cpp or other support libraries) are handled automatically from within the program, full updates require running a command at the console. This makes working with Unsloth Desktop a little clunkier than the typical desktop application. Most systems for working locally with generative AI models have some kind of curated model gallery, and Unsloth Desktop is no exception. Click “Model Hub” in the left-hand menu and you’ll be taken to a catalog of models curated by Unsloth’s team. These can be filtered and organized by capabilities, formats, and how well the model fits on your device. You can also explore a similarly curated list of datasets which can be used for your own model training. Projects and workflows Unsloth Desktop lets you start conversations with a given model ad hoc, and attach files or other context. But for more comprehensive organization, you can create a “project”, which can include files or linked local folders for the LLM to work with. This can be done in lieu of, say, using one’s IDE as the context for a project. One feature Unsloth Desktop has in contrast to LM Studio is the “Data Recipes” feature, a way to create graphical workflows that do multi-step processing from different kinds of data sources. You can use the prebuilt recipes to see how they work and develop your own; the existing receipes cover tasks like OCR document extraction, or crawling GitHub issues/PRs and deriving training data from them. The workflow GUI is reminiscent of how ComfyUI lets you string together blocks to create workflows. An example of an Unsloth Studio data recipe workflow. The GUI used to string together blocks of actions is similar to ComfyUI’s interface. Foundry When you load up a model, Unsloth Desktop presents relatively few knobs to tweak compared to LM Studio. The token window, for instance, is exposed as a knob, but the number of model layers to force to the GPU is handled automatically by default. The default context length of 4096 is way too small for sustained conversation, so you’ll want to goose that by at least twice as much. Unsloth Desktop has the usual conversational chat for Q&A-style model prompting. It also includes specialized interfaces for still-image, video-from-prompt, and text-to-speech generation. These interfaces come pre-equipped with possible model choices. If you select a model that isn’t currently installed, Unsloth Desktop will download it for you, although without a progress meter. (More on this and other problems like it below.) Unsloth Desktop’s tool calling mechanisms have a couple of intriguing features. The “auto-healing tool call” feature is intended to ensure that inference output from tool calls is not broken, and the “nudge tool calls” feature re-runs a tool call if the auto-healing feature doesn’t work. Unfortunately, neither feature is documented or explained well; it isn’t clear how they accomplish what they’re intended to do. I did find that, with these features turned on, thinking and tool calling in Unsloth conversations, regardless of the model, behaved well and generally avoided problems that required restarting the prompt. The biggest issue I ran into involved a chat where I mistakenly set the number of tokens too low for an extended conversation. One truly outstanding feature of Unsloth Desktop is the tools it provides for training models. Instead of cobbling together multiple command-line tools, you can train new models right in Unsloth’s GUI, and then serve predictions with them as well. Training a model in Unsloth Desktop. A few example data sets are included to demonstrate the workflow. The resulting model can then be immediately used to serve preductions inside Unsloth. Foundry For creating image-generation models in particular, Unsloth offers some nice amenities. Training a new image model, for instance, requires a number of data-annotated images. You can assemble both images and annotations right in Unsloth’s GUI, then run the training from there with live statistics as the training proceeds, and then switch to the image-generation UI and create images with the trained model. This all-in-one approach is really appealing. A few minuses A big minus for Unsloth Desktop right now is the patchiness and inconsistency of its documentation. For instance, while there’s mention in the docs of doing your own image training, it isn’t actually documented in any detail. I had to do some experimentation to get it working right. I also ran into various irritating behaviors that required reverse engineering to work around. For instance, when I wanted to try training an image model based on the SDXL model family, the training system crashed with a cryptic error that I realized meant I had to launch Unsloth Desktop in the context of an MSVC developer prompt. (This isn’t documented anywhere either.) The UI itself was sometimes confusing. When selecting a model, the UI didn’t always make it clear to me if clicking the name of the model would begin loading that model, or if it would expand a list of available quantizations for that model. This led to me accidentally starting a model download more than once. Likewise, when you want to do LoRA training and a given base model isn’t available, Unsloth Desktop will download it, but not provide you with any visual feedback that this is even happening or how long it will take — you’ll just get a static “Loading model…” prompt. (Check your network bandwidth stats.) Right now, LM Studio still provides the cleanest, most convenient, and most coherent desktop UI for working with existing models. But Unsloth Desktop has a big attraction: giving you a similar kind of workspace for training new models or developing multi-step recipes. It will be even better once the gaps in its documentation are filled in, and its confusing UI choices are cleaned up. Serdar Yegulalp is a senior writer at InfoWorld. A veteran technology journalist, Serdar has been writing about computers, operating systems, databases, programming, and other information technology topics for 30 years. Before joining InfoWorld in 2013, Serdar wrote for Windows Magazine, InformationWeek, Byte, and a slew of other publications. At InfoWorld, Serdar has covered software development, devops, containerization, machine learning, and artificial intelligence, winning several B2B journalism awards including a 2024 Neal Award and a 2025 Azbee Award for best instructional content and best how-to article, respectively. He currently focuses on software development tools and technologies and major programming languages including Python, Rust, Go, Zig, and Wasm. Tune into his weekly Dev with Serdar videos for programming tips and techniques and close looks at programming libraries and tools. More from this author Show me more