3 jobs · 12 apps
Best local LLM tools to self-host
To run an LLM locally, the core of a private AI stack is Ollama for running models, Open-WebUI (source-available) for chatting with them and RAGFlow for questions over your own files, all running on your own servers so your prompts and documents can stay on your infrastructure. Each pick was chosen by hand for its job and is shown with GitHub data from October 2026.
A local LLM setup has two parts: a model runner that downloads open-weight models and serves them on your own CPU or GPU, and a chat interface on top. Both run on your machine or server, so prompts and documents do not go to a cloud AI provider unless you connect one.
Updated · data from GitHub, refreshed nightly
Top picks
- Rank 1:
Ollama
ollama/ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.
AI & LLM ToolsGoDockerAI-nativeOllama has 182,289 GitHub stars, uses the MIT license, ships a Docker image and had 1,083 commits in the last year; it replaces ChatGPT.
- Rank 2:
LocalAI
mudler/LocalAI
Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).
AI & LLM ToolsGoDockerAI-nativeLocalAI has 49,405 GitHub stars, uses the MIT license, ships a Docker image and had 3,438 commits in the last year; it replaces ChatGPT.
- Rank 3:
Text Generation WebUI
oobabooga/textgen
Gradio web UI for running large language models locally.
AI & LLM ToolsPythonDockerAI-nativeText Generation WebUI has 47,726 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 744 commits in the last year.
1 of 3 · 5 picks
Model runners
Download open-weight models and serve them locally, usually through an OpenAI-compatible API.
- 1
Ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.
AI & LLM ToolsDockerAI-nativeOllama has 182,289 GitHub stars, uses the MIT license, ships a Docker image and had 1,083 commits in the last year; it replaces ChatGPT.
- 2
LocalAI
Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).
AI & LLM ToolsDockerAI-nativeLocalAI has 49,405 GitHub stars, uses the MIT license, ships a Docker image and had 3,438 commits in the last year; it replaces ChatGPT.
- 3
Text Generation WebUI
Gradio web UI for running large language models locally.
AI & LLM ToolsDockerAI-nativeText Generation WebUI has 47,726 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 744 commits in the last year.
- 4
LLM Harbor
Containerized LLM toolkit. Run LLM backends, APIs, frontends, and additional services via a concise CLI.
AI & LLM ToolsDockerAI-nativeLLM Harbor has 3,237 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,288 commits in the last year.
- 5
LLMKube
Kubernetes operator for self-hosted LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and…
AI & LLM ToolsDockerAI-nativeLLMKube has 231 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,039 commits in the last year.
2 of 3 · 5 picks
Chat interfaces
Web chat front ends that connect to a local runner (and, if you choose, to cloud APIs).
- 1
Open-WebUI
User-friendly AI Interface, supports Ollama, OpenAI API.
Source-availableAI & LLM ToolsDockerAI-nativeOpen-WebUI has 154,037 GitHub stars, is source-available under the Open WebUI License (BSD-3-Clause plus branding clause) (not an open-source license), ships a Docker image and had 4,746 commits in the last year; it replaces ChatGPT and Lovable.
- 2
LibreChat
Enhanced ChatGPT-compatible AI chat interface supporting multiple AI providers, with multi-user auth, message search, and plugin support.
AI & LLM ToolsDockerAI-nativeLibreChat has 45,318 GitHub stars, uses the MIT license, ships a Docker image and had 2,074 commits in the last year; it replaces ChatGPT and Lovable.
- 3
LobeHub
Modern design AI chat framework supporting multiple AI providers, one click install MCP Marketplace and Artifacts / Thinking.
Source-availableAI & LLM ToolsDockerAI-nativeLobeHub has 83,008 GitHub stars, is source-available under the LobeHub Community License (not an open-source license), ships a Docker image and had 7,488 commits in the last year; it replaces ChatGPT and Lovable.
- 4
AnythingLLM
All-in-one desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.
AI & LLM ToolsDockerAI-nativeAnythingLLM has 66,738 GitHub stars, uses the MIT license, ships a Docker image and had 887 commits in the last year; it replaces ChatGPT.
- 5
Onyx Community Edition
Chat UI that works with any LLM. It comes loaded with advanced features like agents, web search, RAG, MCP, deep research, Connectors to 40+ knowledge sources, and more.
AI & LLM ToolsDockerAI-nativeOnyx Community Edition has 32,332 GitHub stars, uses the MIT license, ships a Docker image and had 5,771 commits in the last year.
3 of 3 · 2 picks
Use them with your documents
- 1
RAGFlow
Open-source RAG engine with deep document understanding.
AI & LLM ToolsDockerAI-nativeRAGFlow has 91,708 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 5,821 commits in the last year.
- 2
Khoj
Your AI second brain. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI.
AI & LLM ToolsDockerAI-nativeKhoj has 37,565 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 120 commits in the last year; it replaces ChatGPT.
What are the best tools to run an LLM locally?
To run an LLM locally, the core of a private AI stack is Ollama for running models, Open-WebUI (source-available) for chatting with them and RAGFlow for questions over your own files, all running on your own servers so your prompts and documents can stay on your infrastructure. Each pick was chosen by hand for its job and is shown with GitHub data from October 2026.
What hardware do I need to run AI models myself?
Small open models run on a modern CPU with 16 GB of RAM, but a GPU with enough video memory for the model you pick makes answers much faster. Start with a small model in Ollama, measure the speed on your own hardware, then size up.
What is the easiest way to run an LLM locally?
Install Ollama, pull a small model and talk to it from the command line, then add a chat interface such as Open WebUI or LibreChat that connects to Ollama's local API. All of them ship Docker images.
Is Ollama free?
Yes. Ollama is open-source software released under the MIT license, so you can self-host it at no cost. You only pay for the server it runs on, and some projects also offer an optional paid cloud plan.
Can I run this stack with Docker?
All 12 apps here publish Docker images, including Ollama, LocalAI and Text Generation WebUI, so most of the stack can run side by side with Docker Compose on one server. Size the server for the heaviest app and add off-site backups.
The weekly digest
New self-hosted apps, every week
Fresh releases, rising projects and one deploy guide a week. Join home labbers and engineers who self-host.