3 jobs · 12 apps
Best local LLM tools to self-host
To run an LLM locally, the core of a private AI stack is Ollama for running models, Open-WebUI (source-available) for chatting with them and RAGFlow for questions over your own files, all running on your own servers.
Pointed at a model on your own hardware, these tools keep your prompts and documents on your infrastructure.
A local LLM setup has two parts: a model runner that downloads open-weight models and serves them on your own CPU or GPU, and a chat interface on top. Both run on your machine or server, so prompts and documents do not go to a cloud AI provider unless you connect one.
Updated · data from GitHub, refreshed daily
Top picks
- Rank 1:
Ollama
ollama/ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.
Active · last commit Nov 18, 2026
AI & LLM ToolsGoDockerAI-nativeStands out for the most GitHub stars. Ollama has 187,500 GitHub stars, uses the MIT license, ships a Docker image and had 1,199 commits in the last year; it replaces ChatGPT.
- Rank 2:
LocalAI
mudler/LocalAI
Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).
Active · last commit Nov 18, 2026
AI & LLM ToolsGoDockerAI-nativeStands out for the most commits in the last 12 months (5,147). LocalAI has 50,430 GitHub stars, uses the MIT license, ships a Docker image and had 5,147 commits in the last year; it replaces ChatGPT.
- Rank 3:
Text Generation WebUI
oobabooga/textgen
Gradio web UI for running large language models locally.
Maintained · last commit Aug 17, 2026
AI & LLM ToolsPythonDockerAI-nativeStands out for AI features named in its own description. Text Generation WebUI has 47,739 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 688 commits in the last year.
At a glance
Local LLM tools to self-host compared
Picked by hand for each job from apps whose own description or topics match it, in curated order inside each section, with GitHub data from November 2026; an app appears only once.
| App | Stands out for | License | GitHub stars | Commits (12 mo) | Last commit | Docker image |
|---|---|---|---|---|---|---|
| Ollama | the most GitHub stars | MIT | 187,500 | 1,199 | Nov 18, 2026 | Yes |
| LocalAI | the most commits in the last 12 months (5,147) | MIT | 50,430 | 5,147 | Nov 18, 2026 | Yes |
| Text Generation WebUI | AI features named in its own description | AGPL-3.0 | 47,739 | 688 | Aug 17, 2026 | Yes |
| LLM Harbor | a permissive Apache-2.0 license | Apache-2.0 | 3,280 | 1,337 | Nov 11, 2026 | Yes |
| LLMKube | the most recent release (v0.10.18, November 2026) | Apache-2.0 | 267 | 1,480 | Nov 18, 2026 | Yes |
| Open-WebUI | – | source-available, Open WebUI License (BSD-3-Clause plus branding clause) | 159,000 | 6,788 | Nov 18, 2026 | Yes |
| LibreChat | – | MIT | 47,310 | 3,592 | Nov 17, 2026 | Yes |
| LobeHub | – | source-available, LobeHub Community License | 85,748 | 9,314 | Nov 18, 2026 | Yes |
| AnythingLLM | – | MIT | 68,392 | 1,143 | Nov 18, 2026 | Yes |
| Onyx Community Edition | – | MIT | 33,450 | 6,922 | Nov 18, 2026 | Yes |
1 of 3 · 5 picks
Model runners
Download open-weight models and serve them locally, usually through an OpenAI-compatible API.
- 1
Ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.
AI & LLM ToolsDockerAI-nativeStands out for the most GitHub stars. Ollama has 187,500 GitHub stars, uses the MIT license, ships a Docker image and had 1,199 commits in the last year; it replaces ChatGPT.
- 2
LocalAI
Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).
AI & LLM ToolsDockerAI-nativeStands out for the most commits in the last 12 months (5,147). LocalAI has 50,430 GitHub stars, uses the MIT license, ships a Docker image and had 5,147 commits in the last year; it replaces ChatGPT.
- 3
Text Generation WebUI
Gradio web UI for running large language models locally.
AI & LLM ToolsDockerAI-nativeStands out for AI features named in its own description. Text Generation WebUI has 47,739 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 688 commits in the last year.
- 4
LLM Harbor
Containerized LLM toolkit. Run LLM backends, APIs, frontends, and additional services via a concise CLI.
AI & LLM ToolsDockerAI-nativeStands out for a permissive Apache-2.0 license. LLM Harbor has 3,280 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,337 commits in the last year.
- 5
LLMKube
Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.
AI & LLM ToolsDockerAI-nativeStands out for the most recent release (v0.10.18, November 2026). LLMKube has 267 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,480 commits in the last year.
2 of 3 · 5 picks
Chat interfaces
Web chat front ends that connect to a local runner (and, if you choose, to cloud APIs).
- 1
Open-WebUI
User-friendly AI Interface, supports Ollama, OpenAI API.
Source-availableAI & LLM ToolsDockerAI-nativeOpen-WebUI has 159,000 GitHub stars, is source-available under the Open WebUI License (BSD-3-Clause plus branding clause) (not an open-source license), ships a Docker image and had 6,788 commits in the last year; it replaces ChatGPT and Lovable.
- 2
LibreChat
Enhanced ChatGPT-compatible AI chat interface supporting multiple AI providers, with multi-user auth, message search, and plugin support.
AI & LLM ToolsDockerAI-nativeLibreChat has 47,310 GitHub stars, uses the MIT license, ships a Docker image and had 3,592 commits in the last year; it replaces ChatGPT and Lovable.
- 3
LobeHub
Modern design AI chat framework supporting multiple AI providers, one click install MCP Marketplace and Artifacts / Thinking.
Source-availableAI & LLM ToolsDockerAI-nativeLobeHub has 85,748 GitHub stars, is source-available under the LobeHub Community License (not an open-source license), ships a Docker image and had 9,314 commits in the last year; it replaces ChatGPT and Lovable.
- 4
AnythingLLM
All-in-one desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.
AI & LLM ToolsDockerAI-nativeAnythingLLM has 68,392 GitHub stars, uses the MIT license, ships a Docker image and had 1,143 commits in the last year; it replaces ChatGPT.
- 5
Onyx Community Edition
Chat UI that works with any LLM. It comes loaded with advanced features like agents, web search, RAG, MCP, deep research, Connectors to 40+ knowledge sources, and more.
AI & LLM ToolsDockerAI-nativeOnyx Community Edition has 33,450 GitHub stars, uses the MIT license, ships a Docker image and had 6,922 commits in the last year.
3 of 3 · 2 picks
Use them with your documents
- 1
RAGFlow
Open-source RAG engine with deep document understanding.
AI & LLM ToolsDockerAI-nativeRAGFlow has 94,375 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 6,601 commits in the last year.
- 2
Khoj
Your AI second brain. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI.
AI & LLM ToolsDockerAI-nativeKhoj has 38,166 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 91 commits in the last year; it replaces ChatGPT.
What hardware do I need to run AI models myself?
Small open models run on a modern CPU with 16 GB of RAM, but a GPU with enough video memory for the model you pick makes answers much faster. Start with a small model in Ollama, measure the speed on your own hardware, then size up.
What is the easiest way to run an LLM locally?
Install Ollama, pull a small model and talk to it from the command line, then add a chat interface such as Open WebUI or LibreChat that connects to Ollama's local API. All of them ship Docker images.
Is Ollama free?
Yes. Ollama is open-source software released under the MIT license, so you can self-host it at no cost. You only pay for the server it runs on, and some projects also offer an optional paid cloud plan.
Can I run this stack with Docker?
All 12 apps here publish Docker images, including Ollama, LocalAI and Text Generation WebUI, so most of the stack can run side by side with Docker Compose on one server. Size the server for the heaviest app and add off-site backups.
How is this list ranked?
Picked by hand for each job from apps whose own description or topics match it, in curated order inside each section, with GitHub data from November 2026; an app appears only once.
The weekly digest
New self-hosted apps, every week
Fresh releases, rising projects and one deploy guide a week. Join home labbers and engineers who self-host.