Skip to content
appsgit

3 jobs · 12 apps

Best local LLM tools to self-host

To run an LLM locally, the core of a private AI stack is Ollama for running models, Open-WebUI (source-available) for chatting with them and RAGFlow for questions over your own files, all running on your own servers.

Pointed at a model on your own hardware, these tools keep your prompts and documents on your infrastructure.

A local LLM setup has two parts: a model runner that downloads open-weight models and serves them on your own CPU or GPU, and a chat interface on top. Both run on your machine or server, so prompts and documents do not go to a cloud AI provider unless you connect one.

Updated · data from GitHub, refreshed daily

Top picks

  1. Rank 1:

    Ollama

    ollama/ollama

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

    Active · last commit Nov 18, 2026

    AI & LLM ToolsGoDocker

    AI-nativeStands out for the most GitHub stars. Ollama has 187,500 GitHub stars, uses the MIT license, ships a Docker image and had 1,199 commits in the last year; it replaces ChatGPT.

  2. Rank 2:

    LocalAI

    mudler/LocalAI

    Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).

    Active · last commit Nov 18, 2026

    AI & LLM ToolsGoDocker

    AI-nativeStands out for the most commits in the last 12 months (5,147). LocalAI has 50,430 GitHub stars, uses the MIT license, ships a Docker image and had 5,147 commits in the last year; it replaces ChatGPT.

  3. Rank 3:

    Text Generation WebUI

    oobabooga/textgen

    Gradio web UI for running large language models locally.

    Maintained · last commit Aug 17, 2026

    AI & LLM ToolsPythonDocker

    AI-nativeStands out for AI features named in its own description. Text Generation WebUI has 47,739 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 688 commits in the last year.

At a glance

Local LLM tools to self-host compared

Picked by hand for each job from apps whose own description or topics match it, in curated order inside each section, with GitHub data from November 2026; an app appears only once.

Top 10 picks compared, data as of Nov 18, 2026
AppStands out forLicenseGitHub starsCommits (12 mo)Last commitDocker image
Ollamathe most GitHub starsMIT187,5001,199Nov 18, 2026Yes
LocalAIthe most commits in the last 12 months (5,147)MIT50,4305,147Nov 18, 2026Yes
Text Generation WebUIAI features named in its own descriptionAGPL-3.047,739688Aug 17, 2026Yes
LLM Harbora permissive Apache-2.0 licenseApache-2.03,2801,337Nov 11, 2026Yes
LLMKubethe most recent release (v0.10.18, November 2026)Apache-2.02671,480Nov 18, 2026Yes
Open-WebUI–source-available, Open WebUI License (BSD-3-Clause plus branding clause)159,0006,788Nov 18, 2026Yes
LibreChat–MIT47,3103,592Nov 17, 2026Yes
LobeHub–source-available, LobeHub Community License85,7489,314Nov 18, 2026Yes
AnythingLLM–MIT68,3921,143Nov 18, 2026Yes
Onyx Community Edition–MIT33,4506,922Nov 18, 2026Yes

1 of 3 · 5 picks

Model runners

Download open-weight models and serve them locally, usually through an OpenAI-compatible API.

  1. 1

    Ollama

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

    AI-nativeStands out for the most GitHub stars. Ollama has 187,500 GitHub stars, uses the MIT license, ships a Docker image and had 1,199 commits in the last year; it replaces ChatGPT.

  2. 2

    LocalAI

    Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).

    AI-nativeStands out for the most commits in the last 12 months (5,147). LocalAI has 50,430 GitHub stars, uses the MIT license, ships a Docker image and had 5,147 commits in the last year; it replaces ChatGPT.

  3. 3

    Text Generation WebUI

    Gradio web UI for running large language models locally.

    AI-nativeStands out for AI features named in its own description. Text Generation WebUI has 47,739 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 688 commits in the last year.

  4. 4

    LLM Harbor

    Containerized LLM toolkit. Run LLM backends, APIs, frontends, and additional services via a concise CLI.

    AI-nativeStands out for a permissive Apache-2.0 license. LLM Harbor has 3,280 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,337 commits in the last year.

  5. 5

    LLMKube

    Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.

    AI-nativeStands out for the most recent release (v0.10.18, November 2026). LLMKube has 267 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,480 commits in the last year.

2 of 3 · 5 picks

Chat interfaces

Web chat front ends that connect to a local runner (and, if you choose, to cloud APIs).

  1. 1

    Open-WebUI

    User-friendly AI Interface, supports Ollama, OpenAI API.

    AI-nativeOpen-WebUI has 159,000 GitHub stars, is source-available under the Open WebUI License (BSD-3-Clause plus branding clause) (not an open-source license), ships a Docker image and had 6,788 commits in the last year; it replaces ChatGPT and Lovable.

  2. 2

    LibreChat

    Enhanced ChatGPT-compatible AI chat interface supporting multiple AI providers, with multi-user auth, message search, and plugin support.

    AI-nativeLibreChat has 47,310 GitHub stars, uses the MIT license, ships a Docker image and had 3,592 commits in the last year; it replaces ChatGPT and Lovable.

  3. 3

    LobeHub

    Modern design AI chat framework supporting multiple AI providers, one click install MCP Marketplace and Artifacts / Thinking.

    AI-nativeLobeHub has 85,748 GitHub stars, is source-available under the LobeHub Community License (not an open-source license), ships a Docker image and had 9,314 commits in the last year; it replaces ChatGPT and Lovable.

  4. 4

    AnythingLLM

    All-in-one desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.

    AI-nativeAnythingLLM has 68,392 GitHub stars, uses the MIT license, ships a Docker image and had 1,143 commits in the last year; it replaces ChatGPT.

  5. 5

    Onyx Community Edition

    Chat UI that works with any LLM. It comes loaded with advanced features like agents, web search, RAG, MCP, deep research, Connectors to 40+ knowledge sources, and more.

    AI-nativeOnyx Community Edition has 33,450 GitHub stars, uses the MIT license, ships a Docker image and had 6,922 commits in the last year.

3 of 3 · 2 picks

Use them with your documents

  1. 1

    RAGFlow

    Open-source RAG engine with deep document understanding.

    AI-nativeRAGFlow has 94,375 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 6,601 commits in the last year.

  2. 2

    Khoj

    Your AI second brain. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI.

    AI-nativeKhoj has 38,166 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 91 commits in the last year; it replaces ChatGPT.

FAQ

Questions and answers

Still curious? Email info@appsgit.com.

What hardware do I need to run AI models myself?

Small open models run on a modern CPU with 16 GB of RAM, but a GPU with enough video memory for the model you pick makes answers much faster. Start with a small model in Ollama, measure the speed on your own hardware, then size up.

What is the easiest way to run an LLM locally?

Install Ollama, pull a small model and talk to it from the command line, then add a chat interface such as Open WebUI or LibreChat that connects to Ollama's local API. All of them ship Docker images.

Is Ollama free?

Yes. Ollama is open-source software released under the MIT license, so you can self-host it at no cost. You only pay for the server it runs on, and some projects also offer an optional paid cloud plan.

Can I run this stack with Docker?

All 12 apps here publish Docker images, including Ollama, LocalAI and Text Generation WebUI, so most of the stack can run side by side with Docker Compose on one server. Size the server for the heaviest app and add off-site backups.

How is this list ranked?

Picked by hand for each job from apps whose own description or topics match it, in curated order inside each section, with GitHub data from November 2026; an app appears only once.

The weekly digest

New self-hosted apps, every week

Fresh releases, rising projects and one deploy guide a week. Join home labbers and engineers who self-host.

One email a week. No spam, unsubscribe anytime.