Skip to content
appsgit

3 jobs · 12 apps

Best local LLM tools to self-host

To run an LLM locally, the core of a private AI stack is Ollama for running models, Open-WebUI (source-available) for chatting with them and RAGFlow for questions over your own files, all running on your own servers.

Pointed at a model on your own hardware, these tools keep your prompts and documents on your infrastructure.

A local LLM setup has two parts: a model runner that downloads open-weight models and serves them on your own CPU or GPU, and a chat interface on top. Both run on your machine or server, so prompts and documents do not go to a cloud AI provider unless you connect one.

Updated · data from GitHub, refreshed daily

Top picks

  1. Rank 1:

    Ollama

    ollama/ollama

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

    Active · last commit Oct 28, 2026

    AI & LLM ToolsGoDocker

    AI-nativeStands out for the most GitHub stars. Ollama has 185,075 GitHub stars, uses the MIT license, ships a Docker image and had 1,263 commits in the last year; it replaces ChatGPT.

  2. Rank 2:

    LocalAI

    mudler/LocalAI

    Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).

    Active · last commit Oct 28, 2026

    AI & LLM ToolsGoDocker

    AI-nativeStands out for the most commits in the last 12 months (3,909). LocalAI has 50,019 GitHub stars, uses the MIT license, ships a Docker image and had 3,909 commits in the last year; it replaces ChatGPT.

  3. Rank 3:

    Text Generation WebUI

    oobabooga/textgen

    Gradio web UI for running large language models locally.

    Active · last commit Aug 17, 2026

    AI & LLM ToolsPythonDocker

    AI-nativeStands out for AI features named in its own description. Text Generation WebUI has 47,724 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 699 commits in the last year.

At a glance

Local LLM tools to self-host compared

Picked by hand for each job from apps whose own description or topics match it, in curated order inside each section, with GitHub data from October 2026; an app appears only once.

Top 10 picks compared, data as of Oct 28, 2026
AppStands out forLicenseGitHub starsCommits (12 mo)Last commitDocker image
Ollamathe most GitHub starsMIT185,0751,263Oct 28, 2026Yes
LocalAIthe most commits in the last 12 months (3,909)MIT50,0193,909Oct 28, 2026Yes
Text Generation WebUIAI features named in its own descriptionAGPL-3.047,724699Aug 17, 2026Yes
LLM Harbora permissive Apache-2.0 licenseApache-2.03,2561,315Oct 24, 2026Yes
LLMKubethe most recent release (v0.10.13, October 2026)Apache-2.02461,324Oct 28, 2026Yes
Open-WebUI–source-available, Open WebUI License (BSD-3-Clause plus branding clause)157,0046,037Oct 28, 2026Yes
LibreChat–MIT46,4093,220Oct 28, 2026Yes
LobeHub–source-available, LobeHub Community License83,5508,610Oct 28, 2026Yes
AnythingLLM–MIT67,665998Oct 27, 2026Yes
Onyx Community Edition–MIT33,0426,557Oct 28, 2026Yes

1 of 3 · 5 picks

Model runners

Download open-weight models and serve them locally, usually through an OpenAI-compatible API.

  1. 1

    Ollama

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

    AI-nativeStands out for the most GitHub stars. Ollama has 185,075 GitHub stars, uses the MIT license, ships a Docker image and had 1,263 commits in the last year; it replaces ChatGPT.

  2. 2

    LocalAI

    Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).

    AI-nativeStands out for the most commits in the last 12 months (3,909). LocalAI has 50,019 GitHub stars, uses the MIT license, ships a Docker image and had 3,909 commits in the last year; it replaces ChatGPT.

  3. 3

    Text Generation WebUI

    Gradio web UI for running large language models locally.

    AI-nativeStands out for AI features named in its own description. Text Generation WebUI has 47,724 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 699 commits in the last year.

  4. 4

    LLM Harbor

    Containerized LLM toolkit. Run LLM backends, APIs, frontends, and additional services via a concise CLI.

    AI-nativeStands out for a permissive Apache-2.0 license. LLM Harbor has 3,256 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,315 commits in the last year.

  5. 5

    LLMKube

    Kubernetes operator for self-hosted LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and…

    AI-nativeStands out for the most recent release (v0.10.13, October 2026). LLMKube has 246 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 1,324 commits in the last year.

2 of 3 · 5 picks

Chat interfaces

Web chat front ends that connect to a local runner (and, if you choose, to cloud APIs).

  1. 1

    Open-WebUI

    User-friendly AI Interface, supports Ollama, OpenAI API.

    AI-nativeOpen-WebUI has 157,004 GitHub stars, is source-available under the Open WebUI License (BSD-3-Clause plus branding clause) (not an open-source license), ships a Docker image and had 6,037 commits in the last year; it replaces ChatGPT and Lovable.

  2. 2

    LibreChat

    Enhanced ChatGPT-compatible AI chat interface supporting multiple AI providers, with multi-user auth, message search, and plugin support.

    AI-nativeLibreChat has 46,409 GitHub stars, uses the MIT license, ships a Docker image and had 3,220 commits in the last year; it replaces ChatGPT and Lovable.

  3. 3

    LobeHub

    Modern design AI chat framework supporting multiple AI providers, one click install MCP Marketplace and Artifacts / Thinking.

    AI-nativeLobeHub has 83,550 GitHub stars, is source-available under the LobeHub Community License (not an open-source license), ships a Docker image and had 8,610 commits in the last year; it replaces ChatGPT and Lovable.

  4. 4

    AnythingLLM

    All-in-one desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.

    AI-nativeAnythingLLM has 67,665 GitHub stars, uses the MIT license, ships a Docker image and had 998 commits in the last year; it replaces ChatGPT.

  5. 5

    Onyx Community Edition

    Chat UI that works with any LLM. It comes loaded with advanced features like agents, web search, RAG, MCP, deep research, Connectors to 40+ knowledge sources, and more.

    AI-nativeOnyx Community Edition has 33,042 GitHub stars, uses the MIT license, ships a Docker image and had 6,557 commits in the last year.

3 of 3 · 2 picks

Use them with your documents

  1. 1

    RAGFlow

    Open-source RAG engine with deep document understanding.

    AI-nativeRAGFlow has 92,997 GitHub stars, uses the Apache-2.0 license, ships a Docker image and had 6,295 commits in the last year.

  2. 2

    Khoj

    Your AI second brain. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI.

    AI-nativeKhoj has 37,865 GitHub stars, uses the AGPL-3.0 license, ships a Docker image and had 120 commits in the last year; it replaces ChatGPT.

FAQ

Questions and answers

Still curious? Email info@appsgit.com.

What hardware do I need to run AI models myself?

Small open models run on a modern CPU with 16 GB of RAM, but a GPU with enough video memory for the model you pick makes answers much faster. Start with a small model in Ollama, measure the speed on your own hardware, then size up.

What is the easiest way to run an LLM locally?

Install Ollama, pull a small model and talk to it from the command line, then add a chat interface such as Open WebUI or LibreChat that connects to Ollama's local API. All of them ship Docker images.

Is Ollama free?

Yes. Ollama is open-source software released under the MIT license, so you can self-host it at no cost. You only pay for the server it runs on, and some projects also offer an optional paid cloud plan.

Can I run this stack with Docker?

All 12 apps here publish Docker images, including Ollama, LocalAI and Text Generation WebUI, so most of the stack can run side by side with Docker Compose on one server. Size the server for the heaviest app and add off-site backups.

How is this list ranked?

Picked by hand for each job from apps whose own description or topics match it, in curated order inside each section, with GitHub data from October 2026; an app appears only once.

The weekly digest

New self-hosted apps, every week

Fresh releases, rising projects and one deploy guide a week. Join home labbers and engineers who self-host.

One email a week. No spam, unsubscribe anytime.