What Text Generation Web UI offers
Text Generation Web UI (commonly called 'oobabooga' after its creator) is a Gradio-based browser interface for local LLMs. It supports every major model format: GGUFGGUFA file format for distributing quantized language models optimized for efficient local inference, popularized by llama.cpp.Learn more → via llama.cpp, GPTQ and EXL2 via ExLlamaV2, and full-precision models via HuggingFace Transformers.
Beyond basic chat, it includes: a character system for role-play with custom personas, a notebook mode for text completion without the chat wrapper, extensions for text-to-speech and speech-to-text, and a full API server compatible with both the OpenAI format and its own format.
Install on Windows, Mac, or Linux
Clone the repository: `git clone https://github.com/oobabooga/text-generation-webui`. Navigate into it and run the one-click installer for your platform: `start_windows.bat` (Windows), `start_linux.sh` (Linux), or `start_macos.sh` (macOS). The installer creates a conda environment and installs all dependencies automatically.
On first run, it will download PyTorch for your hardware (CUDA, ROCm, or CPU). The setup takes 5–15 minutes. After installation, the web UI opens at `http://127.0.0.1:7860`.
Load a model
Place GGUF model files in the `models/` directory. Go to the Model tab in the web UI, click the refresh icon, and select your model from the dropdown. For GGUF files, select 'llama.cpp' as the loader. Set the number of GPU layers to `99` to fully offload to GPU.
To download models from Hugging Face directly from the UI, use the Download tab. Enter the model repository name (e.g. `TheBloke/Mistral-7B-Instruct-v0.2-GGUF`) and select the specific file you want. Large downloads can be resumed if interrupted.
Use the API server
Enable the OpenAI-compatible API by starting with `--api` flag: `python server.py --api --listen`. This starts an OpenAI-compatible server at port 5000 alongside the web UI. Connect any OpenAI SDK client by setting `base_url='http://127.0.0.1:5000/v1'`.
The API supports streamingStreamingStreaming returns a model's tokens incrementally as they're generated rather than waiting for the complete response to finish.Learn more →, system prompts, and temperatureTemperatureA parameter controlling the randomness of model outputs — lower values produce more focused, deterministic responses; higher values produce more creative, varied text.Learn more → controls. Use it to connect local models to any application that uses the OpenAI SDK — LangChain, LlamaIndex, CrewAI, and others all work without code changes.