mirror of
https://github.com/turboderp-org/exui.git
synced 2026-07-14 11:07:26 +00:00
8524c8cc43
Add support for reading generation_config.json for model parameters Implement model_loaded_callback for parameter updates Add UI controls to switch between model and app defaults Add visual highlighting for modified parameters Add new API endpoints for parameter management Improve session handling of model parameters Update UI styling for parameter controls
ExUI
This is a simple, lightweight browser-based UI for running local inference using ExLlamaV2.
Overview of features
- Friendly, responsive and minimalistic UI
- Persistent sessions
- Multiple instruct formats
- Speculative decoding
- Supports EXL2, GPTQ and FP16 models
- Notepad mode
Screenshots
Running locally
First, clone this repository and install requirements:
git clone https://github.com/turboderp/exui
cd exui
pip install -r requirements.txt
Then run the web server with the included server.py:
python server.py
Your browser should automatically open on the default IP/port. Config and sessions are stored in ~/exui by default.
Prebuilt wheels for ExLlamaV2 are available here. Installing the latest version of Flash Attention is recommended.
Running in Google Colab
An example Colab notebook is provided here.
Installation
More detailed installation instructions can be found here.
More to come
Stay tuned.
