turboderp-org/exui

mirror of https://github.com/turboderp-org/exui.git synced 2026-05-11 16:30:18 +00:00

Files

turboderp 778ffa89fa One more screenshot

2023-11-07 22:34:42 +01:00

1.4 KiB

Raw Blame History

ExUI

This is a simple, lightweight browser-based UI for running local inference using ExLlamaV2.

Overview of features

Friendly, responsive and minimalistic UI
Persistent sessions
Multiple instruct formats
Speculative decoding
Supports EXL2, GPTQ and FP16 models

Screenshots

Running locally

First, clone this repository and install requirements:

git clone https://github.com/turboderp/exui
cd exui
pip install -r requirements.txt

Then run the web server with the included server.py:

python server.py

Your browser should automatically open on the default IP/port. Config and sessions are stored in ~/exui by default.

Prebuilt wheels for ExLlamaV2 are available here. Installing the latest version of Flash Attention is recommended.

More to come

Stay tuned.