open-webui/pipelines

Fork 0

mirror of https://github.com/open-webui/pipelines.git synced 2026-11-05 01:37:24 +00:00

[PR #602] feat: Add mlflow filter pipeline #483

New issue

Open

opened 2026-10-18 21:06:31 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:06:31 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/pipelines/pull/602
Author: @WeichenXu123
Created: 4/20/2026
Status: 🔄 Open

Base: main ← Head: mlflow-filter

📝 Commits (10+)

7932def init
d38b08c init
46362a8 update
14596b8 address comments
e83ef4b update
7355e57 update
6c60c38 update
7d70ee9 clean
8a1a592 update
99b3336 clean

📊 Changes

2 files changed (+165 additions, -1 deletions)

View changed files

📝 .gitignore (+2 -1)
➕ examples/filters/mlflow_filter_pipeline.py (+163 -0)

📄 Description

MLflow Filter Pipeline

Adds a filter pipeline that integrates https://mlflow.org/ tracing with Open WebUI, enabling observability for multi-turn chat sessions.

What it does

inlet: Captures the last user message and session context before each request
outlet: Logs a complete trace per turn — user input, assistant response, model name, and token usage — grouped under the same session in the MLflow UI

Features

Multi-turn session grouping — all turns of a conversation are linked via mlflow.trace.session, viewable with "Group by session" in the MLflow UI
Per-turn tracing — each request/response pair is logged as a separate MLflow trace with latency and status
Token usage tracking — input/output token counts are captured when provided by the backend, automatically aggregated at the trace level
User attribution — traces are tagged with the authenticated user's email via mlflow.trace.user

Requirements

MLflow tracking server running and accessible: mlflow server --host 0.0.0.0 --port 5000
requirements: mlflow>=2.14.0

Configuration (Valves)

mlflow_tracking_uri (default: http://localhost:5000) — MLflow tracking server URI
mlflow_experiment_name (default: open-webui) — Experiment name in MLflow
debug (default: false) — Enable debug logging

Setup

Start MLflow: mlflow server --disable-security-middleware
Upload this pipeline in Open WebUI → Admin → Pipelines
Set MLFLOW_TRACKING_URI env var (Docker users: http://host.docker.internal:5000)
Open the MLflow UI and enable "Group by session" to view full conversations

end-to-end test in local machine:

Launch Mlflow server

MLFLOW_LOGGING_LEVEL=DEBUG mlflow server --disable-security-middleware

Launch Openweb UI server

open-webui serve

Launch pipeline service by docker

build a dockerfile and save it as Dockerfile.mlflow in current directory.

FROM ghcr.io/open-webui/pipelines:main
RUN pip install --no-cache-dir mlflow

build image:

docker build -f Dockerfile.mlflow -t pipelines-mlflow .

launch the docker container:

docker run -p 9099:9099 --add-host=host.docker.internal:host-gateway -v pipelines:/app/pipelines --name pipelines --restart always -e MLFLOW_TRACKING_URI=http://host.docker.internal:5000/  -e DEBUG_MODE=true pipelines-mlflow

OpenWebUI pipeline configure:

OpenWebUI chat:

MLflow traces:

single traces:

chat session:

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/pipelines/pull/602 **Author:** [@WeichenXu123](https://github.com/WeichenXu123) **Created:** 4/20/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `mlflow-filter` --- ### 📝 Commits (10+) - [`7932def`](https://github.com/open-webui/pipelines/commit/7932def67f05a12c7aa52242e8b56f006508196a) init - [`d38b08c`](https://github.com/open-webui/pipelines/commit/d38b08cd16fbe8cb385befddd18735636398a598) init - [`46362a8`](https://github.com/open-webui/pipelines/commit/46362a85031d23b1a678f942cf58b0c48c3e51e2) update - [`14596b8`](https://github.com/open-webui/pipelines/commit/14596b8c8982db7f418a640cf2db3f7be0c9d401) address comments - [`e83ef4b`](https://github.com/open-webui/pipelines/commit/e83ef4b52fd82506be8455ccbc0d03e6c5380892) update - [`7355e57`](https://github.com/open-webui/pipelines/commit/7355e5713d224a4b0180b47b2f436b35c6811726) update - [`6c60c38`](https://github.com/open-webui/pipelines/commit/6c60c38ce0bbf33210e23c6ce65c2dbd82e15aea) update - [`7d70ee9`](https://github.com/open-webui/pipelines/commit/7d70ee9a266bf172064601d24d9d091024ed0fe0) clean - [`8a1a592`](https://github.com/open-webui/pipelines/commit/8a1a5923b2cfc2298d797d1bec4e8af7a6131b10) update - [`99b3336`](https://github.com/open-webui/pipelines/commit/99b33365fbde709ba0223de9fcd44ae345527653) clean ### 📊 Changes **2 files changed** (+165 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+2 -1) ➕ `examples/filters/mlflow_filter_pipeline.py` (+163 -0) </details> ### 📄 Description ## MLflow Filter Pipeline Adds a filter pipeline that integrates https://mlflow.org/ tracing with Open WebUI, enabling observability for multi-turn chat sessions. ## What it does - inlet: Captures the last user message and session context before each request - outlet: Logs a complete trace per turn — user input, assistant response, model name, and token usage — grouped under the same session in the MLflow UI ## Features - Multi-turn session grouping — all turns of a conversation are linked via mlflow.trace.session, viewable with "Group by session" in the MLflow UI - Per-turn tracing — each request/response pair is logged as a separate MLflow trace with latency and status - Token usage tracking — input/output token counts are captured when provided by the backend, automatically aggregated at the trace level - User attribution — traces are tagged with the authenticated user's email via mlflow.trace.user ## Requirements - MLflow tracking server running and accessible: mlflow server --host 0.0.0.0 --port 5000 - requirements: mlflow>=2.14.0 ## Configuration (Valves) - mlflow_tracking_uri (default: http://localhost:5000) — MLflow tracking server URI - mlflow_experiment_name (default: open-webui) — Experiment name in MLflow - debug (default: false) — Enable debug logging ## Setup 1. Start MLflow: `mlflow server --disable-security-middleware` 2. Upload this pipeline in Open WebUI → Admin → Pipelines 3. Set MLFLOW_TRACKING_URI env var (Docker users: http://host.docker.internal:5000) 4. Open the MLflow UI and enable "Group by session" to view full conversations ## end-to-end test in local machine: ### Launch Mlflow server ``` MLFLOW_LOGGING_LEVEL=DEBUG mlflow server --disable-security-middleware ``` ### Launch Openweb UI server ``` open-webui serve ``` ### Launch pipeline service by docker 1. build a dockerfile and save it as `Dockerfile.mlflow` in current directory. ``` FROM ghcr.io/open-webui/pipelines:main RUN pip install --no-cache-dir mlflow ``` 2. build image: ``` docker build -f Dockerfile.mlflow -t pipelines-mlflow . ``` 3. launch the docker container: ``` docker run -p 9099:9099 --add-host=host.docker.internal:host-gateway -v pipelines:/app/pipelines --name pipelines --restart always -e MLFLOW_TRACKING_URI=http://host.docker.internal:5000/ -e DEBUG_MODE=true pipelines-mlflow ``` ### OpenWebUI pipeline configure: <img width="2522" height="1514" alt="image" src="https://github.com/user-attachments/assets/d1009973-25ae-4d2a-9fd3-df14531975ef" /> ### OpenWebUI chat: <img width="3000" height="1468" alt="image" src="https://github.com/user-attachments/assets/0e0d0e3f-1a50-49cd-b29a-27bd2ae67134" /> ### MLflow traces: single traces: <img width="2400" height="1494" alt="image" src="https://github.com/user-attachments/assets/c65e139e-ae4a-4877-8829-c44aca2e4bd8" /> <img width="2414" height="1272" alt="image" src="https://github.com/user-attachments/assets/2c1b04a4-affb-40a9-9560-e9e3bd4eb504" /> chat session: <img width="2628" height="1568" alt="image" src="https://github.com/user-attachments/assets/260de168-d6bd-4f32-8fbd-121ba92d744c" /> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-18 21:06:31 +00:00