[PR #2] [MERGED] Add MLX-LM Server example #283

New issue

Closed

opened 2026-10-18 21:04:38 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:04:38 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/pipelines/pull/2
Author: @justinh-rahb
Created: 5/22/2024
Status: ✅ Merged
Merged: 5/22/2024
Merged by: @tjbck

Base: main ← Head: mlx-lm

📝 Commits (9)

9d9712d Add MLX-LM Server example
9f4803a Move to examples
e263648 Add metadata header
830ae49 Fix depends
940d91c Handle more params
46f4aa1 Adjust default params
45cba3d Fix
bd2ea92 Refac meta
7556800 Refac meta

📊 Changes

1 file changed (+112 additions, -0 deletions)

View changed files

➕ pipelines/examples/mlx_pipeline.py (+112 -0)

📄 Description

Adds a pipeline that uses the MLX-LM library and HTTP server:
https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/SERVER.md

This pipeline starts the server and loads a model from the MLX_MODEL environment variable (default: mistralai/Mistral-7B-Instruct-v0.2)

Requires:

pip install mlx-lm

Warning

Full weights models are large, make sure you've got enough disk space.

Also: FIRST!

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/pipelines/pull/2 **Author:** [@justinh-rahb](https://github.com/justinh-rahb) **Created:** 5/22/2024 **Status:** ✅ Merged **Merged:** 5/22/2024 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `main` ← **Head:** `mlx-lm` --- ### 📝 Commits (9) - [`9d9712d`](https://github.com/open-webui/pipelines/commit/9d9712da98a08a88f9d8c8faf4409a14ba4bd46d) Add MLX-LM Server example - [`9f4803a`](https://github.com/open-webui/pipelines/commit/9f4803a9b651bcd31682eccde6a2a0d7ee35764f) Move to examples - [`e263648`](https://github.com/open-webui/pipelines/commit/e263648f952896be72c85609bf672047e752b867) Add metadata header - [`830ae49`](https://github.com/open-webui/pipelines/commit/830ae49f09e22c169e2295843b95ff58115a4ad9) Fix depends - [`940d91c`](https://github.com/open-webui/pipelines/commit/940d91c21650a05906bbd33faa1ea1e2d724728b) Handle more params - [`46f4aa1`](https://github.com/open-webui/pipelines/commit/46f4aa1ca538a5e215a5c21db8c3085f354a73e4) Adjust default params - [`45cba3d`](https://github.com/open-webui/pipelines/commit/45cba3dc5ddc055c0bb3465a79c7b34ece8c8a63) Fix - [`bd2ea92`](https://github.com/open-webui/pipelines/commit/bd2ea926bba2700c933f6243d93ebbcbd26d0317) Refac meta - [`7556800`](https://github.com/open-webui/pipelines/commit/7556800fdf08e9f6d983eb301d8ad586435d5673) Refac meta ### 📊 Changes **1 file changed** (+112 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `pipelines/examples/mlx_pipeline.py` (+112 -0) </details> ### 📄 Description Adds a pipeline that uses the MLX-LM library and HTTP server: https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/SERVER.md This pipeline starts the server and loads a model from the `MLX_MODEL` environment variable (default: `mistralai/Mistral-7B-Instruct-v0.2`) Requires: ```bash pip install mlx-lm ``` > [!WARNING] > Full weights models are large, make sure you've got enough disk space. Also: ***FIRST!*** --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-18 21:04:38 +00:00