[PR #592] fix: resolve pipeline reload module caching issue #480

New issue

Open

opened 2026-10-18 21:06:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:06:29 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/pipelines/pull/592
Author: @hectorc98
Created: 1/28/2026
Status: 🔄 Open

Base: main ← Head: fix/pipeline-reload-module-caching

📝 Commits (1)

6a7d281 fix: resolve pipeline reload module caching issue

📊 Changes

1 file changed (+7 additions, -1 deletions)

View changed files

📝 main.py (+7 -1)

📄 Description

Fix: Pipeline Reload Module Caching Issue

Problem

The pipeline reload API (POST /pipelines/reload) was returning HTTP 200 responses but not actually reloading updated pipeline code. Users had to restart the entire pod to see pipeline changes take effect, which defeats the purpose of the reload functionality.

Root Cause

Python's module import system caches modules in sys.modules by name. When load_module_from_path() was called during pipeline reload:

importlib.util.spec_from_file_location() would create a module spec
spec.loader.exec_module() would execute the module
However, if the module name already existed in sys.modules, Python would return the cached version instead of re-executing the updated file content

This meant that pipeline code changes were read from disk but the cached module code continued to execute.

Solution

Added sys.modules cache clearing in load_module_from_path() before loading modules:

# Clear module from sys.modules cache to ensure fresh reload
# This fixes the issue where pipeline reload API would return 200 but
# continue using cached module code instead of reloading updated files
if module_name in sys.modules:
    del sys.modules[module_name]

Impact

✅ Pipeline reload API now works correctly - code changes are immediately active after calling /pipelines/reload
✅ No more pod restarts required - development workflow is much smoother
✅ Backward compatible - no breaking changes to existing functionality
✅ Minimal code change - targeted fix with clear documentation

Testing

Verified all module loading flows through the fixed load_module_from_path() function
Confirmed this is the only location in the codebase that needed the fix
Created test script demonstrating the caching issue and fix
All reload paths (API endpoint, file upload, file deletion) benefit from this change

Files Changed

main.py: Added sys.modules cache clearing and documentation

Validation Steps

Deploy updated pipelines server
Upload or modify a pipeline file
Call POST /pipelines/reload
Verify updated pipeline code executes (without pod restart)

This fix resolves a significant development experience issue where pipeline changes appeared to be ignored by the reload API.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/pipelines/pull/592 **Author:** [@hectorc98](https://github.com/hectorc98) **Created:** 1/28/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/pipeline-reload-module-caching` --- ### 📝 Commits (1) - [`6a7d281`](https://github.com/open-webui/pipelines/commit/6a7d281121e3ffe9bb1d024b60f08c02e13cf5c0) fix: resolve pipeline reload module caching issue ### 📊 Changes **1 file changed** (+7 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `main.py` (+7 -1) </details> ### 📄 Description # Fix: Pipeline Reload Module Caching Issue ## Problem The pipeline reload API (`POST /pipelines/reload`) was returning HTTP 200 responses but not actually reloading updated pipeline code. Users had to restart the entire pod to see pipeline changes take effect, which defeats the purpose of the reload functionality. ## Root Cause Python's module import system caches modules in `sys.modules` by name. When `load_module_from_path()` was called during pipeline reload: 1. `importlib.util.spec_from_file_location()` would create a module spec 2. `spec.loader.exec_module()` would execute the module 3. However, if the module name already existed in `sys.modules`, Python would return the cached version instead of re-executing the updated file content This meant that pipeline code changes were read from disk but the cached module code continued to execute. ## Solution Added `sys.modules` cache clearing in `load_module_from_path()` before loading modules: ```python # Clear module from sys.modules cache to ensure fresh reload # This fixes the issue where pipeline reload API would return 200 but # continue using cached module code instead of reloading updated files if module_name in sys.modules: del sys.modules[module_name] ``` ## Impact - ✅ **Pipeline reload API now works correctly** - code changes are immediately active after calling `/pipelines/reload` - ✅ **No more pod restarts required** - development workflow is much smoother - ✅ **Backward compatible** - no breaking changes to existing functionality - ✅ **Minimal code change** - targeted fix with clear documentation ## Testing - Verified all module loading flows through the fixed `load_module_from_path()` function - Confirmed this is the only location in the codebase that needed the fix - Created test script demonstrating the caching issue and fix - All reload paths (API endpoint, file upload, file deletion) benefit from this change ## Files Changed - `main.py`: Added `sys.modules` cache clearing and documentation ## Validation Steps 1. Deploy updated pipelines server 2. Upload or modify a pipeline file 3. Call `POST /pipelines/reload` 4. Verify updated pipeline code executes (without pod restart) This fix resolves a significant development experience issue where pipeline changes appeared to be ignored by the reload API. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-18 21:06:29 +00:00