[PR #131] fix(sync): make upload processing synchronous and resilient #126

New issue

Open

opened 2026-10-18 21:14:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:14:33 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/oikb/pull/131
Author: @weberdigital
Created: 9/25/2026
Status: 🔄 Open

Base: main ← Head: gh/upstream-sync-fix

📝 Commits (1)

a1a98b8 fix(sync): make upload processing synchronous and resilient

📊 Changes

4 files changed (+89 additions, -4 deletions)

View changed files

📝 src/oikb/cli.py (+29 -3)
📝 src/oikb/client.py (+10 -1)
📝 src/oikb/kb_sync.py (+2 -0)
📝 src/oikb/sync.py (+48 -0)

📄 Description

run Open WebUI document processing synchronously (process_in_background=false) so the upload response is only returned once the file is fully indexed; bounds concurrent embedding streams to the upload concurrency instead of unbounded overlap
make the httpx client timeout configurable via upload_timeout in .oikb.yaml and --upload-timeout in the CLI
retry on transient httpx.TimeoutException / TransportError up to 3 times with exponential backoff; before each retry re-check the KB diff and treat an already-indexed file as success instead of re-uploading (which would otherwise trip the duplicate-content guard)
leave non-transient failures (4xx, auth, size) unchanged

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/oikb/pull/131 **Author:** [@weberdigital](https://github.com/weberdigital) **Created:** 9/25/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `gh/upstream-sync-fix` --- ### 📝 Commits (1) - [`a1a98b8`](https://github.com/open-webui/oikb/commit/a1a98b84c5b7d09804b2df852267fd91af9ebd49) fix(sync): make upload processing synchronous and resilient ### 📊 Changes **4 files changed** (+89 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `src/oikb/cli.py` (+29 -3) 📝 `src/oikb/client.py` (+10 -1) 📝 `src/oikb/kb_sync.py` (+2 -0) 📝 `src/oikb/sync.py` (+48 -0) </details> ### 📄 Description - run Open WebUI document processing synchronously (process_in_background=false) so the upload response is only returned once the file is fully indexed; bounds concurrent embedding streams to the upload concurrency instead of unbounded overlap - make the httpx client timeout configurable via upload_timeout in .oikb.yaml and --upload-timeout in the CLI - retry on transient httpx.TimeoutException / TransportError up to 3 times with exponential backoff; before each retry re-check the KB diff and treat an already-indexed file as success instead of re-uploading (which would otherwise trip the duplicate-content guard) - leave non-transient failures (4xx, auth, size) unchanged --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-18 21:14:33 +00:00

bsakaguchi referenced this issue

2026-10-18 21:14:37 +00:00

[PR #141] [MERGED] ci(docker): publish each image flavor on its own #132

bsakaguchi referenced this issue

2026-10-18 21:14:37 +00:00

[PR #142] fix(docker): build the alpine arm64 image without pymupdf #133