[GH-ISSUE #143] `oikb watch` re-syncs in a loop: the watcher treats its own file reads (watchdog `opened`/`closed_no_write`) as changes #80

New issue

Open

opened 2026-10-18 21:14:10 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:14:10 +00:00

Owner

Originally created by @MiskyCAN on GitHub (Oct 11, 2026).
Original GitHub issue: https://github.com/open-webui/oikb/issues/143

Versions: oikb 0.5.0 (also 0.4.0), watchdog 6.0.0, Python 3.14, Linux (inotify)

What happens: once a file in a watched folder is read while oikb watch is running, the watcher syncs about once a second, indefinitely. Every sync reports "N unchanged", and each one sends a POST /api/v1/knowledge/{id}/sync/diff. One of our watchers ran like this continuously (~80,000 diff requests a day). Earlier, on the day that folder was set up, the same loop logged "Diff: +3" every second and left ~1,200 duplicate uploads behind.

Cause: _DebouncedHandler.on_any_event in oikb/watcher.py re-arms the debounce timer for every event type. watchdog 6 reports read-only access on Linux as opened and closed_no_write. A sync reads every file to hash it, so the sync emits exactly the events that schedule the next sync.

Repro (no server needed; uses the handler directly):

import pathlib, tempfile, time
from watchdog.observers import Observer
from oikb.watcher import _DebouncedHandler

folder = pathlib.Path(tempfile.mkdtemp())
(folder / "doc.txt").write_text("one")
syncs = [0]
def sync():                      # what a sync does: read every file
    syncs[0] += 1
    for p in folder.iterdir():
        p.read_bytes()
obs = Observer(); obs.schedule(_DebouncedHandler(sync, debounce_seconds=0.2), str(folder)); obs.start()
time.sleep(0.3)
(folder / "doc.txt").write_text("two")   # one real change
time.sleep(3); obs.stop(); obs.join()
print(syncs[0])                  # 14 here; expected 1

Fix that works for us: ignore the read-only event types in on_any_event. A real change still arrives as created/modified/deleted/moved (and closed after a write).

        if event.is_directory:
            return
        if event.event_type in ("opened", "closed_no_write"):
            return

With that change the repro prints 1.

Originally created by @MiskyCAN on GitHub (Oct 11, 2026). Original GitHub issue: https://github.com/open-webui/oikb/issues/143 **Versions:** oikb 0.5.0 (also 0.4.0), watchdog 6.0.0, Python 3.14, Linux (inotify) **What happens:** once a file in a watched folder is read while `oikb watch` is running, the watcher syncs about once a second, indefinitely. Every sync reports "N unchanged", and each one sends a `POST /api/v1/knowledge/{id}/sync/diff`. One of our watchers ran like this continuously (~80,000 diff requests a day). Earlier, on the day that folder was set up, the same loop logged "Diff: +3" every second and left ~1,200 duplicate uploads behind. **Cause:** `_DebouncedHandler.on_any_event` in `oikb/watcher.py` re-arms the debounce timer for every event type. watchdog 6 reports read-only access on Linux as `opened` and `closed_no_write`. A sync reads every file to hash it, so the sync emits exactly the events that schedule the next sync. **Repro** (no server needed; uses the handler directly): ```python import pathlib, tempfile, time from watchdog.observers import Observer from oikb.watcher import _DebouncedHandler folder = pathlib.Path(tempfile.mkdtemp()) (folder / "doc.txt").write_text("one") syncs = [0] def sync(): # what a sync does: read every file syncs[0] += 1 for p in folder.iterdir(): p.read_bytes() obs = Observer(); obs.schedule(_DebouncedHandler(sync, debounce_seconds=0.2), str(folder)); obs.start() time.sleep(0.3) (folder / "doc.txt").write_text("two") # one real change time.sleep(3); obs.stop(); obs.join() print(syncs[0]) # 14 here; expected 1 ``` **Fix that works for us:** ignore the read-only event types in `on_any_event`. A real change still arrives as created/modified/deleted/moved (and `closed` after a write). ```python if event.is_directory: return if event.event_type in ("opened", "closed_no_write"): return ``` With that change the repro prints 1.

bsakaguchi referenced this issue

2026-10-18 21:14:21 +00:00

[PR #80] [CLOSED] feat(bookstack): support syncing pages from a specific book #101

bsakaguchi referenced this issue

2026-10-18 21:14:23 +00:00

[PR #84] [CLOSED] feat(bookstack): add configurable export options #104