[GH-ISSUE #33] Bug: Root-level files not uploaded during sync of local directory #19

New issue

Open

opened 2026-10-18 21:13:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:13:32 +00:00

Owner

Originally created by @bubgigant on GitHub (Jun 6, 2026).
Original GitHub issue: https://github.com/open-webui/oikb/issues/33

Description

When syncing a local directory to an Open WebUI Knowledge Base, files at the source root (i.e. not inside any subdirectory) are silently skipped. Only files within subdirectories are uploaded.

Repro

Create an OWUI Knowledge Base
Prepare a local directory with files both at the root and inside subdirectories:

/vault/
├── Documentation-X.md          ← root-level (NOT synced)
├── Note-Y.md               ← root-level (NOT synced)
├── Reference-Z.md         ← root-level (NOT synced)
├── Clippings/
│   ├── Ratatouille.md         ← in subdirectory (synced ✓)
│   └── Gazpacho.md            ← in subdirectory (synced ✓)
├── Templates/
│   └── template_Note.md       ← in subdirectory (synced ✓)
└── Excalidraw/
    └── diagram.excalidraw.md  ← in subdirectory (synced ✓)

Configure .oikb.yaml:

defaults:
  interval: 8h
  concurrency: 4

sources:
  - name: obsidian-vault
    source: /vault
    kb-id: <kb-id>
    filter:
      include: ["**/*.md"]
      exclude: [".obsidian/**", ".trash/**", ".git/**"]

Run oikb daemon or oikb sync

Expected

All .md files are uploaded, including those at the root of /vault/.

Observed

Only files inside subdirectories (Clippings, Templates, Excalidraw) are uploaded. Root-level files are silently skipped.

Sync history confirms the discrepancy:

Total .md files (excluding hidden dirs): 282
Files at root (maxdepth 1): 141
Files actually uploaded by oikb: 29 (all inside subdirectories)

Sync history entry:
  files_added: 29
  unmodified: 0
  status: success (with 2 errors for specific files)

Analysis from source code

In sync.py, the upload logic resolves the target directory like this:

directory_id = directory_map.get(path) if path else None
client.upload_file(
    file_content=content,
    filename=filename,
    kb_id=kb_id,
    file_hash=manifest_entry.checksum,
    directory_id=directory_id,
)

For root-level files, path is an empty string "", so directory_id resolves to None. The question is whether:

The manifest builder (connector.build_manifest()) correctly includes root-level files with path="", or
The OWUI /sync/diff endpoint properly returns root-level files in its diff response, or
The file upload with directory_id=None silently fails or is rejected by the OWUI API

The directory_map from the diff response likely only contains mappings for subdirectories (populated from dirs/create calls). Root-level files hit directory_map.get("") → None, which may cause the upload to be skipped or rejected without error logging.

Environment

oikb: v0.3.5 (Docker: ghcr.io/open-webui/oikb:latest)
Open WebUI: v0.9.6
Source: local directory (/vault, mounted from host)
OS: Unraid (Linux, Docker)
Architecture: amd64

Additional context

Syncing works correctly for all files inside subdirectories
No error messages appear in logs for root-level files — they are silently skipped
The oikb reset command was used between sync attempts to clear the KB, confirming this is not an incremental diff issue
Related but distinct from #24 (which reports the opposite: directory structure flattening to root)

Originally created by @bubgigant on GitHub (Jun 6, 2026). Original GitHub issue: https://github.com/open-webui/oikb/issues/33 ### Description When syncing a local directory to an Open WebUI Knowledge Base, files at the source root (i.e. not inside any subdirectory) are silently skipped. Only files within subdirectories are uploaded. ### Repro 1. Create an OWUI Knowledge Base 2. Prepare a local directory with files both at the root and inside subdirectories: ``` /vault/ ├── Documentation-X.md ← root-level (NOT synced) ├── Note-Y.md ← root-level (NOT synced) ├── Reference-Z.md ← root-level (NOT synced) ├── Clippings/ │ ├── Ratatouille.md ← in subdirectory (synced ✓) │ └── Gazpacho.md ← in subdirectory (synced ✓) ├── Templates/ │ └── template_Note.md ← in subdirectory (synced ✓) └── Excalidraw/ └── diagram.excalidraw.md ← in subdirectory (synced ✓) ``` 3. Configure `.oikb.yaml`: ```yaml defaults: interval: 8h concurrency: 4 sources: - name: obsidian-vault source: /vault kb-id: <kb-id> filter: include: ["**/*.md"] exclude: [".obsidian/**", ".trash/**", ".git/**"] ``` 4. Run `oikb daemon` or `oikb sync` ### Expected All `.md` files are uploaded, including those at the root of `/vault/`. ### Observed Only files inside subdirectories (Clippings, Templates, Excalidraw) are uploaded. Root-level files are silently skipped. **Sync history confirms the discrepancy:** - Total `.md` files (excluding hidden dirs): **282** - Files at root (maxdepth 1): **141** - Files actually uploaded by oikb: **29** (all inside subdirectories) ``` Sync history entry: files_added: 29 unmodified: 0 status: success (with 2 errors for specific files) ``` ### Analysis from source code In `sync.py`, the upload logic resolves the target directory like this: ```python directory_id = directory_map.get(path) if path else None client.upload_file( file_content=content, filename=filename, kb_id=kb_id, file_hash=manifest_entry.checksum, directory_id=directory_id, ) ``` For root-level files, `path` is an empty string `""`, so `directory_id` resolves to `None`. The question is whether: 1. **The manifest builder** (`connector.build_manifest()`) correctly includes root-level files with `path=""`, or 2. **The OWUI `/sync/diff` endpoint** properly returns root-level files in its diff response, or 3. **The file upload** with `directory_id=None` silently fails or is rejected by the OWUI API The `directory_map` from the diff response likely only contains mappings for subdirectories (populated from `dirs/create` calls). Root-level files hit `directory_map.get("") → None`, which may cause the upload to be skipped or rejected without error logging. ### Environment - **oikb**: v0.3.5 (Docker: `ghcr.io/open-webui/oikb:latest`) - **Open WebUI**: v0.9.6 - **Source**: local directory (`/vault`, mounted from host) - **OS**: Unraid (Linux, Docker) - **Architecture**: amd64 ### Additional context - Syncing works correctly for all files inside subdirectories - No error messages appear in logs for root-level files — they are silently skipped - The `oikb reset` command was used between sync attempts to clear the KB, confirming this is not an incremental diff issue - Related but distinct from #24 (which reports the opposite: directory structure flattening to root)

bsakaguchi referenced this issue

2026-10-18 21:14:14 +00:00

[PR #19] [MERGED] feat: add Nextcloud connector #83