[GH-ISSUE #117] Failed to work with https://docs.zeroclaw.com/v0.8.5/en/ #70

New issue

Open

opened 2026-10-18 21:14:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-18 21:14:05 +00:00

Owner

Originally created by @luisimasd on GitHub (Sep 7, 2026).
Original GitHub issue: https://github.com/open-webui/oikb/issues/117

Basically it starts relatively ok grabbing pages from https://docs.zeroclaw.com/v0.8.5/en/ but it changes to https://docs.zeroclaw.com/v0.8.5/... building the rest of the links based on that last url, leading to multiple 404 errors.

Noy familiar with python but playing with an IA it mentioned....

Using... docker exec oikb sh -c 'sed -n "1,240p" /app/src/oikb/connectors/web.py'
It mentioned that the problem is around "self.url = url.rstrip("/")" and it should be "self.url = url if url.endswith("/") else url + "/""

Since I am not familiar with python I am not sure if that is correct or not, I leave that to the talented people here who can find out what is going and eventually, if a fix is required.

Environment is:

# docker exec oikb oikb --version
oikb, version 0.4.0

Config file (as of now):

defaults:
  interval: 8h
  concurrency: 2
  filter:
    max-size: 50mb
#  notify:
#    url: https://hooks.slack.com/services/T.../B.../xxx
#    on: error

sources:
  - name: zeroclaw-docs
    source: web:https://docs.zeroclaw.com/v0.8.5/en
    kb-id: 3a7b0abe-3924-4d13-915b-4d3b6b80563a

Example of the issue observed:

root@auto3:/opt/appdata# docker compose up  --force-recreate oikb
[+] Running 1/1
 ✔ Container oikb  Recreated                                                                                                    0.9s
Attaching to oikb
oikb  | oikb daemon listening on port 8080
oikb  |   1 source(s) configured
oikb  |   Auth: disabled (set OIKB_API_KEY to enable)
oikb  |   GET  http://localhost:8080/health
oikb  |   GET  http://localhost:8080/metrics
oikb  |   GET  http://localhost:8080/history
oikb  |   POST http://localhost:8080/sync/{kb_id}
oikb  |
oikb  |   OpenAPI spec: http://localhost:8080/openapi.json
oikb  |   Add as Tool Server in Open WebUI → Settings → Connections
oikb  | 2026-09-07 18:18:15,360 INFO     oikb.daemon  Scheduling web:https://docs.zeroclaw.com/v0.8.5/en every 28800s
oikb  | 2026-09-07 18:18:16,771 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/sitemap.xml "HTTP/1.1 404 Not Found"
oikb  | 2026-09-07 18:18:16,863 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/en "HTTP/1.1 301 Moved Permanently                                   "
oikb  | 2026-09-07 18:18:16,956 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/en/ "HTTP/1.1 200 OK"
oikb  | 2026-09-07 18:18:17,544 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/favicon.svg "HTTP/1.1 200 OK"
oikb  | 2026-09-07 18:18:18,139 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/favicon.png "HTTP/1.1 200 OK"
**oikb  | 2026-09-07 18:18:18,646 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/css/variables-8adf115d.css "HTTP/1                                   .1 404 Not Found"
oikb  | 2026-09-07 18:18:18,651 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/css/general-e96d0476.css "HTTP/1.1                                    404 Not Found"
oikb  | 2026-09-07 18:18:18,739 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/css/chrome.css "HTTP/1.1 200 OK"
oikb  | 2026-09-07 18:18:19,244 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/fonts/fonts-9644e21d.css "HTTP/1.1                                    404 Not Found"
oikb  | 2026-09-07 18:18:19,249 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/highlight-493f70e1.css "HTTP/1.1 4                                   04 Not Found"
oikb  | 2026-09-07 18:18:19,253 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/tomorrow-night-4c0ae647.css "HTTP/                                   1.1 404 Not Found"
oikb  | 2026-09-07 18:18:19,256 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/ayu-highlight-3fdfc3ac.css "HTTP/1                                   .1 404 Not Found"**
oikb  | 2026-09-07 18:18:19,358 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/theme/pc-themes.css "HTTP/1.1 200                                    OK"
oikb  | 2026-09-07 18:18:19,950 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/theme/custom.css "HTTP/1.1 200 OK                                   "
**oikb  | 2026-09-07 18:18:20,456 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/philosophy/index.html "HTTP/1.1 40                                   4 Not Found"**

Originally created by @luisimasd on GitHub (Sep 7, 2026). Original GitHub issue: https://github.com/open-webui/oikb/issues/117 Basically it starts relatively ok grabbing pages from https://docs.zeroclaw.com/v0.8.5/en/ but it changes to https://docs.zeroclaw.com/v0.8.5/... building the rest of the links based on that last url, leading to multiple 404 errors. Noy familiar with python but playing with an IA it mentioned.... Using... `docker exec oikb sh -c 'sed -n "1,240p" /app/src/oikb/connectors/web.py'` It mentioned that the problem is around "`self.url = url.rstrip("/")`" and it should be "`self.url = url if url.endswith("/") else url + "/"`" Since I am not familiar with python I am not sure if that is correct or not, I leave that to the talented people here who can find out what is going and eventually, if a fix is required. Environment is: ``` # docker exec oikb oikb --version oikb, version 0.4.0 ``` Config file (as of now): ``` defaults: interval: 8h concurrency: 2 filter: max-size: 50mb # notify: # url: https://hooks.slack.com/services/T.../B.../xxx # on: error sources: - name: zeroclaw-docs source: web:https://docs.zeroclaw.com/v0.8.5/en kb-id: 3a7b0abe-3924-4d13-915b-4d3b6b80563a ``` Example of the issue observed: ``` root@auto3:/opt/appdata# docker compose up --force-recreate oikb [+] Running 1/1 ✔ Container oikb Recreated 0.9s Attaching to oikb oikb | oikb daemon listening on port 8080 oikb | 1 source(s) configured oikb | Auth: disabled (set OIKB_API_KEY to enable) oikb | GET http://localhost:8080/health oikb | GET http://localhost:8080/metrics oikb | GET http://localhost:8080/history oikb | POST http://localhost:8080/sync/{kb_id} oikb | oikb | OpenAPI spec: http://localhost:8080/openapi.json oikb | Add as Tool Server in Open WebUI → Settings → Connections oikb | 2026-09-07 18:18:15,360 INFO oikb.daemon Scheduling web:https://docs.zeroclaw.com/v0.8.5/en every 28800s oikb | 2026-09-07 18:18:16,771 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/sitemap.xml "HTTP/1.1 404 Not Found" oikb | 2026-09-07 18:18:16,863 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/en "HTTP/1.1 301 Moved Permanently " oikb | 2026-09-07 18:18:16,956 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/en/ "HTTP/1.1 200 OK" oikb | 2026-09-07 18:18:17,544 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/_shared/favicon.svg "HTTP/1.1 200 OK" oikb | 2026-09-07 18:18:18,139 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/_shared/favicon.png "HTTP/1.1 200 OK" **oikb | 2026-09-07 18:18:18,646 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/css/variables-8adf115d.css "HTTP/1 .1 404 Not Found" oikb | 2026-09-07 18:18:18,651 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/css/general-e96d0476.css "HTTP/1.1 404 Not Found" oikb | 2026-09-07 18:18:18,739 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/_shared/css/chrome.css "HTTP/1.1 200 OK" oikb | 2026-09-07 18:18:19,244 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/fonts/fonts-9644e21d.css "HTTP/1.1 404 Not Found" oikb | 2026-09-07 18:18:19,249 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/highlight-493f70e1.css "HTTP/1.1 4 04 Not Found" oikb | 2026-09-07 18:18:19,253 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/tomorrow-night-4c0ae647.css "HTTP/ 1.1 404 Not Found" oikb | 2026-09-07 18:18:19,256 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/ayu-highlight-3fdfc3ac.css "HTTP/1 .1 404 Not Found"** oikb | 2026-09-07 18:18:19,358 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/_shared/theme/pc-themes.css "HTTP/1.1 200 OK" oikb | 2026-09-07 18:18:19,950 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/_shared/theme/custom.css "HTTP/1.1 200 OK " **oikb | 2026-09-07 18:18:20,456 INFO httpx HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/philosophy/index.html "HTTP/1.1 40 4 Not Found"** ```

bsakaguchi referenced this issue

2026-10-18 21:14:25 +00:00

[PR #99] [MERGED] fix: recurse into Google Drive subfolders #110