[GH-ISSUE #4227] Nightly build/test failure - 2026-09-18 #1062

New issue

Open

opened 2026-10-29 21:22:27 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:22:27 +00:00

Owner

Originally created by @github-actions[bot] on GitHub (Sep 18, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4227

Nightly Failure Report

Run: https://github.com/kvcache-ai/Mooncake/actions/runs/35245246893
Branch: refs/heads/main
Timestamp: 2026-09-18T16:11:37.550Z

Please investigate the failed jobs in the workflow run linked above.

Originally created by @github-actions[bot] on GitHub (Sep 18, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4227 ## Nightly Failure Report **Run**: https://github.com/kvcache-ai/Mooncake/actions/runs/35245246893 **Branch**: refs/heads/main **Timestamp**: 2026-09-18T16:11:37.550Z Please investigate the failed jobs in the workflow run linked above.

bsakaguchi added the

nightly-failure

label

2026-10-29 21:22:27 +00:00

bsakaguchi commented

2026-10-29 21:22:27 +00:00

Author

Owner

@he-yufeng commented on GitHub (Sep 19, 2026):

Triage of the three red lanes, all on 3e626f20 (which contains c57745c3, the #4150 link fix):

nightly-test fails at Run CTest unit tests: FilereadWorkerPoolTest.AcceptsTypedTrailingWhitespaceAndCaches again. That is #4213, and the fix is #4210 (still open). Expected until that lands.
nightly-coverage fails at Run Go store binding sanitizer integration tests, but not at linking: no undefined reference anywhere in the log. The Go binary links and starts, then the master dies with Failed to start HTTP metadata server on 0.0.0.0:8080 ("master exited before becoming ready"). That is a port-bind/startup flake, a different root cause from the link issue. So the #4150 fix holds — the link step is past it now, which also closes the loop on #4202.
build-wheels (cuda13, 3.11) shows no failing step of its own; it reads as a cascade of the two above through nightly-gate.

The new signal worth tracking is the 8080 bind failure. If it recurs on the next nightly, the Go sanitizer job probably needs a unique port per run (or a retry when the port is taken); happy to look if it shows up again.

@he-yufeng commented on GitHub (Sep 19, 2026): Triage of the three red lanes, all on `3e626f20` (which contains `c57745c3`, the #4150 link fix): - **nightly-test** fails at `Run CTest unit tests`: `FilereadWorkerPoolTest.AcceptsTypedTrailingWhitespaceAndCaches` again. That is #4213, and the fix is #4210 (still open). Expected until that lands. - **nightly-coverage** fails at `Run Go store binding sanitizer integration tests`, but not at linking: no `undefined reference` anywhere in the log. The Go binary links and starts, then the master dies with `Failed to start HTTP metadata server on 0.0.0.0:8080` ("master exited before becoming ready"). That is a port-bind/startup flake, a different root cause from the link issue. So the #4150 fix holds — the link step is past it now, which also closes the loop on #4202. - **build-wheels (cuda13, 3.11)** shows no failing step of its own; it reads as a cascade of the two above through `nightly-gate`. The new signal worth tracking is the 8080 bind failure. If it recurs on the next nightly, the Go sanitizer job probably needs a unique port per run (or a retry when the port is taken); happy to look if it shows up again.