morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5015] [Bug] Linux Hopper can finish analysis but fail Python bootstrap without the embedded runtime #3088

New issue

Closed

opened 2026-10-31 00:36:27 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-31 00:36:27 +00:00

Owner

Originally created by @gensokyo-dev on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5015

Before submitting

Searched related open/closed Hopper Python and libpython reports, including #494; that macOS overlay report differs from this failure.
Removed credentials, private paths and sensitive target data.

Area / interface / stage

Setup or provider admission; native analysis through production stdio MCP; earliest visible failure is native Python bootstrap.

Observed behavior

Runtime observation in a privately bootstrapped Ubuntu 24.04 environment, not an execution of rea setup. With the official authenticated Hopper 6.6.0 demo, Python 3.12.3, Qt6, Xvfb/Xauth, libX11 and libXtst present, the original bridge timed out after 120 seconds. A screenshot of the owned private display showed completed ELF background analysis followed by Cannot launch script for the REA bootstrap. The bootstrap file existed with mode 0600; the process had no Python shared library mapped.

Adding only Ubuntu's libpython3.12t64 (3.12.3-1ubuntu0.17) made the unchanged baseline complete the same public MCP control workflow. Subsequent baseline SQLite workflows also completed. Launcher bytes and the target digest remained unchanged.

Reproduction actually run

Extract the publisher-authenticated Hopper package and configure a private Ubuntu Base 24.04.5 image with its declared dependencies plus REA's X11 demo-helper packages.
Open an authored 17,360-byte x86_64 ELF through open_binary, then request search_strings. Two unchanged-baseline attempts timed out; one included a read-only capture of the owned private X display confirming the native analysis/script failure above.
Install libpython3.12t64 only inside that image; repeat the same unchanged-baseline control. It passed search, decompilation and verified cleanup.

Expected behavior

The supported headless setup route should supply the matching embedded Python runtime or identify its absence with an actionable dependency outcome. A completed native analysis should not leave callers diagnosing Python bootstrap failure solely from a generic readiness timeout.

Impact / scope

Observed on this minimal Ubuntu image. The default hosted Ubuntu runner may already have the shared library through unrelated packages; that has not been tested here. Actual rea setup on a fresh machine, other distro/package families and Hopper's precise library-discovery mechanism: NOT_ENOUGH_INFO. No general Linux outage or measured regression is claimed.

Version and evidence

REA 6.7.0, c12d9eda87; Node 24.18.0; x86_64 Linux; user-mapped private Ubuntu namespace; one CPU, 8 GiB family cap. Official Hopper launcher SHA-256: b2271008fb82bf14cfae5c3644cdde2b11fc6804bd7c99d67032ec1b1942d53c.

Source-derived follow-up: linuxPackageManagerCommands installs python3 and the X11 helper libraries for Debian-family hosts, but does not explicitly select an embedded Python shared-library package. This is a hypothesis for the supported installer gap, rather than a claim that the full installer was reproduced. Determine the provider's actual ABI/discovery requirements before choosing a portable fix.

Typed outcome / next checks

Proposed: establish a clean real installer reproduction; distinguish missing embedding runtime from ongoing analysis; verify matching runtime installation, explicit doctor scope, cancellation and owned cleanup. Preserve the current publisher-verification and normal demo-activation boundaries.

Originally created by @gensokyo-dev on GitHub (Oct 18, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5015 ### Before submitting - [x] Searched related open/closed Hopper Python and libpython reports, including #494; that macOS overlay report differs from this failure. - [x] Removed credentials, private paths and sensitive target data. ### Area / interface / stage Setup or provider admission; native analysis through production stdio MCP; earliest visible failure is native Python bootstrap. ### Observed behavior **Runtime observation in a privately bootstrapped Ubuntu 24.04 environment, not an execution of `rea setup`.** With the official authenticated Hopper 6.6.0 demo, Python 3.12.3, Qt6, Xvfb/Xauth, libX11 and libXtst present, the original bridge timed out after 120 seconds. A screenshot of the owned private display showed completed ELF background analysis followed by `Cannot launch script` for the REA bootstrap. The bootstrap file existed with mode 0600; the process had no Python shared library mapped. Adding only Ubuntu's `libpython3.12t64` (3.12.3-1ubuntu0.17) made the unchanged baseline complete the same public MCP control workflow. Subsequent baseline SQLite workflows also completed. Launcher bytes and the target digest remained unchanged. ### Reproduction actually run 1. Extract the publisher-authenticated Hopper package and configure a private Ubuntu Base 24.04.5 image with its declared dependencies plus REA's X11 demo-helper packages. 2. Open an authored 17,360-byte x86_64 ELF through `open_binary`, then request `search_strings`. Two unchanged-baseline attempts timed out; one included a read-only capture of the owned private X display confirming the native analysis/script failure above. 3. Install `libpython3.12t64` only inside that image; repeat the same unchanged-baseline control. It passed search, decompilation and verified cleanup. ### Expected behavior The supported headless setup route should supply the matching embedded Python runtime or identify its absence with an actionable dependency outcome. A completed native analysis should not leave callers diagnosing Python bootstrap failure solely from a generic readiness timeout. ### Impact / scope Observed on this minimal Ubuntu image. The default hosted Ubuntu runner may already have the shared library through unrelated packages; that has not been tested here. Actual `rea setup` on a fresh machine, other distro/package families and Hopper's precise library-discovery mechanism: `NOT_ENOUGH_INFO`. No general Linux outage or measured regression is claimed. ### Version and evidence REA 6.7.0, c12d9eda87aa5a022ee7e18ea96d8d03bfd12e96; Node 24.18.0; x86_64 Linux; user-mapped private Ubuntu namespace; one CPU, 8 GiB family cap. Official Hopper launcher SHA-256: `b2271008fb82bf14cfae5c3644cdde2b11fc6804bd7c99d67032ec1b1942d53c`. Source-derived follow-up: [`linuxPackageManagerCommands`](https://github.com/morluto/rea/blob/c12d9eda87aa5a022ee7e18ea96d8d03bfd12e96/src/application/LinuxHopper.ts#L550) installs `python3` and the X11 helper libraries for Debian-family hosts, but does not explicitly select an embedded Python shared-library package. This is a hypothesis for the supported installer gap, rather than a claim that the full installer was reproduced. Determine the provider's actual ABI/discovery requirements before choosing a portable fix. ### Typed outcome / next checks Proposed: establish a clean real installer reproduction; distinguish missing embedding runtime from ongoing analysis; verify matching runtime installation, explicit doctor scope, cancellation and owned cleanup. Preserve the current publisher-verification and normal demo-activation boundaries.

bsakaguchi

2026-10-31 00:36:27 +00:00

closed this issue
added the
triage

bug
labels

bsakaguchi commented

2026-10-31 00:36:28 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 18, 2026):

复现很清晰：同一基线只补装 libpython3.12t64 就走通，说明 REA 在 Linux 上对 Hopper 的 Python bootstrap 存在隐式运行时依赖——分析已完成，卡点纯粹在启动阶段。建议把嵌入式 libpython 纳入 doctor/setup 的依赖预检（对齐现有 X11 demo-helper 包的处理方式），并在文档中写明干净环境需要该包，否则用户会先撞上 "Cannot launch script" 而不是分析本身的报错，排查成本偏高。

@XinwangTao commented on GitHub (Oct 18, 2026): 复现很清晰：同一基线只补装 libpython3.12t64 就走通，说明 REA 在 Linux 上对 Hopper 的 Python bootstrap 存在隐式运行时依赖——分析已完成，卡点纯粹在启动阶段。建议把嵌入式 libpython 纳入 doctor/setup 的依赖预检（对齐现有 X11 demo-helper 包的处理方式），并在文档中写明干净环境需要该包，否则用户会先撞上 "Cannot launch script" 而不是分析本身的报错，排查成本偏高。

bsakaguchi commented

2026-10-31 00:36:29 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 18, 2026):

Investigation checkpoint at 63c970812b8b5c6664400c9360e22c427844820c:

The authenticated Hopper 6.6.0 Deb control lists libc/Qt/X11-related dependencies but no Python shared runtime. REA’s current planned APT command adds python3, but not libpython; this is source/package metadata evidence, not yet a successful installer reproduction. Ubuntu’s retained libpython3.12t64 package provides the libpython3.12 virtual package.
A real public rea setup --yes --install-hopper attempt invoked APT and vendor post-install scripts in a single-ID user namespace, but fontconfig/dbus ownership configuration failed because other UIDs/GIDs were unmapped. That fixture-specific failure is not a REA bug or a completed install.
A clean Ubuntu Base container with full UID/GID support is prepared, with separately bounded native/client scopes and tested success/failure/deadline cleanup. The actual container setup probe has not started: host memory repeatedly fell to 2–3 GiB, below safe admission. Its waiter and owned containers are now stopped, and local receipts remain retained.

Actual clean-installer qualification and any dependency fix remain pending. The earlier controlled minimal-runtime observation remains the runtime evidence; no broader installer or platform claim is added.

@gensokyo-dev commented on GitHub (Oct 18, 2026): Investigation checkpoint at `63c970812b8b5c6664400c9360e22c427844820c`: - The authenticated Hopper 6.6.0 Deb control lists libc/Qt/X11-related dependencies but no Python shared runtime. REA’s current planned APT command adds `python3`, but not `libpython`; this is source/package metadata evidence, not yet a successful installer reproduction. Ubuntu’s retained `libpython3.12t64` package provides the `libpython3.12` virtual package. - A real public `rea setup --yes --install-hopper` attempt invoked APT and vendor post-install scripts in a single-ID user namespace, but fontconfig/dbus ownership configuration failed because other UIDs/GIDs were unmapped. That fixture-specific failure is not a REA bug or a completed install. - A clean Ubuntu Base container with full UID/GID support is prepared, with separately bounded native/client scopes and tested success/failure/deadline cleanup. The actual container setup probe has **not started**: host memory repeatedly fell to 2–3 GiB, below safe admission. Its waiter and owned containers are now stopped, and local receipts remain retained. Actual clean-installer qualification and any dependency fix remain pending. The earlier controlled minimal-runtime observation remains the runtime evidence; no broader installer or platform claim is added.

bsakaguchi commented

2026-10-31 00:36:29 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 18, 2026):

Recheck at latest main 093ce695f0a7c3407c7fde520ed8b20c0981b408 (production tree unchanged):

The full-UID container now ran the actual public setup command. It exited after 5.47 s with download_failed, before APT; a minimal publisher/key fetch probe reproduced ECONNREFUSED. The forwarded host proxy uses loopback, which points at the container itself. This is fixture transport failure, not evidence of a REA installer defect. Both attempts verified payload cleanup and reported zero OOM events.
Correction to the previous local fixture checkpoint: its bootstrap apt-get update returned 0 with failed-index warnings. That does not prove a successful metadata refresh. The private bootstrap now requests APT::Update::Error-Mode=any; that corrected run is still pending.
Static package evidence: vendor post-install only registers desktop/MIME resources. Strings in the package's Hopper core include a sysconfig query for LIBDIR and INSTSONAME; this suggests checking the selected interpreter/shared-library boundary rather than hardcoding Python 3.12 for every supported Ubuntu release. The loader path still needs runtime qualification.
The host proxy accepts connections beyond loopback, so a bridge-gateway mapping can preserve the same transport without changing host configuration; this mapping is not yet verified. Host available memory fell to about 1.2 GiB before its probe was admitted. All owned audit containers and waiters are stopped, with receipts retained.

No production fix/PR or completed clean-install claim yet. The earlier controlled libpython3.12t64 intervention remains the real-provider evidence; the next steps are strict clean setup and native public MCP verification.

@gensokyo-dev commented on GitHub (Oct 18, 2026): Recheck at latest main `093ce695f0a7c3407c7fde520ed8b20c0981b408` (production tree unchanged): - The full-UID container now ran the actual public setup command. It exited after 5.47 s with `download_failed`, **before APT**; a minimal publisher/key fetch probe reproduced `ECONNREFUSED`. The forwarded host proxy uses loopback, which points at the container itself. This is fixture transport failure, not evidence of a REA installer defect. Both attempts verified payload cleanup and reported zero OOM events. - Correction to the previous local fixture checkpoint: its bootstrap `apt-get update` returned 0 with failed-index warnings. That does **not** prove a successful metadata refresh. The private bootstrap now requests `APT::Update::Error-Mode=any`; that corrected run is still pending. - Static package evidence: vendor post-install only registers desktop/MIME resources. Strings in the package's Hopper core include a `sysconfig` query for `LIBDIR` and `INSTSONAME`; this suggests checking the selected interpreter/shared-library boundary rather than hardcoding Python 3.12 for every supported Ubuntu release. The loader path still needs runtime qualification. - The host proxy accepts connections beyond loopback, so a bridge-gateway mapping can preserve the same transport without changing host configuration; this mapping is **not yet verified**. Host available memory fell to about 1.2 GiB before its probe was admitted. All owned audit containers and waiters are stopped, with receipts retained. No production fix/PR or completed clean-install claim yet. The earlier controlled `libpython3.12t64` intervention remains the real-provider evidence; the next steps are strict clean setup and native public MCP verification.

bsakaguchi commented

2026-10-31 00:36:29 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 18, 2026):

Clean-installer reproduction is now confirmed at 093ce695f0a7c3407c7fde520ed8b20c0981b408:

Fresh Ubuntu Base 24.04.5 package installation state, pinned Node 24.18.0, authenticated Hopper 6.6.0, default APT recommendations. APT indexes refreshed successfully with strict error handling; downloaded Deb caches were preloaded, so this is not a cold-network install benchmark.
The actual public rea setup --yes --install-hopper --format json completed with ready / installed_hopper (75.35 s). Afterwards dpkg --audit was empty and Python 3.12 was fully configured, but its reported INSTSONAME=libpython3.12.so.1.0 was absent.
A real public MCP task on the unchanged 17,360-byte control ELF failed with provider_timeout on search_strings. Both paired runs selected the existing public REA_HOPPER_STARTUP_TIMEOUT_MS=30000 option; the earlier default-120-second observations remain separate evidence.
Intervention: apt-get install -y libpython3.12 installed only libpython3.12t64 (3.12.3-1ubuntu0.17), upgrading/removing nothing. The same unchanged REA runtime and ELF then completed all 13 calls, including the string match and decompilation at 0x401126. Hopper returned (arg0 * 0x8 - arg0) + 0xd; independent GNU objdump shows lea *8, sub, add 0xd, ret. Target bytes were unchanged.
Both native MCP attempts verified payload cleanup, with zero OOM events. Guarded elapsed times were 49.54 s (failed) and 24.79 s (completed): these have different outcomes and are not a comparable-task speedup claim. Docker daemon costs are outside the measured container scope.

The owning fix is in progress: share a fresh, bounded system-Python runtime inspection between installer/doctor; select the matching Debian library from the configured distribution interpreter, verify loadability after APT, and preserve unknown/failed observations. The first three focused source files pass 70 tests. Candidate clean-install and real-provider verification are still pending.

@gensokyo-dev commented on GitHub (Oct 18, 2026): Clean-installer reproduction is now confirmed at `093ce695f0a7c3407c7fde520ed8b20c0981b408`: - Fresh Ubuntu Base 24.04.5 package installation state, pinned Node 24.18.0, authenticated Hopper 6.6.0, default APT recommendations. APT indexes refreshed successfully with strict error handling; downloaded Deb caches were preloaded, so this is **not a cold-network install benchmark**. - The actual public `rea setup --yes --install-hopper --format json` completed with `ready` / `installed_hopper` (75.35 s). Afterwards `dpkg --audit` was empty and Python 3.12 was fully configured, but its reported `INSTSONAME=libpython3.12.so.1.0` was absent. - A real public MCP task on the unchanged 17,360-byte control ELF failed with `provider_timeout` on `search_strings`. Both paired runs selected the existing public `REA_HOPPER_STARTUP_TIMEOUT_MS=30000` option; the earlier default-120-second observations remain separate evidence. - Intervention: `apt-get install -y libpython3.12` installed **only** `libpython3.12t64` (3.12.3-1ubuntu0.17), upgrading/removing nothing. The same unchanged REA runtime and ELF then completed all 13 calls, including the string match and decompilation at `0x401126`. Hopper returned `(arg0 * 0x8 - arg0) + 0xd`; independent GNU objdump shows `lea *8`, `sub`, `add 0xd`, `ret`. Target bytes were unchanged. - Both native MCP attempts verified payload cleanup, with zero OOM events. Guarded elapsed times were 49.54 s (failed) and 24.79 s (completed): these have different outcomes and **are not a comparable-task speedup claim**. Docker daemon costs are outside the measured container scope. The owning fix is in progress: share a fresh, bounded system-Python runtime inspection between installer/doctor; select the matching Debian library from the configured distribution interpreter, verify loadability after APT, and preserve unknown/failed observations. The first three focused source files pass 70 tests. Candidate clean-install and real-provider verification are still pending.

bsakaguchi referenced this issue

2026-10-31 01:07:55 +00:00

[PR #3087] [CLOSED] fix(pwntools): surface actionable configuration remediation #5605

bsakaguchi referenced this issue

2026-10-31 01:07:56 +00:00

[PR #3088] [CLOSED] fix(pwntools): report actionable configuration recovery #5608