morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7541] [Bug] Native call observation reports complete coverage after omitting the 65th breakpoint location #4485

New issue

Closed

opened 2026-10-31 00:50:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:50:11 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7541

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

A real node scripts/rea.mjs observe-native-calls <fixture> <request> --json call against a locally compiled Mach-O with 65 implementations of one Objective-C selector returned location_count: 65 but only 64 entries in breakpoints[0].locations. Its coverage.status was complete, and coverage.breakpoint_locations_truncated was absent. The target exited normally with status 0, one call event was recorded, and neither the event limit nor the resource limit was reached.

The request was:

{"breakpoints":[{"kind":"objc-method","selector":"c55Probe","method_type":"instance"}],"duration_ms":5000,"max_events":100,"argument_registers":0}

The result did include a human-readable limitation explaining that only the first 64 locations are listed. An otherwise equivalent fixture with 64 implementations returned all 64 records and no count-cap limitation. Thus the omission is visible through the count/list mismatch and prose, but it is classified as complete in structured coverage.

Reproduction

Use the source revision below with its locked dependencies and compiled CLI available, Node.js 24.22.1 on PATH, and macOS command-line tools providing clang and LLDB. Run the following script from the checkout root. It compiles two console executables, launches only those executables through REA, isolates product state, and deletes its temporary fixture directory afterward.

python3 <<'PY'
import json, os, pathlib, subprocess, tempfile

cli = pathlib.Path('scripts/rea.mjs').resolve()
with tempfile.TemporaryDirectory(prefix='rea-locations-') as directory:
    root = pathlib.Path(directory).resolve()
    env = dict(os.environ)
    for key in ('HOME', 'XDG_CONFIG_HOME', 'XDG_DATA_HOME', 'XDG_CACHE_HOME',
                'XDG_STATE_HOME', 'XDG_RUNTIME_DIR', 'TMPDIR'):
        child = root / key.lower()
        child.mkdir()
        env[key] = str(child)
    request = json.dumps({
        'breakpoints': [{'kind': 'objc-method', 'selector': 'c55Probe',
                         'method_type': 'instance'}],
        'duration_ms': 5000, 'max_events': 100, 'argument_registers': 0,
    })
    for count in (64, 65):
        source = '#import <Foundation/Foundation.h>\n'
        for index in range(count):
            source += (f'@interface Probe{index} : NSObject\n'
                       '- (void)c55Probe;\n@end\n'
                       f'@implementation Probe{index}\n'
                       '- (void)c55Probe {}\n@end\n')
        source += ('int main(void) { @autoreleasepool { '
                   '[[Probe0 new] c55Probe]; } return 0; }\n')
        fixture = root / f'fixture-{count}'
        source_path = fixture.with_suffix('.m')
        source_path.write_text(source)
        subprocess.run(['clang', '-g', '-O0', '-framework', 'Foundation',
                        '-o', str(fixture), str(source_path)],
                       env=env, check=True, timeout=30)
        for attempt in range(1, 3 if count == 65 else 2):
            result = subprocess.run(
                ['node', str(cli), 'observe-native-calls', str(fixture),
                 request, '--json'], env=env, capture_output=True,
                text=True, timeout=45, check=True)
            value = json.loads(result.stdout)['normalized_result']
            breakpoint = value['breakpoints'][0]
            print(json.dumps({
                'fixture': count, 'attempt': attempt,
                'cli_exit': result.returncode,
                'outcome': value['process']['outcome'],
                'exit_status': value['process']['exit_status'],
                'location_count': breakpoint['location_count'],
                'locations_length': len(breakpoint['locations']),
                'coverage': value['coverage'], 'events': len(value['events']),
                'count_limit_notice': any('first 64' in item
                                         for item in value['limitations']),
            }), flush=True)
PY

Compare location_count, locations_length, coverage and count_limit_notice in the three output lines. The control returns 64/64; both 65-method runs return 65/64 with complete coverage and no truncation property.

The reduced 65-method case reproduced in two initial runs and both repeats of the exact script above. Both independently executed 64-method controls returned all 64 locations. Two additional real CLI calls using the broad Foundation init selector each returned 1,629 matches and 64 listed locations with complete coverage.

Expected behavior

The documented resolved-location inventory should include all 65 matching methods when its metadata is far below the documented budget. If REA intentionally retains only 64 records, its completeness classification should explicitly reflect the omitted metadata rather than declare the result complete. Normal process exit should remain distinguishable from location-metadata truncation.

The native-observation guide promises a resolved-location inventory for each requested breakpoint and describes a separate aggregate 8 MiB metadata budget. When that budget causes omissions, the guide specifies the full match count, partial coverage and a true truncation flag. It does not document an additional 64-record cap. Exact locators are in Evidence below.

The current schema comment defines breakpoint_locations_truncated specifically for byte-budget exhaustion. This report does not claim a separate documented promise that the same flag already covers the count cap; using it for all omitted location metadata is one possible correction.

Impact and scope

P2: callers receive an incomplete address/symbol inventory with a misleading structured completeness classification for a valid selector request. In the reduced fixture, -[Probe64 c55Probe] was missing from the returned inventory. All 64 retained records belonged to the selected executable, so matches in unrelated images do not explain the result. The retained location array serialized to 15,478 bytes, well below 8 MiB.

Callers can detect the omission by comparing each locations-array length with its location_count and conservatively treating the inventory as incomplete. The emitted limitation also warns human readers. This workaround detects the problem but does not recover the omitted address records. The executions did not establish lost call events or a failure to install breakpoints; the demonstrated defect is in location reporting and completeness. MCP and other architectures were not exercised.

REA version

Exact stdout from node scripts/rea.mjs --version:

6.8.0

The output ends with a newline. Source baseline: 65f1c447344a423fa2785847dbcea1e9a9e2454a.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1; arm64. /usr/bin/lldb --version reported lldb-2103.0.34.103 and Apple Swift version 6.4 (swiftlang-6.4.0.34.1 clang-2100.3.34.1).

Target and artifact context

Source-authored console Mach-O executables linked against Foundation. Each executable defines 64 or 65 NSObject subclasses with the same instance selector and invokes the first class's method once. No application bundle, GUI launch, external process attachment or network fixture was used.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The exact reproduction script printed:

{"fixture": 64, "attempt": 1, "cli_exit": 0, "outcome": "exited", "exit_status": 0, "location_count": 64, "locations_length": 64, "coverage": {"status": "complete", "event_limit_reached": false, "resource_limit_reached": false, "unresolved_breakpoints": []}, "events": 1, "count_limit_notice": false}
{"fixture": 65, "attempt": 1, "cli_exit": 0, "outcome": "exited", "exit_status": 0, "location_count": 65, "locations_length": 64, "coverage": {"status": "complete", "event_limit_reached": false, "resource_limit_reached": false, "unresolved_breakpoints": []}, "events": 1, "count_limit_notice": true}
{"fixture": 65, "attempt": 2, "cli_exit": 0, "outcome": "exited", "exit_status": 0, "location_count": 65, "locations_length": 64, "coverage": {"status": "complete", "event_limit_reached": false, "resource_limit_reached": false, "unresolved_breakpoints": []}, "events": 1, "count_limit_notice": true}

The complete initial reduced-case Evidence showed -[Probe64 c55Probe] absent in both 65-method location lists; the control contained every fixture method. Both cases recorded the single invocation of -[Probe0 c55Probe].

At the source baseline:

docs/native-investigation.md:979-980 describes the resolved-location inventory. Lines 987-990 specify the 8 MiB metadata limit and its partial-coverage/truncation reporting.
bridge/native/rea_lldb_tracer.py:29 defines the 64-record reporting cap. Lines 494-523 count every matching location but retain only the first 64; only the byte-budget branches set the returned truncation state.
src/native/NativeCallObservation.ts:128-135 derives partial coverage without checking whether fewer location records were retained than counted. Lines 178-183 emit the public truncation property only when the bridge's flag is true. Lines 218-223 add the count-cap limitation separately.
src/domain/native/nativeCallObservation.ts:218-219 currently describes the optional flag as aggregate location-metadata byte-budget exhaustion.
src/cli/nativeCallCommands.ts:23-57 validates and dispatches this public CLI request. src/native/NativeCallObservation.ts:45-100 performs target/input admission, invokes the tracer and projects its result. Successful real CLI executions, rather than only test calls, establish reachability.

Typed outcome or recovery expected

Return the complete location inventory within the documented metadata budget, or return explicit partial coverage for the omitted metadata. Preserve the actual exited-process outcome. The existing full location_count already lets callers calculate the number of omitted records; a new omission-count field is not required to establish this defect.

Analysis or suspected cause

Source-derived: the bridge has two independent reporting bounds, but its truncation state covers only the aggregate byte budget. Once the per-breakpoint list reaches 64 entries, later matches still increase the count while bypassing retention and the truncation branches. The result projection recognizes the mismatch for limitation text without including it in the completeness decision. The 64/65 public CLI comparison corroborates this source explanation.

Additional context

Duplicate searches covered open and closed issues, PRs and issue comments. Related reports have different closure conditions: #7448 concerns empty-target admission; #5306 concerns native UI PID/window classification; #5913 concerns completed facts lost during cancellation/cleanup; #2382 requests source coordinates. #5298 and withdrawn #5299 explicitly reject treating matches outside the executable as missed coverage. This report demonstrates omitted methods within the executable itself and does not revive that theory. No prior report or fix for this per-breakpoint count-cap completeness mismatch was found.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Real CLI runs omitted one of 65 fixture-defined method locations while returning complete coverage; the omission affects returned metadata, with no observed severe loss or blocked tracing workflow.
Workaround status: available
Workaround basis: The observed full location_count and shorter locations array let a consumer detect omissions and conservatively mark metadata incomplete; the limitation also discloses the cap. This detects incompleteness but cannot reconstruct omitted location details from that result.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7541 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior A real `node scripts/rea.mjs observe-native-calls <fixture> <request> --json` call against a locally compiled Mach-O with 65 implementations of one Objective-C selector returned `location_count: 65` but only 64 entries in `breakpoints[0].locations`. Its `coverage.status` was `complete`, and `coverage.breakpoint_locations_truncated` was absent. The target exited normally with status 0, one call event was recorded, and neither the event limit nor the resource limit was reached. The request was: ```json {"breakpoints":[{"kind":"objc-method","selector":"c55Probe","method_type":"instance"}],"duration_ms":5000,"max_events":100,"argument_registers":0} ``` The result did include a human-readable limitation explaining that only the first 64 locations are listed. An otherwise equivalent fixture with 64 implementations returned all 64 records and no count-cap limitation. Thus the omission is visible through the count/list mismatch and prose, but it is classified as complete in structured coverage. ### Reproduction 1. Use the source revision below with its locked dependencies and compiled CLI available, Node.js 24.22.1 on PATH, and macOS command-line tools providing clang and LLDB. Run the following script from the checkout root. It compiles two console executables, launches only those executables through REA, isolates product state, and deletes its temporary fixture directory afterward. ```sh python3 <<'PY' import json, os, pathlib, subprocess, tempfile cli = pathlib.Path('scripts/rea.mjs').resolve() with tempfile.TemporaryDirectory(prefix='rea-locations-') as directory: root = pathlib.Path(directory).resolve() env = dict(os.environ) for key in ('HOME', 'XDG_CONFIG_HOME', 'XDG_DATA_HOME', 'XDG_CACHE_HOME', 'XDG_STATE_HOME', 'XDG_RUNTIME_DIR', 'TMPDIR'): child = root / key.lower() child.mkdir() env[key] = str(child) request = json.dumps({ 'breakpoints': [{'kind': 'objc-method', 'selector': 'c55Probe', 'method_type': 'instance'}], 'duration_ms': 5000, 'max_events': 100, 'argument_registers': 0, }) for count in (64, 65): source = '#import <Foundation/Foundation.h>\n' for index in range(count): source += (f'@interface Probe{index} : NSObject\n' '- (void)c55Probe;\n@end\n' f'@implementation Probe{index}\n' '- (void)c55Probe {}\n@end\n') source += ('int main(void) { @autoreleasepool { ' '[[Probe0 new] c55Probe]; } return 0; }\n') fixture = root / f'fixture-{count}' source_path = fixture.with_suffix('.m') source_path.write_text(source) subprocess.run(['clang', '-g', '-O0', '-framework', 'Foundation', '-o', str(fixture), str(source_path)], env=env, check=True, timeout=30) for attempt in range(1, 3 if count == 65 else 2): result = subprocess.run( ['node', str(cli), 'observe-native-calls', str(fixture), request, '--json'], env=env, capture_output=True, text=True, timeout=45, check=True) value = json.loads(result.stdout)['normalized_result'] breakpoint = value['breakpoints'][0] print(json.dumps({ 'fixture': count, 'attempt': attempt, 'cli_exit': result.returncode, 'outcome': value['process']['outcome'], 'exit_status': value['process']['exit_status'], 'location_count': breakpoint['location_count'], 'locations_length': len(breakpoint['locations']), 'coverage': value['coverage'], 'events': len(value['events']), 'count_limit_notice': any('first 64' in item for item in value['limitations']), }), flush=True) PY ``` 2. Compare `location_count`, `locations_length`, `coverage` and `count_limit_notice` in the three output lines. The control returns 64/64; both 65-method runs return 65/64 with complete coverage and no truncation property. The reduced 65-method case reproduced in two initial runs and both repeats of the exact script above. Both independently executed 64-method controls returned all 64 locations. Two additional real CLI calls using the broad Foundation `init` selector each returned 1,629 matches and 64 listed locations with complete coverage. ### Expected behavior The documented resolved-location inventory should include all 65 matching methods when its metadata is far below the documented budget. If REA intentionally retains only 64 records, its completeness classification should explicitly reflect the omitted metadata rather than declare the result complete. Normal process exit should remain distinguishable from location-metadata truncation. The native-observation guide promises a resolved-location inventory for each requested breakpoint and describes a separate aggregate 8 MiB metadata budget. When that budget causes omissions, the guide specifies the full match count, partial coverage and a true truncation flag. It does not document an additional 64-record cap. Exact locators are in Evidence below. The current schema comment defines `breakpoint_locations_truncated` specifically for byte-budget exhaustion. This report does not claim a separate documented promise that the same flag already covers the count cap; using it for all omitted location metadata is one possible correction. ### Impact and scope P2: callers receive an incomplete address/symbol inventory with a misleading structured completeness classification for a valid selector request. In the reduced fixture, `-[Probe64 c55Probe]` was missing from the returned inventory. All 64 retained records belonged to the selected executable, so matches in unrelated images do not explain the result. The retained location array serialized to 15,478 bytes, well below 8 MiB. Callers can detect the omission by comparing each locations-array length with its `location_count` and conservatively treating the inventory as incomplete. The emitted limitation also warns human readers. This workaround detects the problem but does not recover the omitted address records. The executions did not establish lost call events or a failure to install breakpoints; the demonstrated defect is in location reporting and completeness. MCP and other architectures were not exercised. ### REA version Exact stdout from `node scripts/rea.mjs --version`: ```text 6.8.0 ``` The output ends with a newline. Source baseline: `65f1c447344a423fa2785847dbcea1e9a9e2454a`. ### Runtime and platform Node.js `v24.22.1`; macOS `27.0.1`; `arm64`. `/usr/bin/lldb --version` reported `lldb-2103.0.34.103` and `Apple Swift version 6.4 (swiftlang-6.4.0.34.1 clang-2100.3.34.1)`. ### Target and artifact context Source-authored console Mach-O executables linked against Foundation. Each executable defines 64 or 65 NSObject subclasses with the same instance selector and invokes the first class's method once. No application bundle, GUI launch, external process attachment or network fixture was used. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The exact reproduction script printed: ```jsonl {"fixture": 64, "attempt": 1, "cli_exit": 0, "outcome": "exited", "exit_status": 0, "location_count": 64, "locations_length": 64, "coverage": {"status": "complete", "event_limit_reached": false, "resource_limit_reached": false, "unresolved_breakpoints": []}, "events": 1, "count_limit_notice": false} {"fixture": 65, "attempt": 1, "cli_exit": 0, "outcome": "exited", "exit_status": 0, "location_count": 65, "locations_length": 64, "coverage": {"status": "complete", "event_limit_reached": false, "resource_limit_reached": false, "unresolved_breakpoints": []}, "events": 1, "count_limit_notice": true} {"fixture": 65, "attempt": 2, "cli_exit": 0, "outcome": "exited", "exit_status": 0, "location_count": 65, "locations_length": 64, "coverage": {"status": "complete", "event_limit_reached": false, "resource_limit_reached": false, "unresolved_breakpoints": []}, "events": 1, "count_limit_notice": true} ``` The complete initial reduced-case Evidence showed `-[Probe64 c55Probe]` absent in both 65-method location lists; the control contained every fixture method. Both cases recorded the single invocation of `-[Probe0 c55Probe]`. At the source baseline: - `docs/native-investigation.md:979-980` describes the resolved-location inventory. Lines 987-990 specify the 8 MiB metadata limit and its partial-coverage/truncation reporting. - `bridge/native/rea_lldb_tracer.py:29` defines the 64-record reporting cap. Lines 494-523 count every matching location but retain only the first 64; only the byte-budget branches set the returned truncation state. - `src/native/NativeCallObservation.ts:128-135` derives partial coverage without checking whether fewer location records were retained than counted. Lines 178-183 emit the public truncation property only when the bridge's flag is true. Lines 218-223 add the count-cap limitation separately. - `src/domain/native/nativeCallObservation.ts:218-219` currently describes the optional flag as aggregate location-metadata byte-budget exhaustion. - `src/cli/nativeCallCommands.ts:23-57` validates and dispatches this public CLI request. `src/native/NativeCallObservation.ts:45-100` performs target/input admission, invokes the tracer and projects its result. Successful real CLI executions, rather than only test calls, establish reachability. ### Typed outcome or recovery expected Return the complete location inventory within the documented metadata budget, or return explicit partial coverage for the omitted metadata. Preserve the actual exited-process outcome. The existing full `location_count` already lets callers calculate the number of omitted records; a new omission-count field is not required to establish this defect. ### Analysis or suspected cause Source-derived: the bridge has two independent reporting bounds, but its truncation state covers only the aggregate byte budget. Once the per-breakpoint list reaches 64 entries, later matches still increase the count while bypassing retention and the truncation branches. The result projection recognizes the mismatch for limitation text without including it in the completeness decision. The 64/65 public CLI comparison corroborates this source explanation. ### Additional context Duplicate searches covered open and closed issues, PRs and issue comments. Related reports have different closure conditions: [#7448](https://github.com/morluto/rea/issues/7448) concerns empty-target admission; [#5306](https://github.com/morluto/rea/issues/5306) concerns native UI PID/window classification; [#5913](https://github.com/morluto/rea/issues/5913) concerns completed facts lost during cancellation/cleanup; [#2382](https://github.com/morluto/rea/issues/2382) requests source coordinates. [#5298](https://github.com/morluto/rea/issues/5298) and withdrawn [#5299](https://github.com/morluto/rea/pull/5299) explicitly reject treating matches outside the executable as missed coverage. This report demonstrates omitted methods within the executable itself and does not revive that theory. No prior report or fix for this per-breakpoint count-cap completeness mismatch was found. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Real CLI runs omitted one of 65 fixture-defined method locations while returning complete coverage; the omission affects returned metadata, with no observed severe loss or blocked tracing workflow. - Workaround status: available - Workaround basis: The observed full location_count and shorter locations array let a consumer detect omissions and conservatively mark metadata incomplete; the limitation also discloses the cap. This detects incompleteness but cannot reconstruct omitted location details from that result.

bsakaguchi referenced this issue

2026-10-31 01:13:10 +00:00

[PR #4485] [MERGED] fix(wasm): pass glue filenames to the JavaScript parser #6270

bsakaguchi closed this issue

2026-11-04 12:09:23 +00:00