[GH-ISSUE #7327] [Bug] MCP provenance documentation lists two exceptions although both settings tools are also exempt #4354

New issue

Closed

opened 2026-10-31 00:49:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:49:09 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 26, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7327

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Documentation

Observed behavior

The MCP provenance guide says only export_evidence_bundle and record_unknown omit the result marker. However, inspect_analysis_settings and update_analysis_settings also omit it. In two fresh target-free stdio sessions, both settings tools returned successful results and small validation errors without _meta. The audited source set explicitly includes all four names.

Reproduction

Prerequisite: a source checkout at the revision below with its locked dependencies and compiled runtime already prepared, and Node 24 on PATH. Dependency installation and rebuilding were not rerun for this verification. From the checkout root, run this Python standard-library client; it creates an isolated home/config/cache/data/temp directory, calls only target-free metadata tools, resets one setting to its launch value, and removes its disposable state after the server exits:

python3 <<'PY_REPRO'
import json, os, pathlib, selectors, shutil, subprocess, tempfile

entry = str(pathlib.Path('scripts/rea.mjs').resolve())
node = shutil.which('node')
with tempfile.TemporaryDirectory(prefix='rea-metadata-') as work:
    env = {'PATH': os.path.dirname(node) + ':/usr/bin:/bin',
           'HOME': work, 'REA_LOG_LEVEL': 'silent'}
    for key in ['XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME', 'TMPDIR']:
        env[key] = str(pathlib.Path(work, key.lower()))
        pathlib.Path(env[key]).mkdir()
    p = subprocess.Popen([node, entry, 'mcp'], cwd=work, env=env,
        stdin=subprocess.PIPE, stdout=subprocess.PIPE,
        stderr=subprocess.PIPE, text=True, bufsize=1)
    selector = selectors.DefaultSelector()
    selector.register(p.stdout, selectors.EVENT_READ)
    def request(method, params, request_id=None):
        message = dict(jsonrpc='2.0', method=method, params=params)
        if request_id is not None:
            message['id'] = request_id
        p.stdin.write(json.dumps(message) + '\n')
        p.stdin.flush()
        if request_id is None:
            return
        while True:
            if not selector.select(15):
                raise RuntimeError('Timed out waiting for ' + method)
            line = p.stdout.readline()
            if not line:
                raise RuntimeError('Server closed stdout')
            response = json.loads(line)
            if response.get('id') == request_id:
                return response['result']
    try:
        request('initialize', dict(protocolVersion='2025-06-18',
            capabilities={}, clientInfo=dict(name='metadata-check', version='1')), 1)
        request('notifications/initialized', {})
        cases = [
            ('inspect_analysis_settings', {'settings':['asar.max_header_bytes']}),
            ('update_analysis_settings', {'changes':[
                {'setting':'asar.max_header_bytes', 'action':'reset'}]}),
            ('inspect_analysis_settings', {'unexpected':1}),
            ('update_analysis_settings', {'changes':[]}),
            ('list_unknowns', {}),
            ('list_unknowns', {'unexpected':1}),
        ]
        for request_id, (name, arguments) in enumerate(cases, 2):
            result = request('tools/call', dict(name=name, arguments=arguments), request_id)
            print(json.dumps(dict(tool=name, isError=result.get('isError', False),
                hasMeta='_meta' in result, meta=result.get('_meta'))))
    finally:
        p.terminate()
        try:
            p.wait(timeout=10)
        except subprocess.TimeoutExpired:
            p.kill()
            p.wait(timeout=5)
        selector.close()
        print(json.dumps(dict(exitCode=p.returncode, stderrBytes=len(p.stderr.read()))))
PY_REPRO

Compare the resulting marker presence with the guide's exclusive two-tool list. The first four responses have hasMeta: false; the two list_unknowns controls have meta.provenance.untrusted_content: true. The exact inline client above was executed once; a preceding independently written client observed the same six results in another fresh session.

Expected behavior

The guide's exhaustive exception list should accurately describe the existing audited tool metadata, including the two settings tools. This report requests documentation accuracy for the current behavior.

Impact and scope

P3 documentation mismatch. A reader using this exclusive list to identify marked tools receives incorrect information about the two settings tools. The tested settings calls still succeed or reject invalid input as expected. No target, provider operation, analysis-result change, host-policy consequence, or security-boundary crossing was exercised or demonstrated. The verified scope is the stated revision and the target-free calls below.

REA version

6.8.0; source revision ace4dbf36e.

Runtime and platform

Node.js v24.22.1; macOS arm64; Darwin kernel 27.0.0; stdio MCP requesting protocol 2025-06-18. Python 3 standard-library client. No analysis engine was invoked.

Target and artifact context

Not applicable: fresh local server sessions with no analysis target, artifact, imported bundle, or external provider operation. Inputs are one named runtime setting, its reset operation, empty/unknown-key validation cases, and an empty session metadata listing.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

Tool                       isError  _meta present
inspect_analysis_settings  false    false
update_analysis_settings   false    false
inspect_analysis_settings  true     false
update_analysis_settings   true     false
list_unknowns              false    true
list_unknowns              true     true
Server exit code: 0; stderr bytes: 0

Expected source: docs/mcp-contracts.md:379-382, under “Untrusted target content”, identifies the contract metadata as the owner and names only two tools as exceptions. An exhaustive documentation list should agree with that metadata.
Failure source: the six executed responses above; src/contracts/toolEffects.ts:787-800 lists four exceptions, and :803-814 derives returnsTargetContent from membership. src/server/registerAnalysisSettingsTools.ts:8-31 registers both settings contracts without requiring a target. src/server/ToolResultDelivery.ts:198-218 conditionally adds the marker, and src/server/EvidenceMcpServerCore.ts:508-519 applies the same predicate to SDK errors. tests/conformance/contracts/targetContentProvenance.test.ts:22-28 explicitly expects the four-name set; that test was read, not run.
Evidence provenance: observed for the target-free stdio responses; static for the documentation/source discrepancy and registration/delivery explanation.
Local verification: reproduced in two fresh sessions, including one execution of the exact inline client above. Both servers exited with code 0 and empty stderr.
Reproduction completeness: complete for the documentation mismatch and the listed metadata calls.

Additional context

Restoration check (proposed, not executed): the guide's stated exception list agrees with the audited four-tool set and the observed settings responses. Repeat the small target-free calls; retain list_unknowns as the marked success/error control. Documentation accuracy is the restoration criterion; this report establishes no need for a runtime policy change.

Duplicate search covered the supplied open/closed issue index and full bodies, plus the discussion and closure histories of #1150, #1908, #2398 and #3221. #1150 was closed unmerged and its discussion disagreed with extra confirmation/warnings; #1908 introduced the metadata and audited the then-existing exceptions; #2398 repaired partial-error flag propagation; #3221 concerned oversized SDK diagnostics. #7049 introduced the settings tools. None of these reports covers the current list omission, and no matching report was found in the supplied corpus. A current upstream issue/PR search before submission also found no exact duplicate.

Triage assessment

Impact level: P3
Assessment status: supported
Impact basis: The verified failure is an inaccurate exclusive documentation list. The metadata calls themselves behaved consistently with their audited source contracts; no consequential runtime effect was observed.
Workaround status: available
Workaround basis: Consult the four-name audited source set or inspect the actual result metadata, both demonstrated here without a code change.

Originally created by @coygeek on GitHub (Oct 26, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7327 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Documentation ## Observed behavior The MCP provenance guide says only `export_evidence_bundle` and `record_unknown` omit the result marker. However, `inspect_analysis_settings` and `update_analysis_settings` also omit it. In two fresh target-free stdio sessions, both settings tools returned successful results and small validation errors without `_meta`. The audited source set explicitly includes all four names. ## Reproduction Prerequisite: a source checkout at the revision below with its locked dependencies and compiled runtime already prepared, and Node 24 on PATH. Dependency installation and rebuilding were not rerun for this verification. From the checkout root, run this Python standard-library client; it creates an isolated home/config/cache/data/temp directory, calls only target-free metadata tools, resets one setting to its launch value, and removes its disposable state after the server exits: ```sh python3 <<'PY_REPRO' import json, os, pathlib, selectors, shutil, subprocess, tempfile entry = str(pathlib.Path('scripts/rea.mjs').resolve()) node = shutil.which('node') with tempfile.TemporaryDirectory(prefix='rea-metadata-') as work: env = {'PATH': os.path.dirname(node) + ':/usr/bin:/bin', 'HOME': work, 'REA_LOG_LEVEL': 'silent'} for key in ['XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME', 'TMPDIR']: env[key] = str(pathlib.Path(work, key.lower())) pathlib.Path(env[key]).mkdir() p = subprocess.Popen([node, entry, 'mcp'], cwd=work, env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, bufsize=1) selector = selectors.DefaultSelector() selector.register(p.stdout, selectors.EVENT_READ) def request(method, params, request_id=None): message = dict(jsonrpc='2.0', method=method, params=params) if request_id is not None: message['id'] = request_id p.stdin.write(json.dumps(message) + '\n') p.stdin.flush() if request_id is None: return while True: if not selector.select(15): raise RuntimeError('Timed out waiting for ' + method) line = p.stdout.readline() if not line: raise RuntimeError('Server closed stdout') response = json.loads(line) if response.get('id') == request_id: return response['result'] try: request('initialize', dict(protocolVersion='2025-06-18', capabilities={}, clientInfo=dict(name='metadata-check', version='1')), 1) request('notifications/initialized', {}) cases = [ ('inspect_analysis_settings', {'settings':['asar.max_header_bytes']}), ('update_analysis_settings', {'changes':[ {'setting':'asar.max_header_bytes', 'action':'reset'}]}), ('inspect_analysis_settings', {'unexpected':1}), ('update_analysis_settings', {'changes':[]}), ('list_unknowns', {}), ('list_unknowns', {'unexpected':1}), ] for request_id, (name, arguments) in enumerate(cases, 2): result = request('tools/call', dict(name=name, arguments=arguments), request_id) print(json.dumps(dict(tool=name, isError=result.get('isError', False), hasMeta='_meta' in result, meta=result.get('_meta')))) finally: p.terminate() try: p.wait(timeout=10) except subprocess.TimeoutExpired: p.kill() p.wait(timeout=5) selector.close() print(json.dumps(dict(exitCode=p.returncode, stderrBytes=len(p.stderr.read())))) PY_REPRO ``` Compare the resulting marker presence with the guide's exclusive two-tool list. The first four responses have `hasMeta: false`; the two `list_unknowns` controls have `meta.provenance.untrusted_content: true`. The exact inline client above was executed once; a preceding independently written client observed the same six results in another fresh session. ## Expected behavior The guide's exhaustive exception list should accurately describe the existing audited tool metadata, including the two settings tools. This report requests documentation accuracy for the current behavior. ## Impact and scope P3 documentation mismatch. A reader using this exclusive list to identify marked tools receives incorrect information about the two settings tools. The tested settings calls still succeed or reject invalid input as expected. No target, provider operation, analysis-result change, host-policy consequence, or security-boundary crossing was exercised or demonstrated. The verified scope is the stated revision and the target-free calls below. ## REA version 6.8.0; source revision ace4dbf36e8c8f36a0e192aff9a11484ed428358. ## Runtime and platform Node.js v24.22.1; macOS arm64; Darwin kernel 27.0.0; stdio MCP requesting protocol 2025-06-18. Python 3 standard-library client. No analysis engine was invoked. ## Target and artifact context Not applicable: fresh local server sessions with no analysis target, artifact, imported bundle, or external provider operation. Inputs are one named runtime setting, its reset operation, empty/unknown-key validation cases, and an empty session metadata listing. ## Interface MCP ## Earliest failure stage Transport or result projection ## Relevant output or evidence ```text Tool isError _meta present inspect_analysis_settings false false update_analysis_settings false false inspect_analysis_settings true false update_analysis_settings true false list_unknowns false true list_unknowns true true Server exit code: 0; stderr bytes: 0 ``` - Expected source: `docs/mcp-contracts.md:379-382`, under “Untrusted target content”, identifies the contract metadata as the owner and names only two tools as exceptions. An exhaustive documentation list should agree with that metadata. - Failure source: the six executed responses above; `src/contracts/toolEffects.ts:787-800` lists four exceptions, and `:803-814` derives `returnsTargetContent` from membership. `src/server/registerAnalysisSettingsTools.ts:8-31` registers both settings contracts without requiring a target. `src/server/ToolResultDelivery.ts:198-218` conditionally adds the marker, and `src/server/EvidenceMcpServerCore.ts:508-519` applies the same predicate to SDK errors. `tests/conformance/contracts/targetContentProvenance.test.ts:22-28` explicitly expects the four-name set; that test was read, not run. - Evidence provenance: observed for the target-free stdio responses; static for the documentation/source discrepancy and registration/delivery explanation. - Local verification: reproduced in two fresh sessions, including one execution of the exact inline client above. Both servers exited with code 0 and empty stderr. - Reproduction completeness: complete for the documentation mismatch and the listed metadata calls. ## Additional context Restoration check (proposed, not executed): the guide's stated exception list agrees with the audited four-tool set and the observed settings responses. Repeat the small target-free calls; retain `list_unknowns` as the marked success/error control. Documentation accuracy is the restoration criterion; this report establishes no need for a runtime policy change. Duplicate search covered the supplied open/closed issue index and full bodies, plus the discussion and closure histories of #1150, #1908, #2398 and #3221. #1150 was closed unmerged and its discussion disagreed with extra confirmation/warnings; #1908 introduced the metadata and audited the then-existing exceptions; #2398 repaired partial-error flag propagation; #3221 concerned oversized SDK diagnostics. #7049 introduced the settings tools. None of these reports covers the current list omission, and no matching report was found in the supplied corpus. A current upstream issue/PR search before submission also found no exact duplicate. ## Triage assessment - Impact level: P3 - Assessment status: supported - Impact basis: The verified failure is an inaccurate exclusive documentation list. The metadata calls themselves behaved consistently with their audited source contracts; no consequential runtime effect was observed. - Workaround status: available - Workaround basis: Consult the four-name audited source set or inspect the actual result metadata, both demonstrated here without a code change.

bsakaguchi referenced this issue

2026-10-31 01:12:37 +00:00

[PR #4354] [MERGED] test(adb): cover staged APK names and evidence identity #6199

bsakaguchi closed this issue

2026-11-04 11:07:27 +00:00