morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6118] [Bug] Protobuf schema admission hides missing message types and malformed descriptors #3748

New issue

Closed

opened 2026-10-31 00:43:38 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:43:38 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6118

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The public inspect-web-network-capture CLI returns the same correction issue for a valid descriptor set that lacks the selected message type and for a malformed descriptor wire stream. Both failures identify only protobuf.schema and say the descriptor or message is invalid or over budget. They omit which caller choice failed and the producer's known reason.

The valid descriptor with its defined message succeeds. Both rejected calls correctly return invalid_request and retain partial capture Evidence; this report concerns diagnostic and recovery information, not lost source bytes or an untagged public response.

Reproduction

Executed against the built checkout with its installed dependencies. From the repository root, pipe the following script into node --import tsx --input-type=module. It uses the repository's synthetic HAR and descriptor fixtures, runs three actual CLI subprocesses and removes its disposable files. The assertions describe the current incorrect result.

import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { spawnSync } from 'node:child_process';
const { protobufHar, protobufCaptureDescriptor } = await import(new URL('./tests/fixtures/protobufCapture.ts', `file://${process.cwd()}/`));
const root = mkdtempSync(join(tmpdir(), 'rea-schema-repro-'));
const env = {...process.env, REA_LOG_LEVEL:'silent'};
for (const [key,name] of [['HOME','home'],['TMPDIR','temp'],['XDG_CONFIG_HOME','config'],['XDG_DATA_HOME','data'],['XDG_CACHE_HOME','cache'],['XDG_STATE_HOME','state']]) {
  const path=join(root,name); mkdirSync(path); env[key]=path;
}
try {
  const capture=join(root,'capture.har'), descriptor=join(root,'capture.pb');
  writeFileSync(capture,protobufHar());
  const failures=[];
  for (const [label,bytes,message] of [
    ['valid',protobufCaptureDescriptor(),'capture.Message'],
    ['missing-message',protobufCaptureDescriptor(),'capture.Missing'],
    ['malformed-descriptor',Buffer.from([0x0a,0xff]),'capture.Message'],
  ]) {
    writeFileSync(descriptor,bytes);
    const result=spawnSync(process.execPath,['scripts/rea.mjs','inspect-web-network-capture',capture,'har','--record','0','--protobuf-schema',descriptor,'--protobuf-message',message,'--json'],{env,encoding:'utf8'});
    const output=JSON.parse(result.stdout);
    assert.equal(result.status,label==='valid'?0:1,result.stderr);
    assert.equal(result.stderr,'');
    if (label!=='valid') failures.push(output.details.issues);
    console.log(JSON.stringify({label,exit:result.status,code:output.code??null,issues:output.details?.issues??null,partialSourceRetained:!!output.details?.partial_observation}));
  }
  assert.deepEqual(failures[0],failures[1]);
} finally { rmSync(root,{recursive:true,force:true}); }

Expected behavior

Tell the caller whether to correct protobuf.schema.message_type or replace/fix protobuf.schema.descriptor_set_path. For malformed descriptors, retain the available safe wire reason/offset instead of collapsing it to an unrelated type-name or budget possibility. Preserve partial source Evidence and the existing separate budget-failure path.

The documented independent schema inputs and argument correction metadata support targeted correction. The repository boundary contract requires meaningful failure reasons.

Impact and scope

P2: a bounded diagnostic/recovery defect. A typo in a fully qualified message name is indistinguishable from corrupt descriptor bytes, so a caller cannot select the targeted correction from the returned issue. The valid schema/control works, and partial capture Evidence survives both failures. Manually checking the descriptor and type is possible; frequency and effects on real client automation are unknown.

Observed scope is the public CLI with generated local HAR/descriptor inputs. MCP shares the source path but was not exercised for these cases. No network target or external analysis provider ran.

REA version

rea-agents source/package version 6.7.0; revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; repository-pinned Node.js v24.18.0. Built CLI; installed locked tsx and protobuf dependencies for fixture generation.

Target and artifact context

One local synthetic HAR 1.2 with a protobuf/gRPC-web response. The valid compiled FileDescriptorSet defines capture.Message. Rejected cases select capture.Missing or substitute descriptor bytes 0a ff.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Executed compact observations:

{"label":"valid","exit":0,"code":null,"issues":null,"partialSourceRetained":false}
{"label":"missing-message","exit":1,"code":"invalid_request","issues":[{"path":["protobuf","schema"],"reason":"invalid_format","message":"The selected local FileDescriptorSet or message type is invalid or exceeds decoding budgets"}],"partialSourceRetained":true}
{"label":"malformed-descriptor","exit":1,"code":"invalid_request","issues":[{"path":["protobuf","schema"],"reason":"invalid_format","message":"The selected local FileDescriptorSet or message type is invalid or exceeds decoding budgets"}],"partialSourceRetained":true}

The two failed calls have identical details.issues; their exit status is 1 and stderr is empty. Both public responses contain details.partial_observation. Their surrounding input/Evidence identities differ as expected.

Typed outcome or recovery expected

Keep invalid_request, but attach a specific issue to the message-type or descriptor-path field and retain the meaningful source failure. Retaining an internal cause alone would not make the current public issue actionable.

Analysis or suspected cause

Source-derived: ProtobufSchema.read detects malformed wire data and missing type names separately, then throws ordinary Errors. The capture catch rethrows CaptureFormatError but replaces other failures with one generic AnalysisInputError without cause. The wire error offset/reason is already discarded at the schema producer. Projection faithfully exposes the generic issue it receives.

Additional context

Fetched upstream main aa51e4343963fdd0b97ec2ebdc3c41579fd9cb38: the affected schema read/catch branches are unchanged; neighboring changes concern MIME/framing metadata. Upstream was source-compared, not executed. #5926 concerns redacted MIME applicability, #5942 concerns base64 decoded-byte accounting, #5386 concerns trailer values, and original feature PR #5031 does not fix this admission distinction. No restoration patch or known-good revision was tested.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6118 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The public `inspect-web-network-capture` CLI returns the same correction issue for a valid descriptor set that lacks the selected message type and for a malformed descriptor wire stream. Both failures identify only `protobuf.schema` and say the descriptor or message is invalid or over budget. They omit which caller choice failed and the producer's known reason. The valid descriptor with its defined message succeeds. Both rejected calls correctly return `invalid_request` and retain partial capture Evidence; this report concerns diagnostic and recovery information, not lost source bytes or an untagged public response. ### Reproduction Executed against the built checkout with its installed dependencies. From the repository root, pipe the following script into `node --import tsx --input-type=module`. It uses the repository's synthetic HAR and descriptor fixtures, runs three actual CLI subprocesses and removes its disposable files. The assertions describe the current incorrect result. ```javascript import assert from 'node:assert/strict'; import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { spawnSync } from 'node:child_process'; const { protobufHar, protobufCaptureDescriptor } = await import(new URL('./tests/fixtures/protobufCapture.ts', `file://${process.cwd()}/`)); const root = mkdtempSync(join(tmpdir(), 'rea-schema-repro-')); const env = {...process.env, REA_LOG_LEVEL:'silent'}; for (const [key,name] of [['HOME','home'],['TMPDIR','temp'],['XDG_CONFIG_HOME','config'],['XDG_DATA_HOME','data'],['XDG_CACHE_HOME','cache'],['XDG_STATE_HOME','state']]) { const path=join(root,name); mkdirSync(path); env[key]=path; } try { const capture=join(root,'capture.har'), descriptor=join(root,'capture.pb'); writeFileSync(capture,protobufHar()); const failures=[]; for (const [label,bytes,message] of [ ['valid',protobufCaptureDescriptor(),'capture.Message'], ['missing-message',protobufCaptureDescriptor(),'capture.Missing'], ['malformed-descriptor',Buffer.from([0x0a,0xff]),'capture.Message'], ]) { writeFileSync(descriptor,bytes); const result=spawnSync(process.execPath,['scripts/rea.mjs','inspect-web-network-capture',capture,'har','--record','0','--protobuf-schema',descriptor,'--protobuf-message',message,'--json'],{env,encoding:'utf8'}); const output=JSON.parse(result.stdout); assert.equal(result.status,label==='valid'?0:1,result.stderr); assert.equal(result.stderr,''); if (label!=='valid') failures.push(output.details.issues); console.log(JSON.stringify({label,exit:result.status,code:output.code??null,issues:output.details?.issues??null,partialSourceRetained:!!output.details?.partial_observation})); } assert.deepEqual(failures[0],failures[1]); } finally { rmSync(root,{recursive:true,force:true}); } ``` ### Expected behavior Tell the caller whether to correct `protobuf.schema.message_type` or replace/fix `protobuf.schema.descriptor_set_path`. For malformed descriptors, retain the available safe wire reason/offset instead of collapsing it to an unrelated type-name or budget possibility. Preserve partial source Evidence and the existing separate budget-failure path. The [documented independent schema inputs](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/web-network-captures.md#L147-L167) and [argument correction metadata](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorCore.ts#L80-L96) support targeted correction. The [repository boundary contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/AGENTS.md#L31-L31) requires meaningful failure reasons. ### Impact and scope P2: a bounded diagnostic/recovery defect. A typo in a fully qualified message name is indistinguishable from corrupt descriptor bytes, so a caller cannot select the targeted correction from the returned issue. The valid schema/control works, and partial capture Evidence survives both failures. Manually checking the descriptor and type is possible; frequency and effects on real client automation are unknown. Observed scope is the public CLI with generated local HAR/descriptor inputs. MCP shares the source path but was not exercised for these cases. No network target or external analysis provider ran. ### REA version `rea-agents` source/package version 6.7.0; revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; repository-pinned Node.js v24.18.0. Built CLI; installed locked tsx and protobuf dependencies for fixture generation. ### Target and artifact context One local synthetic HAR 1.2 with a protobuf/gRPC-web response. The valid compiled FileDescriptorSet defines capture.Message. Rejected cases select capture.Missing or substitute descriptor bytes 0a ff. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence Executed compact observations: ```json {"label":"valid","exit":0,"code":null,"issues":null,"partialSourceRetained":false} {"label":"missing-message","exit":1,"code":"invalid_request","issues":[{"path":["protobuf","schema"],"reason":"invalid_format","message":"The selected local FileDescriptorSet or message type is invalid or exceeds decoding budgets"}],"partialSourceRetained":true} {"label":"malformed-descriptor","exit":1,"code":"invalid_request","issues":[{"path":["protobuf","schema"],"reason":"invalid_format","message":"The selected local FileDescriptorSet or message type is invalid or exceeds decoding budgets"}],"partialSourceRetained":true} ``` The two failed calls have identical `details.issues`; their exit status is 1 and stderr is empty. Both public responses contain `details.partial_observation`. Their surrounding input/Evidence identities differ as expected. ### Typed outcome or recovery expected Keep `invalid_request`, but attach a specific issue to the message-type or descriptor-path field and retain the meaningful source failure. Retaining an internal `cause` alone would not make the current public issue actionable. ### Analysis or suspected cause Source-derived: [ProtobufSchema.read](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/browser/history/protobuf/ProtobufSchema.ts#L145-L161) detects malformed wire data and missing type names separately, then throws ordinary Errors. [The capture catch](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/browser/history/protobuf/ProtobufCapture.ts#L410-L428) rethrows CaptureFormatError but replaces other failures with one generic AnalysisInputError without cause. The wire error offset/reason is already discarded at the schema producer. [Projection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorProjection.ts#L333-L344) faithfully exposes the generic issue it receives. ### Additional context Fetched upstream main `aa51e4343963fdd0b97ec2ebdc3c41579fd9cb38`: the affected schema read/catch branches are unchanged; neighboring changes concern MIME/framing metadata. Upstream was source-compared, not executed. #5926 concerns redacted MIME applicability, #5942 concerns base64 decoded-byte accounting, #5386 concerns trailer values, and original feature PR #5031 does not fix this admission distinction. No restoration patch or known-good revision was tested.

bsakaguchi closed this issue

2026-10-31 00:43:38 +00:00

bsakaguchi commented

2026-10-31 00:43:39 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

确认，两种根本不同的失败被压成同一条模糊错误：一个是用户选择的 message type 在 descriptor 里不存在（schema 本身有效），另一个是 descriptor wire stream 损坏（格式问题）——前者该补 message type，后者该换 descriptor，用户拿到同一条 "invalid or over budget" 完全无法区分。建议把 admission 拆成两层可区分的诊断：wire 有效性（descriptor 能否被解析）与引用存在性（所选 message type 是否定义于其中），缺失时列出可用的 message type 名，并透传 producer 侧已知的解析原因。

@XinwangTao commented on GitHub (Oct 22, 2026): 确认，两种根本不同的失败被压成同一条模糊错误：一个是用户选择的 message type 在 descriptor 里不存在（schema 本身有效），另一个是 descriptor wire stream 损坏（格式问题）——前者该补 message type，后者该换 descriptor，用户拿到同一条 "invalid or over budget" 完全无法区分。建议把 admission 拆成两层可区分的诊断：wire 有效性（descriptor 能否被解析）与引用存在性（所选 message type 是否定义于其中），缺失时列出可用的 message type 名，并透传 producer 侧已知的解析原因。

bsakaguchi commented

2026-10-31 00:43:39 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Fixed on main in github.com/morluto/rea@b3c3b21e43. Schema admission distinguishes malformed descriptor bytes from missing selected message, identifies the corrective input field, and retains selected capture facts.

Validation: src/browser/history/protobuf/ProtobufCapture.test.ts passed in the retained audit receipt. Both failure kinds, field-specific tagged issues, available wire offset and unchanged selected-record assertions.

Scope: Controlled production module tests; original external-provider/public-adapter replay was not rerun. Closure is scoped to this deterministic owning-boundary defect, not general provider qualification.

@morluto commented on GitHub (Oct 23, 2026): Fixed on main in https://github.com/morluto/rea/commit/b3c3b21e43cab4e71b460d08829d8c6e4e9b9c21. Schema admission distinguishes malformed descriptor bytes from missing selected message, identifies the corrective input field, and retains selected capture facts. Validation: `src/browser/history/protobuf/ProtobufCapture.test.ts` passed in the retained audit receipt. Both failure kinds, field-specific tagged issues, available wire offset and unchanged selected-record assertions. Scope: Controlled production module tests; original external-provider/public-adapter replay was not rerun. Closure is scoped to this deterministic owning-boundary defect, not general provider qualification.

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3748

No description provided.

Rows
Columns