morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[PR #7213] [CLOSED] feat(browser): page retained web bundle endpoints and inferences #7721

New issue

Closed

opened 2026-11-03 13:39:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:53 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7213
Author: @HarsH-1877
Created: 10/26/2026
Status: ❌ Closed

Base: main ← Head: feat/web-bundle-endpoint-views-6198

📄 Description

Summary

Add a dedicated web-bundle analysis view projection to inspect_analysis_view and analyze_web_bundle --detail summary --view ... allowing callers and agents to query compact summaries or page endpoint observations (collection: "endpoints") and static inferences (collection: "inferences", such as endpoint_candidate) from completed analyze_web_bundle Evidence without re-analyzing page captures.

Closes #6198

Problem and expected behavior

Previously, completed analyze_web_bundle Evidence could only be projected via javascript-routes views. While route trees could be queried, callers had no bounded way to retrieve discovered endpoint observations (e.g. call:fetch, call:sendBeacon) or static inferences (such as endpoint_candidate) from retained Evidence without transferring the complete bundle analysis report.

With this change:

inspect_analysis_view supports view: { kind: "web-bundle", collection: "endpoints" | "inferences" | "summary", offset, limit, ... } as well as view: { kind: "summary" } for analyze_web_bundle parents.
Endpoints can be filtered by indices, values, script_keys, mechanisms, and projected by fields.
Inferences can be filtered by indices, kinds, values, script_keys, and projected by fields.
Producer ordering, duplicate endpoints across distinct source locations, partial producer completeness, and parent artifact identity are strictly preserved.
Original source_index is preserved outside field selection to guarantee parent row identity across boundaries.
The 1 MiB transport wire budget is enforced with typed AnalysisResourceConstraintError and byte_limited: true markers.

Change and scope

Follow the contribution scope.

Implemented projectWebBundleView in src/domain/analysisView/webBundleView.ts handling endpoints, inferences, and summary collections.
Exported webBundleViewRequestSchema, webBundleSummaryViewResultSchema, and webBundlePageViewResultSchema from src/domain/analysisView/analysisView.ts, wiring them into analysisViewRequestSchema, analysisViewResultSchema, and retainedViewProjectors.
Extended webRoutesSummaryDeliveryShape in src/domain/analysisView/javascriptRoutesView.ts and bundleOutputSchema in src/contracts/browserToolContracts.ts to admit webBundleViewRequestSchema alongside javaScriptRoutesViewRequestSchema.
Documented the web-bundle view kind and added a representative example in src/contracts/analysisViewToolContracts.ts.
Added unit tests in src/domain/analysisView/webBundleView.test.ts and MCP boundary integration tests in tests/boundary/mcp/webBundleViewsMcp.test.ts validating advertised MCP schemas with Ajv.

Contract and boundary impact

Semantic owner and earliest changed stage: src/domain/analysisView/webBundleView.ts owns the projection of completed analyze_web_bundle Evidence.
Contribution scope fit and linked maintainer decision: Normal analysis view projection enhancement within established boundaries; ordinary feature addition.
CLI and MCP/tool-catalog contract: inspect_analysis_view accepts kind: "web-bundle" and analyze_web_bundle --detail summary --view ... accepts web bundle view shapes.
Provider, bridge, target-format, or platform compatibility: Platform-independent; operates purely on normalized WebBundleAnalysis Evidence records.
Evidence, artifact, provenance, or reconstruction contract: Preserves parent_evidence_id, parent_digest, parent.artifact ?? null, target_url, target_origin, producer completeness, source_index, and source locations.
Target selection, authority, upstream behavior, and cleanup details: Read-only projection over completed Evidence.
Generated metadata impact: Generated contracts and documentation checked via npm run docs:check.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated:
- src/domain/analysisView/webBundleView.test.ts (13 tests covering pagination, duplicate preservation, field/kind filtering, partial completeness, empty collections, artifact identity, 1 MiB wire limit error, and multi-item byte budget truncation).
- tests/boundary/mcp/webBundleViewsMcp.test.ts (2 tests testing MCP SDK tool calls over retained-evidence, inline, and evidence-file sources with advertised schema validation).
Base reproduction or other evidence: Verified that inspect_analysis_view previously rejected web-bundle view kinds with invalid_value.

User-visible CLI/MCP output:

{
  "kind": "web-bundle",
  "parent_operation": "analyze_web_bundle",
  "target_url": "https://example.test/app",
  "target_origin": "https://example.test",
  "view": {
    "kind": "web-bundle",
    "collection": "endpoints",
    "offset": 0,
    "limit": 2
  },
  "completeness": { "status": "partial" },
  "items": [
    {
      "source_index": 0,
      "value": "/api/users",
      "mechanism": "call:fetch",
      "location": { "script_key": "app.js", "line": 10, "column": 2 }
    },
    {
      "source_index": 1,
      "value": "/api/users",
      "mechanism": "call:fetch",
      "location": { "script_key": "vendor.js", "line": 42, "column": 8 }
    }
  ],
  "coverage": {
    "status": "page",
    "examined": 2,
    "total": 3,
    "next_offset": 2,
    "exhausted": false
  },
  "byte_limited": false
}

Remaining proof gaps: None for projection logic; real CDP browser capture is covered by existing browser acceptance suites.

For boundary fixes: Covered by boundary regressions for inspect_analysis_view and analyze_web_bundle.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npm run check:fast — passed (lint and typecheck clean).
npm run docs:check — passed (README and catalog consistency verified).
npm run test:local -- src/domain/analysisView/webBundleView.test.ts src/domain/analysisView/javascriptRoutesView.test.ts — passed (25/25 tests).
npm run test:focused -- tests/boundary/mcp/webBundleViewsMcp.test.ts — passed (2/2 tests).
npm run test:focused -- tests/acceptance/javascriptRoutesCli.test.ts — passed (6/6 tests).

Compatibility, safety, and release

Breaking changes or migration steps: None; additive view kind and result schema.
Real Hopper/Ghidra, browser, or OS coverage: Browser tool and MCP boundary tests pass on Windows.
Package or release metadata impact: None.
Security, privacy, process, or containment review: Does not access disk or external services outside of reading resolved Evidence; limits payload serialization to 1 MiB wire budget.

Author readiness

Human owner: @HarsH-1877

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7213 **Author:** [@HarsH-1877](https://github.com/HarsH-1877) **Created:** 10/26/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/web-bundle-endpoint-views-6198` --- ### 📄 Description ## Summary Add a dedicated `web-bundle` analysis view projection to `inspect_analysis_view` and `analyze_web_bundle --detail summary --view ...` allowing callers and agents to query compact summaries or page endpoint observations (`collection: "endpoints"`) and static inferences (`collection: "inferences"`, such as `endpoint_candidate`) from completed `analyze_web_bundle` Evidence without re-analyzing page captures. Closes #6198 ## Problem and expected behavior Previously, completed `analyze_web_bundle` Evidence could only be projected via `javascript-routes` views. While route trees could be queried, callers had no bounded way to retrieve discovered endpoint observations (e.g. `call:fetch`, `call:sendBeacon`) or static inferences (such as `endpoint_candidate`) from retained Evidence without transferring the complete bundle analysis report. With this change: - `inspect_analysis_view` supports `view: { kind: "web-bundle", collection: "endpoints" | "inferences" | "summary", offset, limit, ... }` as well as `view: { kind: "summary" }` for `analyze_web_bundle` parents. - Endpoints can be filtered by `indices`, `values`, `script_keys`, `mechanisms`, and projected by `fields`. - Inferences can be filtered by `indices`, `kinds`, `values`, `script_keys`, and projected by `fields`. - Producer ordering, duplicate endpoints across distinct source locations, partial producer completeness, and parent artifact identity are strictly preserved. - Original `source_index` is preserved outside field selection to guarantee parent row identity across boundaries. - The 1 MiB transport wire budget is enforced with typed `AnalysisResourceConstraintError` and `byte_limited: true` markers. ## Change and scope Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). 1. Implemented `projectWebBundleView` in `src/domain/analysisView/webBundleView.ts` handling `endpoints`, `inferences`, and `summary` collections. 2. Exported `webBundleViewRequestSchema`, `webBundleSummaryViewResultSchema`, and `webBundlePageViewResultSchema` from `src/domain/analysisView/analysisView.ts`, wiring them into `analysisViewRequestSchema`, `analysisViewResultSchema`, and `retainedViewProjectors`. 3. Extended `webRoutesSummaryDeliveryShape` in `src/domain/analysisView/javascriptRoutesView.ts` and `bundleOutputSchema` in `src/contracts/browserToolContracts.ts` to admit `webBundleViewRequestSchema` alongside `javaScriptRoutesViewRequestSchema`. 4. Documented the `web-bundle` view kind and added a representative example in `src/contracts/analysisViewToolContracts.ts`. 5. Added unit tests in `src/domain/analysisView/webBundleView.test.ts` and MCP boundary integration tests in `tests/boundary/mcp/webBundleViewsMcp.test.ts` validating advertised MCP schemas with Ajv. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/domain/analysisView/webBundleView.ts` owns the projection of completed `analyze_web_bundle` Evidence. - Contribution scope fit and linked maintainer decision: Normal analysis view projection enhancement within established boundaries; ordinary feature addition. - CLI and MCP/tool-catalog contract: `inspect_analysis_view` accepts `kind: "web-bundle"` and `analyze_web_bundle --detail summary --view ...` accepts web bundle view shapes. - Provider, bridge, target-format, or platform compatibility: Platform-independent; operates purely on normalized `WebBundleAnalysis` Evidence records. - Evidence, artifact, provenance, or reconstruction contract: Preserves `parent_evidence_id`, `parent_digest`, `parent.artifact ?? null`, `target_url`, `target_origin`, producer completeness, `source_index`, and source locations. - Target selection, authority, upstream behavior, and cleanup details: Read-only projection over completed Evidence. - Generated metadata impact: Generated contracts and documentation checked via `npm run docs:check`. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - `src/domain/analysisView/webBundleView.test.ts` (13 tests covering pagination, duplicate preservation, field/kind filtering, partial completeness, empty collections, artifact identity, 1 MiB wire limit error, and multi-item byte budget truncation). - `tests/boundary/mcp/webBundleViewsMcp.test.ts` (2 tests testing MCP SDK tool calls over `retained-evidence`, `inline`, and `evidence-file` sources with advertised schema validation). - Base reproduction or other evidence: Verified that `inspect_analysis_view` previously rejected `web-bundle` view kinds with `invalid_value`. - User-visible CLI/MCP output: ```json { "kind": "web-bundle", "parent_operation": "analyze_web_bundle", "target_url": "https://example.test/app", "target_origin": "https://example.test", "view": { "kind": "web-bundle", "collection": "endpoints", "offset": 0, "limit": 2 }, "completeness": { "status": "partial" }, "items": [ { "source_index": 0, "value": "/api/users", "mechanism": "call:fetch", "location": { "script_key": "app.js", "line": 10, "column": 2 } }, { "source_index": 1, "value": "/api/users", "mechanism": "call:fetch", "location": { "script_key": "vendor.js", "line": 42, "column": 8 } } ], "coverage": { "status": "page", "examined": 2, "total": 3, "next_offset": 2, "exhausted": false }, "byte_limited": false } ``` - Remaining proof gaps: None for projection logic; real CDP browser capture is covered by existing browser acceptance suites. For boundary fixes: Covered by boundary regressions for `inspect_analysis_view` and `analyze_web_bundle`. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run check:fast` — passed (lint and typecheck clean). - `npm run docs:check` — passed (README and catalog consistency verified). - `npm run test:local -- src/domain/analysisView/webBundleView.test.ts src/domain/analysisView/javascriptRoutesView.test.ts` — passed (25/25 tests). - `npm run test:focused -- tests/boundary/mcp/webBundleViewsMcp.test.ts` — passed (2/2 tests). - `npm run test:focused -- tests/acceptance/javascriptRoutesCli.test.ts` — passed (6/6 tests). ## Compatibility, safety, and release - Breaking changes or migration steps: None; additive view kind and result schema. - Real Hopper/Ghidra, browser, or OS coverage: Browser tool and MCP boundary tests pass on Windows. - Package or release metadata impact: None. - Security, privacy, process, or containment review: Does not access disk or external services outside of reading resolved Evidence; limits payload serialization to 1 MiB wire budget. ## Author readiness Human owner: @HarsH-1877 - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:39:53 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7721

No description provided.

Rows
Columns