[GH-ISSUE #4184] [Bug] Unrelated SQLite catalog names exhaust selected-table reply budget #1640

New issue

Closed

opened 2026-10-16 21:08:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:08:18 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4184

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-sqlite-database catalog-60.db --table selected --json refuses a small ordinary-table inspection because unrelated catalog names exceed an aggregate 16 MiB check. The same selected schema and row succeed with 50 unrelated tables and fail with 60. The larger database is below the 256 MiB input limit, and native SQLite queries succeed under the documented individual value/record and SQL-statement limits. REA returns a complete-reply resource constraint before returning any selected evidence.

Reproduction

From the built source checkout at the revision below, capture the CLI entrypoint and change to a fresh disposable directory. These commands do not change the checkout.

REA_ENTRYPOINT="$PWD/scripts/rea.mjs"
repro_dir="$(mktemp -d)"
cd "$repro_dir"
mkdir home
export HOME="$PWD/home"
export XDG_CONFIG_HOME="$PWD/home/config"
export XDG_CACHE_HOME="$PWD/home/cache"

Save this fixture as make-databases.cjs, then run node make-databases.cjs. Both generated databases pass quick_check. Each contains the eight-character table selected, one TEXT column and one small row, plus either 50 or 60 unrelated ordinary tables. No long identifier is passed as a process argument.

const { DatabaseSync } = require('node:sqlite');
for (const count of [50, 60]) {
  const db = new DatabaseSync(`catalog-${count}.db`);
  db.exec("CREATE TABLE selected (v TEXT); INSERT INTO selected VALUES ('selected'); BEGIN;");
  for (let i = 0; i < count; i++) {
    const name = `unrelated_${i}_` + 'x'.repeat(300 * 1024);
    db.exec(`CREATE TABLE "${name}" (v TEXT)`);
  }
  db.exec('COMMIT');
  console.log(count, db.prepare('PRAGMA quick_check').get());
  db.close();
}

Run the same five public CLI commands used in the reproduction:

node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-50.db --table selected --json > selected-50.json
printf 'selected-50 exit %s\n' "$?"
node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-60.db --table selected --json > selected-60.json
printf 'selected-60 exit %s\n' "$?"
node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-50.db --table absent --json > absent-50.json
printf 'absent-50 exit %s\n' "$?"
node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-60.db --table absent --json > absent-60.json
printf 'absent-60 exit %s\n' "$?"
node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-60.db --schema-scope database --json > database-60.json
printf 'database-60 exit %s\n' "$?"

The observed exit codes are 0, 1, 1, 1, 1. The first call returns complete selected_table schema containing only selected, with one TEXT value selected. The second returns the resource constraint below. --table absent returns invalid_request with details.issues[0].path: ["table"] and Selected ordinary table does not exist with that exact name: absent for the smaller catalog, but returns the reply-budget resource constraint for the larger catalog. Explicit whole-database scope also refuses the larger catalog. All five CLI stderr streams were empty. Independent verifier and review replays produced the same outcome sequence; this does not establish frequency on other environments.

Expected behavior

Both --table selected requests should return the complete selected-table schema and one row, subject to genuine documented resource limits. The SQLite contract defines selected-table scope as that table and its indexes and triggers. Its resource section distinguishes a 256 MiB combined input, individual SQLite value/record and SQL-statement limits, a 16 MiB complete worker JSON reply, worker heap and deadline bounds. It does not define a 16 MiB aggregate limit for unrelated catalog names.

The larger database is 55,611,392 bytes with no WAL. Its longest table name is 307,213 bytes and its longest CREATE declaration is 307,237 bytes. The selected declaration remains CREATE TABLE selected (v TEXT). Separate names are separate SQLite values, not one SQLite record. The failed CLI request returns no selected reply, so no exact byte count for that hypothetical reply is claimed.

Impact and scope

P2, supported: an otherwise valid selected-table observation fails for this unusual schema shape. The observed failure is an explicit bounded refusal; no modification of the original database or incorrect successful evidence was observed. The absent-table result is another consequence of the same preselection check, not a separate bug.

Workaround status is unverified. No executed REA option completed this selection while preserving the original database evidence. Renaming or exporting tables changes the inspected artifact. MCP shares the service/provider by source inspection, but was not exercised for this report. No general cross-platform, cross-runtime or installed-release applicability is claimed.

REA version

Source-built checkout at 62bf09809ef7bb224028916424f8f07652a20f4f. The source CLI reports 6.5.0; this is not a reproduction of the published package. Current public main was fetched before submission; the relevant SQLite implementation, CLI, contracts and documentation are unchanged. Current-main applicability is source-inspected, not a separate current-main runtime reproduction.

Runtime and platform

Node.js v24.18.0; npm 11.16.0 for the verifier build; macOS arm64; reported SQLite engine 3.53.1. The verifier rebuilt the independent checkout with npm run build:cached. The CLI used isolated HOME/XDG directories and quiescent local databases. No external provider or service was involved.

Target and artifact context

Two synthetic local SQLite databases generated inline above, 46,346,240 and 55,611,392 bytes, with no WAL. Both have identical selected schema, columns and row values. The aggregate UTF-8 lengths of all main table-list names are 15,360,661 and 18,432,791 bytes respectively.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

The failing selected-table command returns the following fields directly, without a cleanup wrapper. This is an excerpt of the observed CLI error, not a complete envelope.

{
  "code": "resource_constraint",
  "category": "resource_constraint",
  "message": "Complete SQLite evidence exceeds the 16 MiB JSON reply budget; no partial success was returned.",
  "retryable": false,
  "details": {
    "operation": "inspect_sqlite_database",
    "resource": "transport",
    "reported_limits": {
      "sqlite_value_or_record_bytes": 16777216,
      "worker_reply_bytes": 16777216
    }
  }
}

An executed native control reopened each database with new DatabaseSync(file, { readOnly: true, allowExtension: false, limits: { length: 16777216, sqlLength: 16777216 } }), applied trusted_schema=OFF, query_only=ON, mmap_size=0, cell_size_check=ON and BEGIN, and successfully enumerated PRAGMA main.table_list, read the selected declaration and columns, and selected its row. Both returned identical selected facts. This separates the aggregate-name refusal from a native SQLite record or SQL limit.

Evidence provenance is observed, with reproduced public CLI behavior and separate source-derived cause analysis. The failed-case reply size, installed-package behavior and MCP execution are not measured or established.

Typed outcome or recovery expected

Proposed closure check: rerun both fixtures and the same five CLI commands. Both --table selected calls should return complete observations containing only selected and its one row. Both --table absent calls should identify the exact-table selection error at ["table"]. The larger explicit whole-database request may continue to fail under the genuine complete-reply limit. Preserve the independent native value/record, SQL-statement, input, heap and deadline bounds.

Analysis or suspected cause

Source-derived: SqliteDatabaseInspection.ts constructs a separate OutputBudget and retains every main table name before selection lookup or schema filtering. That class counts JSON bytes and throws the observed message. Selected-table declaration filtering happens later. The source comment describes all names as a lower bound on eventual schema output, but selected-table output excludes unrelated tables. The successful native controls and small selected-table control distinguish this from a genuinely oversized selected result. No implementation change is needed to reproduce the failure, and this report does not prescribe a particular fix.

Additional context

The open input-budget classification report #4127 requires an input exceeding 256 MiB; this input is below that limit. The open path-identity report #3324 concerns a symlink followed by a parent component; these simple filenames select the intended databases. Closed SQLite reports #2612, #2613 and #2614 concern completed observations during cleanup, unknown-column correction metadata and absent-file classification. Their triggers and closure checks differ from the selected-scope refusal here. No matching report was found in the canonical open-issue inventory or targeted closed searches.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4184 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect-sqlite-database catalog-60.db --table selected --json` refuses a small ordinary-table inspection because unrelated catalog names exceed an aggregate 16 MiB check. The same selected schema and row succeed with 50 unrelated tables and fail with 60. The larger database is below the 256 MiB input limit, and native SQLite queries succeed under the documented individual value/record and SQL-statement limits. REA returns a complete-reply resource constraint before returning any selected evidence. ### Reproduction From the built source checkout at the revision below, capture the CLI entrypoint and change to a fresh disposable directory. These commands do not change the checkout. ```sh REA_ENTRYPOINT="$PWD/scripts/rea.mjs" repro_dir="$(mktemp -d)" cd "$repro_dir" mkdir home export HOME="$PWD/home" export XDG_CONFIG_HOME="$PWD/home/config" export XDG_CACHE_HOME="$PWD/home/cache" ``` Save this fixture as `make-databases.cjs`, then run `node make-databases.cjs`. Both generated databases pass `quick_check`. Each contains the eight-character table `selected`, one TEXT column and one small row, plus either 50 or 60 unrelated ordinary tables. No long identifier is passed as a process argument. ```js const { DatabaseSync } = require('node:sqlite'); for (const count of [50, 60]) { const db = new DatabaseSync(`catalog-${count}.db`); db.exec("CREATE TABLE selected (v TEXT); INSERT INTO selected VALUES ('selected'); BEGIN;"); for (let i = 0; i < count; i++) { const name = `unrelated_${i}_` + 'x'.repeat(300 * 1024); db.exec(`CREATE TABLE "${name}" (v TEXT)`); } db.exec('COMMIT'); console.log(count, db.prepare('PRAGMA quick_check').get()); db.close(); } ``` Run the same five public CLI commands used in the reproduction: ```sh node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-50.db --table selected --json > selected-50.json printf 'selected-50 exit %s\n' "$?" node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-60.db --table selected --json > selected-60.json printf 'selected-60 exit %s\n' "$?" node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-50.db --table absent --json > absent-50.json printf 'absent-50 exit %s\n' "$?" node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-60.db --table absent --json > absent-60.json printf 'absent-60 exit %s\n' "$?" node "$REA_ENTRYPOINT" inspect-sqlite-database catalog-60.db --schema-scope database --json > database-60.json printf 'database-60 exit %s\n' "$?" ``` The observed exit codes are `0, 1, 1, 1, 1`. The first call returns complete `selected_table` schema containing only `selected`, with one TEXT value `selected`. The second returns the resource constraint below. `--table absent` returns `invalid_request` with `details.issues[0].path: ["table"]` and `Selected ordinary table does not exist with that exact name: absent` for the smaller catalog, but returns the reply-budget resource constraint for the larger catalog. Explicit whole-database scope also refuses the larger catalog. All five CLI stderr streams were empty. Independent verifier and review replays produced the same outcome sequence; this does not establish frequency on other environments. ### Expected behavior Both `--table selected` requests should return the complete selected-table schema and one row, subject to genuine documented resource limits. The [SQLite contract](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/docs/sqlite-databases.md#returned-evidence) defines selected-table scope as that table and its indexes and triggers. Its resource section distinguishes a 256 MiB combined input, individual SQLite value/record and SQL-statement limits, a 16 MiB complete worker JSON reply, worker heap and deadline bounds. It does not define a 16 MiB aggregate limit for unrelated catalog names. The larger database is 55,611,392 bytes with no WAL. Its longest table name is 307,213 bytes and its longest CREATE declaration is 307,237 bytes. The selected declaration remains `CREATE TABLE selected (v TEXT)`. Separate names are separate SQLite values, not one SQLite record. The failed CLI request returns no selected reply, so no exact byte count for that hypothetical reply is claimed. ### Impact and scope P2, supported: an otherwise valid selected-table observation fails for this unusual schema shape. The observed failure is an explicit bounded refusal; no modification of the original database or incorrect successful evidence was observed. The absent-table result is another consequence of the same preselection check, not a separate bug. Workaround status is unverified. No executed REA option completed this selection while preserving the original database evidence. Renaming or exporting tables changes the inspected artifact. MCP shares the service/provider by source inspection, but was not exercised for this report. No general cross-platform, cross-runtime or installed-release applicability is claimed. ### REA version Source-built checkout at [62bf09809ef7bb224028916424f8f07652a20f4f](https://github.com/morluto/rea/commit/62bf09809ef7bb224028916424f8f07652a20f4f). The source CLI reports `6.5.0`; this is not a reproduction of the published package. Current public main was fetched before submission; the relevant SQLite implementation, CLI, contracts and documentation are unchanged. Current-main applicability is source-inspected, not a separate current-main runtime reproduction. ### Runtime and platform Node.js v24.18.0; npm 11.16.0 for the verifier build; macOS arm64; reported SQLite engine 3.53.1. The verifier rebuilt the independent checkout with `npm run build:cached`. The CLI used isolated HOME/XDG directories and quiescent local databases. No external provider or service was involved. ### Target and artifact context Two synthetic local SQLite databases generated inline above, 46,346,240 and 55,611,392 bytes, with no WAL. Both have identical selected schema, columns and row values. The aggregate UTF-8 lengths of all `main` table-list names are 15,360,661 and 18,432,791 bytes respectively. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence The failing selected-table command returns the following fields directly, without a cleanup wrapper. This is an excerpt of the observed CLI error, not a complete envelope. ```json { "code": "resource_constraint", "category": "resource_constraint", "message": "Complete SQLite evidence exceeds the 16 MiB JSON reply budget; no partial success was returned.", "retryable": false, "details": { "operation": "inspect_sqlite_database", "resource": "transport", "reported_limits": { "sqlite_value_or_record_bytes": 16777216, "worker_reply_bytes": 16777216 } } } ``` An executed native control reopened each database with `new DatabaseSync(file, { readOnly: true, allowExtension: false, limits: { length: 16777216, sqlLength: 16777216 } })`, applied `trusted_schema=OFF`, `query_only=ON`, `mmap_size=0`, `cell_size_check=ON` and `BEGIN`, and successfully enumerated `PRAGMA main.table_list`, read the selected declaration and columns, and selected its row. Both returned identical selected facts. This separates the aggregate-name refusal from a native SQLite record or SQL limit. Evidence provenance is observed, with reproduced public CLI behavior and separate source-derived cause analysis. The failed-case reply size, installed-package behavior and MCP execution are not measured or established. ### Typed outcome or recovery expected Proposed closure check: rerun both fixtures and the same five CLI commands. Both `--table selected` calls should return complete observations containing only `selected` and its one row. Both `--table absent` calls should identify the exact-table selection error at `["table"]`. The larger explicit whole-database request may continue to fail under the genuine complete-reply limit. Preserve the independent native value/record, SQL-statement, input, heap and deadline bounds. ### Analysis or suspected cause Source-derived: [SqliteDatabaseInspection.ts](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/sqlite/SqliteDatabaseInspection.ts#L217-L306) constructs a separate `OutputBudget` and retains every `main` table name before selection lookup or schema filtering. That class counts JSON bytes and throws the observed message. Selected-table declaration filtering happens later. The source comment describes all names as a lower bound on eventual schema output, but selected-table output excludes unrelated tables. The successful native controls and small selected-table control distinguish this from a genuinely oversized selected result. No implementation change is needed to reproduce the failure, and this report does not prescribe a particular fix. ### Additional context The open input-budget classification report [#4127](https://github.com/morluto/rea/issues/4127) requires an input exceeding 256 MiB; this input is below that limit. The open path-identity report [#3324](https://github.com/morluto/rea/issues/3324) concerns a symlink followed by a parent component; these simple filenames select the intended databases. Closed SQLite reports #2612, #2613 and #2614 concern completed observations during cleanup, unknown-column correction metadata and absent-file classification. Their triggers and closure checks differ from the selected-scope refusal here. No matching report was found in the canonical open-issue inventory or targeted closed searches.

bsakaguchi closed this issue

2026-10-31 00:31:54 +00:00

bsakaguchi referenced this issue

2026-10-31 00:54:18 +00:00

[PR #1640] [MERGED] fix(test): preserve Ghidra interior-label controls on Mach-O #4717