morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4710] [Bug] SQLite inventory row sampling duplicates values when decoded column names collide #2459

New issue

Closed

opened 2026-10-17 21:05:00 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-17 21:05:00 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4710

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-sqlite-database with --schema-scope inventory --table t and omitted columns returns successful but incorrect row Evidence for an authored SQLite snapshot. The first column has malformed UTF-8 name bytes FF6F6C; a separate valid column has name �ol, bytes EFBFBD6F6C. Both names decode to �ol. Automatic selection returns columns ["�ol", "�ol"] and values ["replacement-column", "replacement-column"], losing the first column's value from the sample. A read-only SELECT * returns ["first-column", "replacement-column"] from the unchanged file.

Reproduction

Use the built source checkout and pinned runtime listed below. From the checkout root, run these commands. scripts/rea.mjs is also the entrypoint inside the separately installed npm package.

work="$(mktemp -d)"
cat > "$work/fixture.mjs" <<'JS'
import { DatabaseSync } from 'node:sqlite';
import { readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
const root = process.argv[2];
const good = join(root, 'good.sqlite');
const db = new DatabaseSync(good);
db.exec(`CREATE TABLE t("col" TEXT, "�ol" TEXT);
  INSERT INTO t VALUES ('first-column', 'replacement-column');`);
db.close();
const bytes = readFileSync(good);
const marker = Buffer.from('"col"');
const offset = bytes.indexOf(marker);
if (offset < 0 || bytes.indexOf(marker, offset + 1) !== -1) throw Error('ambiguous marker');
bytes[offset + 1] = 0xff;
writeFileSync(join(root, 'malformed.sqlite'), bytes);
JS
node "$work/fixture.mjs" "$work"
node_bin="$(command -v node)"
mkdir "$work/home-good" "$work/home-malformed"
env -i HOME="$work/home-good" PATH="$(dirname "$node_bin"):/usr/bin:/bin" NO_COLOR=1 "$node_bin" scripts/rea.mjs inspect-sqlite-database "$work/good.sqlite" --schema-scope inventory --table t --json
env -i HOME="$work/home-malformed" PATH="$(dirname "$node_bin"):/usr/bin:/bin" NO_COLOR=1 "$node_bin" scripts/rea.mjs inspect-sqlite-database "$work/malformed.sqlite" --schema-scope inventory --table t --json

Both commands exit 0. The valid control retains both original values; the malformed copy repeats the second value. The script changes one byte in the stored declaration after closing SQLite and preserves the row payload.

Run this read-only oracle on the malformed copy to distinguish incorrect projection from display replacement:

node --input-type=module - "$work/malformed.sqlite" <<'JS'
import { DatabaseSync } from 'node:sqlite';
const db = new DatabaseSync(process.argv[2], { readOnly: true });
const names = db.prepare("SELECT cid, hex(name) AS name_hex FROM pragma_table_xinfo('t')").all();
const statement = db.prepare('SELECT * FROM t');
statement.setReturnArrays(true);
console.log(JSON.stringify({ names, rows: statement.all() }));
db.close();
JS

The oracle returns name bytes FF6F6C and EFBFBD6F6C, with row values first-column and replacement-column.

Discriminating controls used a fresh HOME per CLI invocation. On the malformed collision file, inventory with --column '�ol' returns the single correct value replacement-column. Omitting --schema-scope inventory, or selecting --schema-scope database, refuses the file with a typed path/invalid_format error for malformed UTF-8 schema SQL. A separate table with a malformed name and an unaffected safe column returns the correct subset when safe is explicitly selected. Automatic projection without a colliding valid identifier correctly returns a typed format refusal.

Expected behavior

Omitting columns selects all visible columns. A successful sample must preserve each selected column's identity and value. If faithful selection is unavailable for malformed identifiers, a meaningful typed malformed-format refusal is acceptable. Compact inventory should continue to omit SQL declarations and expanded column details, with columns_completeness: "not_requested". This report does not require inventory to expand or validate every stored declaration.

Impact and scope

The failure produces misleading successful row evidence on the authored malformed-name collision. It leaves the source bytes unchanged. Both tested builds reproduced the collision once in an eleven-case public CLI matrix per build. The exact reproduction blocks above were also replayed separately as described below. This establishes behavior on these fixtures, not the frequency of malformed names in ordinary databases. Other encodings, runtimes and MCP execution were not tested.

Impact level: P2
Assessment status: supported
Impact basis: A bounded malformed-name collision silently substitutes one selected column's value for another in successful Evidence.
Workaround status: unverified
Workaround basis: The tested expanded scopes refuse the artifact, and explicit valid-column selection returns a correct subset. Neither returns all affected columns faithfully. A complete recovery or export workaround was not verified.

REA version

Fresh source build at main commit 264799b0ff501b2d2da36153fd3afbc063ab9ef4, reporting 6.5.0, and a separate install of the latest published npm package rea-agents@6.5.0, also reporting 6.5.0. The runtime failure reproduced on both.

Runtime and platform

Node v24.18.0 and npm 11.16.0, matching the source toolchain pins; macOS 27.0.1 arm64; observed SQLite 3.53.1. CLI invocations used isolated HOME directories with no ambient client configuration. The source build completed with npm ci and npm run build:cached.

Target and artifact context

An authored, closed, 8192-byte UTF-8 SQLite database with one ordinary table, two TEXT columns, one row and no WAL or rollback journal. The malformed copy changes only the first column name's initial byte. No private database or active analysis target was involved.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Expected source: docs/sqlite-databases.md:44-60 documents independent schema scope and row selection, including all visible columns by default. src/domain/sqlite/sqliteDatabase.ts:25-85 permits explicit inventory with a table and omitted columns.
Failure source: Observed public CLI results on both builds; src/sqlite/SqliteDatabaseInspection.ts:186-209, :297-318, :348-365 and :426-455 locate column discovery, inventory omission and row projection.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Both automatic inventory requests exit 0 and return confidence: "observed". Reduced row fields are:

{
  "valid_control": {
    "columns": ["col", "�ol"],
    "text_values": ["first-column", "replacement-column"]
  },
  "malformed_copy": {
    "columns": ["�ol", "�ol"],
    "text_values": ["replacement-column", "replacement-column"]
  }
}

Every successful inventory control retained scope: "inventory", completeness: "complete", columns: [], and columns_completeness: "not_requested", with no SQL declaration. That completeness describes inventory scope; it does not justify the incorrect row values. The two-column collision results also retain column_count: 2.

The eleven-case matrix per build covered valid and malformed single-column tables, valid and malformed tables with an unaffected column, the valid and malformed collision controls, explicit selections, and default/database guards. All 22 outcomes were parsed and asserted. Before/after SHA-256 values matched for all six fixtures. A separate read-only oracle confirmed distinct raw names and retained row values. Exact reproduction replay used newly authored files and fresh homes for each build.

Typed outcome or recovery expected

For automatic selection on the collision fixture, return faithful data for both distinct stored columns or a meaningful typed format refusal. Preserve successful distinct values on the valid control, correct explicit valid-column subsets, the existing expanded-scope malformed-SQL guard, and compact inventory's not_requested semantics.

Analysis or suspected cause

Source-derived reasoning: inventory omits stored SQL decoding but reads selected-table columns internally through PRAGMA table_xinfo. The binding supplies decoded strings, so both names become �ol. Automatic projection quotes that string twice, and both expressions resolve to the valid second column. The result dimensions still match and the incorrect sample passes validation. Expanded scopes decode stored SQL first and refuse this malformed declaration.

Additional context

The observed defect requires explicit inventory, table selection, omitted columns and a malformed/valid identifier collision. The default-scope format refusal is correct. Open and closed issue and PR searches found no matching report. Existing SQLite reports about selected-schema budgets, path resolution, symlink identity and rejected-column metadata have different triggers and outcomes. No last-known-good version was established.

Originally created by @coygeek on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4710 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect-sqlite-database` with `--schema-scope inventory --table t` and omitted columns returns successful but incorrect row Evidence for an authored SQLite snapshot. The first column has malformed UTF-8 name bytes `FF6F6C`; a separate valid column has name `�ol`, bytes `EFBFBD6F6C`. Both names decode to `�ol`. Automatic selection returns columns `["�ol", "�ol"]` and values `["replacement-column", "replacement-column"]`, losing the first column's value from the sample. A read-only `SELECT *` returns `["first-column", "replacement-column"]` from the unchanged file. ### Reproduction Use the built source checkout and pinned runtime listed below. From the checkout root, run these commands. `scripts/rea.mjs` is also the entrypoint inside the separately installed npm package. ```sh work="$(mktemp -d)" cat > "$work/fixture.mjs" <<'JS' import { DatabaseSync } from 'node:sqlite'; import { readFileSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; const root = process.argv[2]; const good = join(root, 'good.sqlite'); const db = new DatabaseSync(good); db.exec(`CREATE TABLE t("col" TEXT, "�ol" TEXT); INSERT INTO t VALUES ('first-column', 'replacement-column');`); db.close(); const bytes = readFileSync(good); const marker = Buffer.from('"col"'); const offset = bytes.indexOf(marker); if (offset < 0 || bytes.indexOf(marker, offset + 1) !== -1) throw Error('ambiguous marker'); bytes[offset + 1] = 0xff; writeFileSync(join(root, 'malformed.sqlite'), bytes); JS node "$work/fixture.mjs" "$work" node_bin="$(command -v node)" mkdir "$work/home-good" "$work/home-malformed" env -i HOME="$work/home-good" PATH="$(dirname "$node_bin"):/usr/bin:/bin" NO_COLOR=1 "$node_bin" scripts/rea.mjs inspect-sqlite-database "$work/good.sqlite" --schema-scope inventory --table t --json env -i HOME="$work/home-malformed" PATH="$(dirname "$node_bin"):/usr/bin:/bin" NO_COLOR=1 "$node_bin" scripts/rea.mjs inspect-sqlite-database "$work/malformed.sqlite" --schema-scope inventory --table t --json ``` Both commands exit 0. The valid control retains both original values; the malformed copy repeats the second value. The script changes one byte in the stored declaration after closing SQLite and preserves the row payload. Run this read-only oracle on the malformed copy to distinguish incorrect projection from display replacement: ```sh node --input-type=module - "$work/malformed.sqlite" <<'JS' import { DatabaseSync } from 'node:sqlite'; const db = new DatabaseSync(process.argv[2], { readOnly: true }); const names = db.prepare("SELECT cid, hex(name) AS name_hex FROM pragma_table_xinfo('t')").all(); const statement = db.prepare('SELECT * FROM t'); statement.setReturnArrays(true); console.log(JSON.stringify({ names, rows: statement.all() })); db.close(); JS ``` The oracle returns name bytes `FF6F6C` and `EFBFBD6F6C`, with row values `first-column` and `replacement-column`. Discriminating controls used a fresh HOME per CLI invocation. On the malformed collision file, inventory with `--column '�ol'` returns the single correct value `replacement-column`. Omitting `--schema-scope inventory`, or selecting `--schema-scope database`, refuses the file with a typed path/`invalid_format` error for malformed UTF-8 schema SQL. A separate table with a malformed name and an unaffected `safe` column returns the correct subset when `safe` is explicitly selected. Automatic projection without a colliding valid identifier correctly returns a typed format refusal. ### Expected behavior Omitting columns selects all visible columns. A successful sample must preserve each selected column's identity and value. If faithful selection is unavailable for malformed identifiers, a meaningful typed malformed-format refusal is acceptable. Compact inventory should continue to omit SQL declarations and expanded column details, with `columns_completeness: "not_requested"`. This report does not require inventory to expand or validate every stored declaration. ### Impact and scope The failure produces misleading successful row evidence on the authored malformed-name collision. It leaves the source bytes unchanged. Both tested builds reproduced the collision once in an eleven-case public CLI matrix per build. The exact reproduction blocks above were also replayed separately as described below. This establishes behavior on these fixtures, not the frequency of malformed names in ordinary databases. Other encodings, runtimes and MCP execution were not tested. - Impact level: P2 - Assessment status: supported - Impact basis: A bounded malformed-name collision silently substitutes one selected column's value for another in successful Evidence. - Workaround status: unverified - Workaround basis: The tested expanded scopes refuse the artifact, and explicit valid-column selection returns a correct subset. Neither returns all affected columns faithfully. A complete recovery or export workaround was not verified. ### REA version Fresh source build at main commit `264799b0ff501b2d2da36153fd3afbc063ab9ef4`, reporting `6.5.0`, and a separate install of the latest published npm package `rea-agents@6.5.0`, also reporting `6.5.0`. The runtime failure reproduced on both. ### Runtime and platform Node v24.18.0 and npm 11.16.0, matching the source toolchain pins; macOS 27.0.1 arm64; observed SQLite 3.53.1. CLI invocations used isolated HOME directories with no ambient client configuration. The source build completed with `npm ci` and `npm run build:cached`. ### Target and artifact context An authored, closed, 8192-byte UTF-8 SQLite database with one ordinary table, two TEXT columns, one row and no WAL or rollback journal. The malformed copy changes only the first column name's initial byte. No private database or active analysis target was involved. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence - Expected source: `docs/sqlite-databases.md:44-60` documents independent schema scope and row selection, including all visible columns by default. `src/domain/sqlite/sqliteDatabase.ts:25-85` permits explicit inventory with a table and omitted columns. - Failure source: Observed public CLI results on both builds; `src/sqlite/SqliteDatabaseInspection.ts:186-209`, `:297-318`, `:348-365` and `:426-455` locate column discovery, inventory omission and row projection. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete Both automatic inventory requests exit 0 and return `confidence: "observed"`. Reduced row fields are: ```json { "valid_control": { "columns": ["col", "�ol"], "text_values": ["first-column", "replacement-column"] }, "malformed_copy": { "columns": ["�ol", "�ol"], "text_values": ["replacement-column", "replacement-column"] } } ``` Every successful inventory control retained `scope: "inventory"`, `completeness: "complete"`, `columns: []`, and `columns_completeness: "not_requested"`, with no SQL declaration. That completeness describes inventory scope; it does not justify the incorrect row values. The two-column collision results also retain `column_count: 2`. The eleven-case matrix per build covered valid and malformed single-column tables, valid and malformed tables with an unaffected column, the valid and malformed collision controls, explicit selections, and default/database guards. All 22 outcomes were parsed and asserted. Before/after SHA-256 values matched for all six fixtures. A separate read-only oracle confirmed distinct raw names and retained row values. Exact reproduction replay used newly authored files and fresh homes for each build. ### Typed outcome or recovery expected For automatic selection on the collision fixture, return faithful data for both distinct stored columns or a meaningful typed format refusal. Preserve successful distinct values on the valid control, correct explicit valid-column subsets, the existing expanded-scope malformed-SQL guard, and compact inventory's `not_requested` semantics. ### Analysis or suspected cause Source-derived reasoning: inventory omits stored SQL decoding but reads selected-table columns internally through `PRAGMA table_xinfo`. The binding supplies decoded strings, so both names become `�ol`. Automatic projection quotes that string twice, and both expressions resolve to the valid second column. The result dimensions still match and the incorrect sample passes validation. Expanded scopes decode stored SQL first and refuse this malformed declaration. ### Additional context The observed defect requires explicit inventory, table selection, omitted columns and a malformed/valid identifier collision. The default-scope format refusal is correct. Open and closed issue and PR searches found no matching report. Existing SQLite reports about selected-schema budgets, path resolution, symlink identity and rejected-column metadata have different triggers and outcomes. No last-known-good version was established.

bsakaguchi closed this issue

2026-10-17 21:05:01 +00:00

bsakaguchi commented

2026-10-17 21:05:03 +00:00

Author

Owner

@xu91102 commented on GitHub (Oct 17, 2026):

Independently reproduced on a fresh build of main c6b62754b095a134d6d67877508fa34311a2eda2, Node24.18.0 / SQLite3.53.1 on macOS arm64, through the actual CLI and authored closed snapshots. The raw names FF6F6C / EFBFBD6F6C remain distinct; the read-only original row is first-column, second-column, while automatic inventory returned second-column twice. Seven CLI controls, explicit valid subsets, expanded malformed-SQL rejection and unchanged hashes for all four fixtures were verified.

The correction is pushed to my fork: 700b169. It rejects ambiguous automatic projection in the existing row-selection owner before quoted identifiers can select the same native column. Independent review exposed the ASCII-equivalent variant (FF4F4C decoding to �OL beside valid �ol), which I also reproduced and fixed using the existing SQLite ASCII identifier key. Exact explicit selections, deliberate duplicate valid columns, Unicode distinctions and compact inventory omission remain intact. Related4021/4216/2701 actual diffs do not fix this collision.

Final semantic coverage:10 files122PASS3conditionalSKIP, plus README5PASS; a later single formatting-only line reflow was exactly verified, followed by final check/docs/discovery and a fresh seven-call CLI matrix. Actual compiled CLI and official SDK stdioMCP regressions verify typed refusals, ping and same-connection recovery, explicit subsets, text/structured parity and unchanged source bytes. Independent Standards/Spec review passed after that correction; normal commit/push preserved frozen bytes. Windows/Linux, external-database prevalence and performance are not qualified. No upstream PR or upstream CI exists because ordinary creation remains permission-blocked; no human readiness is attested.

Update (2026-10-17): main commit 748ac7f0df independently fixes the metadata identity boundary, as described in the maintainer comment above. I rebuilt that main commit and reran the original exact-name and ASCII-variant matrices through the public CLI on macOS arm64 / Node 24.18.0 / SQLite 3.53.1: 14 calls passed the expected controls. Automatic selection refuses malformed metadata; explicit valid subsets select the correct second column; valid rows and expanded-schema refusal remain intact. All eight fixture file hashes stayed unchanged and readonly SELECT * still returned both distinct values.

The prepared fork branch is now retained read-only. No upstream PR was created from it, and this is not a claim that my patch was merged or that the previous full local/MCP suite ran on the new main. No further duplicate PR is planned for this root cause.

@xu91102 commented on GitHub (Oct 17, 2026): Independently reproduced on a fresh build of main `c6b62754b095a134d6d67877508fa34311a2eda2`, Node24.18.0 / SQLite3.53.1 on macOS arm64, through the actual CLI and authored closed snapshots. The raw names `FF6F6C` / `EFBFBD6F6C` remain distinct; the read-only original row is `first-column, second-column`, while automatic inventory returned `second-column` twice. Seven CLI controls, explicit valid subsets, expanded malformed-SQL rejection and unchanged hashes for all four fixtures were verified. The correction is pushed to my fork: [700b169](https://github.com/xu91102/rea/commit/700b169c8eeb49aabeb7d5621db986022e5f7c5f). It rejects ambiguous automatic projection in the existing row-selection owner before quoted identifiers can select the same native column. Independent review exposed the ASCII-equivalent variant (`FF4F4C` decoding to �OL beside valid �ol), which I also reproduced and fixed using the existing SQLite ASCII identifier key. Exact explicit selections, deliberate duplicate valid columns, Unicode distinctions and compact inventory omission remain intact. Related4021/4216/2701 actual diffs do not fix this collision. Final semantic coverage:10 files122PASS3conditionalSKIP, plus README5PASS; a later single formatting-only line reflow was exactly verified, followed by final check/docs/discovery and a fresh seven-call CLI matrix. Actual compiled CLI and official SDK stdioMCP regressions verify typed refusals, ping and same-connection recovery, explicit subsets, text/structured parity and unchanged source bytes. Independent Standards/Spec review passed after that correction; normal commit/push preserved frozen bytes. Windows/Linux, external-database prevalence and performance are not qualified. No upstream PR or upstream CI exists because ordinary creation remains permission-blocked; no human readiness is attested. Update (2026-10-17): main commit 748ac7f0df340f0d0569eca2e803a59cd51ffdbd independently fixes the metadata identity boundary, as described in the maintainer comment above. I rebuilt that main commit and reran the original exact-name and ASCII-variant matrices through the public CLI on macOS arm64 / Node 24.18.0 / SQLite 3.53.1: 14 calls passed the expected controls. Automatic selection refuses malformed metadata; explicit valid subsets select the correct second column; valid rows and expanded-schema refusal remain intact. All eight fixture file hashes stayed unchanged and readonly SELECT * still returned both distinct values. The prepared fork branch is now retained read-only. No upstream PR was created from it, and this is not a claim that my patch was merged or that the previous full local/MCP suite ran on the new main. No further duplicate PR is planned for this root cause.

bsakaguchi commented

2026-10-17 21:05:03 +00:00

Author

Owner

@morluto commented on GitHub (Oct 17, 2026):

Published 748ac7f0d on main.

Root cause / structural fix: column metadata crossed the binding as lossy display strings, which were reused as SQL identifiers. Read raw metadata bytes through temp.pragma_table_xinfo(?, 'main') on the private connection with an empty temp schema, then decode strictly at that owner. Stored main tables/views cannot shadow the selected built-in. Inventory-only explicit subsets first compare encoded name bytes, so an unrelated malformed identifier cannot alias a valid requested column. Automatic selection refuses malformed names; valid U+FFFD, caller order/repeats, compact inventory omission and expanded-scope guards are preserved. Retained columns sort by native cid without copying large metadata into SQLite sorter records.

Executed validation: 5 new identity controls covering UTF-8/UTF-16LE/UTF-16BE and malformed-name collision/subset outcomes; 15 selected producer/provider controls across 3 files (33 skipped, including large/stress cases); repeated focused built identity controls; check:fast; docs:check (5 checks); commit/push hooks. Public CLI baseline returned the duplicated second value; fixed automatic selection returned invalid_request with issues[0].reason=invalid_format and the malformed column-name detail. Public CLI explicit valid subset returned its correct value with columns_completeness=not_requested. SHA-256 of the malformed selected source remained identical to the baseline Evidence digest.

Context7/DeepWiki queried and checked against the actual pinned runtime. SQLite PRAGMA documentation supports raw-byte projection through table-valued PRAGMAs with the schema passed last. DeepWiki incorrectly claimed a stored main table shadows the temp-qualified built-in; a 9-case in-memory probe demonstrated the opposite for stored main tables and views. Existing production-boundary shadowing controls passed. This relies on the owned connection keeping temp empty, not on trusting stored schema names.

Costs: two equivalent completed snapshot tasks (inventory/all columns and inventory/reordered repeated subset); all 6 complete JSON replies byte-identical, 976 raw cl100k tokens each. Baseline c7e9e096d.

Phase/build	Elapsed s	CPU s	Sampled process-family peak RSS bytes
First/baseline	0.059898	0.045103	58966016
First/candidate	0.056525	0.043264	58982400
Warm median/baseline	0.057036	0.043991	58040320
Warm median/candidate	0.056790	0.045332	57335808

Two alternating warm repeats, no true cold-cache or performance claim. wait4 CPU and sampled PPID-family RSS can miss transients. Whole-agent, full CLI/MCP schema/retry, private snapshot acquisition/worker, large-file and Windows costs unmeasured. No heavy memory stress/profiling run.

Scope: generated inert offline SQLite files on macOS arm64 / Node24.18.0; actual built-in SQLite, no live application, Windows or all-engine-version qualification. Rebased over independent Cutter/JavaScript changes; affected source/test SHA-256 unchanged. Owned comparison worktree verified inactive/clean and removed normally.

@morluto commented on GitHub (Oct 17, 2026): Published `748ac7f0d` on main. Root cause / structural fix: column metadata crossed the binding as lossy display strings, which were reused as SQL identifiers. Read raw metadata bytes through `temp.pragma_table_xinfo(?, 'main')` on the private connection with an empty temp schema, then decode strictly at that owner. Stored main tables/views cannot shadow the selected built-in. Inventory-only explicit subsets first compare encoded name bytes, so an unrelated malformed identifier cannot alias a valid requested column. Automatic selection refuses malformed names; valid U+FFFD, caller order/repeats, compact inventory omission and expanded-scope guards are preserved. Retained columns sort by native cid without copying large metadata into SQLite sorter records. Executed validation: 5 new identity controls covering UTF-8/UTF-16LE/UTF-16BE and malformed-name collision/subset outcomes; 15 selected producer/provider controls across 3 files (33 skipped, including large/stress cases); repeated focused built identity controls; check:fast; docs:check (5 checks); commit/push hooks. Public CLI baseline returned the duplicated second value; fixed automatic selection returned invalid_request with issues[0].reason=invalid_format and the malformed column-name detail. Public CLI explicit valid subset returned its correct value with columns_completeness=not_requested. SHA-256 of the malformed selected source remained identical to the baseline Evidence digest. Context7/DeepWiki queried and checked against the actual pinned runtime. [SQLite PRAGMA documentation](https://sqlite.org/pragma.html#pragfunc) supports raw-byte projection through table-valued PRAGMAs with the schema passed last. DeepWiki incorrectly claimed a stored main table shadows the temp-qualified built-in; a 9-case in-memory probe demonstrated the opposite for stored main tables and views. Existing production-boundary shadowing controls passed. This relies on the owned connection keeping temp empty, not on trusting stored schema names. Costs: two equivalent completed snapshot tasks (inventory/all columns and inventory/reordered repeated subset); all 6 complete JSON replies byte-identical, 976 raw cl100k tokens each. Baseline c7e9e096d. | Phase/build | Elapsed s | CPU s | Sampled process-family peak RSS bytes | | --- | ---: | ---: | ---: | | First/baseline | 0.059898 | 0.045103 | 58966016 | | First/candidate | 0.056525 | 0.043264 | 58982400 | | Warm median/baseline | 0.057036 | 0.043991 | 58040320 | | Warm median/candidate | 0.056790 | 0.045332 | 57335808 | Two alternating warm repeats, no true cold-cache or performance claim. wait4 CPU and sampled PPID-family RSS can miss transients. Whole-agent, full CLI/MCP schema/retry, private snapshot acquisition/worker, large-file and Windows costs unmeasured. No heavy memory stress/profiling run. Scope: generated inert offline SQLite files on macOS arm64 / Node24.18.0; actual built-in SQLite, no live application, Windows or all-engine-version qualification. Rebased over independent Cutter/JavaScript changes; affected source/test SHA-256 unchanged. Owned comparison worktree verified inactive/clean and removed normally.

bsakaguchi commented

2026-10-17 21:05:03 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 17, 2026):

Solid root-cause analysis — reusing lossy display strings as SQL identifiers is exactly the kind of bug that hides until two byte sequences decode to the same name. Reading raw metadata bytes through temp.pragma_table_xinfo on the private connection and decoding strictly at the owner is the right boundary fix, and keeping explicit subsets compared by encoded bytes closes the alias hole. One UX suggestion: when automatic selection refuses a malformed column name or an ambiguity is detected, include the offending raw bytes (e.g. FF 6F 6C) in the diagnostic message — that makes it actionable for users inspecting the schema, rather than just seeing a rejection.

@XinwangTao commented on GitHub (Oct 17, 2026): Solid root-cause analysis — reusing lossy display strings as SQL identifiers is exactly the kind of bug that hides until two byte sequences decode to the same name. Reading raw metadata bytes through `temp.pragma_table_xinfo` on the private connection and decoding strictly at the owner is the right boundary fix, and keeping explicit subsets compared by encoded bytes closes the alias hole. One UX suggestion: when automatic selection refuses a malformed column name or an ambiguity is detected, include the offending raw bytes (e.g. `FF 6F 6C`) in the diagnostic message — that makes it actionable for users inspecting the schema, rather than just seeing a rejection.

bsakaguchi commented

2026-10-17 21:05:04 +00:00

Author

Owner

@morluto commented on GitHub (Oct 17, 2026):

Fixed on main by 748ac7f0df (column identity is preserved before selection).

Rechecked at 969aafcf2: all five SQLite column-identity regressions pass. A fresh authored SQLite fixture through the built public CLI preserves both valid columns, rejects malformed automatic selection with invalid_request / invalid_format, and returns the correct value for an explicitly selected valid column. This verifies current source/build, not the published npm package.

@morluto commented on GitHub (Oct 17, 2026): Fixed on main by 748ac7f0df340f0d0569eca2e803a59cd51ffdbd (column identity is preserved before selection). Rechecked at 969aafcf2: all five SQLite column-identity regressions pass. A fresh authored SQLite fixture through the built public CLI preserves both valid columns, rejects malformed automatic selection with invalid_request / invalid_format, and returns the correct value for an explicitly selected valid column. This verifies current source/build, not the published npm package.

bsakaguchi referenced this issue

2026-10-31 00:29:51 +00:00

[GH-ISSUE #2975] [Bug] macOS: after the Hopper app exits, the MCP session cannot open any target until the server restarts #1046

bsakaguchi commented

2026-10-31 00:34:13 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 18, 2026):

Closed loop looks good — encoded-name comparison for explicit subsets plus refusal of malformed names in automatic selection covers both sides of the collision. One follow-up on the UX point: does the current rejection message include the offending raw bytes (like FF 6F 6C) in the diagnostic? Schema inspection tools usually display decoded names, so surfacing the raw bytes would make the refusal actionable without digging into the schema by hand.

@XinwangTao commented on GitHub (Oct 18, 2026): Closed loop looks good — encoded-name comparison for explicit subsets plus refusal of malformed names in automatic selection covers both sides of the collision. One follow-up on the UX point: does the current rejection message include the offending raw bytes (like FF 6F 6C) in the diagnostic? Schema inspection tools usually display decoded names, so surfacing the raw bytes would make the refusal actionable without digging into the schema by hand.

bsakaguchi referenced this issue

2026-10-31 01:05:23 +00:00

[PR #2499] [MERGED] fix(hopper): stop Linux demo children when the REA owner exits #5276

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2459

No description provided.

Rows
Columns