morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6795] [Bug] SQLite selected-table metadata aliases distinct names after UTF-8 replacement decoding #4144

New issue

Closed

opened 2026-10-31 00:47:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:47:09 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6795

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-sqlite-database exits 0 and returns incorrect schema facts for an exactly selected table when an unrelated table's invalid UTF-8 name decodes to the same JavaScript string. The valid table �ol has one column and is neither STRICT nor WITHOUT ROWID. REA returns its correct declaration, column definition, and row, but reports column_count: 2, strict: true, and without_rowid: true from the other table. The schema and column completeness fields both say complete.

The two table names have distinct bytes, EFBFBD6F6C and FF6F6C. A native lookup filtered by exact name bytes returns the valid table's correct facts. PRAGMA integrity_check returns ok. Reversing table creation order changes which metadata survives and makes the observed error disappear.

Reproduction

From the tested source checkout with dependencies and the built runtime prepared, use the supported Node runtime. These are the reduced fixture-creation and CLI steps performed; the public paths below replace the investigation directory. Start with an empty /tmp/rea-repro directory.

mkdir -p /tmp/rea-repro
node --input-type=module <<'JS'
import { DatabaseSync } from 'node:sqlite';
import { readFileSync, writeFileSync } from 'node:fs';
const path = '/tmp/rea-repro/selected.db';
const db = new DatabaseSync(path);
db.exec(`
  CREATE TABLE "col"(id INTEGER PRIMARY KEY,value TEXT) WITHOUT ROWID, STRICT;
  INSERT INTO "col" VALUES(1,'unrelated');
  CREATE TABLE "�ol"(value TEXT);
  INSERT INTO "�ol" VALUES('selected');
`);
db.close();
// Change the unrelated table's name in its three stored schema occurrences.
// This is a self-authored fixture; no user database is modified.
const bytes = readFileSync(path);
for (let i = 0; i < bytes.length - 3; i++) {
  if (bytes.subarray(i, i + 3).equals(Buffer.from('col'))) bytes[i] = 0xff;
}
writeFileSync(path, bytes);
const reader = new DatabaseSync(path, { readOnly: true });
console.log(reader.prepare('PRAGMA integrity_check').all());
console.log(reader.prepare(`
  SELECT hex(CAST(name AS BLOB)) AS name_hex,ncol,wr,strict
  FROM temp.pragma_table_list
  WHERE schema='main' AND CAST(name AS BLOB)=CAST(? AS BLOB)
`).all('�ol'));
console.log(reader.prepare('SELECT * FROM "�ol"').all());
reader.close();
JS
node scripts/rea.mjs inspect-sqlite-database /tmp/rea-repro/selected.db --table '�ol' --json

The native control returns ok, ncol: 1, wr: 0, strict: 0, and the row selected. The CLI succeeds with the contradictory metadata below. The error was observed once through the production CLI and once through the production snapshot inspector on the same fixture. An otherwise equivalent fixture with the opposite creation order and a clean selected-table export both returned the correct metadata through the CLI. All three CLI source-file hashes remained unchanged; stderr was empty.

Expected behavior

Keep exact table identity when joining PRAGMA metadata to stored declarations. Return the selected table's own without_rowid: false, strict: false, and column_count: 1, or explicitly refuse an unresolved identity. Never silently attach another table's facts to a successful, complete observation. The repository's identity and uncertainty contract requires source values and unknown identity to survive normalization.

Impact and scope

P2: a bounded schema-inspection input produces silently misattributed Evidence. An analyst can receive incorrect table-layout, constraint, and column-count facts even though the selected declaration and rows are correct. The fixture is a quiescent, integrity-checking SQLite database with one deliberately malformed unrelated identifier and one valid identifier containing literal U+FFFD. No source mutation was observed during inspection.

A direct SQLite metadata query filtered by exact name bytes returned the correct facts for the original file. A clean export also returned correct metadata, but changes artifact provenance and page layout. Real-world frequency, other encodings/platforms, and MCP transport behavior were not established.

REA version

Package version 6.7.0; tested source and prepared built CLI at bd4f5e9c197ba942ed026c9fed6ff3fae663901d.

Runtime and platform

Node.js v24.22.1; Linux 7.2.8-9-omarchy x86_64; built-in SQLite 3.53.4. No external analysis provider.

Target and artifact context

Self-authored 12 KiB UTF-8 SQLite file with two ordinary tables and one row in each; no WAL, generated columns, views, or application extensions. The unrelated table is STRICT and WITHOUT ROWID; the selected table is neither.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Native byte-qualified metadata on the failing file:

[
  {"name_hex":"EFBFBD6F6C","ncol":1,"wr":0,"strict":0},
  {"name_hex":"FF6F6C","ncol":2,"wr":1,"strict":1}
]

Observed CLI normalized_result subset:

{
  "schema": {
    "scope": "selected_table",
    "completeness": "complete",
    "tables": [{
      "name": "�ol",
      "sql": "CREATE TABLE \"�ol\"(value TEXT)",
      "without_rowid": true,
      "strict": true,
      "column_count": 2,
      "columns_completeness": "complete",
      "columns": [{"cid":0,"name":"value","declared_type":"TEXT"}]
    }]
  },
  "rows": {
    "columns": ["value"],
    "values": [[{"type":"text","value":"selected","bytes_hex":"73656c6563746564","encoding":"UTF-8"}]],
    "returned_rows": 1
  }
}

The public CLI returned confidence: "observed". A native exact-byte query and PRAGMA table_xinfo('�ol') independently showed one ordinary column. The failing file's SHA-256 was identical before and after the CLI call.

Typed outcome or recovery expected

Accurately attributed selected-table facts. If exact identity cannot be established, return an explicit typed refusal or limitation rather than successful complete Evidence containing another table's metadata.

Analysis or suspected cause

Source-derived: tableRow.name accepts already-decoded text. The PRAGMA main.table_list loop keys its map by that string, so the invalid name and literal replacement-character name collide. Later, the selected declaration receives the surviving row's flags and count. Stored declaration names and column names use raw bytes, but that identity protection is missing from this metadata producer.

The observed creation-order control supports the overwrite explanation. A fix was not attempted.

Additional context

The supplied open-issue inventory contained no matching report. Open #6604 concerns missing collations; #5294 concerns cleanup markers; #6607 concerns omitted database identity in worker resource errors. None describes this table-metadata collision. Live open/closed searches also found closed #4710, which concerns automatic row projection through colliding column names within one table. This report concerns a separate table-list join that attaches another table’s flags and column count to an exact valid table selection; the selected rows remain correct. No regression range was established. Verification finished before drafting; the bug was not re-executed after this draft was created.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The observed production CLI emits successful, complete Evidence with another table's STRICT, WITHOUT ROWID, and column-count facts for a valid exact selection; the demonstrated scope is limited to colliding decoded names in an authored fixture.
Workaround status: available
Workaround basis: A native SQLite metadata query filtered by exact name bytes returned the original selected table's correct facts without changing the source hash. A clean selected-table export also returned correct CLI metadata, with different artifact provenance and page layout.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6795 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior `inspect-sqlite-database` exits 0 and returns incorrect schema facts for an exactly selected table when an unrelated table's invalid UTF-8 name decodes to the same JavaScript string. The valid table `�ol` has one column and is neither `STRICT` nor `WITHOUT ROWID`. REA returns its correct declaration, column definition, and row, but reports `column_count: 2`, `strict: true`, and `without_rowid: true` from the other table. The schema and column completeness fields both say `complete`. The two table names have distinct bytes, `EFBFBD6F6C` and `FF6F6C`. A native lookup filtered by exact name bytes returns the valid table's correct facts. `PRAGMA integrity_check` returns `ok`. Reversing table creation order changes which metadata survives and makes the observed error disappear. ## Reproduction From the tested source checkout with dependencies and the built runtime prepared, use the supported Node runtime. These are the reduced fixture-creation and CLI steps performed; the public paths below replace the investigation directory. Start with an empty `/tmp/rea-repro` directory. ```sh mkdir -p /tmp/rea-repro node --input-type=module <<'JS' import { DatabaseSync } from 'node:sqlite'; import { readFileSync, writeFileSync } from 'node:fs'; const path = '/tmp/rea-repro/selected.db'; const db = new DatabaseSync(path); db.exec(` CREATE TABLE "col"(id INTEGER PRIMARY KEY,value TEXT) WITHOUT ROWID, STRICT; INSERT INTO "col" VALUES(1,'unrelated'); CREATE TABLE "�ol"(value TEXT); INSERT INTO "�ol" VALUES('selected'); `); db.close(); // Change the unrelated table's name in its three stored schema occurrences. // This is a self-authored fixture; no user database is modified. const bytes = readFileSync(path); for (let i = 0; i < bytes.length - 3; i++) { if (bytes.subarray(i, i + 3).equals(Buffer.from('col'))) bytes[i] = 0xff; } writeFileSync(path, bytes); const reader = new DatabaseSync(path, { readOnly: true }); console.log(reader.prepare('PRAGMA integrity_check').all()); console.log(reader.prepare(` SELECT hex(CAST(name AS BLOB)) AS name_hex,ncol,wr,strict FROM temp.pragma_table_list WHERE schema='main' AND CAST(name AS BLOB)=CAST(? AS BLOB) `).all('�ol')); console.log(reader.prepare('SELECT * FROM "�ol"').all()); reader.close(); JS node scripts/rea.mjs inspect-sqlite-database /tmp/rea-repro/selected.db --table '�ol' --json ``` The native control returns `ok`, `ncol: 1`, `wr: 0`, `strict: 0`, and the row `selected`. The CLI succeeds with the contradictory metadata below. The error was observed once through the production CLI and once through the production snapshot inspector on the same fixture. An otherwise equivalent fixture with the opposite creation order and a clean selected-table export both returned the correct metadata through the CLI. All three CLI source-file hashes remained unchanged; stderr was empty. ## Expected behavior Keep exact table identity when joining PRAGMA metadata to stored declarations. Return the selected table's own `without_rowid: false`, `strict: false`, and `column_count: 1`, or explicitly refuse an unresolved identity. Never silently attach another table's facts to a successful, complete observation. The repository's [identity and uncertainty contract](https://github.com/morluto/rea/blob/bd4f5e9c197ba942ed026c9fed6ff3fae663901d/AGENTS.md) requires source values and unknown identity to survive normalization. ## Impact and scope P2: a bounded schema-inspection input produces silently misattributed Evidence. An analyst can receive incorrect table-layout, constraint, and column-count facts even though the selected declaration and rows are correct. The fixture is a quiescent, integrity-checking SQLite database with one deliberately malformed unrelated identifier and one valid identifier containing literal U+FFFD. No source mutation was observed during inspection. A direct SQLite metadata query filtered by exact name bytes returned the correct facts for the original file. A clean export also returned correct metadata, but changes artifact provenance and page layout. Real-world frequency, other encodings/platforms, and MCP transport behavior were not established. ## REA version Package version 6.7.0; tested source and prepared built CLI at `bd4f5e9c197ba942ed026c9fed6ff3fae663901d`. ## Runtime and platform Node.js v24.22.1; Linux 7.2.8-9-omarchy x86_64; built-in SQLite 3.53.4. No external analysis provider. ## Target and artifact context Self-authored 12 KiB UTF-8 SQLite file with two ordinary tables and one row in each; no WAL, generated columns, views, or application extensions. The unrelated table is `STRICT` and `WITHOUT ROWID`; the selected table is neither. ## Interface CLI ## Earliest failure stage Capture or observation ## Relevant output or evidence Native byte-qualified metadata on the failing file: ```json [ {"name_hex":"EFBFBD6F6C","ncol":1,"wr":0,"strict":0}, {"name_hex":"FF6F6C","ncol":2,"wr":1,"strict":1} ] ``` Observed CLI `normalized_result` subset: ```json { "schema": { "scope": "selected_table", "completeness": "complete", "tables": [{ "name": "�ol", "sql": "CREATE TABLE \"�ol\"(value TEXT)", "without_rowid": true, "strict": true, "column_count": 2, "columns_completeness": "complete", "columns": [{"cid":0,"name":"value","declared_type":"TEXT"}] }] }, "rows": { "columns": ["value"], "values": [[{"type":"text","value":"selected","bytes_hex":"73656c6563746564","encoding":"UTF-8"}]], "returned_rows": 1 } } ``` The public CLI returned `confidence: "observed"`. A native exact-byte query and `PRAGMA table_xinfo('�ol')` independently showed one ordinary column. The failing file's SHA-256 was identical before and after the CLI call. ## Typed outcome or recovery expected Accurately attributed selected-table facts. If exact identity cannot be established, return an explicit typed refusal or limitation rather than successful complete Evidence containing another table's metadata. ## Analysis or suspected cause Source-derived: [`tableRow.name` accepts already-decoded text](https://github.com/morluto/rea/blob/bd4f5e9c197ba942ed026c9fed6ff3fae663901d/src/sqlite/SqliteDatabaseInspection.ts#L35). The [`PRAGMA main.table_list` loop keys its map by that string](https://github.com/morluto/rea/blob/bd4f5e9c197ba942ed026c9fed6ff3fae663901d/src/sqlite/SqliteDatabaseInspection.ts#L267), so the invalid name and literal replacement-character name collide. Later, [the selected declaration receives the surviving row's flags and count](https://github.com/morluto/rea/blob/bd4f5e9c197ba942ed026c9fed6ff3fae663901d/src/sqlite/SqliteDatabaseInspection.ts#L385). Stored declaration names and column names use raw bytes, but that identity protection is missing from this metadata producer. The observed creation-order control supports the overwrite explanation. A fix was not attempted. ## Additional context The supplied open-issue inventory contained no matching report. Open #6604 concerns missing collations; #5294 concerns cleanup markers; #6607 concerns omitted database identity in worker resource errors. None describes this table-metadata collision. Live open/closed searches also found closed #4710, which concerns automatic row projection through colliding column names within one table. This report concerns a separate table-list join that attaches another table’s flags and column count to an exact valid table selection; the selected rows remain correct. No regression range was established. Verification finished before drafting; the bug was not re-executed after this draft was created. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The observed production CLI emits successful, complete Evidence with another table's STRICT, WITHOUT ROWID, and column-count facts for a valid exact selection; the demonstrated scope is limited to colliding decoded names in an authored fixture. - Workaround status: available - Workaround basis: A native SQLite metadata query filtered by exact name bytes returned the original selected table's correct facts without changing the source hash. A clean selected-table export also returned correct CLI metadata, with different artifact provenance and page layout.

bsakaguchi closed this issue

2026-10-31 00:47:09 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:43 +00:00

[PR #4144] [CLOSED] fix(managed): match Windows module paths independently of the host #6084

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425