morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6656] [Bug] Safetensors inspection rejects null optional metadata accepted by the reference reader #4034

New issue

Closed

opened 2026-10-31 00:46:14 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:46:14 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6656

[Bug] Safetensors inspection rejects null optional metadata accepted by the reference reader

Labels: bug, triage

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

ModelArtifactService.inspect rejects an 82-byte safetensors file containing one ordinary U8 tensor when the optional root metadata property is "__metadata__": null. It returns BinaryTargetError, constraint file_format_rejected, with reason Safetensors header requires unsigned integer declarations.

The published safetensors 0.8.0 reference reader deserializes the identical bytes as tensor x, dtype U8, shape [1], data [7]. A control omitting the metadata property succeeds in both REA and the reference reader. This was observed at the shared application-service boundary; CLI and MCP transports were not executed.

Reproduction

In the source checkout below with locked dependencies available, save this as /tmp/repro-null-metadata.mts:

import { writeFile } from 'node:fs/promises';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const [checkout,output]=process.argv.slice(2).map(x=>resolve(x));
const {ModelArtifactProvider}=await import(pathToFileURL(join(checkout,'src/model/ModelArtifactProvider.ts')).href);
const {ModelArtifactService}=await import(pathToFileURL(join(checkout,'src/application/model/ModelArtifactService.ts')).href);
const header=Buffer.from('{"__metadata__":null,"x":{"dtype":"U8","shape":[1],"data_offsets":[0,1]}}');
const length=Buffer.alloc(8);length.writeBigUInt64LE(BigInt(header.length));
const path=join(output,'null-metadata.safetensors');
await writeFile(path,Buffer.concat([length,header,Buffer.from([7])]));
const service=new ModelArtifactService(new ModelArtifactProvider());
try {
 const result=await service.inspect({path,detail:'complete'});
 console.log(JSON.stringify(result.ok?{ok:true,tensors:result.value.normalized_result.tensors}:{ok:false,tag:result.error._tag,constraint:result.error.constraint,reason:result.error.reason}));
}finally {await service.close();}

Run from the checkout, supplying an existing disposable output directory:

node_modules/.bin/tsx /tmp/repro-null-metadata.mts "$PWD" /tmp

The portable driver above was executed and produced the error below. An earlier paired service probe exercised this input and the absent-property control; the reference reader read both files. No model execution or external references are involved.

Expected behavior

Treat explicit null optional metadata like absent metadata, while inspecting the valid tensor declarations and bytes. The reference implementation's HashMetadata models __metadata__ as Option<HashMap<String, String>>. Successful execution of safetensors 0.8.0 confirms acceptance of this exact fixture. The format notes defer the JSON subset to the reference implementation.

Impact and scope

A bounded, reference-reader-compatible model file cannot be inspected even though its known tensor declarations and payload are valid. Omitting only the null metadata property in a separately constructed copy succeeds and preserves the tensor. Frequency in third-party model files and other optional representations were not measured. No released-version regression is claimed.

REA version

Source commit 23def04096869b85c4d3c14a73ed09168483c9f6, package version 6.7.0, unreleased main. Published tag rea-agents-6.7.0 points to 27dc42d10249f0410904a75c0997842eb03b931c and does not contain src/model/SafetensorsHeader.ts; this report concerns the main revision only.

Runtime and platform

Node.js v26.12.1; Linux x86_64, Omarchy 4.0.6. Source execution used locked tsx. Reference comparison: Python 3.14.8 and safetensors 0.8.0's published Linux x86_64 wheel, extracted into disposable storage without a global installation. No Hopper or Ghidra.

Target and artifact context

Synthetic 82-byte safetensors file: eight-byte little-endian header length, 73 bytes of UTF-8 JSON, and one payload byte. Header:

{"__metadata__":null,"x":{"dtype":"U8","shape":[1],"data_offsets":[0,1]}}

Interface

Other — shared application service and production file reader.

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

REA portable driver output:

{"ok":false,"tag":"BinaryTargetError","constraint":"file_format_rejected","reason":"Safetensors header requires unsigned integer declarations"}

safetensors.deserialize(path.read_bytes()) on the same file returned:

{"name":"x","dtype":"U8","shape":[1],"data":[7]}

The absent-property control returned REA metadata: [] and a tensor with shape: ["1"], byte_length: 1, parameter_count: "1", and storage_validation: "consistent".

Evidence provenance: observed. Local verification: reproduced. Reproduction completeness: complete for the source-service boundary and independent reference comparison; public CLI/MCP adapters remain untested.

Typed outcome or recovery expected

Successful model artifact with no metadata entries and the known one-byte U8 tensor.

Analysis or suspected cause

Source-derived explanation: SafetensorsHeader.ts:65-108 recognizes strings and containers, then treats all other JSON tokens as unsigned integers. The valid optional null value reaches that integer-only fallback before metadata handling. Lines 204-206 also assume every present metadata property is an object.

Additional context

Searches covered open and closed reports for safetensors/GGUF/model inspection, __metadata__, safetensors null, and the exact integer-declaration error. Related #6637 concerns extra tensor-entry fields and a different exact-field-count refusal. This fixture contains exactly the three required tensor fields and fails earlier on the root metadata value. #6627 concerns large dimensions; this fixture has shape [1]. Closed #5895 concerns length-prefix detection; this fixture passes detection and reaches JSON parsing.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The shared inspection service rejects a small model accepted by the published reference reader; the null-property failure and ordinary control were directly observed.
Workaround status: available
Workaround basis: A separately constructed control omitting only the null optional metadata property succeeds in both readers with the same U8 declaration and payload; modifying the original file is unnecessary.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6656 # [Bug] Safetensors inspection rejects null optional metadata accepted by the reference reader Labels: `bug`, `triage` ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior `ModelArtifactService.inspect` rejects an 82-byte safetensors file containing one ordinary U8 tensor when the optional root metadata property is `"__metadata__": null`. It returns `BinaryTargetError`, constraint `file_format_rejected`, with reason `Safetensors header requires unsigned integer declarations`. The published safetensors 0.8.0 reference reader deserializes the identical bytes as tensor `x`, dtype `U8`, shape `[1]`, data `[7]`. A control omitting the metadata property succeeds in both REA and the reference reader. This was observed at the shared application-service boundary; CLI and MCP transports were not executed. ## Reproduction In the source checkout below with locked dependencies available, save this as `/tmp/repro-null-metadata.mts`: ```ts import { writeFile } from 'node:fs/promises'; import { join, resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; const [checkout,output]=process.argv.slice(2).map(x=>resolve(x)); const {ModelArtifactProvider}=await import(pathToFileURL(join(checkout,'src/model/ModelArtifactProvider.ts')).href); const {ModelArtifactService}=await import(pathToFileURL(join(checkout,'src/application/model/ModelArtifactService.ts')).href); const header=Buffer.from('{"__metadata__":null,"x":{"dtype":"U8","shape":[1],"data_offsets":[0,1]}}'); const length=Buffer.alloc(8);length.writeBigUInt64LE(BigInt(header.length)); const path=join(output,'null-metadata.safetensors'); await writeFile(path,Buffer.concat([length,header,Buffer.from([7])])); const service=new ModelArtifactService(new ModelArtifactProvider()); try { const result=await service.inspect({path,detail:'complete'}); console.log(JSON.stringify(result.ok?{ok:true,tensors:result.value.normalized_result.tensors}:{ok:false,tag:result.error._tag,constraint:result.error.constraint,reason:result.error.reason})); }finally {await service.close();} ``` Run from the checkout, supplying an existing disposable output directory: ```sh node_modules/.bin/tsx /tmp/repro-null-metadata.mts "$PWD" /tmp ``` The portable driver above was executed and produced the error below. An earlier paired service probe exercised this input and the absent-property control; the reference reader read both files. No model execution or external references are involved. ## Expected behavior Treat explicit null optional metadata like absent metadata, while inspecting the valid tensor declarations and bytes. The reference implementation's [HashMetadata](https://github.com/huggingface/safetensors/blob/e246a2560645b7525f5775669ed816eb57c5bcc8/safetensors/src/tensor.rs#L508-L527) models `__metadata__` as `Option<HashMap<String, String>>`. Successful execution of safetensors 0.8.0 confirms acceptance of this exact fixture. The [format notes](https://github.com/huggingface/safetensors/blob/e246a2560645b7525f5775669ed816eb57c5bcc8/README.md#format) defer the JSON subset to the reference implementation. ## Impact and scope A bounded, reference-reader-compatible model file cannot be inspected even though its known tensor declarations and payload are valid. Omitting only the null metadata property in a separately constructed copy succeeds and preserves the tensor. Frequency in third-party model files and other optional representations were not measured. No released-version regression is claimed. ## REA version Source commit `23def04096869b85c4d3c14a73ed09168483c9f6`, package version `6.7.0`, unreleased main. Published tag `rea-agents-6.7.0` points to `27dc42d10249f0410904a75c0997842eb03b931c` and does not contain `src/model/SafetensorsHeader.ts`; this report concerns the main revision only. ## Runtime and platform Node.js v26.12.1; Linux x86_64, Omarchy 4.0.6. Source execution used locked tsx. Reference comparison: Python 3.14.8 and safetensors 0.8.0's published Linux x86_64 wheel, extracted into disposable storage without a global installation. No Hopper or Ghidra. ## Target and artifact context Synthetic 82-byte safetensors file: eight-byte little-endian header length, 73 bytes of UTF-8 JSON, and one payload byte. Header: ```json {"__metadata__":null,"x":{"dtype":"U8","shape":[1],"data_offsets":[0,1]}} ``` ## Interface Other — shared application service and production file reader. ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence REA portable driver output: ```json {"ok":false,"tag":"BinaryTargetError","constraint":"file_format_rejected","reason":"Safetensors header requires unsigned integer declarations"} ``` `safetensors.deserialize(path.read_bytes())` on the same file returned: ```json {"name":"x","dtype":"U8","shape":[1],"data":[7]} ``` The absent-property control returned REA `metadata: []` and a tensor with `shape: ["1"]`, `byte_length: 1`, `parameter_count: "1"`, and `storage_validation: "consistent"`. Evidence provenance: observed. Local verification: reproduced. Reproduction completeness: complete for the source-service boundary and independent reference comparison; public CLI/MCP adapters remain untested. ## Typed outcome or recovery expected Successful model artifact with no metadata entries and the known one-byte U8 tensor. ## Analysis or suspected cause Source-derived explanation: [SafetensorsHeader.ts:65-108](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/model/SafetensorsHeader.ts#L65-L108) recognizes strings and containers, then treats all other JSON tokens as unsigned integers. The valid optional `null` value reaches that integer-only fallback before metadata handling. [Lines 204-206](https://github.com/morluto/rea/blob/23def04096869b85c4d3c14a73ed09168483c9f6/src/model/SafetensorsHeader.ts#L204-L206) also assume every present metadata property is an object. ## Additional context Searches covered open and closed reports for safetensors/GGUF/model inspection, `__metadata__`, safetensors null, and the exact integer-declaration error. Related [#6637](https://github.com/morluto/rea/issues/6637) concerns extra tensor-entry fields and a different exact-field-count refusal. This fixture contains exactly the three required tensor fields and fails earlier on the root metadata value. [#6627](https://github.com/morluto/rea/issues/6627) concerns large dimensions; this fixture has shape `[1]`. Closed [#5895](https://github.com/morluto/rea/issues/5895) concerns length-prefix detection; this fixture passes detection and reaches JSON parsing. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The shared inspection service rejects a small model accepted by the published reference reader; the null-property failure and ordinary control were directly observed. - Workaround status: available - Workaround basis: A separately constructed control omitting only the null optional metadata property succeeds in both readers with the same U8 declaration and payload; modifying the original file is unnecessary.

bsakaguchi closed this issue

2026-10-31 00:46:14 +00:00

bsakaguchi commented

2026-10-31 00:46:14 +00:00

Author

Owner

@Sriram-PR commented on GitHub (Oct 23, 2026):

I'll take this one. Reproduced on main (a281455f) through ModelArtifactService.inspect with detail: "complete": your 82-byte file fails with Safetensors header requires unsigned integer declarations, and the files with __metadata__ left out or set to {"a":"b"} both pass. The header JSON reader in SafetensorsHeader.ts has no case for null, so it falls through to the integer branch.

Plan: let the reader parse null and treat "__metadata__": null as no metadata, which matches the reference Option<HashMap<String, String>>. null anywhere else (a tensor entry, a metadata value, inside shape or offsets) still gets rejected, like the reference does. Tests for the null metadata case plus those rejections.

@Sriram-PR commented on GitHub (Oct 23, 2026): I'll take this one. Reproduced on main (a281455f) through `ModelArtifactService.inspect` with `detail: "complete"`: your 82-byte file fails with `Safetensors header requires unsigned integer declarations`, and the files with `__metadata__` left out or set to `{"a":"b"}` both pass. The header JSON reader in `SafetensorsHeader.ts` has no case for `null`, so it falls through to the integer branch. Plan: let the reader parse `null` and treat `"__metadata__": null` as no metadata, which matches the reference `Option<HashMap<String, String>>`. `null` anywhere else (a tensor entry, a metadata value, inside shape or offsets) still gets rejected, like the reference does. Tests for the null metadata case plus those rejections.

bsakaguchi referenced this issue

2026-10-31 01:11:06 +00:00

[PR #4034] [MERGED] fix(ci): reject false browser environment cancellation passes #6005

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4034

No description provided.

Rows
Columns