morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6437] [Bug] inspect-python-pyc loses the sign of negative-zero float constants in JSON #3852

New issue

Closed

opened 2026-10-31 00:44:38 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:44:38 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6437

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

node scripts/rea.mjs inspect-python-pyc "$WORK/two.pyc" --json exits 0 and reports both stored float constants 0.0 and -0.0 as {"kind":"float","value":0}. The result has confidence: "observed", complete coverage, and no limitation identifying sign loss. CPython's independent marshal decode distinguishes the two constants in that same file.

Reproduction

Use a fresh checkout of 9eb829aca34be36ff112826337996f6139f82eff, Node.js and npm, and CPython 3.11 for the fixture/oracle. From the repository root:

npm ci && npm run build:cached
WORK="$(mktemp -d)"
mkdir -p "$WORK/home"
python3.11 - "$WORK" <<'PYTHON'
import marshal, math, pathlib, py_compile, sys
root = pathlib.Path(sys.argv[1])
source = root / "two.py"
source.write_text("A = 0.0\nB = -0.0\n")
target = root / "two.pyc"
py_compile.compile(str(source), cfile=str(target), dfile="two.py", doraise=True)
code = marshal.loads(target.read_bytes()[16:])
print("CPython constants:", [repr(x) for x in code.co_consts])
print("Float signs:", [math.copysign(1, x) for x in code.co_consts
                       if isinstance(x, float)])
PYTHON
HOME="$WORK/home" node scripts/rea.mjs inspect-python-pyc "$WORK/two.pyc" --json > "$WORK/result.json"
python3.11 - "$WORK/result.json" <<'PYTHON'
import json, sys
result = json.load(open(sys.argv[1]))
print(json.dumps(result["normalized_result"]["code_objects"][0]["consts"]))
print("Coverage:", result["normalized_result"]["coverage"]["status"])
print("Confidence:", result["confidence"])
PYTHON

The two-constant case reproduced on all three CLI calls. A separate module containing only B = -0.0 also reproduced on its single call. The observed checkout already had dependencies and runtime built; the first line supplies bootstrap instructions for a fresh checkout and was not rerun for these observations. Neither REA nor the oracle executes the target bytecode.

Expected behavior

Keep positive and negative zero distinguishable in the serialized float summary, or explicitly represent the sign as unknown instead of reporting an observed positive-zero value.

The documented result includes constant summaries and states representation limitations for strings, large integers and non-finite floats. It does not explicitly discuss signed zero. The value-preservation rule requires preserving source values and explicit unknowns when normalization loses information. This is a supported CPython 3.11 input within the documented version range.

Impact and scope

P2: a bounded wrong scalar is presented as observed evidence. An analyst cannot distinguish the stored positive and negative float zeros from this result. The demonstrated CPython marshal decode recovers the sign for these self-authored files, providing an external-inspection workaround when the matching interpreter is available. It does not repair REA's normalized result.

Real-world frequency is unknown. Only CPython 3.11 fixtures and CLI JSON on the platform below were exercised. MCP, other Python versions, other output formats and other platforms were not tested. No security boundary crossing or input mutation was observed.

REA version

6.7.0 (source 9eb829aca3)

Runtime and platform

Node.js v24.22.1; macOS 27.0.1; arm64; CPython 3.11.17 for fixture generation and independent decoding. No native analysis engine was needed. The repository's exact Node 24.18.0 pin was not exercised.

Target and artifact context

Self-authored CPython 3.11 timestamp-based pyc, magic 3495, with top-level constants (0.0, -0.0, None). No live process or external target is required.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

CPython decoding of the same artifact:

CPython constants: ['0.0', '-0.0', 'None']
Float signs: [1.0, -1.0]

CLI result subset:

[{"kind": "float", "value": 0}, {"kind": "float", "value": 0}, {"kind": "none"}]
Coverage: complete
Confidence: observed

An additional direct call to the built production parser returned float values for which Object.is(value, -0) was [false, true]. JSON.stringify of those same constant summaries emitted the collapsed array above. The public CLI reproduction establishes the defect; this parser probe locates the loss at serialization.

Typed outcome or recovery expected

Return a JSON-safe float representation that retains the negative-zero sign, or an explicit unknown sign. Preserve the positive-zero control and the none constant. A generic successful result with two indistinguishable observed scalars leaves the evidence loss unresolved.

Analysis or suspected cause

Source-derived and consistent with the parser probe: jsonFloat returns finite numbers unchanged, and the float summary uses it. JavaScript's JSON serialization then emits 0 for negative zero. The float schema has string alternatives for non-finite floats but no signed-zero alternative. No repair was tested.

Additional context

Open/closed issue searches covered the command name, jsonFloat, PythonPycParser, pyc/float, marshal/zero, and negative/signed zero. Related #4834 and #5705 concern plist readers and different projection paths; #6415 concerns unhashable code objects in pyc containers; #2296 requests Python decompilation. None reports this float-summary behavior. No last-known-good revision was established.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6437 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `node scripts/rea.mjs inspect-python-pyc "$WORK/two.pyc" --json` exits 0 and reports both stored float constants `0.0` and `-0.0` as `{"kind":"float","value":0}`. The result has `confidence: "observed"`, complete coverage, and no limitation identifying sign loss. CPython's independent marshal decode distinguishes the two constants in that same file. ### Reproduction Use a fresh checkout of `9eb829aca34be36ff112826337996f6139f82eff`, Node.js and npm, and CPython 3.11 for the fixture/oracle. From the repository root: ```sh npm ci && npm run build:cached WORK="$(mktemp -d)" mkdir -p "$WORK/home" python3.11 - "$WORK" <<'PYTHON' import marshal, math, pathlib, py_compile, sys root = pathlib.Path(sys.argv[1]) source = root / "two.py" source.write_text("A = 0.0\nB = -0.0\n") target = root / "two.pyc" py_compile.compile(str(source), cfile=str(target), dfile="two.py", doraise=True) code = marshal.loads(target.read_bytes()[16:]) print("CPython constants:", [repr(x) for x in code.co_consts]) print("Float signs:", [math.copysign(1, x) for x in code.co_consts if isinstance(x, float)]) PYTHON HOME="$WORK/home" node scripts/rea.mjs inspect-python-pyc "$WORK/two.pyc" --json > "$WORK/result.json" python3.11 - "$WORK/result.json" <<'PYTHON' import json, sys result = json.load(open(sys.argv[1])) print(json.dumps(result["normalized_result"]["code_objects"][0]["consts"])) print("Coverage:", result["normalized_result"]["coverage"]["status"]) print("Confidence:", result["confidence"]) PYTHON ``` The two-constant case reproduced on all three CLI calls. A separate module containing only `B = -0.0` also reproduced on its single call. The observed checkout already had dependencies and runtime built; the first line supplies bootstrap instructions for a fresh checkout and was not rerun for these observations. Neither REA nor the oracle executes the target bytecode. ### Expected behavior Keep positive and negative zero distinguishable in the serialized float summary, or explicitly represent the sign as unknown instead of reporting an observed positive-zero value. The [documented result](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/docs/python-pyc.md#L44-L57) includes constant summaries and states representation limitations for strings, large integers and non-finite floats. It does not explicitly discuss signed zero. The [value-preservation rule](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/AGENTS.md#L31) requires preserving source values and explicit unknowns when normalization loses information. This is a supported CPython 3.11 input within the [documented version range](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/docs/python-pyc.md#L28-L31). ### Impact and scope P2: a bounded wrong scalar is presented as observed evidence. An analyst cannot distinguish the stored positive and negative float zeros from this result. The demonstrated CPython marshal decode recovers the sign for these self-authored files, providing an external-inspection workaround when the matching interpreter is available. It does not repair REA's normalized result. Real-world frequency is unknown. Only CPython 3.11 fixtures and CLI JSON on the platform below were exercised. MCP, other Python versions, other output formats and other platforms were not tested. No security boundary crossing or input mutation was observed. ### REA version 6.7.0 (source 9eb829aca34be36ff112826337996f6139f82eff) ### Runtime and platform Node.js v24.22.1; macOS 27.0.1; arm64; CPython 3.11.17 for fixture generation and independent decoding. No native analysis engine was needed. The repository's exact Node 24.18.0 pin was not exercised. ### Target and artifact context Self-authored CPython 3.11 timestamp-based pyc, magic 3495, with top-level constants `(0.0, -0.0, None)`. No live process or external target is required. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence CPython decoding of the same artifact: ```text CPython constants: ['0.0', '-0.0', 'None'] Float signs: [1.0, -1.0] ``` CLI result subset: ```text [{"kind": "float", "value": 0}, {"kind": "float", "value": 0}, {"kind": "none"}] Coverage: complete Confidence: observed ``` An additional direct call to the built production parser returned float values for which `Object.is(value, -0)` was `[false, true]`. `JSON.stringify` of those same constant summaries emitted the collapsed array above. The public CLI reproduction establishes the defect; this parser probe locates the loss at serialization. ### Typed outcome or recovery expected Return a JSON-safe float representation that retains the negative-zero sign, or an explicit unknown sign. Preserve the positive-zero control and the `none` constant. A generic successful result with two indistinguishable observed scalars leaves the evidence loss unresolved. ### Analysis or suspected cause Source-derived and consistent with the parser probe: [jsonFloat](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/python/PythonPycParser.ts#L1469-L1477) returns finite numbers unchanged, and the [float summary](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/python/PythonPycParser.ts#L1557-L1558) uses it. JavaScript's JSON serialization then emits `0` for negative zero. The [float schema](https://github.com/morluto/rea/blob/9eb829aca34be36ff112826337996f6139f82eff/src/domain/python/pythonPyc.ts#L114-L125) has string alternatives for non-finite floats but no signed-zero alternative. No repair was tested. ### Additional context Open/closed issue searches covered the command name, `jsonFloat`, `PythonPycParser`, pyc/float, marshal/zero, and negative/signed zero. Related #4834 and #5705 concern plist readers and different projection paths; #6415 concerns unhashable code objects in pyc containers; #2296 requests Python decompilation. None reports this float-summary behavior. No last-known-good revision was established.

bsakaguchi closed this issue

2026-10-31 00:44:38 +00:00

bsakaguchi commented

2026-10-31 00:44:38 +00:00

Author

Owner

@Lew1sWong commented on GitHub (Oct 23, 2026):

Reproduced on current main (e32a0494d), macOS arm64, CPython 3.11: inspect-python-pyc reports both 0.0 and -0.0 as {"kind":"float","value":0}, and -0j as real 0 / imag 0, with complete coverage. I'd like to work on a fix and will open a PR shortly.

@Lew1sWong commented on GitHub (Oct 23, 2026): Reproduced on current main (e32a0494d), macOS arm64, CPython 3.11: `inspect-python-pyc` reports both `0.0` and `-0.0` as `{"kind":"float","value":0}`, and `-0j` as real `0` / imag `0`, with complete coverage. I'd like to work on a fix and will open a PR shortly.

bsakaguchi commented

2026-10-31 00:44:39 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Resolved on main in 06032237c. jsonFloat emits the explicit Python representation -0.0 for negative zero in floats and complex components, and the executed JSON roundtrip control distinguishes it from positive zero. Validation: src/python/PythonPycParser.test.ts — keeps signed zero and non-finite float and complex constants JSON-safe; recorded execution at e1d9ebd3c. Parser/schema/JSON serialization controls only; no packaged CLI rerun or independent CPython execution.

@morluto commented on GitHub (Oct 23, 2026): Resolved on main in [06032237c](https://github.com/morluto/rea/commit/06032237c6755e3dd5f9a8db732c89958b238cc7). jsonFloat emits the explicit Python representation -0.0 for negative zero in floats and complex components, and the executed JSON roundtrip control distinguishes it from positive zero. Validation: `src/python/PythonPycParser.test.ts` — keeps signed zero and non-finite float and complex constants JSON-safe; recorded execution at `e1d9ebd3c`. Parser/schema/JSON serialization controls only; no packaged CLI rerun or independent CPython execution.

bsakaguchi referenced this issue

2026-10-31 01:10:39 +00:00

[PR #3852] [MERGED] fix(javascript): mark resource-skipped files parse_status skipped #5947

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3852

No description provided.

Rows
Columns