morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6551] [Bug] environment-compare treats numeric format identifiers as clocks #3941

New issue

Closed

opened 2026-10-31 00:45:26 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:45:26 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6551

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The production environment comparison reports full agreement between {"format":1196444237} and {"format":1448695129}, counting the differing integers as one normalized clock. These are the little-endian FourCC identifiers for MJPG and YUYV, so the input describes different video formats. The same integers under a size field correctly produce a value difference. A timestamp control receives the intended clock normalization.

Reproduction

With the source revision below and its locked dependencies installed, save this as probe.mjs outside the checkout and run node --import ./node_modules/tsx/dist/loader.mjs /path/to/probe.mjs "$PWD". This three-case production-service probe was executed once; CLI/MCP transport and a real video application were not executed.

import { pathToFileURL } from 'node:url';
import { join } from 'node:path';
const mod = async p => import(pathToFileURL(join(process.argv[2], p)).href);
const { compareEnvironment } = await mod('src/application/environment/environmentCompare.ts');
const { timedCaptureEvidence } = await mod('tests/fixtures/analysisView.ts');
const { environmentStateServeFixture, environmentServeSource } = await mod('tests/fixtures/environmentServe.ts');
const fourcc = s => Buffer.from(s).readUInt32LE(0);
for (const key of ['format', 'size', 'timestamp']) {
  const fixture = environmentStateServeFixture(timedCaptureEvidence([
    ['GET', '/api/video', { [key]: fourcc('MJPG') }],
  ]));
  const result = compareEnvironment({
    source: environmentServeSource(fixture),
    captureValue: fixture.capture,
    probes: [{ label: 'fourcc', value: timedCaptureEvidence([
      ['GET', '/api/video', { [key]: fourcc('YUYV') }],
    ]) }],
    operation: 'environment-compare',
  });
  if (!result.ok) throw result.error;
  console.log(JSON.stringify({ key, summary: result.value.summary,
    differences: result.value.probes[0].results[0].differences }));
}

Expected behavior

A numeric field named format should retain its value difference. The comparison documentation restricts epoch-number clock normalization to time-named fields; format names a format, rather than a time. The size and actual timestamp controls should retain their respective value comparison and clock normalization.

Impact and scope

The demonstrated video-format change disappears from the fidelity report, which reports agreement 1 and no differences. Numeric values in the recognized epoch range under non-time field names ending in at can reach this branch; broader application frequency is unknown. Manually comparing original response values can expose the difference; no automatic workaround was tested.

REA version

Main source revision 6c4d880f2dd9ca7d42593be916f90ec19b8a09d7. Environment comparison is absent from published rea-agents 6.7.0; this report concerns main.

Runtime and platform

Node.js v24.22.1; Linux x86_64; checkout TypeScript loader.

Target and artifact context

Generated capture Evidence for one successful JSON GET /api/video response. No network requests, live devices, or third-party analysis providers.

Interface

Other

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

The executed cases returned:

field       agreement   normalized.clocks   differences
format      1           1                   []
size        0           0                   value at /size: live 1448695129, env 1196444237
timestamp   1           1                   []

The numbers were generated from the FourCC strings using Buffer.from(value).readUInt32LE(0).

Analysis or suspected cause

Source-derived: the TIME_FIELD predicate and its use match any suffix at, which also matches format. Both integers then have the same inferred epoch-seconds format, and their content difference is discarded.

Additional context

Proposed restoration check, not executed: the format case should report a value difference while a legitimate timestamp field can still normalize by clock format. No fix or post-draft rerun was performed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The executed production-service probe hides a consequential format change in a bounded fidelity comparison.
Workaround status: unverified
Workaround basis: Manual inspection of original values is possible, but no automatic comparison bypass was exercised.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6551 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The production environment comparison reports full agreement between `{"format":1196444237}` and `{"format":1448695129}`, counting the differing integers as one normalized clock. These are the little-endian FourCC identifiers for `MJPG` and `YUYV`, so the input describes different video formats. The same integers under a `size` field correctly produce a value difference. A `timestamp` control receives the intended clock normalization. ### Reproduction With the source revision below and its locked dependencies installed, save this as `probe.mjs` outside the checkout and run `node --import ./node_modules/tsx/dist/loader.mjs /path/to/probe.mjs "$PWD"`. This three-case production-service probe was executed once; CLI/MCP transport and a real video application were not executed. ```js import { pathToFileURL } from 'node:url'; import { join } from 'node:path'; const mod = async p => import(pathToFileURL(join(process.argv[2], p)).href); const { compareEnvironment } = await mod('src/application/environment/environmentCompare.ts'); const { timedCaptureEvidence } = await mod('tests/fixtures/analysisView.ts'); const { environmentStateServeFixture, environmentServeSource } = await mod('tests/fixtures/environmentServe.ts'); const fourcc = s => Buffer.from(s).readUInt32LE(0); for (const key of ['format', 'size', 'timestamp']) { const fixture = environmentStateServeFixture(timedCaptureEvidence([ ['GET', '/api/video', { [key]: fourcc('MJPG') }], ])); const result = compareEnvironment({ source: environmentServeSource(fixture), captureValue: fixture.capture, probes: [{ label: 'fourcc', value: timedCaptureEvidence([ ['GET', '/api/video', { [key]: fourcc('YUYV') }], ]) }], operation: 'environment-compare', }); if (!result.ok) throw result.error; console.log(JSON.stringify({ key, summary: result.value.summary, differences: result.value.probes[0].results[0].differences })); } ``` ### Expected behavior A numeric field named `format` should retain its value difference. The [comparison documentation](https://github.com/morluto/rea/blob/6c4d880f2dd9ca7d42593be916f90ec19b8a09d7/docs/agent-environments.md#measure-fidelity-against-the-application) restricts epoch-number clock normalization to time-named fields; `format` names a format, rather than a time. The `size` and actual timestamp controls should retain their respective value comparison and clock normalization. ### Impact and scope The demonstrated video-format change disappears from the fidelity report, which reports agreement 1 and no differences. Numeric values in the recognized epoch range under non-time field names ending in `at` can reach this branch; broader application frequency is unknown. Manually comparing original response values can expose the difference; no automatic workaround was tested. ### REA version Main source revision `6c4d880f2dd9ca7d42593be916f90ec19b8a09d7`. Environment comparison is absent from published `rea-agents` 6.7.0; this report concerns main. ### Runtime and platform Node.js v24.22.1; Linux x86_64; checkout TypeScript loader. ### Target and artifact context Generated capture Evidence for one successful JSON `GET /api/video` response. No network requests, live devices, or third-party analysis providers. ### Interface Other ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence The executed cases returned: ```text field agreement normalized.clocks differences format 1 1 [] size 0 0 value at /size: live 1448695129, env 1196444237 timestamp 1 1 [] ``` The numbers were generated from the FourCC strings using `Buffer.from(value).readUInt32LE(0)`. ### Analysis or suspected cause Source-derived: the [TIME_FIELD predicate and its use](https://github.com/morluto/rea/blob/6c4d880f2dd9ca7d42593be916f90ec19b8a09d7/src/application/environment/environmentCompareDiff.ts#L53-L79) match any suffix `at`, which also matches `format`. Both integers then have the same inferred epoch-seconds format, and their content difference is discarded. ### Additional context Proposed restoration check, not executed: the `format` case should report a value difference while a legitimate timestamp field can still normalize by clock format. No fix or post-draft rerun was performed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The executed production-service probe hides a consequential format change in a bounded fidelity comparison. - Workaround status: unverified - Workaround basis: Manual inspection of original values is possible, but no automatic comparison bypass was exercised.

bsakaguchi closed this issue

2026-10-31 00:45:26 +00:00

bsakaguchi commented

2026-10-31 00:45:27 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 23, 2026):

Observed behavior

A distinct public HAR-to-CLI reproduction confirms the reported field-name admission failure on fresh main. It uses the built-in HAR inspector and manifest projector rather than fixture-generated Evidence. The non-clock format case hides the numeric difference, the identical values under size differ, and genuine numeric clock controls updated and created_at retain normalization. No live application or video device was exercised.

Reproduction

Run this block from the main source checkout identified below with its pinned toolchain, locked dependencies, compiled runtime, and Python 3. I ran the block verbatim after a fresh npm run build:cached; all 16 public CLI invocations exited 0. The synthetic recorded domain is never contacted. HOME, XDG paths, and temporary files are isolated, and the block cleans its files.

python3 - <<'PYTHON'
import json, os, pathlib, shutil, subprocess, tempfile
repo = pathlib.Path.cwd()
node = shutil.which('node')
assert node and (repo / 'scripts/rea.mjs').is_file()
with tempfile.TemporaryDirectory(prefix='compare-clock-') as directory:
    root = pathlib.Path(directory)
    for name in ('home', 'config', 'cache', 'data', 'temp'):
        (root / name).mkdir()
    env = {'PATH': os.environ['PATH'], 'HOME': str(root / 'home'),
           'XDG_CONFIG_HOME': str(root / 'config'), 'XDG_CACHE_HOME': str(root / 'cache'),
           'XDG_DATA_HOME': str(root / 'data'), 'TMPDIR': str(root / 'temp')}
    started = subprocess.check_output(
        ['date', '-u', '+%Y-%m-%dT%H:%M:%SZ'], env=env, text=True).strip()
    def cli(args, destination):
        result = subprocess.run([node, str(repo / 'scripts/rea.mjs'), *args,
                                 '--format', 'json'], cwd=root, env=env,
                                capture_output=True, text=True, timeout=45)
        assert result.returncode == 0, (args, result.stdout, result.stderr)
        destination.write_text(result.stdout)
        return json.loads(result.stdout)
    def har(body):
        text = json.dumps(body, separators=(',', ':'))
        size = len(text.encode())
        return {'log': {'version': '1.2',
            'creator': {'name': 'comparison-fixture', 'version': '1'},
            'entries': [{'startedDateTime': started, 'time': 1,
                'request': {'method': 'GET', 'url': 'https://app.example/api/settings',
                    'httpVersion': 'HTTP/1.1', 'cookies': [], 'headers': [],
                    'queryString': [], 'headersSize': -1, 'bodySize': 0},
                'response': {'status': 200, 'statusText': 'OK', 'httpVersion': 'HTTP/1.1',
                    'cookies': [], 'headers': [{'name': 'Content-Type', 'value': 'application/json'}],
                    'content': {'size': size, 'mimeType': 'application/json', 'text': text},
                    'redirectURL': '', 'headersSize': -1, 'bodySize': size},
                'cache': {}, 'timings': {'send': 0, 'wait': 1, 'receive': 0}}]}}
    for field in ('format', 'size', 'updated', 'created_at'):
        case = root / field
        case.mkdir()
        captures = {}
        for name, value in (('capture', 1700000000), ('probe', 1700000001)):
            path = case / (name + '.har')
            path.write_text(json.dumps(har({field: value})))
            captures[name] = cli(['inspect-web-network-capture', str(path), 'har'],
                                 case / (name + '.json'))
        view = case / 'view.json'
        view.write_text(json.dumps({'source': {'kind': 'inline', 'evidence': captures['capture']},
                                    'view': {'kind': 'environment-manifest'}}))
        cli(['inspect-analysis-view', str(view)], case / 'manifest.json')
        report = cli(['environment-compare', str(case / 'manifest.json'),
                      '--evidence', str(case / 'capture.json'),
                      '--probe', str(case / 'probe.json')], case / 'report.json')
        step = report['probes'][0]['results'][0]
        print(json.dumps({'field': field, 'state': report['environment']['state'],
            'agreement': report['summary']['agreement'], 'equal': step['equal'],
            'clocks': report['summary']['normalised']['clocks'],
            'differences': step['differences']}))
        # Workaround for this small response: compare the retained JSON facts directly.
        def body(evidence):
            return json.loads(evidence['normalized_result']['records'][0]
                              ['reported']['response']['content']['text'])
        left, right = body(captures['capture']), body(captures['probe'])
        assert left == {field: 1700000000} and right == {field: 1700000001}
        print(json.dumps({'raw_body': field, 'equal': left == right,
                          'capture': left, 'probe': right}))
PYTHON

Impact and scope

For this small response, the executed direct JSON comparison of retained Evidence bodies exposes both exact numbers and their inequality without changing Evidence. This is a verified bounded workaround for detecting this difference; it does not restore the automated agreement score or replace general environment comparison with its id mapping and clock semantics. NOT_ENOUGH_INFO: real API prevalence or practicality for larger responses was not measured.

REA version

Fresh main source b58aa63796fe6465d241db9cb3c72c918c02890c; package and public launcher report 6.7.0. Separately installed latest published rea-agents@6.7.0: launcher reports 6.7.0, but the command is absent from help and this same workflow fails at inspect-analysis-view because environment-manifest is not an admitted view kind. This confirms main reproduction, not published-release affectedness.

Runtime and platform

Node.js v24.22.1; npm 11.16.0; macOS 27.0.1 arm64. Main and release were exercised in isolated local environments, with no optional provider or external application.

Relevant output or evidence

Observed public CLI output:

{"field": "format", "state": "served", "agreement": 1, "equal": true, "clocks": 1, "differences": []}
{"raw_body": "format", "equal": false, "capture": {"format": 1700000000}, "probe": {"format": 1700000001}}
{"field": "size", "state": "served", "agreement": 0, "equal": false, "clocks": 0, "differences": [{"category": "value", "pointer": "/size", "live": "1700000001", "env": "1700000000"}]}
{"raw_body": "size", "equal": false, "capture": {"size": 1700000000}, "probe": {"size": 1700000001}}
{"field": "updated", "state": "served", "agreement": 1, "equal": true, "clocks": 1, "differences": []}
{"raw_body": "updated", "equal": false, "capture": {"updated": 1700000000}, "probe": {"updated": 1700000001}}
{"field": "created_at", "state": "served", "agreement": 1, "equal": true, "clocks": 1, "differences": []}
{"raw_body": "created_at", "equal": false, "capture": {"created_at": 1700000000}, "probe": {"created_at": 1700000001}}

The existing src/application/environment/environmentCompareDiff.test.ts file also passed all 16 tests. The main source's numeric clock gate still matches the terminal at in format; that source-derived explanation agrees with the public result. All original numeric values survived HAR inspection, so the bounded workaround checks retained source facts rather than modified inputs.

Typed outcome or recovery expected

The same four-case reproduction should report a value difference at /format, clocks 0, and agreement 0; keep the size difference and keep updated and created_at at clocks 1 and agreement 1. This proposed restoration check preserves genuine numeric clock normalization; no fix was applied.

@coygeek commented on GitHub (Oct 23, 2026): ### Observed behavior A distinct public HAR-to-CLI reproduction confirms the reported field-name admission failure on fresh main. It uses the built-in HAR inspector and manifest projector rather than fixture-generated Evidence. The non-clock `format` case hides the numeric difference, the identical values under `size` differ, and genuine numeric clock controls `updated` and `created_at` retain normalization. No live application or video device was exercised. ### Reproduction Run this block from the main source checkout identified below with its pinned toolchain, locked dependencies, compiled runtime, and Python 3. I ran the block verbatim after a fresh `npm run build:cached`; all 16 public CLI invocations exited 0. The synthetic recorded domain is never contacted. HOME, XDG paths, and temporary files are isolated, and the block cleans its files. ```sh python3 - <<'PYTHON' import json, os, pathlib, shutil, subprocess, tempfile repo = pathlib.Path.cwd() node = shutil.which('node') assert node and (repo / 'scripts/rea.mjs').is_file() with tempfile.TemporaryDirectory(prefix='compare-clock-') as directory: root = pathlib.Path(directory) for name in ('home', 'config', 'cache', 'data', 'temp'): (root / name).mkdir() env = {'PATH': os.environ['PATH'], 'HOME': str(root / 'home'), 'XDG_CONFIG_HOME': str(root / 'config'), 'XDG_CACHE_HOME': str(root / 'cache'), 'XDG_DATA_HOME': str(root / 'data'), 'TMPDIR': str(root / 'temp')} started = subprocess.check_output( ['date', '-u', '+%Y-%m-%dT%H:%M:%SZ'], env=env, text=True).strip() def cli(args, destination): result = subprocess.run([node, str(repo / 'scripts/rea.mjs'), *args, '--format', 'json'], cwd=root, env=env, capture_output=True, text=True, timeout=45) assert result.returncode == 0, (args, result.stdout, result.stderr) destination.write_text(result.stdout) return json.loads(result.stdout) def har(body): text = json.dumps(body, separators=(',', ':')) size = len(text.encode()) return {'log': {'version': '1.2', 'creator': {'name': 'comparison-fixture', 'version': '1'}, 'entries': [{'startedDateTime': started, 'time': 1, 'request': {'method': 'GET', 'url': 'https://app.example/api/settings', 'httpVersion': 'HTTP/1.1', 'cookies': [], 'headers': [], 'queryString': [], 'headersSize': -1, 'bodySize': 0}, 'response': {'status': 200, 'statusText': 'OK', 'httpVersion': 'HTTP/1.1', 'cookies': [], 'headers': [{'name': 'Content-Type', 'value': 'application/json'}], 'content': {'size': size, 'mimeType': 'application/json', 'text': text}, 'redirectURL': '', 'headersSize': -1, 'bodySize': size}, 'cache': {}, 'timings': {'send': 0, 'wait': 1, 'receive': 0}}]}} for field in ('format', 'size', 'updated', 'created_at'): case = root / field case.mkdir() captures = {} for name, value in (('capture', 1700000000), ('probe', 1700000001)): path = case / (name + '.har') path.write_text(json.dumps(har({field: value}))) captures[name] = cli(['inspect-web-network-capture', str(path), 'har'], case / (name + '.json')) view = case / 'view.json' view.write_text(json.dumps({'source': {'kind': 'inline', 'evidence': captures['capture']}, 'view': {'kind': 'environment-manifest'}})) cli(['inspect-analysis-view', str(view)], case / 'manifest.json') report = cli(['environment-compare', str(case / 'manifest.json'), '--evidence', str(case / 'capture.json'), '--probe', str(case / 'probe.json')], case / 'report.json') step = report['probes'][0]['results'][0] print(json.dumps({'field': field, 'state': report['environment']['state'], 'agreement': report['summary']['agreement'], 'equal': step['equal'], 'clocks': report['summary']['normalised']['clocks'], 'differences': step['differences']})) # Workaround for this small response: compare the retained JSON facts directly. def body(evidence): return json.loads(evidence['normalized_result']['records'][0] ['reported']['response']['content']['text']) left, right = body(captures['capture']), body(captures['probe']) assert left == {field: 1700000000} and right == {field: 1700000001} print(json.dumps({'raw_body': field, 'equal': left == right, 'capture': left, 'probe': right})) PYTHON ``` ### Impact and scope For this small response, the executed direct JSON comparison of retained Evidence bodies exposes both exact numbers and their inequality without changing Evidence. This is a verified bounded workaround for detecting this difference; it does not restore the automated agreement score or replace general environment comparison with its id mapping and clock semantics. NOT_ENOUGH_INFO: real API prevalence or practicality for larger responses was not measured. ### REA version Fresh main source `b58aa63796fe6465d241db9cb3c72c918c02890c`; package and public launcher report 6.7.0. Separately installed latest published `rea-agents@6.7.0`: launcher reports 6.7.0, but the command is absent from help and this same workflow fails at `inspect-analysis-view` because `environment-manifest` is not an admitted view kind. This confirms main reproduction, not published-release affectedness. ### Runtime and platform Node.js v24.22.1; npm 11.16.0; macOS 27.0.1 arm64. Main and release were exercised in isolated local environments, with no optional provider or external application. ### Relevant output or evidence Observed public CLI output: ```jsonl {"field": "format", "state": "served", "agreement": 1, "equal": true, "clocks": 1, "differences": []} {"raw_body": "format", "equal": false, "capture": {"format": 1700000000}, "probe": {"format": 1700000001}} {"field": "size", "state": "served", "agreement": 0, "equal": false, "clocks": 0, "differences": [{"category": "value", "pointer": "/size", "live": "1700000001", "env": "1700000000"}]} {"raw_body": "size", "equal": false, "capture": {"size": 1700000000}, "probe": {"size": 1700000001}} {"field": "updated", "state": "served", "agreement": 1, "equal": true, "clocks": 1, "differences": []} {"raw_body": "updated", "equal": false, "capture": {"updated": 1700000000}, "probe": {"updated": 1700000001}} {"field": "created_at", "state": "served", "agreement": 1, "equal": true, "clocks": 1, "differences": []} {"raw_body": "created_at", "equal": false, "capture": {"created_at": 1700000000}, "probe": {"created_at": 1700000001}} ``` The existing `src/application/environment/environmentCompareDiff.test.ts` file also passed all 16 tests. The main source's numeric clock gate still matches the terminal `at` in `format`; that source-derived explanation agrees with the public result. All original numeric values survived HAR inspection, so the bounded workaround checks retained source facts rather than modified inputs. ### Typed outcome or recovery expected The same four-case reproduction should report a `value` difference at `/format`, clocks 0, and agreement 0; keep the `size` difference and keep `updated` and `created_at` at clocks 1 and agreement 1. This proposed restoration check preserves genuine numeric clock normalization; no fix was applied.

bsakaguchi referenced this issue

2026-10-31 01:11:10 +00:00

[PR #4040] [MERGED] fix(browser): preserve frame identity across captures #6013

bsakaguchi referenced this issue

2026-11-01 21:44:21 +00:00

[PR #5560] fix(browser): ensure cancellation deadline and teardown on hanging CDP calls #6962