morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #24] Keep installed REA skills and localized product claims current #12

New issue

Closed

opened 2026-10-15 18:19:52 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-15 18:19:52 +00:00

Owner

Originally created by @morluto on GitHub (Jul 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/24

Problem

Existing installations can retain an older minimal skill while the canonical packaged skill evolves. Product counts can drift across localized documentation.

Scope

Detect skill drift; upgrade transactionally with backup/readback; version the skill; test older upgrades; generate or centrally validate tool counts and workflow claims.

Acceptance criteria

Setup updates stale managed skills without touching unrelated skills.
Package verification proves canonical readback.
Tool-family counts cannot diverge across code/docs.

Roadmap expansion

#72
Reject stale counts, manually drifted ledgers, and mismatched tool inventories in CI.
Verify installed skill version, upgrade, and readback.
Keep generated artifacts reproducible and sanitized.

This contract-hardening slice is no longer blocked by #21; broader Evidence roadmap work remains tracked independently.

Originally created by @morluto on GitHub (Jul 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/24 ## Problem Existing installations can retain an older minimal skill while the canonical packaged skill evolves. Product counts can drift across localized documentation. ## Scope Detect skill drift; upgrade transactionally with backup/readback; version the skill; test older upgrades; generate or centrally validate tool counts and workflow claims. ## Acceptance criteria - Setup updates stale managed skills without touching unrelated skills. - Package verification proves canonical readback. - Tool-family counts cannot diverge across code/docs. ## Roadmap expansion - [x] #72 - [ ] Reject stale counts, manually drifted ledgers, and mismatched tool inventories in CI. - [ ] Verify installed skill version, upgrade, and readback. - [ ] Keep generated artifacts reproducible and sanitized. This contract-hardening slice is no longer blocked by #21; broader Evidence roadmap work remains tracked independently.

bsakaguchi

2026-10-15 18:19:52 +00:00

closed this issue
added the
area: build

roadmap

documentation

P2: medium
labels

bsakaguchi commented

2026-10-15 18:19:54 +00:00

Author

Owner

@morluto commented on GitHub (Jul 16, 2026):

Current drift findings from REA 1.7.0 and `main`

A live investigation found three simultaneously visible catalog generations:

Surface	Skill version	MCP tool count
Installed `~/.agents/skills/rea-analysis/SKILL.md`	12	68
Published `rea-agents@1.7.0` package/server	14	78
Current `main` (11 commits after the 1.7.0 tag)	18	84

The 84-tool state is unreleased development state, so the published 78 versus checkout 84 difference is expected. The installed v12 skill is genuinely stale relative to the active v14/78 server.

Doctor detects the drift but still reports healthy

rea doctor reports:

healthy: true
skill:
  installed_version: "12"
  state: stale
  remediation: Run rea setup to update the installed REA skill.

Skill state is identity metadata but is not added to checks, so it cannot affect doctorHealthy. This weakens the skill's own readiness workflow: an agent can see healthy: true, continue, and operate with instructions that omit live capabilities or describe an older catalog.

Consider either:

adding stale skill/catalog identity as a config_drift check, making healthy false; or
introducing an explicit ready_with_drift/degraded state so provider readiness and agent-instruction alignment remain distinguishable.

Doctor should also report the installed skill's declared tool count beside expected package count/digest. A version comparison alone does not identify whether the mismatch affects available workflows.

Concrete release-verifier drift on current main

scripts/verify-published-package.mjs currently hard-codes 80 tools, while current main runtime and contract tests expect 84. The release workflow invokes this verifier only after npm publish, so the stale assertion can discover the mismatch only after the package has already shipped.

This should be corrected by:

deriving the expected count/digest from the packed canonical product catalog instead of another literal;
running the packed-artifact identity/catalog check before publish;
retaining the post-publish check as registry propagation verification;
testing an old-package -> upgrade -> setup -> restart lifecycle, including skill backup/readback and live-state unknowns.

Relationship to #261

The stale skill did not create the trace_feature.max_operations validation failure in #261, but it compounds capability discovery failures: host-rendered schemas, live server capabilities, and installed workflow instructions can each describe a different effective surface. #261 should own bounded-argument discovery/error repair; this issue should own cross-version skill/catalog alignment and release verification.

@morluto commented on GitHub (Jul 16, 2026): ## Current drift findings from REA 1.7.0 and `main` A live investigation found three simultaneously visible catalog generations: | Surface | Skill version | MCP tool count | | --- | ---: | ---: | | Installed `~/.agents/skills/rea-analysis/SKILL.md` | 12 | 68 | | Published `rea-agents@1.7.0` package/server | 14 | 78 | | Current `main` (11 commits after the 1.7.0 tag) | 18 | 84 | The 84-tool state is unreleased development state, so the published 78 versus checkout 84 difference is expected. The installed v12 skill is genuinely stale relative to the active v14/78 server. ### Doctor detects the drift but still reports healthy `rea doctor` reports: ```text healthy: true skill: installed_version: "12" state: stale remediation: Run rea setup to update the installed REA skill. ``` Skill state is identity metadata but is not added to `checks`, so it cannot affect `doctorHealthy`. This weakens the skill's own readiness workflow: an agent can see `healthy: true`, continue, and operate with instructions that omit live capabilities or describe an older catalog. Consider either: - adding stale skill/catalog identity as a `config_drift` check, making `healthy` false; or - introducing an explicit `ready_with_drift`/degraded state so provider readiness and agent-instruction alignment remain distinguishable. Doctor should also report the installed skill's declared tool count beside expected package count/digest. A version comparison alone does not identify whether the mismatch affects available workflows. ### Concrete release-verifier drift on current main `scripts/verify-published-package.mjs` currently hard-codes 80 tools, while current `main` runtime and contract tests expect 84. The release workflow invokes this verifier only after `npm publish`, so the stale assertion can discover the mismatch only after the package has already shipped. This should be corrected by: 1. deriving the expected count/digest from the packed canonical product catalog instead of another literal; 2. running the packed-artifact identity/catalog check before publish; 3. retaining the post-publish check as registry propagation verification; 4. testing an old-package -> upgrade -> setup -> restart lifecycle, including skill backup/readback and live-state unknowns. ### Relationship to #261 The stale skill did not create the `trace_feature.max_operations` validation failure in #261, but it compounds capability discovery failures: host-rendered schemas, live server capabilities, and installed workflow instructions can each describe a different effective surface. #261 should own bounded-argument discovery/error repair; this issue should own cross-version skill/catalog alignment and release verification.

bsakaguchi commented

2026-10-15 18:19:55 +00:00

Author

Owner

@morluto commented on GitHub (Jul 16, 2026):

REA 2.0 implementation slice: skill metadata is generated from CATALOG_IDENTITY with version, tool_count, and catalog_digest; doctor marks missing/stale identity unhealthy with rea setup remediation; isolated package verification proves backup-first stale-skill replacement and aligned readback. Broader #21/#72 work remains out of scope.

@morluto commented on GitHub (Jul 16, 2026): REA 2.0 implementation slice: skill metadata is generated from CATALOG_IDENTITY with version, tool_count, and catalog_digest; doctor marks missing/stale identity unhealthy with rea setup remediation; isolated package verification proves backup-first stale-skill replacement and aligned readback. Broader #21/#72 work remains out of scope.

bsakaguchi commented

2026-10-15 18:19:55 +00:00

Author

Owner

@morluto commented on GitHub (Jul 17, 2026):

Completed on current . Installed-skill metadata is generated with version, tool count, and catalog digest; doctor detects missing/stale installs; setup uses backup plus readback verification; and packaged installation is covered by . Broader evidence/ledger automation remains in its own issues.

@morluto commented on GitHub (Jul 17, 2026): Completed on current . Installed-skill metadata is generated with version, tool count, and catalog digest; doctor detects missing/stale installs; setup uses backup plus readback verification; and packaged installation is covered by . Broader evidence/ledger automation remains in its own issues.

bsakaguchi commented

2026-10-15 18:19:55 +00:00

Author

Owner

@morluto commented on GitHub (Jul 17, 2026):

Correction to the preceding comment: completed on current main. Installed-skill metadata includes generated version, tool count, and catalog digest; doctor detects missing or stale installs; setup performs backup and readback verification; and scripts/verify-package.mjs covers packaged installation.

@morluto commented on GitHub (Jul 17, 2026): Correction to the preceding comment: completed on current main. Installed-skill metadata includes generated version, tool count, and catalog digest; doctor detects missing or stale installs; setup performs backup and readback verification; and scripts/verify-package.mjs covers packaged installation.

bsakaguchi referenced this issue

2026-10-15 18:25:27 +00:00

[GH-ISSUE #1389] [Design] Add rsleigh as a headless static-analysis provider candidate #483

bsakaguchi referenced this issue

2026-10-15 18:28:29 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-16 21:11:14 +00:00

[PR #12] [MERGED] [codex] Resolve open quality and logging issues #1806