morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1554] [Bug] Full MCP input catalog blocks ordinary Copilot CLI chat with gpt-4.1 metadata #523

New issue

Open

opened 2026-10-15 18:25:50 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-15 18:25:50 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 10, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1554

With GitHub Copilot CLI 1.0.95 configured for an offline OpenAI-compatible BYOK endpoint and gpt-4.1 agent model metadata, enabling the complete REA MCP catalog makes ordinary chat fail before the first model HTTP request. REA starts, native MCP discovery succeeds, and all 139 input schemas validate against their declared dialect. No REA analysis is needed to trigger the failure.

Reproduction and scope

Linux x64, Node 24.18.0, @github/copilot@1.0.95 (upstream tag a7ae5b0ce17beddfa5930812bb064138fd3a1cb5).
REA source build at 2d99e1e3, with its default full input-schema profile.
Configure REA through rea setup --client copilot_cli in an isolated COPILOT_HOME.
Use Copilot's documented offline BYOK environment settings: COPILOT_OFFLINE=true, a loopback COPILOT_PROVIDER_BASE_URL, provider type openai, wire API completions, and model ID/wire model gpt-4.1.
Run copilot --disable-builtin-mcps --no-ask-user --output-format json --model gpt-4.1 --prompt "Say hello briefly.".

The native JSONL output reports:

warningType: compaction_static_context_blocked
Static system messages and tool definitions exceed the model's usable context budget. Reduce static context or switch to a larger-context model.
errorType: query
No response was returned. Send your message again to retry.

Controlled observations

Disabling only REA restores chat.
Requesting one million or ten million prompt tokens through the documented COPILOT_PROVIDER_MAX_PROMPT_TOKENS override does not restore this model configuration. The actual effective budget is unknown; these experiments do not establish whether the override is ignored, clamped, or applied elsewhere.
Omitting output schemas alone still fails. Omitting only root input examples still fails.
A private diagnostic projection removing input annotation prose and examples, while preserving validation keywords, allows chat. Replacing inputs with minimal schemas also allows chat, but that is only a diagnostic and weakens the advertisement.
Selecting gpt-5.4 agent model metadata allows ordinary chat with all 139 original input and output schemas intact. Native skill activation and real REA JavaScript analysis also succeed; Copilot spills the larger Evidence result and its native view tool retrieves the complete JSON.

The endpoint in these experiments is a local model-protocol fixture. This establishes native CLI behavior, not live GitHub/OpenAI API acceptance, reasoning quality, Windows support, or a known numeric context capacity. Serialized catalog bytes are not treated as model token counts.

Expected outcome / follow-up

Investigate the client's effective model budget and preserve a reproducible real-client regression. Any future REA mitigation must preserve the complete canonical tool inventory, useful guidance, and parameter-validation semantics; merely deleting tools or weakening nested schemas is not a completed fix. Keep the smaller-model failure documented until an actual supported workflow succeeds.

Originally created by @N0zoM1z0 on GitHub (Oct 10, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1554 With GitHub Copilot CLI 1.0.95 configured for an offline OpenAI-compatible BYOK endpoint and `gpt-4.1` agent model metadata, enabling the complete REA MCP catalog makes ordinary chat fail before the first model HTTP request. REA starts, native MCP discovery succeeds, and all 139 input schemas validate against their declared dialect. No REA analysis is needed to trigger the failure. **Reproduction and scope** - Linux x64, Node 24.18.0, `@github/copilot@1.0.95` (upstream tag a7ae5b0ce17beddfa5930812bb064138fd3a1cb5). - REA source build at 2d99e1e3, with its default full input-schema profile. - Configure REA through `rea setup --client copilot_cli` in an isolated `COPILOT_HOME`. - Use Copilot's documented offline BYOK environment settings: `COPILOT_OFFLINE=true`, a loopback `COPILOT_PROVIDER_BASE_URL`, provider type `openai`, wire API `completions`, and model ID/wire model `gpt-4.1`. - Run `copilot --disable-builtin-mcps --no-ask-user --output-format json --model gpt-4.1 --prompt "Say hello briefly."`. The native JSONL output reports: ```text warningType: compaction_static_context_blocked Static system messages and tool definitions exceed the model's usable context budget. Reduce static context or switch to a larger-context model. errorType: query No response was returned. Send your message again to retry. ``` **Controlled observations** - Disabling only REA restores chat. - Requesting one million or ten million prompt tokens through the documented `COPILOT_PROVIDER_MAX_PROMPT_TOKENS` override does not restore this model configuration. The actual effective budget is unknown; these experiments do not establish whether the override is ignored, clamped, or applied elsewhere. - Omitting output schemas alone still fails. Omitting only root input examples still fails. - A private diagnostic projection removing input annotation prose and examples, while preserving validation keywords, allows chat. Replacing inputs with minimal schemas also allows chat, but that is only a diagnostic and weakens the advertisement. - Selecting `gpt-5.4` agent model metadata allows ordinary chat with all 139 original input and output schemas intact. Native skill activation and real REA JavaScript analysis also succeed; Copilot spills the larger Evidence result and its native `view` tool retrieves the complete JSON. The endpoint in these experiments is a local model-protocol fixture. This establishes native CLI behavior, not live GitHub/OpenAI API acceptance, reasoning quality, Windows support, or a known numeric context capacity. Serialized catalog bytes are not treated as model token counts. **Expected outcome / follow-up** Investigate the client's effective model budget and preserve a reproducible real-client regression. Any future REA mitigation must preserve the complete canonical tool inventory, useful guidance, and parameter-validation semantics; merely deleting tools or weakening nested schemas is not a completed fix. Keep the smaller-model failure documented until an actual supported workflow succeeds.

bsakaguchi added the

area: mcp

bug

P2: medium

labels

2026-10-15 18:25:50 +00:00

bsakaguchi commented

2026-10-15 18:25:51 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

Taking the unassigned #1554 follow-up. I will reproduce the native Copilot 1.0.95 GPT-4.1 path on current main and compare the existing full/compact input-schema profiles, retaining all canonical tools and parameter validation. This is a loopback model-protocol check; no live provider or Windows claim. I will avoid the setup environment-preservation work already in #1607.

@N0zoM1z0 commented on GitHub (Oct 10, 2026): Taking the unassigned #1554 follow-up. I will reproduce the native Copilot 1.0.95 GPT-4.1 path on current main and compare the existing full/compact input-schema profiles, retaining all canonical tools and parameter validation. This is a loopback model-protocol check; no live provider or Windows claim. I will avoid the setup environment-preservation work already in #1607.

bsakaguchi commented

2026-10-15 18:25:51 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

#1624 is merged after all 19 checks passed on d39d1617. Copilot 1.0.95 / GPT-4.1 with the existing explicit compact profile passed native ordinary chat and skill-driven MCP analysis with all 170 tools, Unicode paths, complete Evidence retrieved through the native spill reader, and verified empty process lineages. The optional verifier and installation guide now document this workflow; GPT-5.4 / full also passed.

GPT-4.1 / full still reproduces compaction_static_context_blocked before HTTP. Compact advertisements can reduce nested constraints while the server validates canonical inputs, so this is an alternate workflow rather than a full-profile fix; keeping #1554 open. Linux loopback model only, no live API or Windows claim.

@N0zoM1z0 commented on GitHub (Oct 10, 2026): #1624 is merged after all 19 checks passed on `d39d1617`. Copilot 1.0.95 / GPT-4.1 with the existing explicit compact profile passed native ordinary chat and skill-driven MCP analysis with all 170 tools, Unicode paths, complete Evidence retrieved through the native spill reader, and verified empty process lineages. The optional verifier and installation guide now document this workflow; GPT-5.4 / full also passed. GPT-4.1 / full still reproduces `compaction_static_context_blocked` before HTTP. Compact advertisements can reduce nested constraints while the server validates canonical inputs, so this is an alternate workflow rather than a full-profile fix; keeping #1554 open. Linux loopback model only, no live API or Windows claim.

bsakaguchi commented

2026-10-15 18:25:51 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

A new native Copilot 1.0.95 control preserves the full 170-tool input/output catalog and keeps the actual HTTP wire model gpt-4.1: an explicit BYOK model ID (rea-fixture-gpt-4.1) with the documented prompt-limit setting passes ordinary chat and skill-driven analysis, including complete spilled Evidence readback and verified empty process lineages. Every emitted request was checked for model: gpt-4.1. The same isolated HOME/profile with the built-in gpt-4.1 ID still reproduces compaction_static_context_blocked before HTTP.

This distinguishes model-ID/capability selection from schema rejection without weakening advertisements. It does not prove the effective numeric budget or live API acceptance, and the built-in-ID failure remains open. Preparing a separate verifier/installation-guide follow-up after #1646; no setup-environment changes overlapping #1607.

@N0zoM1z0 commented on GitHub (Oct 10, 2026): A new native Copilot 1.0.95 control preserves the full 170-tool input/output catalog and keeps the actual HTTP wire model `gpt-4.1`: an explicit BYOK model ID (`rea-fixture-gpt-4.1`) with the documented prompt-limit setting passes ordinary chat and skill-driven analysis, including complete spilled Evidence readback and verified empty process lineages. Every emitted request was checked for `model: gpt-4.1`. The same isolated HOME/profile with the built-in `gpt-4.1` ID still reproduces `compaction_static_context_blocked` before HTTP. This distinguishes model-ID/capability selection from schema rejection without weakening advertisements. It does not prove the effective numeric budget or live API acceptance, and the built-in-ID failure remains open. Preparing a separate verifier/installation-guide follow-up after #1646; no setup-environment changes overlapping #1607.

bsakaguchi commented

2026-10-15 18:25:51 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

#1647 is merged after 19/19 exact-head CI checks. The native Copilot verifier now separates and verifies client model IDs and actual wire models, with disposable HOME/USERPROFILE. On source 641f3193, full-profile rea-gpt-4.1 chat and analysis both passed while every HTTP request still used gpt-4.1; default full-profile and existing compact controls also passed. Analysis retrieved complete real REA Evidence.

The built-in gpt-4.1 ID/full-profile control still failed with compaction_static_context_blocked before any HTTP request. This issue stays open. These are Linux installed-native-client results with a deterministic loopback endpoint; effective token capacity, live APIs, and native Windows/macOS client acceptance remain unverified.

@N0zoM1z0 commented on GitHub (Oct 10, 2026): #1647 is merged after 19/19 exact-head CI checks. The native Copilot verifier now separates and verifies client model IDs and actual wire models, with disposable HOME/USERPROFILE. On source `641f3193`, full-profile `rea-gpt-4.1` chat and analysis both passed while every HTTP request still used `gpt-4.1`; default full-profile and existing compact controls also passed. Analysis retrieved complete real REA Evidence. The built-in `gpt-4.1` ID/full-profile control still failed with `compaction_static_context_blocked` before any HTTP request. This issue stays open. These are Linux installed-native-client results with a deterministic loopback endpoint; effective token capacity, live APIs, and native Windows/macOS client acceptance remain unverified.

bsakaguchi commented

2026-10-15 18:25:51 +00:00

Author

Owner

@bhaveshmadisetty commented on GitHub (Oct 11, 2026):

@N0zoM1z0 I’d like to help investigate the remaining built-in gpt-4.1 / full-profile failure, building on #1624 and #1647.

My proposed scope is to reproduce the failing control and the passing custom-model-ID control with the same REA build, catalog, endpoint, wire model, and requested prompt limit. I’ll compare Copilot’s model metadata and effective context-budget selection, then reduce the failure to a minimal MCP reproduction.

I’ll also check whether the same failure persists in a newer Copilot CLI version. If the cause belongs to Copilot’s budget handling, I’ll prepare an upstream reproduction; any REA change would preserve the complete tool inventory and canonical validation.

Are you already investigating this specific boundary? I haven’t reproduced it locally yet and want to avoid duplicating your work.

@bhaveshmadisetty commented on GitHub (Oct 11, 2026): @N0zoM1z0 I’d like to help investigate the remaining built-in gpt-4.1 / full-profile failure, building on #1624 and #1647. My proposed scope is to reproduce the failing control and the passing custom-model-ID control with the same REA build, catalog, endpoint, wire model, and requested prompt limit. I’ll compare Copilot’s model metadata and effective context-budget selection, then reduce the failure to a minimal MCP reproduction. I’ll also check whether the same failure persists in a newer Copilot CLI version. If the cause belongs to Copilot’s budget handling, I’ll prepare an upstream reproduction; any REA change would preserve the complete tool inventory and canonical validation. Are you already investigating this specific boundary? I haven’t reproduced it locally yet and want to avoid duplicating your work.

bsakaguchi referenced this issue

2026-10-16 21:13:43 +00:00

[PR #523] [MERGED] fix(workflows): retain container presence in export shape comparisons #2049

bsakaguchi referenced this issue

2026-10-16 21:15:17 +00:00

[PR #698] [MERGED] fix(comparison): retain container-only export shape paths #2178

bsakaguchi commented

2026-10-31 00:29:16 +00:00

Author

Owner

@morluto commented on GitHub (Oct 26, 2026):

Thanks for preserving the real-client controls and their limits. Since this report, #5975 made compact input advertisements the default, which changes the configuration that produced the failure. I am keeping the issue open because the reported Copilot/model combination has not been shown here to succeed with the current default; a focused rerun would establish whether the smaller-model workflow is resolved.

@morluto commented on GitHub (Oct 26, 2026): Thanks for preserving the real-client controls and their limits. Since this report, #5975 made compact input advertisements the default, which changes the configuration that produced the failure. I am keeping the issue open because the reported Copilot/model combination has not been shown here to succeed with the current default; a focused rerun would establish whether the smaller-model workflow is resolved.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#523

No description provided.

Rows
Columns