Skip to content

Weekly research scan [2026-07-21] - #143

Merged
jgoldfoot merged 2 commits into
mainfrom
research-scan-2026-07-21
Jul 28, 2026
Merged

Weekly research scan [2026-07-21]#143
jgoldfoot merged 2 commits into
mainfrom
research-scan-2026-07-21

Conversation

@jgoldfoot

Copy link
Copy Markdown
Owner

Draft — for human review. Do not merge without verifying the sources below.

Weekly research scan for the 1–2 weeks to 2026-07-21. Four verified, material findings. All claims are confirmed against primary sources (arXiv abstract pages, official spec changelogs, the benchmark's own site). Anything I could not confirm at a primary source was cut rather than approximated — see Cut / not proposed.

npm test122 passed, 7 suites. prettier --check clean. No changes to package.json, package-lock.json, or .github/workflows.

Findings

# Category Claim Primary source
1 [STALE] MCP revision 2026-07-28 makes the protocol stateless (removes initialize handshake + Mcp-Session-Id) and moves Tasks out of core into the io.modelcontextprotocol/tasks extension. Whitepaper described Tasks as "experimental, added 2025-11-25". spec changelog, release-candidate post
2 [STALE] OSWorld 1.0 superseded by OSWorld 2.0 (site notice 2026-06-26). Whitepaper §9.1 claimed CUAs "today complete approximately 45%" and projected toward 100%. osworld-v1.xlang.ai
3 [NEW] OSWorld 2.0: 108 long-horizon workflows; best frontier agent (Claude Opus 4.8, max thinking, batched tool calls) completes 20.6% at a 500-step budget, 54.8% partial. Median human time ~1.6h/task; ~318 tool calls vs ~30 in OSWorld 1.0. arXiv:2606.29537
4 [CITATION] WebMCP is a Draft Community Group Report (20 July 2026) from the W3C Web Machine Learning CG, exposing document.modelContext. It is not a W3C Standard and not on the Standards Track. Framework described it as MCP "maturing into" browser-native integration. webmachinelearning.github.io/webmcp
5 [CITATION] A2A spec is v1.0.0; governed by the Linux Foundation since June 2025, not simply "Google, 2025". a2a-protocol.org, LF announcement
6 [CITATION] Odysseys and OpAgent both verified real; added authors + dates. Odysseys: 200 live-web tasks, strongest model 44.5%. OpAgent: 71.6% WebArena, submitted 14 Feb 2026. arXiv:2604.24964, arXiv:2602.13559

Proposed edits

docs/whitepaper.md

  • MCP primitives block: Tasks removed from the core list and moved to a new Extensions subsection alongside MCP Apps. This incidentally fixes a pre-existing mismatch where the text said "three primitives" but listed four.
  • New paragraph on protocol-level statelessness and its Layer 5 deployment consequence (sticky sessions no longer required), plus the 12-month deprecation policy and the deprecation of Roots/Sampling/Logging.
  • §9.1 retitled "Preparing for 100% CUA Execution" → "The Long-Horizon Gap" and rewritten around the OSWorld 2.0 result. The argument now runs: short-horizon UI competence and long-horizon completion are different problems, and Layers 2–5 reduce the per-step ambiguity that compounds across a long horizon. This strengthens the framework's case rather than weakening it, but it does reverse the section's original "execution is nearly solved" premise — the substantive judgement call in this PR, and the thing most worth your review.
  • WebMCP discovery paragraph and the market-signal bullet corrected per finding 4.
  • References: OSWorld 2.0 added; MCP/WebMCP/A2A/Odysseys/OpAgent entries updated; list renumbering repaired.

README.md — matching citation updates.

DRIFT.md — one new [QUESTION], see below.

Cut / not proposed

  • A current OSWorld 1.0 SOTA number. Secondary aggregators report 85–85.6% and name a model I could not corroborate anywhere ("Claude Mythos 5"). No primary leaderboard confirmed it, so I cut the figure entirely rather than swap one unverified number for a better-sounding one. §9.1 now cites only verified anchors: the 12.24% original baseline against a 72.36% human baseline, and the 20.6% OSWorld 2.0 result.
  • WebMCP browser-support claims. Secondary sources contradicted each other (Chrome 146 ships / Chrome 149 origin trial) and all used navigator.modelContext, which the primary spec contradicts — it is document.modelContext. No browser-version claims proposed.
  • WCAG 3.0 / ARIA. Checked; WCAG 3.0 remains a Working Draft (March 2026 update) with Recommendation not expected before 2028. Nothing material for the five layers.

Flagged for your decision (DRIFT.md)

Both README and whitepaper recommend <link rel="alternate" type="application/mcp+json" href="/mcp-server" /> for MCP discovery. I could not confirm from any primary source that this media type is registered or that this pattern is specified in either MCP or WebMCP. It may be an original BiModal Design proposal — legitimate to publish, but it currently reads as established practice. Logged as a [QUESTION] in DRIFT.md rather than edited, since rewriting an authored recommendation is your call, not the scan's.

Conventions respected

Benchmark numbers remain directional projections synthesized from agent benchmarks, never presented as a BiModal A/B test. The 20.6% and 44.5% figures are reported as what those benchmarks measured about agents, not about this framework. data-agent-* remains supplementary. npm, not pnpm. Drift in the single DRIFT.md.

Update framework claims against primary-source-verified developments.

- MCP: document the 2026-07-28 revision (stateless core, Extensions
  framework). Tasks moved from experimental core feature to the
  io.modelcontextprotocol/tasks extension.
- OSWorld: 1.0 superseded by OSWorld 2.0 (arXiv:2606.29537). Replace the
  unverifiable "~45% today" figure and the "projecting toward 100%"
  framing with the verified 20.6% long-horizon result.
- WebMCP: correct to a distinct W3C Community Group Draft Report exposing
  document.modelContext, not a later stage of MCP. Not a W3C Standard.
- A2A: spec v1.0.0, Linux Foundation governance.
- Citations: add OSWorld 2.0; add authors/dates to Odysseys and OpAgent.
- DRIFT.md: flag the unverified application/mcp+json discovery pattern.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jgoldfoot
jgoldfoot marked this pull request as ready for review July 28, 2026 15:22
@jgoldfoot
jgoldfoot merged commit d5baa73 into main Jul 28, 2026
15 of 16 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31043ac499

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread DRIFT.md
Comment on lines +12 to +15
### [QUESTION] Unverified MCP discovery pattern (`application/mcp+json`)

Both README.md (Quick Start, Layer 5 snippet) and docs/whitepaper.md
(Standardized WebMCP Discovery) recommend announcing an MCP server with:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep research questions out of DRIFT

This entry explicitly says the documents agree and only the primary-source status of their shared recommendation is uncertain, so it is not a cross-artifact contradiction. Keeping it under Open turns DRIFT.md into a research/decision log rather than a record of genuine unresolved drift; either fix or relabel the recommendation in the source documents, or track the uncertainty elsewhere.

AGENTS.md reference: AGENTS.md:L252-L255

Useful? React with 👍 / 👎.

Comment thread docs/whitepaper.md
Comment on lines +1111 to +1113
round-robin load balancer. The revision also adds a formal feature lifecycle
with a minimum twelve-month deprecation window, and deprecates the Roots,
Sampling, and Logging features.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark deprecated capabilities in the implementation guidance

For readers targeting the newly documented 2026-07-28 MCP revision, this deprecation notice conflicts with the actionable Pattern 11 in docs/implementation-guide.md:957-1015 and examples/mcp-client-capabilities.md:1-75, which still recommend Roots and Sampling without a protocol-version or deprecation warning—they only advise pinning the SDK. Update or version-pin that pattern and example in the same change so current implementers are not directed toward capabilities the whitepaper now says are being retired.

AGENTS.md reference: AGENTS.md:L197-L203

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant