MCP v2 compatibility benchmarks
The Model Context Protocol published its 2026-07-28 specification in July 2026. MCPVault uses the new official SDK, which we call MCP v2. One process accepts both protocol generations, so existing client configurations continue to work.
We benchmarked the upgrade before shipping it. Three pairings, one real vault (381 notes), everything over stdio.
Per request, warm connection · median ms
Full numbers (median and p95, in ms)
| metric | MCPVault today | MCP v2 with today’s apps | MCP v2 with new apps |
|---|---|---|---|
| cold start (n=8) | 107.3 · p95 155.5 | 109.8 · p95 116.8 | 105.8 · p95 107.8 |
| tools/list (n=50) | 0.15 · p95 0.28 | 0.14 · p95 0.28 | 0.17 · p95 0.34 |
| get_vault_stats (n=50) | 12.13 · p95 13.95 | 11.4 · p95 12.2 | 11.66 · p95 12.33 |
| read_note (n=50) | 0.46 · p95 0.84 | 0.43 · p95 0.58 | 0.45 · p95 0.65 |
| list_directory (n=50) | 0.45 · p95 0.66 | 0.43 · p95 0.49 | 0.46 · p95 0.64 |
| search_notes (n=20) | 85.53 · p95 107.44 | 84.33 · p95 86.4 | 85.62 · p95 88.15 |
Why MCPVault moved
- One process accepts clients using either protocol generation.
- New MCP fixes and security updates land on the 2.x SDK line.
- The specification supports JSON Schema 2020-12 for more precise tool input definitions.
Stateless HTTP
The 2026-07-28 specification removes per-client server sessions from the HTTP transport. This can simplify running an MCP server behind a load balancer. MCPVault does not expose an HTTP transport today. A separate, opt-in HTTP package is tracked in issue #49. Local stdio will remain the default.
Method
Each pairing spawns mcpvault <vault> --read-only over stdio. Cold start is the median of 8 full connect cycles, spawn to first response. Per-request numbers are medians on a warm connection after one warmup call (50 iterations per tool, 20 for search_notes). Measured 2026-08-17 on macOS, Node 26, SDK 1.30.0 vs 2.0.0. The scripts live in the repo's benchmarks folder, so you can rerun them against your own vault.
For the skeptics
Fair questions a developer should ask about a project benchmarking itself.
Sub-millisecond tool calls? That looks made up.
It's stdio on the same machine: no network, and the file is already in the OS cache. The number isolates server processing, the only part the SDK swap could change.
Eight cold starts and twenty searches is a tiny sample.
True. It's enough to compare medians of a quiet local process, and it's why we claim a tie, never a speedup. Every p95 is in the table above.
MCP v2 wins some rows. Convenient.
Noise. Those differences flip between runs. The claim is only that MCP v2 isn't slower.
You benchmarked your own project and concluded it's fine.
The scripts are in the repo and print the exact numbers on this page. Rerun them on your own vault; if you get different results, open an issue.
Two protocol versions in one server sounds like per-request overhead.
The version is settled once, when the client connects. Requests never re-check it, which is what the flat per-request numbers show.
So nothing at all got slower? Really?
One thing: pinning an MCP v2 client to the new protocol version adds about 110 ms, once, at connect. Only our test harness does that; shipping apps don't pin. Publishing it because you'd find it anyway.