MCP Gateway: avoid blocking list calls on startup by RajeshKumar11 · Pull Request #298040 · microsoft/vscode

RajeshKumar11 · 2026-02-26T17:13:58Z

Summary

Avoid blocking MCP gateway list calls (tools/list, resources/list, resources/templates/list) on slow server startup
Refresh servers in the background when cache state is Unknown/Outdated

Rationale

These list endpoints were awaiting server startup, causing large TTFT when the gateway is enabled. Returning cached data immediately and refreshing in the background restores the feature without the startup delay.

Copilot

Pull request overview

This pull request optimizes the MCP Gateway performance by avoiding blocking list calls during server startup, addressing the 1min+ TTFT issue reported in #297780. The core change is to return cached data immediately when available and trigger server refreshes in the background, rather than waiting for all servers to be ready before responding.

Changes:

Removed blocking await Promise.all() call that waited for all servers to be ready before listing tools
Added conditional cache state checks that trigger background server refreshes for Unknown/Outdated states
Modified three list methods to return immediately with cached/live data while refreshing stale servers asynchronously

Copilot · 2026-02-26T17:19:46Z

src/vs/workbench/contrib/mcp/common/mcpGatewayToolBrokerChannel.ts

+			if (cacheState === McpServerCacheState.Unknown || cacheState === McpServerCacheState.Outdated) {
+				// Avoid blocking the tool list on slow server startup; refresh in the background.
+				void this._ensureServerReady(server);
+			}
 			if (cacheState !== McpServerCacheState.Live && cacheState !== McpServerCacheState.Cached && cacheState !== McpServerCacheState.RefreshingFromCached) {
 				continue;


The logic for checking cache state and conditionally refreshing servers in the background is duplicated across three methods (_listTools, _listResources, _listResourceTemplates). Consider extracting this logic into a private helper method to improve maintainability and ensure consistency when the logic needs to change in the future.

Addressed — the duplicated logic has been extracted into _waitForStartup and _shouldUseCachedData, which are shared by _listTools, _listResources, and _listResourceTemplates.

Copilot · 2026-02-26T17:19:47Z

src/vs/workbench/contrib/mcp/common/mcpGatewayToolBrokerChannel.ts

+			if (cacheState === McpServerCacheState.Unknown || cacheState === McpServerCacheState.Outdated) {
+				// Avoid blocking the tool list on slow server startup; refresh in the background.
+				void this._ensureServerReady(server);
+			}
 			if (cacheState !== McpServerCacheState.Live && cacheState !== McpServerCacheState.Cached && cacheState !== McpServerCacheState.RefreshingFromCached) {
 				continue;


With the changes in this PR, the test "starts server when cache state is unknown" on line 123 may no longer be accurate. The server startup is now triggered asynchronously in the background (void this._ensureServerReady) and won't block the listTools call. The test should be updated to verify that: 1) the background refresh is initiated, and 2) the call returns immediately without waiting for the server to be ready. Consider adding an async wait or verification that the server starts eventually in the background.

Updated. The test now asserts that listTools returns the tools (not an empty array) after the grace period, and verifies startCalls === 1. A second test (returns empty tools and does not re-wait if server does not start within grace period) was added to cover the timeout path using a server whose start() never resolves and a short grace period.

connor4312 · 2026-02-26T18:21:22Z

This is nice but I think it's not practical atm for scenarios since Claude SDK and Copilot CLI's both don't support 'dynamic' tools that change after the connection is established. Wonder what you think @TylerLeonhardt

TylerLeonhardt · 2026-02-26T18:45:12Z

@connor4312 it actually may work in the claude case because we react to when there are new mcp tools added (we essentially restart the session).

connor4312 · 2026-02-26T19:12:24Z

The downside is just that the first message sent may lack appropriate tools. Perhaps we have some reasonable timeout here instead. Like 5 seconds -- but it should per server, so each new message should not wait 5s if a server takes forever to start, we should only give it that grace period when we first ask to start it.

Instead of immediately returning empty results for Unknown-state servers and refreshing in the background, wait up to 5 seconds for the server to become live on the first list call. Subsequent calls find the already- resolved promise in _startupGrace and return immediately, so only the first message pays the startup cost. - _waitForStartup: races _ensureServerReady against the configurable grace period (default 5 000 ms); result is cached per server - _shouldUseCachedData: awaits the grace period for Unknown servers, falls back to background-refresh for Outdated servers - _listTools: refactored to Promise.all to parallelise per-server waits - Tests: updated 'starts server when cache state is unknown' to assert tools are returned after the grace period; added new test for the timeout path using createNeverStartingServer

RajeshKumar11 · 2026-02-27T09:23:39Z

Thanks for the discussion @connor4312 @TylerLeonhardt — updated with the per-server grace period approach.

The new behaviour:

On the first //\ call for an Unknown-state server, we race server startup against a 5 s timeout. If the server comes up in time, its tools are included in that response.
On subsequent calls the stored promise is already resolved, so no wait occurs — the server either contributes its tools (if it started) or is skipped.
Outdated servers continue to refresh in the background and return cached data immediately (no grace period needed since cached data is available).

The timeout is injected via the constructor (default 5 000 ms) so it can be overridden in tests without fake timers.

connor4312 · 2026-02-27T20:20:11Z

src/vs/workbench/contrib/mcp/common/mcpGatewayToolBrokerChannel.ts

+		if (cacheState === McpServerCacheState.Outdated) {
+			// Has cached data — refresh in the background without blocking.
+			void this._ensureServerReady(server);
+		}


I think we should actually take the same codepath for outdated servers as we do for unknown servers. Just because it started up once before quickly doesn't mean it'll do so again.

Good point — updated. Outdated now takes the same codepath as Unknown: both race server startup against the 5s per-server grace period. The separate background-fire path for Outdated is removed.

Per review feedback: Outdated servers should use the same per-server startup grace period as Unknown servers. A prior fast startup does not guarantee a fast restart, so both states now race startup against the 5s timeout before returning results.

connor4312

nice work, thanks for the contribution!

connor4312 · 2026-03-02T17:40:21Z

@RajeshKumar11 looks like there is a compile error

Error: [core-ci                  ] src/vs/workbench/contrib/mcp/test/common/mcpGatewayToolBrokerChannel.test.ts(245,3): error TS2322: Type '() => Promise<{ state: McpConnectionState.Kind; }>' is not assignable to type '(opts?: IMcpServerStartOpts | undefined) => Promise<McpConnectionState>'.

MCP Gateway: avoid blocking list calls on startup

6e417b3

Copilot AI review requested due to automatic review settings February 26, 2026 17:13

Copilot started reviewing on behalf of RajeshKumar11 February 26, 2026 17:15 View session

Copilot AI reviewed Feb 26, 2026

View reviewed changes

austindyoung mentioned this pull request Feb 26, 2026

[mirror] microsoft/vscode#298040 MCP Gateway: avoid blocking list calls on startup austindyoung/vscode-driftfence-mirror-fork#217

Open

vs-code-engineering bot assigned connor4312 Feb 26, 2026

RajeshKumar11 added 4 commits February 27, 2026 08:17

Merge branch 'main' into fix/mcp-gateway-nonblocking-297780-v2

86ec1ce

Merge branch 'main' into fix/mcp-gateway-nonblocking-297780-v2

3235b1e

Merge branch 'main' into fix/mcp-gateway-nonblocking-297780-v2

72e7228

Merge branch 'main' into fix/mcp-gateway-nonblocking-297780-v2

df5dd11

connor4312 reviewed Feb 27, 2026

View reviewed changes

RajeshKumar11 added 3 commits March 1, 2026 10:07

Merge branch 'main' into fix/mcp-gateway-nonblocking-297780-v2

5cb3cc3

Merge branch 'main' into fix/mcp-gateway-nonblocking-297780-v2

517f7a3

connor4312 approved these changes Mar 2, 2026

View reviewed changes

connor4312 enabled auto-merge (squash) March 2, 2026 17:29

vs-code-engineering bot added this to the March 2026 milestone Mar 2, 2026

TylerLeonhardt mentioned this pull request Mar 3, 2026

implement mcp gateway for cli (gated behind sessions flag) microsoft/vscode-copilot-chat#4142

Merged

connor4312 mentioned this pull request Mar 3, 2026

Fix compile error in mcpGatewayToolBrokerChannel test #299072

Merged

connor4312 closed this pull request by merging all changes into microsoft:main in 5e0dc2f Mar 4, 2026

Conversation

RajeshKumar11 commented Feb 26, 2026

Summary

Rationale

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

RajeshKumar11 Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

RajeshKumar11 Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

connor4312 commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TylerLeonhardt commented Feb 26, 2026

Uh oh!

connor4312 commented Feb 26, 2026

Uh oh!

RajeshKumar11 commented Feb 27, 2026

Uh oh!

connor4312 Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

RajeshKumar11 Mar 1, 2026

Choose a reason for hiding this comment

Uh oh!

connor4312 left a comment

Choose a reason for hiding this comment

Uh oh!

connor4312 commented Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

connor4312 commented Feb 26, 2026 •

edited

Loading