Fix fetch_webpage crash on shell scripts due to undefined query embedding

[Copilot]

fetch_webpage crashes VS Code's extension host when fetching .sh files (but not .html) because the chunking endpoint returns non-empty chunks for shell scripts. When no query is provided, computeEmbeddings('') returns { values: [] }, making queryEmbedding = undefined. computeChunkScores then calls distance(chunk.embedding, undefined) which throws a TypeError that becomes an unhandled rejection, crashing the process.

Changes

• urlChunkEmbeddingsIndex.ts — primary fix:
  • Skip query embedding computation entirely when query is empty/whitespace; return chunks unscored
  • Guard against computeEmbeddings returning undefined (when API returns empty values array)
  • Fix computeEmbeddings return type from Promise<Embedding> → Promise<Embedding | undefined>
  • Extract toChunksWithoutScores() helper to remove duplication

// Before: parallel Promise.all could produce undefined queryEmbedding, then crash in computeChunkScores
const [queryEmbedding, fileChunksAndEmbeddings] = await Promise.all([
    this.computeEmbeddings(embeddingType, query, 'query', token), // '' → values[0] = undefined
    this.getEmbeddingsForFiles(...)
]);
return this.computeChunkScores(fileChunksAndEmbeddings, queryEmbedding); // 💥 TypeError

// After: early-exit if no query or no embedding
if (!query.trim()) return this.toChunksWithoutScores(fileChunksAndEmbeddings);
const queryEmbedding = await this.computeEmbeddings(...);
if (!queryEmbedding) return this.toChunksWithoutScores(fileChunksAndEmbeddings);
return this.computeChunkScores(fileChunksAndEmbeddings, queryEmbedding);

• urlChunkEmbeddingsIndex.spec.ts — new unit tests covering empty query, undefined embedding, and the .sh regression scenario

Original prompt

---

This section details on the original issue you should resolve

<issue_title>fetch_webpage crashes VS code</issue_title>
<issue_description>- Copilot Chat Extension Version: 0.37.6

• VS Code Version: 1.109.4
• OS Version: Linux
• Feature (e.g. agent/edit/ask mode): agent
• Selected model: multiple
• Logs: crash.log

Steps to Reproduce:

1. Ask copilot to "Fetch contents of https://pkgs.netbird.io/install.sh"
   • You can optionally nudge AI to use the correct tool by adding "using fetch_webpage"
2. As long as agent chooses to use fetch_webpage tool, it always crashes entire VS code

Only happens with this specific URL, I have tried https://example.org and NVM install script https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh and those work okay.

---

Update

This seems to be related to some file post-processing, rather than fetching. Copilot thinks it's related to embeddings computed as filesAndTheirChunks.

I did manage to reproduce this locally with empty file or dummy file contents like Nothing to see here hosted locally (using python3 -m http.server <port>).

Interesting thing is that if the file was called install.html, the command worked correctly, but when the same file was called install.sh, it crashed VS Code.

Similar story with https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh. It works when fetched from original URL, but as soon as I download the install script and fetch it locally, it also fails, so the information in the URL probably also plays some role.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes fetch_webpage crashes VS code vscode#296098

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[TylerLeonhardt]

@copilot Did you confirm that before this fix, the tests failed? I'd expect you to be able to repeat the failure, apply the fix, get the passing tests

[Copilot]

@copilot Did you confirm that before this fix, the tests failed? I'd expect you to be able to repeat the failure, apply the fix, get the passing tests

Yes, confirmed. With the original code (before fix), 4 of 6 tests fail with the exact crash from the issue:

FAIL  returns chunks without scores when query is empty
FAIL  returns chunks without scores when query is whitespace only
FAIL  returns chunks without scores when query embedding computation returns empty values
FAIL  does not crash for shell script URLs with empty query

TypeError: Cannot read properties of undefined (reading 'type')
 ❯ distance src/platform/embeddings/common/embeddingsComputer.ts:152:49
 ❯ UrlChunkEmbeddingsIndex.computeChunkScores src/platform/urlChunkSearch/node/urlChunkEmbeddingsIndex.ts:109:5
 ❯ UrlChunkEmbeddingsIndex.findInUrls src/platform/urlChunkSearch/node/urlChunkEmbeddingsIndex.ts:75:15

After applying the fix, all 6 tests pass. I just re-verified by reverting urlChunkEmbeddingsIndex.ts to 57332e8, running the tests (4 fail), restoring the fix, and running again (6 pass).