mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-01-01 14:39:16 +00:00
Compare commits
8 Commits
fix/trimen
...
v0.0.7-nig
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2eda442a1c | ||
|
|
3e082ae89a | ||
|
|
51207043d0 | ||
|
|
2403061bab | ||
|
|
1ffcb51052 | ||
|
|
c33d162ff2 | ||
|
|
bbfe94cfe2 | ||
|
|
2d0884b04d |
@@ -268,6 +268,11 @@ In addition to a project settings file, a project's `.gemini` directory can cont
|
||||
"loadMemoryFromIncludeDirectories": true
|
||||
```
|
||||
|
||||
- **`tavilyApiKey`** (string):
|
||||
- **Description:** API key for Tavily web search service. Required to enable the `web_search` tool functionality. If not configured, the web search tool will be disabled and skipped.
|
||||
- **Default:** `undefined` (web search disabled)
|
||||
- **Example:** `"tavilyApiKey": "tvly-your-api-key-here"`
|
||||
|
||||
### Example `settings.json`:
|
||||
|
||||
```json
|
||||
@@ -276,6 +281,7 @@ In addition to a project settings file, a project's `.gemini` directory can cont
|
||||
"sandbox": "docker",
|
||||
"toolDiscoveryCommand": "bin/get_tools",
|
||||
"toolCallCommand": "bin/call_tool",
|
||||
"tavilyApiKey": "$TAVILY_API_KEY",
|
||||
"mcpServers": {
|
||||
"mainServer": {
|
||||
"command": "bin/mcp_server.py"
|
||||
@@ -373,6 +379,11 @@ The CLI automatically loads environment variables from an `.env` file. The loadi
|
||||
- **`CODE_ASSIST_ENDPOINT`**:
|
||||
- Specifies the endpoint for the code assist server.
|
||||
- This is useful for development and testing.
|
||||
- **`TAVILY_API_KEY`**:
|
||||
- Your API key for the Tavily web search service.
|
||||
- Required to enable the `web_search` tool functionality.
|
||||
- If not configured, the web search tool will be disabled and skipped.
|
||||
- Example: `export TAVILY_API_KEY="tvly-your-api-key-here"`
|
||||
|
||||
## Command-Line Arguments
|
||||
|
||||
@@ -430,6 +441,9 @@ Arguments passed directly when running the CLI can override other configurations
|
||||
- Displays the version of the CLI.
|
||||
- **`--openai-logging`**:
|
||||
- Enables logging of OpenAI API calls for debugging and analysis. This flag overrides the `enableOpenAILogging` setting in `settings.json`.
|
||||
- **`--tavily-api-key <api_key>`**:
|
||||
- Sets the Tavily API key for web search functionality for this session.
|
||||
- Example: `gemini --tavily-api-key tvly-your-api-key-here`
|
||||
|
||||
## Context Files (Hierarchical Instructional Context)
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ This documentation is organized into the following sections:
|
||||
- **[Multi-File Read Tool](./tools/multi-file.md):** Documentation for the `read_many_files` tool.
|
||||
- **[Shell Tool](./tools/shell.md):** Documentation for the `run_shell_command` tool.
|
||||
- **[Web Fetch Tool](./tools/web-fetch.md):** Documentation for the `web_fetch` tool.
|
||||
- **[Web Search Tool](./tools/web-search.md):** Documentation for the `google_web_search` tool.
|
||||
- **[Web Search Tool](./tools/web-search.md):** Documentation for the `web_search` tool.
|
||||
- **[Memory Tool](./tools/memory.md):** Documentation for the `save_memory` tool.
|
||||
- **[Contributing & Development Guide](../CONTRIBUTING.md):** Information for contributors and developers, including setup, building, testing, and coding conventions.
|
||||
- **[NPM Workspaces and Publishing](./npm.md):** Details on how the project's packages are managed and published.
|
||||
|
||||
@@ -1,36 +1,43 @@
|
||||
# Web Search Tool (`google_web_search`)
|
||||
# Web Search Tool (`web_search`)
|
||||
|
||||
This document describes the `google_web_search` tool.
|
||||
This document describes the `web_search` tool.
|
||||
|
||||
## Description
|
||||
|
||||
Use `google_web_search` to perform a web search using Google Search via the Gemini API. The `google_web_search` tool returns a summary of web results with sources.
|
||||
Use `web_search` to perform a web search using the Tavily API. The tool returns a concise answer with sources when possible.
|
||||
|
||||
### Arguments
|
||||
|
||||
`google_web_search` takes one argument:
|
||||
`web_search` takes one argument:
|
||||
|
||||
- `query` (string, required): The search query.
|
||||
|
||||
## How to use `google_web_search` with the Gemini CLI
|
||||
## How to use `web_search`
|
||||
|
||||
The `google_web_search` tool sends a query to the Gemini API, which then performs a web search. `google_web_search` will return a generated response based on the search results, including citations and sources.
|
||||
`web_search` calls the Tavily API directly. You must configure the `TAVILY_API_KEY` through one of the following methods:
|
||||
|
||||
1. **Settings file**: Add `"tavilyApiKey": "your-key-here"` to your `settings.json`
|
||||
2. **Environment variable**: Set `TAVILY_API_KEY` in your environment or `.env` file
|
||||
3. **Command line**: Use `--tavily-api-key your-key-here` when running the CLI
|
||||
|
||||
If the key is not configured, the tool will be disabled and skipped.
|
||||
|
||||
Usage:
|
||||
|
||||
```
|
||||
google_web_search(query="Your query goes here.")
|
||||
web_search(query="Your query goes here.")
|
||||
```
|
||||
|
||||
## `google_web_search` examples
|
||||
## `web_search` examples
|
||||
|
||||
Get information on a topic:
|
||||
|
||||
```
|
||||
google_web_search(query="latest advancements in AI-powered code generation")
|
||||
web_search(query="latest advancements in AI-powered code generation")
|
||||
```
|
||||
|
||||
## Important notes
|
||||
|
||||
- **Response returned:** The `google_web_search` tool returns a processed summary, not a raw list of search results.
|
||||
- **Citations:** The response includes citations to the sources used to generate the summary.
|
||||
- **Response returned:** The `web_search` tool returns a concise answer when available, with a list of source links.
|
||||
- **Citations:** Source links are appended as a numbered list.
|
||||
- **API key:** Configure `TAVILY_API_KEY` via settings.json, environment variables, .env files, or command line arguments. If not configured, the tool is not registered.
|
||||
|
||||
@@ -9,6 +9,11 @@ import { strict as assert } from 'assert';
|
||||
import { TestRig, printDebugInfo, validateModelOutput } from './test-helper.js';
|
||||
|
||||
test('should be able to search the web', async () => {
|
||||
// Skip if Tavily key is not configured
|
||||
if (!process.env.TAVILY_API_KEY) {
|
||||
console.warn('Skipping web search test: TAVILY_API_KEY not set');
|
||||
return;
|
||||
}
|
||||
const rig = new TestRig();
|
||||
await rig.setup('should be able to search the web');
|
||||
|
||||
@@ -27,7 +32,7 @@ test('should be able to search the web', async () => {
|
||||
throw error; // Re-throw if not a network error
|
||||
}
|
||||
|
||||
const foundToolCall = await rig.waitForToolCall('google_web_search');
|
||||
const foundToolCall = await rig.waitForToolCall('web_search');
|
||||
|
||||
// Add debugging information
|
||||
if (!foundToolCall) {
|
||||
@@ -35,12 +40,11 @@ test('should be able to search the web', async () => {
|
||||
|
||||
// Check if the tool call failed due to network issues
|
||||
const failedSearchCalls = allTools.filter(
|
||||
(t) =>
|
||||
t.toolRequest.name === 'google_web_search' && !t.toolRequest.success,
|
||||
(t) => t.toolRequest.name === 'web_search' && !t.toolRequest.success,
|
||||
);
|
||||
if (failedSearchCalls.length > 0) {
|
||||
console.warn(
|
||||
'google_web_search tool was called but failed, possibly due to network issues',
|
||||
'web_search tool was called but failed, possibly due to network issues',
|
||||
);
|
||||
console.warn(
|
||||
'Failed calls:',
|
||||
@@ -50,20 +54,20 @@ test('should be able to search the web', async () => {
|
||||
}
|
||||
}
|
||||
|
||||
assert.ok(foundToolCall, 'Expected to find a call to google_web_search');
|
||||
assert.ok(foundToolCall, 'Expected to find a call to web_search');
|
||||
|
||||
// Validate model output - will throw if no output, warn if missing expected content
|
||||
const hasExpectedContent = validateModelOutput(
|
||||
result,
|
||||
['weather', 'london'],
|
||||
'Google web search test',
|
||||
'Web search test',
|
||||
);
|
||||
|
||||
// If content was missing, log the search queries used
|
||||
if (!hasExpectedContent) {
|
||||
const searchCalls = rig
|
||||
.readToolLogs()
|
||||
.filter((t) => t.toolRequest.name === 'google_web_search');
|
||||
.filter((t) => t.toolRequest.name === 'web_search');
|
||||
if (searchCalls.length > 0) {
|
||||
console.warn(
|
||||
'Search queries used:',
|
||||
22
package-lock.json
generated
22
package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "@qwen-code/qwen-code",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "@qwen-code/qwen-code",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"workspaces": [
|
||||
"packages/*"
|
||||
],
|
||||
@@ -7421,6 +7421,15 @@
|
||||
"json5": "lib/cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/jsonrepair": {
|
||||
"version": "3.13.0",
|
||||
"resolved": "https://registry.npmjs.org/jsonrepair/-/jsonrepair-3.13.0.tgz",
|
||||
"integrity": "sha512-5YRzlAQ7tuzV1nAJu3LvDlrKtBFIALHN2+a+I1MGJCt3ldRDBF/bZuvIPzae8Epot6KBXd0awRZZcuoeAsZ/mw==",
|
||||
"license": "ISC",
|
||||
"bin": {
|
||||
"jsonrepair": "bin/cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/jsx-ast-utils": {
|
||||
"version": "3.3.5",
|
||||
"resolved": "https://registry.npmjs.org/jsx-ast-utils/-/jsx-ast-utils-3.3.5.tgz",
|
||||
@@ -11691,7 +11700,7 @@
|
||||
},
|
||||
"packages/cli": {
|
||||
"name": "@qwen-code/qwen-code",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"dependencies": {
|
||||
"@google/genai": "1.9.0",
|
||||
"@iarna/toml": "^2.2.5",
|
||||
@@ -11895,7 +11904,7 @@
|
||||
},
|
||||
"packages/core": {
|
||||
"name": "@qwen-code/qwen-code-core",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"dependencies": {
|
||||
"@google/genai": "1.9.0",
|
||||
"@modelcontextprotocol/sdk": "^1.11.0",
|
||||
@@ -11916,6 +11925,7 @@
|
||||
"html-to-text": "^9.0.5",
|
||||
"https-proxy-agent": "^7.0.6",
|
||||
"ignore": "^7.0.0",
|
||||
"jsonrepair": "^3.13.0",
|
||||
"marked": "^15.0.12",
|
||||
"micromatch": "^4.0.8",
|
||||
"open": "^10.1.2",
|
||||
@@ -12042,7 +12052,7 @@
|
||||
},
|
||||
"packages/test-utils": {
|
||||
"name": "@qwen-code/qwen-code-test-utils",
|
||||
"version": "0.1.18",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"license": "Apache-2.0",
|
||||
"devDependencies": {
|
||||
"typescript": "^5.3.3"
|
||||
@@ -12053,7 +12063,7 @@
|
||||
},
|
||||
"packages/vscode-ide-companion": {
|
||||
"name": "qwen-code-vscode-ide-companion",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"license": "LICENSE",
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.15.1",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@qwen-code/qwen-code",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
@@ -13,7 +13,7 @@
|
||||
"url": "git+https://github.com/QwenLM/qwen-code.git"
|
||||
},
|
||||
"config": {
|
||||
"sandboxImageUri": "ghcr.io/qwenlm/qwen-code:0.0.6"
|
||||
"sandboxImageUri": "ghcr.io/qwenlm/qwen-code:0.0.7-nightly.1"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "node scripts/start.js",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@qwen-code/qwen-code",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"description": "Qwen Code",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
@@ -25,7 +25,7 @@
|
||||
"dist"
|
||||
],
|
||||
"config": {
|
||||
"sandboxImageUri": "ghcr.io/qwenlm/qwen-code:0.0.6"
|
||||
"sandboxImageUri": "ghcr.io/qwenlm/qwen-code:0.0.7-nightly.1"
|
||||
},
|
||||
"dependencies": {
|
||||
"@google/genai": "1.9.0",
|
||||
|
||||
@@ -69,6 +69,7 @@ export interface CliArgs {
|
||||
proxy: string | undefined;
|
||||
includeDirectories: string[] | undefined;
|
||||
loadMemoryFromIncludeDirectories: boolean | undefined;
|
||||
tavilyApiKey: string | undefined;
|
||||
}
|
||||
|
||||
export async function parseArguments(): Promise<CliArgs> {
|
||||
@@ -215,6 +216,10 @@ export async function parseArguments(): Promise<CliArgs> {
|
||||
type: 'string',
|
||||
description: 'OpenAI base URL (for custom endpoints)',
|
||||
})
|
||||
.option('tavily-api-key', {
|
||||
type: 'string',
|
||||
description: 'Tavily API key for web search functionality',
|
||||
})
|
||||
.option('proxy', {
|
||||
type: 'string',
|
||||
description:
|
||||
@@ -334,6 +339,11 @@ export async function loadCliConfig(
|
||||
process.env.OPENAI_BASE_URL = argv.openaiBaseUrl;
|
||||
}
|
||||
|
||||
// Handle Tavily API key from command line
|
||||
if (argv.tavilyApiKey) {
|
||||
process.env.TAVILY_API_KEY = argv.tavilyApiKey;
|
||||
}
|
||||
|
||||
// Set the context filename in the server's memoryTool module BEFORE loading memory
|
||||
// TODO(b/343434939): This is a bit of a hack. The contextFileName should ideally be passed
|
||||
// directly to the Config constructor in core, and have core handle setGeminiMdFilename.
|
||||
@@ -513,6 +523,8 @@ export async function loadCliConfig(
|
||||
],
|
||||
contentGenerator: settings.contentGenerator,
|
||||
cliVersion,
|
||||
tavilyApiKey:
|
||||
argv.tavilyApiKey || settings.tavilyApiKey || process.env.TAVILY_API_KEY,
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -147,6 +147,9 @@ export interface Settings {
|
||||
includeDirectories?: string[];
|
||||
|
||||
loadMemoryFromIncludeDirectories?: boolean;
|
||||
|
||||
// Web search API keys
|
||||
tavilyApiKey?: string;
|
||||
}
|
||||
|
||||
export interface SettingsError {
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import React, { useState, useEffect } from 'react';
|
||||
import { Box, Text, useInput, Static } from 'ink';
|
||||
import React, { useState, useEffect, useMemo } from 'react';
|
||||
import { Box, Text, useInput } from 'ink';
|
||||
import Spinner from 'ink-spinner';
|
||||
import Link from 'ink-link';
|
||||
import qrcode from 'qrcode-terminal';
|
||||
@@ -26,17 +26,93 @@ interface QwenOAuthProgressProps {
|
||||
authMessage?: string | null;
|
||||
}
|
||||
|
||||
interface StaticItem {
|
||||
key: string;
|
||||
type:
|
||||
| 'title'
|
||||
| 'instructions'
|
||||
| 'url'
|
||||
| 'qr-instructions'
|
||||
| 'qr-code'
|
||||
| 'auth-content';
|
||||
url?: string;
|
||||
qrCode?: string;
|
||||
/**
|
||||
* Static QR Code Display Component
|
||||
* Renders the QR code and URL once and doesn't re-render unless the URL changes
|
||||
*/
|
||||
function QrCodeDisplay({
|
||||
verificationUrl,
|
||||
qrCodeData,
|
||||
}: {
|
||||
verificationUrl: string;
|
||||
qrCodeData: string | null;
|
||||
}): React.JSX.Element | null {
|
||||
if (!qrCodeData) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<Box
|
||||
borderStyle="round"
|
||||
borderColor={Colors.AccentBlue}
|
||||
flexDirection="column"
|
||||
padding={1}
|
||||
width="100%"
|
||||
>
|
||||
<Text bold color={Colors.AccentBlue}>
|
||||
Qwen OAuth Authentication
|
||||
</Text>
|
||||
|
||||
<Box marginTop={1}>
|
||||
<Text>Please visit this URL to authorize:</Text>
|
||||
</Box>
|
||||
|
||||
<Link url={verificationUrl} fallback={false}>
|
||||
<Text color={Colors.AccentGreen} bold>
|
||||
{verificationUrl}
|
||||
</Text>
|
||||
</Link>
|
||||
|
||||
<Box marginTop={1}>
|
||||
<Text>Or scan the QR code below:</Text>
|
||||
</Box>
|
||||
|
||||
<Box marginTop={1}>
|
||||
<Text>{qrCodeData}</Text>
|
||||
</Box>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Dynamic Status Display Component
|
||||
* Shows the loading spinner, timer, and status messages
|
||||
*/
|
||||
function StatusDisplay({
|
||||
timeRemaining,
|
||||
dots,
|
||||
}: {
|
||||
timeRemaining: number;
|
||||
dots: string;
|
||||
}): React.JSX.Element {
|
||||
const formatTime = (seconds: number): string => {
|
||||
const minutes = Math.floor(seconds / 60);
|
||||
const remainingSeconds = seconds % 60;
|
||||
return `${minutes}:${remainingSeconds.toString().padStart(2, '0')}`;
|
||||
};
|
||||
|
||||
return (
|
||||
<Box
|
||||
borderStyle="round"
|
||||
borderColor={Colors.AccentBlue}
|
||||
flexDirection="column"
|
||||
padding={1}
|
||||
width="100%"
|
||||
>
|
||||
<Box marginTop={1}>
|
||||
<Text>
|
||||
<Spinner type="dots" /> Waiting for authorization{dots}
|
||||
</Text>
|
||||
</Box>
|
||||
|
||||
<Box marginTop={1} justifyContent="space-between">
|
||||
<Text color={Colors.Gray}>
|
||||
Time remaining: {formatTime(timeRemaining)}
|
||||
</Text>
|
||||
<Text color={Colors.AccentPurple}>(Press ESC to cancel)</Text>
|
||||
</Box>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
export function QwenOAuthProgress({
|
||||
@@ -60,33 +136,29 @@ export function QwenOAuthProgress({
|
||||
}
|
||||
});
|
||||
|
||||
// Generate QR code when device auth is available
|
||||
// Generate QR code once when device auth is available
|
||||
useEffect(() => {
|
||||
if (!deviceAuth) {
|
||||
setQrCodeData(null);
|
||||
if (!deviceAuth?.verification_uri_complete) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Only generate QR code if we don't have one yet for this URL
|
||||
if (qrCodeData === null) {
|
||||
const generateQR = () => {
|
||||
try {
|
||||
qrcode.generate(
|
||||
deviceAuth.verification_uri_complete,
|
||||
{ small: true },
|
||||
(qrcode: string) => {
|
||||
setQrCodeData(qrcode);
|
||||
},
|
||||
);
|
||||
} catch (error) {
|
||||
console.error('Failed to generate QR code:', error);
|
||||
setQrCodeData(null);
|
||||
}
|
||||
};
|
||||
const generateQR = () => {
|
||||
try {
|
||||
qrcode.generate(
|
||||
deviceAuth.verification_uri_complete,
|
||||
{ small: true },
|
||||
(qrcode: string) => {
|
||||
setQrCodeData(qrcode);
|
||||
},
|
||||
);
|
||||
} catch (error) {
|
||||
console.error('Failed to generate QR code:', error);
|
||||
setQrCodeData(null);
|
||||
}
|
||||
};
|
||||
|
||||
generateQR();
|
||||
}
|
||||
}, [deviceAuth, qrCodeData]);
|
||||
generateQR();
|
||||
}, [deviceAuth?.verification_uri_complete]);
|
||||
|
||||
// Countdown timer
|
||||
useEffect(() => {
|
||||
@@ -115,11 +187,17 @@ export function QwenOAuthProgress({
|
||||
return () => clearInterval(dotsTimer);
|
||||
}, []);
|
||||
|
||||
const formatTime = (seconds: number): string => {
|
||||
const minutes = Math.floor(seconds / 60);
|
||||
const remainingSeconds = seconds % 60;
|
||||
return `${minutes}:${remainingSeconds.toString().padStart(2, '0')}`;
|
||||
};
|
||||
// Memoize the QR code display to prevent unnecessary re-renders
|
||||
const qrCodeDisplay = useMemo(() => {
|
||||
if (!deviceAuth?.verification_uri_complete) return null;
|
||||
|
||||
return (
|
||||
<QrCodeDisplay
|
||||
verificationUrl={deviceAuth.verification_uri_complete}
|
||||
qrCodeData={qrCodeData}
|
||||
/>
|
||||
);
|
||||
}, [deviceAuth?.verification_uri_complete, qrCodeData]);
|
||||
|
||||
// Handle timeout state
|
||||
if (authStatus === 'timeout') {
|
||||
@@ -151,6 +229,7 @@ export function QwenOAuthProgress({
|
||||
);
|
||||
}
|
||||
|
||||
// Show loading state when no device auth is available yet
|
||||
if (!deviceAuth) {
|
||||
return (
|
||||
<Box
|
||||
@@ -167,7 +246,8 @@ export function QwenOAuthProgress({
|
||||
</Box>
|
||||
<Box marginTop={1} justifyContent="space-between">
|
||||
<Text color={Colors.Gray}>
|
||||
Time remaining: {formatTime(timeRemaining)}
|
||||
Time remaining: {Math.floor(timeRemaining / 60)}:
|
||||
{(timeRemaining % 60).toString().padStart(2, '0')}
|
||||
</Text>
|
||||
<Text color={Colors.AccentPurple}>(Press ESC to cancel)</Text>
|
||||
</Box>
|
||||
@@ -176,77 +256,12 @@ export function QwenOAuthProgress({
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
{qrCodeData && (
|
||||
<Static
|
||||
items={
|
||||
[
|
||||
{
|
||||
key: 'auth-content',
|
||||
type: 'auth-content' as const,
|
||||
url: deviceAuth.verification_uri_complete,
|
||||
qrCode: qrCodeData,
|
||||
},
|
||||
] as StaticItem[]
|
||||
}
|
||||
style={{
|
||||
width: '100%',
|
||||
}}
|
||||
>
|
||||
{(item: StaticItem) => (
|
||||
<Box
|
||||
borderStyle="round"
|
||||
borderColor={Colors.AccentBlue}
|
||||
flexDirection="column"
|
||||
padding={1}
|
||||
width="100%"
|
||||
key={item.key}
|
||||
>
|
||||
<Text bold color={Colors.AccentBlue}>
|
||||
Qwen OAuth Authentication
|
||||
</Text>
|
||||
<Box flexDirection="column" width="100%">
|
||||
{/* Static QR Code Display */}
|
||||
{qrCodeDisplay}
|
||||
|
||||
<Box marginTop={1}>
|
||||
<Text>Please visit this URL to authorize:</Text>
|
||||
</Box>
|
||||
|
||||
<Link url={item.url || ''} fallback={false}>
|
||||
<Text color={Colors.AccentGreen} bold>
|
||||
{item.url || ''}
|
||||
</Text>
|
||||
</Link>
|
||||
|
||||
<Box marginTop={1}>
|
||||
<Text>Or scan the QR code below:</Text>
|
||||
</Box>
|
||||
|
||||
<Box marginTop={1}>
|
||||
<Text>{item.qrCode || ''}</Text>
|
||||
</Box>
|
||||
</Box>
|
||||
)}
|
||||
</Static>
|
||||
)}
|
||||
<Box
|
||||
borderStyle="round"
|
||||
borderColor={Colors.AccentBlue}
|
||||
flexDirection="column"
|
||||
padding={1}
|
||||
width="100%"
|
||||
>
|
||||
<Box marginTop={1}>
|
||||
<Text>
|
||||
<Spinner type="dots" /> Waiting for authorization{dots}
|
||||
</Text>
|
||||
</Box>
|
||||
|
||||
<Box marginTop={1} justifyContent="space-between">
|
||||
<Text color={Colors.Gray}>
|
||||
Time remaining: {formatTime(timeRemaining)}
|
||||
</Text>
|
||||
<Text color={Colors.AccentPurple}>(Press ESC to cancel)</Text>
|
||||
</Box>
|
||||
</Box>
|
||||
</>
|
||||
{/* Dynamic Status Display */}
|
||||
<StatusDisplay timeRemaining={timeRemaining} dots={dots} />
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -562,6 +562,10 @@ export async function start_sandbox(
|
||||
if (process.env.OPENAI_API_KEY) {
|
||||
args.push('--env', `OPENAI_API_KEY=${process.env.OPENAI_API_KEY}`);
|
||||
}
|
||||
// copy TAVILY_API_KEY for web search tool
|
||||
if (process.env.TAVILY_API_KEY) {
|
||||
args.push('--env', `TAVILY_API_KEY=${process.env.TAVILY_API_KEY}`);
|
||||
}
|
||||
if (process.env.OPENAI_BASE_URL) {
|
||||
args.push('--env', `OPENAI_BASE_URL=${process.env.OPENAI_BASE_URL}`);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@qwen-code/qwen-code-core",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"description": "Qwen Code Core",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
@@ -39,6 +39,7 @@
|
||||
"html-to-text": "^9.0.5",
|
||||
"https-proxy-agent": "^7.0.6",
|
||||
"ignore": "^7.0.0",
|
||||
"jsonrepair": "^3.13.0",
|
||||
"marked": "^15.0.12",
|
||||
"micromatch": "^4.0.8",
|
||||
"open": "^10.1.2",
|
||||
|
||||
@@ -211,6 +211,8 @@ export interface ConfigParameters {
|
||||
};
|
||||
cliVersion?: string;
|
||||
loadMemoryFromIncludeDirectories?: boolean;
|
||||
// Web search providers
|
||||
tavilyApiKey?: string;
|
||||
}
|
||||
|
||||
export class Config {
|
||||
@@ -286,6 +288,7 @@ export class Config {
|
||||
};
|
||||
private readonly cliVersion?: string;
|
||||
private readonly loadMemoryFromIncludeDirectories: boolean = false;
|
||||
private readonly tavilyApiKey?: string;
|
||||
|
||||
constructor(params: ConfigParameters) {
|
||||
this.sessionId = params.sessionId;
|
||||
@@ -363,6 +366,9 @@ export class Config {
|
||||
this.loadMemoryFromIncludeDirectories =
|
||||
params.loadMemoryFromIncludeDirectories ?? false;
|
||||
|
||||
// Web search
|
||||
this.tavilyApiKey = params.tavilyApiKey;
|
||||
|
||||
if (params.contextFileName) {
|
||||
setGeminiMdFilename(params.contextFileName);
|
||||
}
|
||||
@@ -695,6 +701,11 @@ export class Config {
|
||||
return this.summarizeToolOutput;
|
||||
}
|
||||
|
||||
// Web search provider configuration
|
||||
getTavilyApiKey(): string | undefined {
|
||||
return this.tavilyApiKey;
|
||||
}
|
||||
|
||||
getIdeModeFeature(): boolean {
|
||||
return this.ideModeFeature;
|
||||
}
|
||||
@@ -805,7 +816,10 @@ export class Config {
|
||||
registerCoreTool(ReadManyFilesTool, this);
|
||||
registerCoreTool(ShellTool, this);
|
||||
registerCoreTool(MemoryTool);
|
||||
registerCoreTool(WebSearchTool, this);
|
||||
// Conditionally register web search tool only if Tavily API key is set
|
||||
if (this.getTavilyApiKey()) {
|
||||
registerCoreTool(WebSearchTool, this);
|
||||
}
|
||||
|
||||
await registry.discoverAllTools();
|
||||
return registry;
|
||||
|
||||
@@ -1160,6 +1160,90 @@ describe('OpenAIContentGenerator', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('should handle MCP tools with parametersJsonSchema', async () => {
|
||||
const mockResponse = {
|
||||
id: 'chatcmpl-123',
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: 'assistant', content: 'Response' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
created: 1677652288,
|
||||
model: 'gpt-4',
|
||||
};
|
||||
|
||||
mockOpenAIClient.chat.completions.create.mockResolvedValue(mockResponse);
|
||||
|
||||
const request: GenerateContentParameters = {
|
||||
contents: [{ role: 'user', parts: [{ text: 'Test' }] }],
|
||||
model: 'gpt-4',
|
||||
config: {
|
||||
tools: [
|
||||
{
|
||||
callTool: vi.fn(),
|
||||
tool: () =>
|
||||
Promise.resolve({
|
||||
functionDeclarations: [
|
||||
{
|
||||
name: 'list-items',
|
||||
description: 'Get a list of items',
|
||||
parametersJsonSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
page_number: {
|
||||
type: 'number',
|
||||
description: 'Page number',
|
||||
},
|
||||
page_size: {
|
||||
type: 'number',
|
||||
description: 'Number of items per page',
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
$schema: 'http://json-schema.org/draft-07/schema#',
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
} as unknown as CallableTool,
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
await generator.generateContent(request, 'test-prompt-id');
|
||||
|
||||
expect(mockOpenAIClient.chat.completions.create).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
tools: [
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'list-items',
|
||||
description: 'Get a list of items',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
page_number: {
|
||||
type: 'number',
|
||||
description: 'Page number',
|
||||
},
|
||||
page_size: {
|
||||
type: 'number',
|
||||
description: 'Number of items per page',
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
$schema: 'http://json-schema.org/draft-07/schema#',
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('should handle nested parameter objects', async () => {
|
||||
const mockResponse = {
|
||||
id: 'chatcmpl-123',
|
||||
|
||||
@@ -26,6 +26,7 @@ import { logApiError, logApiResponse } from '../telemetry/loggers.js';
|
||||
import { ApiErrorEvent, ApiResponseEvent } from '../telemetry/types.js';
|
||||
import { Config } from '../config/config.js';
|
||||
import { openaiLogger } from '../utils/openaiLogger.js';
|
||||
import { safeJsonParse } from '../utils/safeJsonParse.js';
|
||||
|
||||
// OpenAI API type definitions for logging
|
||||
interface OpenAIToolCall {
|
||||
@@ -365,8 +366,6 @@ export class OpenAIContentGenerator implements ContentGenerator {
|
||||
);
|
||||
}
|
||||
|
||||
// console.log('createParams', createParams);
|
||||
|
||||
const stream = (await this.client.chat.completions.create(
|
||||
createParams,
|
||||
)) as AsyncIterable<OpenAI.Chat.ChatCompletionChunk>;
|
||||
@@ -741,6 +740,16 @@ export class OpenAIContentGenerator implements ContentGenerator {
|
||||
return convertTypes(converted) as Record<string, unknown> | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Converts Gemini tools to OpenAI format for API compatibility.
|
||||
* Handles both Gemini tools (using 'parameters' field) and MCP tools (using 'parametersJsonSchema' field).
|
||||
*
|
||||
* Gemini tools use a custom parameter format that needs conversion to OpenAI JSON Schema format.
|
||||
* MCP tools already use JSON Schema format in the parametersJsonSchema field and can be used directly.
|
||||
*
|
||||
* @param geminiTools - Array of Gemini tools to convert
|
||||
* @returns Promise resolving to array of OpenAI-compatible tools
|
||||
*/
|
||||
private async convertGeminiToolsToOpenAI(
|
||||
geminiTools: ToolListUnion,
|
||||
): Promise<OpenAI.Chat.ChatCompletionTool[]> {
|
||||
@@ -761,14 +770,31 @@ export class OpenAIContentGenerator implements ContentGenerator {
|
||||
if (actualTool.functionDeclarations) {
|
||||
for (const func of actualTool.functionDeclarations) {
|
||||
if (func.name && func.description) {
|
||||
let parameters: Record<string, unknown> | undefined;
|
||||
|
||||
// Handle both Gemini tools (parameters) and MCP tools (parametersJsonSchema)
|
||||
if (func.parametersJsonSchema) {
|
||||
// MCP tool format - use parametersJsonSchema directly
|
||||
if (func.parametersJsonSchema) {
|
||||
// Create a shallow copy to avoid mutating the original object
|
||||
const paramsCopy = {
|
||||
...(func.parametersJsonSchema as Record<string, unknown>),
|
||||
};
|
||||
parameters = paramsCopy;
|
||||
}
|
||||
} else if (func.parameters) {
|
||||
// Gemini tool format - convert parameters to OpenAI format
|
||||
parameters = this.convertGeminiParametersToOpenAI(
|
||||
func.parameters as Record<string, unknown>,
|
||||
);
|
||||
}
|
||||
|
||||
openAITools.push({
|
||||
type: 'function',
|
||||
function: {
|
||||
name: func.name,
|
||||
description: func.description,
|
||||
parameters: this.convertGeminiParametersToOpenAI(
|
||||
(func.parameters || {}) as Record<string, unknown>,
|
||||
),
|
||||
parameters,
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1151,12 +1177,7 @@ export class OpenAIContentGenerator implements ContentGenerator {
|
||||
if (toolCall.function) {
|
||||
let args: Record<string, unknown> = {};
|
||||
if (toolCall.function.arguments) {
|
||||
try {
|
||||
args = JSON.parse(toolCall.function.arguments);
|
||||
} catch (error) {
|
||||
console.error('Failed to parse function arguments:', error);
|
||||
args = {};
|
||||
}
|
||||
args = safeJsonParse(toolCall.function.arguments, {});
|
||||
}
|
||||
|
||||
parts.push({
|
||||
@@ -1272,14 +1293,7 @@ export class OpenAIContentGenerator implements ContentGenerator {
|
||||
if (accumulatedCall.name) {
|
||||
let args: Record<string, unknown> = {};
|
||||
if (accumulatedCall.arguments) {
|
||||
try {
|
||||
args = JSON.parse(accumulatedCall.arguments);
|
||||
} catch (error) {
|
||||
console.error(
|
||||
'Failed to parse final tool call arguments:',
|
||||
error,
|
||||
);
|
||||
}
|
||||
args = safeJsonParse(accumulatedCall.arguments, {});
|
||||
}
|
||||
|
||||
parts.push({
|
||||
|
||||
@@ -82,10 +82,12 @@ export class QwenLogger {
|
||||
return undefined;
|
||||
if (!QwenLogger.instance) {
|
||||
QwenLogger.instance = new QwenLogger(config);
|
||||
process.on(
|
||||
'exit',
|
||||
QwenLogger.instance.shutdown.bind(QwenLogger.instance),
|
||||
);
|
||||
}
|
||||
|
||||
process.on('exit', QwenLogger.instance.shutdown.bind(QwenLogger.instance));
|
||||
|
||||
return QwenLogger.instance;
|
||||
}
|
||||
|
||||
|
||||
@@ -4,35 +4,24 @@
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { GroundingMetadata } from '@google/genai';
|
||||
import { BaseTool, Icon, ToolResult } from './tools.js';
|
||||
import { Type } from '@google/genai';
|
||||
import { SchemaValidator } from '../utils/schemaValidator.js';
|
||||
|
||||
import { getErrorMessage } from '../utils/errors.js';
|
||||
import { Config } from '../config/config.js';
|
||||
import { getResponseText } from '../utils/generateContentResponseUtilities.js';
|
||||
|
||||
interface GroundingChunkWeb {
|
||||
uri?: string;
|
||||
title?: string;
|
||||
interface TavilyResultItem {
|
||||
title: string;
|
||||
url: string;
|
||||
content?: string;
|
||||
score?: number;
|
||||
published_date?: string;
|
||||
}
|
||||
|
||||
interface GroundingChunkItem {
|
||||
web?: GroundingChunkWeb;
|
||||
// Other properties might exist if needed in the future
|
||||
}
|
||||
|
||||
interface GroundingSupportSegment {
|
||||
startIndex: number;
|
||||
endIndex: number;
|
||||
text?: string; // text is optional as per the example
|
||||
}
|
||||
|
||||
interface GroundingSupportItem {
|
||||
segment?: GroundingSupportSegment;
|
||||
groundingChunkIndices?: number[];
|
||||
confidenceScores?: number[]; // Optional as per example
|
||||
interface TavilySearchResponse {
|
||||
query: string;
|
||||
answer?: string;
|
||||
results: TavilyResultItem[];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -42,7 +31,6 @@ export interface WebSearchToolParams {
|
||||
/**
|
||||
* The search query.
|
||||
*/
|
||||
|
||||
query: string;
|
||||
}
|
||||
|
||||
@@ -50,25 +38,23 @@ export interface WebSearchToolParams {
|
||||
* Extends ToolResult to include sources for web search.
|
||||
*/
|
||||
export interface WebSearchToolResult extends ToolResult {
|
||||
sources?: GroundingMetadata extends { groundingChunks: GroundingChunkItem[] }
|
||||
? GroundingMetadata['groundingChunks']
|
||||
: GroundingChunkItem[];
|
||||
sources?: Array<{ title: string; url: string }>;
|
||||
}
|
||||
|
||||
/**
|
||||
* A tool to perform web searches using Google Search via the Gemini API.
|
||||
* A tool to perform web searches using Tavily API.
|
||||
*/
|
||||
export class WebSearchTool extends BaseTool<
|
||||
WebSearchToolParams,
|
||||
WebSearchToolResult
|
||||
> {
|
||||
static readonly Name: string = 'google_web_search';
|
||||
static readonly Name: string = 'web_search';
|
||||
|
||||
constructor(private readonly config: Config) {
|
||||
super(
|
||||
WebSearchTool.Name,
|
||||
'GoogleSearch',
|
||||
'Performs a web search using Google Search (via the Gemini API) and returns the results. This tool is useful for finding information on the internet based on a query.',
|
||||
'TavilySearch',
|
||||
'Performs a web search using the Tavily API and returns a concise answer with sources. Requires the TAVILY_API_KEY environment variable.',
|
||||
Icon.Globe,
|
||||
{
|
||||
type: Type.OBJECT,
|
||||
@@ -106,7 +92,7 @@ export class WebSearchTool extends BaseTool<
|
||||
|
||||
async execute(
|
||||
params: WebSearchToolParams,
|
||||
signal: AbortSignal,
|
||||
_signal: AbortSignal,
|
||||
): Promise<WebSearchToolResult> {
|
||||
const validationError = this.validateToolParams(params);
|
||||
if (validationError) {
|
||||
@@ -115,79 +101,83 @@ export class WebSearchTool extends BaseTool<
|
||||
returnDisplay: validationError,
|
||||
};
|
||||
}
|
||||
const geminiClient = this.config.getGeminiClient();
|
||||
|
||||
const apiKey = this.config.getTavilyApiKey() || process.env.TAVILY_API_KEY;
|
||||
if (!apiKey) {
|
||||
return {
|
||||
llmContent:
|
||||
'Web search is disabled because TAVILY_API_KEY is not configured. Please set it in your settings.json, .env file, or via --tavily-api-key command line argument to enable web search.',
|
||||
returnDisplay:
|
||||
'Web search disabled. Configure TAVILY_API_KEY to enable Tavily search.',
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await geminiClient.generateContent(
|
||||
[{ role: 'user', parts: [{ text: params.query }] }],
|
||||
{ tools: [{ googleSearch: {} }] },
|
||||
signal,
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 15000);
|
||||
const response = await fetch('https://api.tavily.com/search', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
api_key: apiKey,
|
||||
query: params.query,
|
||||
search_depth: 'advanced',
|
||||
max_results: 5,
|
||||
include_answer: true,
|
||||
}),
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text().catch(() => '');
|
||||
throw new Error(
|
||||
`Tavily API error: ${response.status} ${response.statusText}${text ? ` - ${text}` : ''}`,
|
||||
);
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TavilySearchResponse;
|
||||
|
||||
const sources = (data.results || []).map((r) => ({
|
||||
title: r.title,
|
||||
url: r.url,
|
||||
}));
|
||||
|
||||
const sourceListFormatted = sources.map(
|
||||
(s, i) => `[${i + 1}] ${s.title || 'Untitled'} (${s.url})`,
|
||||
);
|
||||
|
||||
const responseText = getResponseText(response);
|
||||
const groundingMetadata = response.candidates?.[0]?.groundingMetadata;
|
||||
const sources = groundingMetadata?.groundingChunks as
|
||||
| GroundingChunkItem[]
|
||||
| undefined;
|
||||
const groundingSupports = groundingMetadata?.groundingSupports as
|
||||
| GroundingSupportItem[]
|
||||
| undefined;
|
||||
let content = data.answer?.trim() || '';
|
||||
if (!content) {
|
||||
// Fallback: build a concise summary from top results
|
||||
content = sources
|
||||
.slice(0, 3)
|
||||
.map((s, i) => `${i + 1}. ${s.title} - ${s.url}`)
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
if (!responseText || !responseText.trim()) {
|
||||
if (sourceListFormatted.length > 0) {
|
||||
content += `\n\nSources:\n${sourceListFormatted.join('\n')}`;
|
||||
}
|
||||
|
||||
if (!content.trim()) {
|
||||
return {
|
||||
llmContent: `No search results or information found for query: "${params.query}"`,
|
||||
returnDisplay: 'No information found.',
|
||||
};
|
||||
}
|
||||
|
||||
let modifiedResponseText = responseText;
|
||||
const sourceListFormatted: string[] = [];
|
||||
|
||||
if (sources && sources.length > 0) {
|
||||
sources.forEach((source: GroundingChunkItem, index: number) => {
|
||||
const title = source.web?.title || 'Untitled';
|
||||
const uri = source.web?.uri || 'No URI';
|
||||
sourceListFormatted.push(`[${index + 1}] ${title} (${uri})`);
|
||||
});
|
||||
|
||||
if (groundingSupports && groundingSupports.length > 0) {
|
||||
const insertions: Array<{ index: number; marker: string }> = [];
|
||||
groundingSupports.forEach((support: GroundingSupportItem) => {
|
||||
if (support.segment && support.groundingChunkIndices) {
|
||||
const citationMarker = support.groundingChunkIndices
|
||||
.map((chunkIndex: number) => `[${chunkIndex + 1}]`)
|
||||
.join('');
|
||||
insertions.push({
|
||||
index: support.segment.endIndex,
|
||||
marker: citationMarker,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// Sort insertions by index in descending order to avoid shifting subsequent indices
|
||||
insertions.sort((a, b) => b.index - a.index);
|
||||
|
||||
const responseChars = modifiedResponseText.split(''); // Use new variable
|
||||
insertions.forEach((insertion) => {
|
||||
// Fixed arrow function syntax
|
||||
responseChars.splice(insertion.index, 0, insertion.marker);
|
||||
});
|
||||
modifiedResponseText = responseChars.join(''); // Assign back to modifiedResponseText
|
||||
}
|
||||
|
||||
if (sourceListFormatted.length > 0) {
|
||||
modifiedResponseText +=
|
||||
'\n\nSources:\n' + sourceListFormatted.join('\n'); // Fixed string concatenation
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
llmContent: `Web search results for "${params.query}":\n\n${modifiedResponseText}`,
|
||||
llmContent: `Web search results for "${params.query}":\n\n${content}`,
|
||||
returnDisplay: `Search results for "${params.query}" returned.`,
|
||||
sources,
|
||||
};
|
||||
} catch (error: unknown) {
|
||||
const errorMessage = `Error during web search for query "${params.query}": ${getErrorMessage(error)}`;
|
||||
const errorMessage = `Error during web search for query "${params.query}": ${getErrorMessage(
|
||||
error,
|
||||
)}`;
|
||||
console.error(errorMessage, error);
|
||||
return {
|
||||
llmContent: `Error: ${errorMessage}`,
|
||||
|
||||
149
packages/core/src/utils/safeJsonParse.test.ts
Normal file
149
packages/core/src/utils/safeJsonParse.test.ts
Normal file
@@ -0,0 +1,149 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2025 Qwen
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { safeJsonParse } from './safeJsonParse.js';
|
||||
|
||||
describe('safeJsonParse', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe('valid JSON parsing', () => {
|
||||
it('should parse valid JSON correctly', () => {
|
||||
const validJson = '{"name": "test", "value": 123}';
|
||||
const result = safeJsonParse(validJson);
|
||||
|
||||
expect(result).toEqual({ name: 'test', value: 123 });
|
||||
});
|
||||
|
||||
it('should parse valid JSON arrays', () => {
|
||||
const validArray = '["item1", "item2", "item3"]';
|
||||
const result = safeJsonParse(validArray);
|
||||
|
||||
expect(result).toEqual(['item1', 'item2', 'item3']);
|
||||
});
|
||||
|
||||
it('should parse valid JSON with nested objects', () => {
|
||||
const validNested =
|
||||
'{"config": {"paths": ["testlogs/*.py"], "options": {"recursive": true}}}';
|
||||
const result = safeJsonParse(validNested);
|
||||
|
||||
expect(result).toEqual({
|
||||
config: {
|
||||
paths: ['testlogs/*.py'],
|
||||
options: { recursive: true },
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('malformed JSON with jsonrepair fallback', () => {
|
||||
it('should handle malformed JSON with single quotes', () => {
|
||||
const malformedJson = "{'name': 'test', 'value': 123}";
|
||||
const result = safeJsonParse(malformedJson);
|
||||
|
||||
expect(result).toEqual({ name: 'test', value: 123 });
|
||||
});
|
||||
|
||||
it('should handle malformed JSON with unquoted keys', () => {
|
||||
const malformedJson = '{name: "test", value: 123}';
|
||||
const result = safeJsonParse(malformedJson);
|
||||
|
||||
expect(result).toEqual({ name: 'test', value: 123 });
|
||||
});
|
||||
|
||||
it('should handle malformed JSON with trailing commas', () => {
|
||||
const malformedJson = '{"name": "test", "value": 123,}';
|
||||
const result = safeJsonParse(malformedJson);
|
||||
|
||||
expect(result).toEqual({ name: 'test', value: 123 });
|
||||
});
|
||||
|
||||
it('should handle malformed JSON with comments', () => {
|
||||
const malformedJson = '{"name": "test", // comment\n "value": 123}';
|
||||
const result = safeJsonParse(malformedJson);
|
||||
|
||||
expect(result).toEqual({ name: 'test', value: 123 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('fallback behavior', () => {
|
||||
it('should return fallback value for empty string', () => {
|
||||
const emptyString = '';
|
||||
const fallback = { default: 'value' };
|
||||
|
||||
const result = safeJsonParse(emptyString, fallback);
|
||||
|
||||
expect(result).toEqual(fallback);
|
||||
});
|
||||
|
||||
it('should return fallback value for null input', () => {
|
||||
const nullInput = null as unknown as string;
|
||||
const fallback = { default: 'value' };
|
||||
|
||||
const result = safeJsonParse(nullInput, fallback);
|
||||
|
||||
expect(result).toEqual(fallback);
|
||||
});
|
||||
|
||||
it('should return fallback value for undefined input', () => {
|
||||
const undefinedInput = undefined as unknown as string;
|
||||
const fallback = { default: 'value' };
|
||||
|
||||
const result = safeJsonParse(undefinedInput, fallback);
|
||||
|
||||
expect(result).toEqual(fallback);
|
||||
});
|
||||
|
||||
it('should return empty object as default fallback', () => {
|
||||
const invalidJson = 'invalid json';
|
||||
|
||||
const result = safeJsonParse(invalidJson);
|
||||
|
||||
// jsonrepair returns the original string for completely invalid JSON
|
||||
expect(result).toEqual('invalid json');
|
||||
});
|
||||
|
||||
it('should return custom fallback when parsing fails', () => {
|
||||
const invalidJson = 'invalid json';
|
||||
const customFallback = { error: 'parsing failed', data: null };
|
||||
|
||||
const result = safeJsonParse(invalidJson, customFallback);
|
||||
|
||||
// jsonrepair returns the original string for completely invalid JSON
|
||||
expect(result).toEqual('invalid json');
|
||||
});
|
||||
});
|
||||
|
||||
describe('type safety', () => {
|
||||
it('should preserve generic type when parsing valid JSON', () => {
|
||||
const validJson = '{"name": "test", "value": 123}';
|
||||
const result = safeJsonParse<{ name: string; value: number }>(validJson);
|
||||
|
||||
expect(result).toEqual({ name: 'test', value: 123 });
|
||||
// TypeScript should infer the correct type
|
||||
expect(typeof result.name).toBe('string');
|
||||
expect(typeof result.value).toBe('number');
|
||||
});
|
||||
|
||||
it('should return fallback type when parsing fails', () => {
|
||||
const invalidJson = 'invalid json';
|
||||
const fallback = { error: 'fallback' } as const;
|
||||
|
||||
const result = safeJsonParse(invalidJson, fallback);
|
||||
|
||||
// jsonrepair returns the original string for completely invalid JSON
|
||||
expect(result).toEqual('invalid json');
|
||||
// TypeScript should preserve the fallback type
|
||||
expect(typeof result).toBe('string');
|
||||
});
|
||||
});
|
||||
});
|
||||
45
packages/core/src/utils/safeJsonParse.ts
Normal file
45
packages/core/src/utils/safeJsonParse.ts
Normal file
@@ -0,0 +1,45 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2025 Qwen
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { jsonrepair } from 'jsonrepair';
|
||||
|
||||
/**
|
||||
* Safely parse JSON string with jsonrepair fallback for malformed JSON.
|
||||
* This function attempts to parse JSON normally first, and if that fails,
|
||||
* it uses jsonrepair to fix common JSON formatting issues before parsing.
|
||||
*
|
||||
* @param jsonString - The JSON string to parse
|
||||
* @param fallbackValue - The value to return if parsing fails completely
|
||||
* @returns The parsed object or the fallback value
|
||||
*/
|
||||
export function safeJsonParse<T = Record<string, unknown>>(
|
||||
jsonString: string,
|
||||
fallbackValue: T = {} as T,
|
||||
): T {
|
||||
if (!jsonString || typeof jsonString !== 'string') {
|
||||
return fallbackValue;
|
||||
}
|
||||
|
||||
try {
|
||||
// First attempt: try normal JSON.parse
|
||||
return JSON.parse(jsonString) as T;
|
||||
} catch (error) {
|
||||
try {
|
||||
// Second attempt: use jsonrepair to fix common JSON issues
|
||||
const repairedJson = jsonrepair(jsonString);
|
||||
|
||||
// jsonrepair always returns a string, so we need to parse it
|
||||
return JSON.parse(repairedJson) as T;
|
||||
} catch (repairError) {
|
||||
console.error('Failed to parse JSON even with jsonrepair:', {
|
||||
originalError: error,
|
||||
repairError,
|
||||
jsonString,
|
||||
});
|
||||
return fallbackValue;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@qwen-code/qwen-code-test-utils",
|
||||
"version": "0.1.18",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"private": true,
|
||||
"main": "src/index.ts",
|
||||
"license": "Apache-2.0",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"name": "qwen-code-vscode-ide-companion",
|
||||
"displayName": "Qwen Code Companion",
|
||||
"description": "Enable Qwen Code with direct access to your VS Code workspace.",
|
||||
"version": "0.0.6",
|
||||
"version": "0.0.7-nightly.1",
|
||||
"publisher": "qwenlm",
|
||||
"icon": "assets/icon.png",
|
||||
"repository": {
|
||||
|
||||
Reference in New Issue
Block a user