abbyswag/qwen-code
1
0
Fork 0
mirror of https://github.com/QwenLM/qwen-code.git synced 2025-12-19 09:33:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: restructure docs

Browse Source
This commit is contained in:
pomelo-nwu
2025-12-04 18:26:05 +08:00
parent 6729980b47
commit 17785c418d
72 changed files with 0 additions and 412 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
docs/developers/core/index.md Normal file
View File

@@ -0,0 +1,55 @@
# Qwen Code Core
Qwen Code's core package (`packages/core`) is the backend portion of Qwen Code, handling communication with model APIs, managing tools, and processing requests sent from `packages/cli`. For a general overview of Qwen Code, see the [main documentation page](../index.md).
## Navigating this section
- **[Core tools API](./tools-api.md):** Information on how tools are defined, registered, and used by the core.
- **[Memory Import Processor](./memport.md):** Documentation for the modular QWEN.md import feature using @file.md syntax.
## Role of the core
While the `packages/cli` portion of Qwen Code provides the user interface, `packages/core` is responsible for:
- **Model API interaction:** Securely communicating with the configured model provider, sending user prompts, and receiving model responses.
- **Prompt engineering:** Constructing effective prompts for the model, potentially incorporating conversation history, tool definitions, and instructional context from context files (e.g., `QWEN.md`).
- **Tool management & orchestration:**
- Registering available tools (e.g., file system tools, shell command execution).
- Interpreting tool use requests from the model.
- Executing the requested tools with the provided arguments.
- Returning tool execution results to the model for further processing.
- **Session and state management:** Keeping track of the conversation state, including history and any relevant context required for coherent interactions.
- **Configuration:** Managing core-specific configurations, such as API key access, model selection, and tool settings.
## Security considerations
The core plays a vital role in security:
- **API key management:** It handles provider credentials and ensures they're used securely when communicating with APIs.
- **Tool execution:** When tools interact with the local system (e.g., `run_shell_command`), the core (and its underlying tool implementations) must do so with appropriate caution, often involving sandboxing mechanisms to prevent unintended modifications.
## Chat history compression
To ensure that long conversations don't exceed the token limits of the selected model, the core includes a chat history compression feature.
When a conversation approaches the token limit for the configured model, the core automatically compresses the conversation history before sending it to the model. This compression is designed to be lossless in terms of the information conveyed, but it reduces the overall number of tokens used.
You can find token limits for each provider's models in their documentation.
## Model fallback
Qwen Code includes a model fallback mechanism to ensure that you can continue to use the CLI even if the default model is rate-limited.
If you are using the default "pro" model and the CLI detects that you are being rate-limited, it automatically switches to the "flash" model for the current session. This allows you to continue working without interruption.
## File discovery service
The file discovery service is responsible for finding files in the project that are relevant to the current context. It is used by the `@` command and other tools that need to access files.
## Memory discovery service
The memory discovery service is responsible for finding and loading the context files (default: `QWEN.md`) that provide context to the model. It searches for these files in a hierarchical manner, starting from the current working directory and moving up to the project root and the user's home directory. It also searches in subdirectories.
This allows you to have global, project-level, and component-level context files, which are all combined to provide the model with the most relevant information.
You can use the [`/memory` command](../cli/commands.md) to `show`, `add`, and `refresh` the content of loaded context files.

215
docs/developers/core/memport.md Normal file
View File

@@ -0,0 +1,215 @@
# Memory Import Processor
The Memory Import Processor is a feature that allows you to modularize your context files (e.g., `QWEN.md`) by importing content from other files using the `@file.md` syntax.
## Overview
This feature enables you to break down large context files (e.g., `QWEN.md`) into smaller, more manageable components that can be reused across different contexts. The import processor supports both relative and absolute paths, with built-in safety features to prevent circular imports and ensure file access security.
## Syntax
Use the `@` symbol followed by the path to the file you want to import:
```markdown
# Main QWEN.md file
This is the main content.
@./components/instructions.md
More content here.
@./shared/configuration.md
```
## Supported Path Formats
### Relative Paths
- `@./file.md` - Import from the same directory
- `@../file.md` - Import from parent directory
- `@./components/file.md` - Import from subdirectory
### Absolute Paths
- `@/absolute/path/to/file.md` - Import using absolute path
## Examples
### Basic Import
```markdown
# My QWEN.md
Welcome to my project!
@./getting-started.md
## Features
@./features/overview.md
```
### Nested Imports
The imported files can themselves contain imports, creating a nested structure:
```markdown
# main.md
@./header.md
@./content.md
@./footer.md
```
```markdown
# header.md
# Project Header
@./shared/title.md
```
## Safety Features
### Circular Import Detection
The processor automatically detects and prevents circular imports:
```markdown
# file-a.md
@./file-b.md
# file-b.md
@./file-a.md <!-- This will be detected and prevented -->
```
### File Access Security
The `validateImportPath` function ensures that imports are only allowed from specified directories, preventing access to sensitive files outside the allowed scope.
### Maximum Import Depth
To prevent infinite recursion, there's a configurable maximum import depth (default: 5 levels).
## Error Handling
### Missing Files
If a referenced file doesn't exist, the import will fail gracefully with an error comment in the output.
### File Access Errors
Permission issues or other file system errors are handled gracefully with appropriate error messages.
## Code Region Detection
The import processor uses the `marked` library to detect code blocks and inline code spans, ensuring that `@` imports inside these regions are properly ignored. This provides robust handling of nested code blocks and complex Markdown structures.
## Import Tree Structure
The processor returns an import tree that shows the hierarchy of imported files. This helps users debug problems with their context files by showing which files were read and their import relationships.
Example tree structure:
```
Memory Files
L project: QWEN.md
L a.md
L b.md
L c.md
L d.md
L e.md
L f.md
L included.md
```
The tree preserves the order that files were imported and shows the complete import chain for debugging purposes.
## Comparison to Claude Code's `/memory` (`claude.md`) Approach
Claude Code's `/memory` feature (as seen in `claude.md`) produces a flat, linear document by concatenating all included files, always marking file boundaries with clear comments and path names. It does not explicitly present the import hierarchy, but the LLM receives all file contents and paths, which is sufficient for reconstructing the hierarchy if needed.
Note: The import tree is mainly for clarity during development and has limited relevance to LLM consumption.
## API Reference
### `processImports(content, basePath, debugMode?, importState?)`
Processes import statements in context file content.
**Parameters:**
- `content` (string): The content to process for imports
- `basePath` (string): The directory path where the current file is located
- `debugMode` (boolean, optional): Whether to enable debug logging (default: false)
- `importState` (ImportState, optional): State tracking for circular import prevention
**Returns:** Promise&lt;ProcessImportsResult&gt; - Object containing processed content and import tree
### `ProcessImportsResult`
```typescript
interface ProcessImportsResult {
content: string; // The processed content with imports resolved
importTree: MemoryFile; // Tree structure showing the import hierarchy
}
```
### `MemoryFile`
```typescript
interface MemoryFile {
path: string; // The file path
imports?: MemoryFile[]; // Direct imports, in the order they were imported
}
```
### `validateImportPath(importPath, basePath, allowedDirectories)`
Validates import paths to ensure they are safe and within allowed directories.
**Parameters:**
- `importPath` (string): The import path to validate
- `basePath` (string): The base directory for resolving relative paths
- `allowedDirectories` (string[]): Array of allowed directory paths
**Returns:** boolean - Whether the import path is valid
### `findProjectRoot(startDir)`
Finds the project root by searching for a `.git` directory upwards from the given start directory. Implemented as an **async** function using non-blocking file system APIs to avoid blocking the Node.js event loop.
**Parameters:**
- `startDir` (string): The directory to start searching from
**Returns:** Promise&lt;string&gt; - The project root directory (or the start directory if no `.git` is found)
## Best Practices
1. **Use descriptive file names** for imported components
2. **Keep imports shallow** - avoid deeply nested import chains
3. **Document your structure** - maintain a clear hierarchy of imported files
4. **Test your imports** - ensure all referenced files exist and are accessible
5. **Use relative paths** when possible for better portability
## Troubleshooting
### Common Issues
1. **Import not working**: Check that the file exists and the path is correct
2. **Circular import warnings**: Review your import structure for circular references
3. **Permission errors**: Ensure the files are readable and within allowed directories
4. **Path resolution issues**: Use absolute paths if relative paths aren't resolving correctly
### Debug Mode
Enable debug mode to see detailed logging of the import process:
```typescript
const result = await processImports(content, basePath, true);
```

79
docs/developers/core/tools-api.md Normal file
View File

@@ -0,0 +1,79 @@
# Qwen Code Core: Tools API
The Qwen Code core (`packages/core`) features a robust system for defining, registering, and executing tools. These tools extend the capabilities of the model, allowing it to interact with the local environment, fetch web content, and perform various actions beyond simple text generation.
## Core Concepts
- **Tool (`tools.ts`):** An interface and base class (`BaseTool`) that defines the contract for all tools. Each tool must have:
- `name`: A unique internal name (used in API calls to the model).
- `displayName`: A user-friendly name.
- `description`: A clear explanation of what the tool does, which is provided to the model.
- `parameterSchema`: A JSON schema defining the parameters that the tool accepts. This is crucial for the model to understand how to call the tool correctly.
- `validateToolParams()`: A method to validate incoming parameters.
- `getDescription()`: A method to provide a human-readable description of what the tool will do with specific parameters before execution.
- `shouldConfirmExecute()`: A method to determine if user confirmation is required before execution (e.g., for potentially destructive operations).
- `execute()`: The core method that performs the tool's action and returns a `ToolResult`.
- **`ToolResult` (`tools.ts`):** An interface defining the structure of a tool's execution outcome:
- `llmContent`: The factual content to be included in the history sent back to the LLM for context. This can be a simple string or a `PartListUnion` (an array of `Part` objects and strings) for rich content.
- `returnDisplay`: A user-friendly string (often Markdown) or a special object (like `FileDiff`) for display in the CLI.
- **Returning Rich Content:** Tools are not limited to returning simple text. The `llmContent` can be a `PartListUnion`, which is an array that can contain a mix of `Part` objects (for images, audio, etc.) and `string`s. This allows a single tool execution to return multiple pieces of rich content.
- **Tool Registry (`tool-registry.ts`):** A class (`ToolRegistry`) responsible for:
- **Registering Tools:** Holding a collection of all available built-in tools (e.g., `ListFiles`, `ReadFile`).
- **Discovering Tools:** It can also discover tools dynamically:
- **Command-based Discovery:** If `tools.toolDiscoveryCommand` is configured in settings, this command is executed. It's expected to output JSON describing custom tools, which are then registered as `DiscoveredTool` instances.
- **MCP-based Discovery:** If `mcp.mcpServerCommand` is configured, the registry can connect to a Model Context Protocol (MCP) server to list and register tools (`DiscoveredMCPTool`).
- **Providing Schemas:** Exposing the `FunctionDeclaration` schemas of all registered tools to the model, so it knows what tools are available and how to use them.
- **Retrieving Tools:** Allowing the core to get a specific tool by name for execution.
## Built-in Tools
The core comes with a suite of pre-defined tools, typically found in `packages/core/src/tools/`. These include:
- **File System Tools:**
- `ListFiles` (`ls.ts`): Lists directory contents.
- `ReadFile` (`read-file.ts`): Reads the content of a single file. It takes an `absolute_path` parameter, which must be an absolute path.
- `WriteFile` (`write-file.ts`): Writes content to a file.
- `ReadManyFiles` (`read-many-files.ts`): Reads and concatenates content from multiple files or glob patterns (used by the `@` command in CLI).
- `Grep` (`grep.ts`): Searches for patterns in files.
- `Glob` (`glob.ts`): Finds files matching glob patterns.
- `Edit` (`edit.ts`): Performs in-place modifications to files (often requiring confirmation).
- **Execution Tools:**
- `Shell` (`shell.ts`): Executes arbitrary shell commands (requires careful sandboxing and user confirmation).
- **Web Tools:**
- `WebFetch` (`web-fetch.ts`): Fetches content from a URL.
- `WebSearch` (`web-search.ts`): Performs a web search.
- **Memory Tools:**
- `SaveMemory` (`memoryTool.ts`): Interacts with the AI's memory.
- **Planning Tools:**
- `Task` (`task.ts`): Delegates tasks to specialized subagents.
- `TodoWrite` (`todoWrite.ts`): Creates and manages a structured task list.
- `ExitPlanMode` (`exitPlanMode.ts`): Exits plan mode and returns to normal operation.
Each of these tools extends `BaseTool` and implements the required methods for its specific functionality.
## Tool Execution Flow
1. **Model Request:** The model, based on the user's prompt and the provided tool schemas, decides to use a tool and returns a `FunctionCall` part in its response, specifying the tool name and arguments.
2. **Core Receives Request:** The core parses this `FunctionCall`.
3. **Tool Retrieval:** It looks up the requested tool in the `ToolRegistry`.
4. **Parameter Validation:** The tool's `validateToolParams()` method is called.
5. **Confirmation (if needed):**
- The tool's `shouldConfirmExecute()` method is called.
- If it returns details for confirmation, the core communicates this back to the CLI, which prompts the user.
- The user's decision (e.g., proceed, cancel) is sent back to the core.
6. **Execution:** If validated and confirmed (or if no confirmation is needed), the core calls the tool's `execute()` method with the provided arguments and an `AbortSignal` (for potential cancellation).
7. **Result Processing:** The `ToolResult` from `execute()` is received by the core.
8. **Response to Model:** The `llmContent` from the `ToolResult` is packaged as a `FunctionResponse` and sent back to the model so it can continue generating a user-facing response.
9. **Display to User:** The `returnDisplay` from the `ToolResult` is sent to the CLI to show the user what the tool did.
## Extending with Custom Tools
While direct programmatic registration of new tools by users isn't explicitly detailed as a primary workflow in the provided files for typical end-users, the architecture supports extension through:
- **Command-based Discovery:** Advanced users or project administrators can define a `tools.toolDiscoveryCommand` in `settings.json`. This command, when run by the core, should output a JSON array of `FunctionDeclaration` objects. The core will then make these available as `DiscoveredTool` instances. The corresponding `tools.toolCallCommand` would then be responsible for actually executing these custom tools.
- **MCP Server(s):** For more complex scenarios, one or more MCP servers can be set up and configured via the `mcpServers` setting in `settings.json`. The core can then discover and use tools exposed by these servers. As mentioned, if you have multiple MCP servers, the tool names will be prefixed with the server name from your configuration (e.g., `serverAlias__actualToolName`).
This tool system provides a flexible and powerful way to augment the model's capabilities, making Qwen Code a versatile assistant for a wide range of tasks.