abbyswag/qwen-code
1
0
Fork 0
mirror of https://github.com/QwenLM/qwen-code.git synced 2025-12-19 09:33:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: restructure docs

Browse Source
This commit is contained in:
pomelo-nwu
2025-12-04 18:26:05 +08:00
parent 6729980b47
commit 17785c418d
72 changed files with 0 additions and 412 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/developers/development/_meta.ts Normal file
View File

@@ -0,0 +1,8 @@
export default {
architecture: 'Architecture',
npm: 'NPM',
deployment: 'Deployment',
telemetry: 'Telemetry',
'integration-tests': 'Integration Tests',
'issue-and-pr-automation': 'Issue and PR Automation',
};

54
docs/developers/development/architecture.md Normal file
View File

@@ -0,0 +1,54 @@
# Qwen Code Architecture Overview
This document provides a high-level overview of Qwen Code's architecture.
## Core components
Qwen Code is primarily composed of two main packages, along with a suite of tools that can be used by the system in the course of handling command-line input:
1. **CLI package (`packages/cli`):**
- **Purpose:** This contains the user-facing portion of Qwen Code, such as handling the initial user input, presenting the final output, and managing the overall user experience.
- **Key functions contained in the package:**
- [Input processing](./cli/commands.md)
- History management
- Display rendering
- [Theme and UI customization](./cli/themes.md)
- [CLI configuration settings](./cli/configuration.md)
2. **Core package (`packages/core`):**
- **Purpose:** This acts as the backend for Qwen Code. It receives requests sent from `packages/cli`, orchestrates interactions with the configured model API, and manages the execution of available tools.
- **Key functions contained in the package:**
- API client for communicating with the Google Gemini API
- Prompt construction and management
- Tool registration and execution logic
- State management for conversations or sessions
- Server-side configuration
3. **Tools (`packages/core/src/tools/`):**
- **Purpose:** These are individual modules that extend the capabilities of the Gemini model, allowing it to interact with the local environment (e.g., file system, shell commands, web fetching).
- **Interaction:** `packages/core` invokes these tools based on requests from the Gemini model.
## Interaction Flow
A typical interaction with Qwen Code follows this flow:
1. **User input:** The user types a prompt or command into the terminal, which is managed by `packages/cli`.
2. **Request to core:** `packages/cli` sends the user's input to `packages/core`.
3. **Request processed:** The core package:
- Constructs an appropriate prompt for the configured model API, possibly including conversation history and available tool definitions.
- Sends the prompt to the model API.
4. **Model API response:** The model API processes the prompt and returns a response. This response might be a direct answer or a request to use one of the available tools.
5. **Tool execution (if applicable):**
- When the model API requests a tool, the core package prepares to execute it.
- If the requested tool can modify the file system or execute shell commands, the user is first given details of the tool and its arguments, and the user must approve the execution.
- Read-only operations, such as reading files, might not require explicit user confirmation to proceed.
- Once confirmed, or if confirmation is not required, the core package executes the relevant action within the relevant tool, and the result is sent back to the model API by the core package.
- The model API processes the tool result and generates a final response.
6. **Response to CLI:** The core package sends the final response back to the CLI package.
7. **Display to user:** The CLI package formats and displays the response to the user in the terminal.
## Key Design Principles
- **Modularity:** Separating the CLI (frontend) from the Core (backend) allows for independent development and potential future extensions (e.g., different frontends for the same backend).
- **Extensibility:** The tool system is designed to be extensible, allowing new capabilities to be added.
- **User experience:** The CLI focuses on providing a rich and interactive terminal experience.

117
docs/developers/development/deployment.md Normal file
View File

@@ -0,0 +1,117 @@
# Qwen Code Execution and Deployment
This document describes how to run Qwen Code and explains the deployment architecture that Qwen Code uses.
## Running Qwen Code
There are several ways to run Qwen Code. The option you choose depends on how you intend to use it.
---
### 1. Standard installation (Recommended for typical users)
This is the recommended way for end-users to install Qwen Code. It involves downloading the Qwen Code package from the NPM registry.
- **Global install:**
```bash
npm install -g @qwen-code/qwen-code
```
Then, run the CLI from anywhere:
```bash
qwen
```
- **NPX execution:**
```bash
# Execute the latest version from NPM without a global install
npx @qwen-code/qwen-code
```
---
### 2. Running in a sandbox (Docker/Podman)
For security and isolation, Qwen Code can be run inside a container. This is the default way that the CLI executes tools that might have side effects.
- **Directly from the Registry:**
You can run the published sandbox image directly. This is useful for environments where you only have Docker and want to run the CLI.
```bash
# Run the published sandbox image
docker run --rm -it ghcr.io/qwenlm/qwen-code:0.0.11
```
- **Using the `--sandbox` flag:**
If you have Qwen Code installed locally (using the standard installation described above), you can instruct it to run inside the sandbox container.
```bash
qwen --sandbox -y -p "your prompt here"
```
---
### 3. Running from source (Recommended for Qwen Code contributors)
Contributors to the project will want to run the CLI directly from the source code.
- **Development Mode:**
This method provides hot-reloading and is useful for active development.
```bash
# From the root of the repository
npm run start
```
- **Production-like mode (Linked package):**
This method simulates a global installation by linking your local package. It's useful for testing a local build in a production workflow.
```bash
# Link the local cli package to your global node_modules
npm link packages/cli
# Now you can run your local version using the `qwen` command
qwen
```
---
### 4. Running the latest Qwen Code commit from GitHub
You can run the most recently committed version of Qwen Code directly from the GitHub repository. This is useful for testing features still in development.
```bash
# Execute the CLI directly from the main branch on GitHub
npx https://github.com/QwenLM/qwen-code
```
## Deployment architecture
The execution methods described above are made possible by the following architectural components and processes:
**NPM packages**
Qwen Code project is a monorepo that publishes core packages to the NPM registry:
- `@qwen-code/qwen-code-core`: The backend, handling logic and tool execution.
- `@qwen-code/qwen-code`: The user-facing frontend.
These packages are used when performing the standard installation and when running Qwen Code from the source.
**Build and packaging processes**
There are two distinct build processes used, depending on the distribution channel:
- **NPM publication:** For publishing to the NPM registry, the TypeScript source code in `@qwen-code/qwen-code-core` and `@qwen-code/qwen-code` is transpiled into standard JavaScript using the TypeScript Compiler (`tsc`). The resulting `dist/` directory is what gets published in the NPM package. This is a standard approach for TypeScript libraries.
- **GitHub `npx` execution:** When running the latest version of Qwen Code directly from GitHub, a different process is triggered by the `prepare` script in `package.json`. This script uses `esbuild` to bundle the entire application and its dependencies into a single, self-contained JavaScript file. This bundle is created on-the-fly on the user's machine and is not checked into the repository.
**Docker sandbox image**
The Docker-based execution method is supported by the `qwen-code-sandbox` container image. This image is published to a container registry and contains a pre-installed, global version of Qwen Code.
## Release process
The release process is automated through GitHub Actions. The release workflow performs the following actions:
1. Build the NPM packages using `tsc`.
2. Publish the NPM packages to the artifact registry.
3. Create GitHub releases with bundled assets.

137
docs/developers/development/integration-tests.md Normal file
View File

@@ -0,0 +1,137 @@
# Integration Tests
This document provides information about the integration testing framework used in this project.
## Overview
The integration tests are designed to validate the end-to-end functionality of Qwen Code. They execute the built binary in a controlled environment and verify that it behaves as expected when interacting with the file system.
These tests are located in the `integration-tests` directory and are run using a custom test runner.
## Running the tests
The integration tests are not run as part of the default `npm run test` command. They must be run explicitly using the `npm run test:integration:all` script.
The integration tests can also be run using the following shortcut:
```bash
npm run test:e2e
```
## Running a specific set of tests
To run a subset of test files, you can use `npm run <integration test command> <file_name1> ....` where &lt;integration test command&gt; is either `test:e2e` or `test:integration*` and `<file_name>` is any of the `.test.js` files in the `integration-tests/` directory. For example, the following command runs `list_directory.test.js` and `write_file.test.js`:
```bash
npm run test:e2e list_directory write_file
```
### Running a single test by name
To run a single test by its name, use the `--test-name-pattern` flag:
```bash
npm run test:e2e -- --test-name-pattern "reads a file"
```
### Running all tests
To run the entire suite of integration tests, use the following command:
```bash
npm run test:integration:all
```
### Sandbox matrix
The `all` command will run tests for `no sandboxing`, `docker` and `podman`.
Each individual type can be run using the following commands:
```bash
npm run test:integration:sandbox:none
```
```bash
npm run test:integration:sandbox:docker
```
```bash
npm run test:integration:sandbox:podman
```
## Diagnostics
The integration test runner provides several options for diagnostics to help track down test failures.
### Keeping test output
You can preserve the temporary files created during a test run for inspection. This is useful for debugging issues with file system operations.
To keep the test output set the `KEEP_OUTPUT` environment variable to `true`.
```bash
KEEP_OUTPUT=true npm run test:integration:sandbox:none
```
When output is kept, the test runner will print the path to the unique directory for the test run.
### Verbose output
For more detailed debugging, set the `VERBOSE` environment variable to `true`.
```bash
VERBOSE=true npm run test:integration:sandbox:none
```
When using `VERBOSE=true` and `KEEP_OUTPUT=true` in the same command, the output is streamed to the console and also saved to a log file within the test's temporary directory.
The verbose output is formatted to clearly identify the source of the logs:
```
--- TEST: <log dir>:<test-name> ---
... output from the qwen command ...
--- END TEST: <log dir>:<test-name> ---
```
## Linting and formatting
To ensure code quality and consistency, the integration test files are linted as part of the main build process. You can also manually run the linter and auto-fixer.
### Running the linter
To check for linting errors, run the following command:
```bash
npm run lint
```
You can include the `:fix` flag in the command to automatically fix any fixable linting errors:
```bash
npm run lint:fix
```
## Directory structure
The integration tests create a unique directory for each test run inside the `.integration-tests` directory. Within this directory, a subdirectory is created for each test file, and within that, a subdirectory is created for each individual test case.
This structure makes it easy to locate the artifacts for a specific test run, file, or case.
```
.integration-tests/
└── <run-id>/
└── <test-file-name>.test.js/
└── <test-case-name>/
├── output.log
└── ...other test artifacts...
```
## Continuous integration
To ensure the integration tests are always run, a GitHub Actions workflow is defined in `.github/workflows/e2e.yml`. This workflow automatically runs the integrations tests for pull requests against the `main` branch, or when a pull request is added to a merge queue.
The workflow runs the tests in different sandboxing environments to ensure Qwen Code is tested across each:
- `sandbox:none`: Runs the tests without any sandboxing.
- `sandbox:docker`: Runs the tests in a Docker container.
- `sandbox:podman`: Runs the tests in a Podman container.

84
docs/developers/development/issue-and-pr-automation.md Normal file
View File

@@ -0,0 +1,84 @@
# Automation and Triage Processes
This document provides a detailed overview of the automated processes we use to manage and triage issues and pull requests. Our goal is to provide prompt feedback and ensure that contributions are reviewed and integrated efficiently. Understanding this automation will help you as a contributor know what to expect and how to best interact with our repository bots.
## Guiding Principle: Issues and Pull Requests
First and foremost, almost every Pull Request (PR) should be linked to a corresponding Issue. The issue describes the "what" and the "why" (the bug or feature), while the PR is the "how" (the implementation). This separation helps us track work, prioritize features, and maintain clear historical context. Our automation is built around this principle.
---
## Detailed Automation Workflows
Here is a breakdown of the specific automation workflows that run in our repository.
### 1. When you open an Issue: `Automated Issue Triage`
This is the first bot you will interact with when you create an issue. Its job is to perform an initial analysis and apply the correct labels.
- **Workflow File**: `.github/workflows/qwen-automated-issue-triage.yml`
- **When it runs**: Immediately after an issue is created or reopened.
- **What it does**:
- It uses a Qwen model to analyze the issue's title and body against a detailed set of guidelines.
- **Applies one `area/*` label**: Categorizes the issue into a functional area of the project (e.g., `area/ux`, `area/models`, `area/platform`).
- **Applies one `kind/*` label**: Identifies the type of issue (e.g., `kind/bug`, `kind/enhancement`, `kind/question`).
- **Applies one `priority/*` label**: Assigns a priority from P0 (critical) to P3 (low) based on the described impact.
- **May apply `status/need-information`**: If the issue lacks critical details (like logs or reproduction steps), it will be flagged for more information.
- **May apply `status/need-retesting`**: If the issue references a CLI version that is more than six versions old, it will be flagged for retesting on a current version.
- **What you should do**:
- Fill out the issue template as completely as possible. The more detail you provide, the more accurate the triage will be.
- If the `status/need-information` label is added, please provide the requested details in a comment.
### 2. When you open a Pull Request: `Continuous Integration (CI)`
This workflow ensures that all changes meet our quality standards before they can be merged.
- **Workflow File**: `.github/workflows/ci.yml`
- **When it runs**: On every push to a pull request.
- **What it does**:
- **Lint**: Checks that your code adheres to our project's formatting and style rules.
- **Test**: Runs our full suite of automated tests across macOS, Windows, and Linux, and on multiple Node.js versions. This is the most time-consuming part of the CI process.
- **Post Coverage Comment**: After all tests have successfully passed, a bot will post a comment on your PR. This comment provides a summary of how well your changes are covered by tests.
- **What you should do**:
- Ensure all CI checks pass. A green checkmark ✅ will appear next to your commit when everything is successful.
- If a check fails (a red "X" ❌), click the "Details" link next to the failed check to view the logs, identify the problem, and push a fix.
### 3. Ongoing Triage for Pull Requests: `PR Auditing and Label Sync`
This workflow runs periodically to ensure all open PRs are correctly linked to issues and have consistent labels.
- **Workflow File**: `.github/workflows/qwen-scheduled-pr-triage.yml`
- **When it runs**: Every 15 minutes on all open pull requests.
- **What it does**:
- **Checks for a linked issue**: The bot scans your PR description for a keyword that links it to an issue (e.g., `Fixes #123`, `Closes #456`).
- **Adds `status/need-issue`**: If no linked issue is found, the bot will add the `status/need-issue` label to your PR. This is a clear signal that an issue needs to be created and linked.
- **Synchronizes labels**: If an issue _is_ linked, the bot ensures the PR's labels perfectly match the issue's labels. It will add any missing labels and remove any that don't belong, and it will remove the `status/need-issue` label if it was present.
- **What you should do**:
- **Always link your PR to an issue.** This is the most important step. Add a line like `Resolves #<issue-number>` to your PR description.
- This will ensure your PR is correctly categorized and moves through the review process smoothly.
### 4. Ongoing Triage for Issues: `Scheduled Issue Triage`
This is a fallback workflow to ensure that no issue gets missed by the triage process.
- **Workflow File**: `.github/workflows/qwen-scheduled-issue-triage.yml`
- **When it runs**: Every hour on all open issues.
- **What it does**:
- It actively seeks out issues that either have no labels at all or still have the `status/need-triage` label.
- It then triggers the same powerful QwenCode-based analysis as the initial triage bot to apply the correct labels.
- **What you should do**:
- You typically don't need to do anything. This workflow is a safety net to ensure every issue is eventually categorized, even if the initial triage fails.
### 5. Release Automation
This workflow handles the process of packaging and publishing new versions of Qwen Code.
- **Workflow File**: `.github/workflows/release.yml`
- **When it runs**: On a daily schedule for "nightly" releases, and manually for official patch/minor releases.
- **What it does**:
- Automatically builds the project, bumps the version numbers, and publishes the packages to npm.
- Creates a corresponding release on GitHub with generated release notes.
- **What you should do**:
- As a contributor, you don't need to do anything for this process. You can be confident that once your PR is merged into the `main` branch, your changes will be included in the very next nightly release.
We hope this detailed overview is helpful. If you have any questions about our automation or processes, please don't hesitate to ask!

280
docs/developers/development/npm.md Normal file
View File

@@ -0,0 +1,280 @@
# Package Overview
This monorepo contains two main packages: `@qwen-code/qwen-code` and `@qwen-code/qwen-code-core`.
## `@qwen-code/qwen-code`
This is the main package for Qwen Code. It is responsible for the user interface, command parsing, and all other user-facing functionality.
When this package is published, it is bundled into a single executable file. This bundle includes all of the package's dependencies, including `@qwen-code/qwen-code-core`. This means that whether a user installs the package with `npm install -g @qwen-code/qwen-code` or runs it directly with `npx @qwen-code/qwen-code`, they are using this single, self-contained executable.
## `@qwen-code/qwen-code-core`
This package contains the core logic for the CLI. It is responsible for making API requests to configured providers, handling authentication, and managing the local cache.
This package is not bundled. When it is published, it is published as a standard Node.js package with its own dependencies. This allows it to be used as a standalone package in other projects, if needed. All transpiled js code in the `dist` folder is included in the package.
# Release Process
This project follows a structured release process to ensure that all packages are versioned and published correctly. The process is designed to be as automated as possible.
## How To Release
Releases are managed through the [release.yml](https://github.com/QwenLM/qwen-code/actions/workflows/release.yml) GitHub Actions workflow. To perform a manual release for a patch or hotfix:
1. Navigate to the **Actions** tab of the repository.
2. Select the **Release** workflow from the list.
3. Click the **Run workflow** dropdown button.
4. Fill in the required inputs:
- **Version**: The exact version to release (e.g., `v0.2.1`).
- **Ref**: The branch or commit SHA to release from (defaults to `main`).
- **Dry Run**: Leave as `true` to test the workflow without publishing, or set to `false` to perform a live release.
5. Click **Run workflow**.
## Nightly Releases
In addition to manual releases, this project has an automated nightly release process to provide the latest "bleeding edge" version for testing and development.
### Process
Every night at midnight UTC, the [Release workflow](https://github.com/QwenLM/qwen-code/actions/workflows/release.yml) runs automatically on a schedule. It performs the following steps:
1. Checks out the latest code from the `main` branch.
2. Installs all dependencies.
3. Runs the full suite of `preflight` checks and integration tests.
4. If all tests succeed, it calculates the next nightly version number (e.g., `v0.2.1-nightly.20230101`).
5. It then builds and publishes the packages to npm with the `nightly` dist-tag.
6. Finally, it creates a GitHub Release for the nightly version.
### Failure Handling
If any step in the nightly workflow fails, it will automatically create a new issue in the repository with the labels `bug` and `nightly-failure`. The issue will contain a link to the failed workflow run for easy debugging.
### How to Use the Nightly Build
To install the latest nightly build, use the `@nightly` tag:
```bash
npm install -g @qwen-code/qwen-code@nightly
```
We also run a Google cloud build called [release-docker.yml](../.gcp/release-docker.yml). Which publishes the sandbox docker to match your release. This will also be moved to GH and combined with the main release file once service account permissions are sorted out.
### After the Release
After the workflow has successfully completed, you can monitor its progress in the [GitHub Actions tab](https://github.com/QwenLM/qwen-code/actions/workflows/release.yml). Once complete, you should:
1. Go to the [pull requests page](https://github.com/QwenLM/qwen-code/pulls) of the repository.
2. Create a new pull request from the `release/vX.Y.Z` branch to `main`.
3. Review the pull request (it should only contain version updates in `package.json` files) and merge it. This keeps the version in `main` up-to-date.
## Release Validation
After pushing a new release smoke testing should be performed to ensure that the packages are working as expected. This can be done by installing the packages locally and running a set of tests to ensure that they are functioning correctly.
- `npx -y @qwen-code/qwen-code@latest --version` to validate the push worked as expected if you were not doing a rc or dev tag
- `npx -y @qwen-code/qwen-code@<release tag> --version` to validate the tag pushed appropriately
- _This is destructive locally_ `npm uninstall @qwen-code/qwen-code && npm uninstall -g @qwen-code/qwen-code && npm cache clean --force && npm install @qwen-code/qwen-code@<version>`
- Smoke testing a basic run through of exercising a few llm commands and tools is recommended to ensure that the packages are working as expected. We'll codify this more in the future.
## When to merge the version change, or not?
The above pattern for creating patch or hotfix releases from current or older commits leaves the repository in the following state:
1. The Tag (`vX.Y.Z-patch.1`): This tag correctly points to the original commit on main
that contains the stable code you intended to release. This is crucial. Anyone checking
out this tag gets the exact code that was published.
2. The Branch (`release-vX.Y.Z-patch.1`): This branch contains one new commit on top of the
tagged commit. That new commit only contains the version number change in package.json
(and other related files like package-lock.json).
This separation is good. It keeps your main branch history clean of release-specific
version bumps until you decide to merge them.
This is the critical decision, and it depends entirely on the nature of the release.
### Merge Back for Stable Patches and Hotfixes
You almost always want to merge the `release-<tag>` branch back into `main` for any
stable patch or hotfix release.
- Why? The primary reason is to update the version in main's package.json. If you release
v1.2.1 from an older commit but never merge the version bump back, your main branch's
package.json will still say "version": "1.2.0". The next developer who starts work for
the next feature release (v1.3.0) will be branching from a codebase that has an
incorrect, older version number. This leads to confusion and requires manual version
bumping later.
- The Process: After the release-v1.2.1 branch is created and the package is successfully
published, you should open a pull request to merge release-v1.2.1 into main. This PR
will contain just one commit: "chore: bump version to v1.2.1". It's a clean, simple
integration that keeps your main branch in sync with the latest released version.
### Do NOT Merge Back for Pre-Releases (RC, Beta, Dev)
You typically do not merge release branches for pre-releases back into `main`.
- Why? Pre-release versions (e.g., v1.3.0-rc.1, v1.3.0-rc.2) are, by definition, not
stable and are temporary. You don't want to pollute your main branch's history with a
series of version bumps for release candidates. The package.json in main should reflect
the latest stable release version, not an RC.
- The Process: The release-v1.3.0-rc.1 branch is created, the npm publish --tag rc happens,
and then... the branch has served its purpose. You can simply delete it. The code for
the RC is already on main (or a feature branch), so no functional code is lost. The
release branch was just a temporary vehicle for the version number.
## Local Testing and Validation: Changes to the Packaging and Publishing Process
If you need to test the release process without actually publishing to NPM or creating a public GitHub release, you can trigger the workflow manually from the GitHub UI.
1. Go to the [Actions tab](https://github.com/QwenLM/qwen-code/actions/workflows/release.yml) of the repository.
2. Click on the "Run workflow" dropdown.
3. Leave the `dry_run` option checked (`true`).
4. Click the "Run workflow" button.
This will run the entire release process but will skip the `npm publish` and `gh release create` steps. You can inspect the workflow logs to ensure everything is working as expected.
It is crucial to test any changes to the packaging and publishing process locally before committing them. This ensures that the packages will be published correctly and that they will work as expected when installed by a user.
To validate your changes, you can perform a dry run of the publishing process. This will simulate the publishing process without actually publishing the packages to the npm registry.
```bash
npm_package_version=9.9.9 SANDBOX_IMAGE_REGISTRY="registry" SANDBOX_IMAGE_NAME="thename" npm run publish:npm --dry-run
```
This command will do the following:
1. Build all the packages.
2. Run all the prepublish scripts.
3. Create the package tarballs that would be published to npm.
4. Print a summary of the packages that would be published.
You can then inspect the generated tarballs to ensure that they contain the correct files and that the `package.json` files have been updated correctly. The tarballs will be created in the root of each package's directory (e.g., `packages/cli/qwen-code-0.1.6.tgz`).
By performing a dry run, you can be confident that your changes to the packaging process are correct and that the packages will be published successfully.
## Release Deep Dive
The main goal of the release process is to take the source code from the packages/ directory, build it, and assemble a
clean, self-contained package in a temporary `bundle` directory at the root of the project. This `bundle` directory is what
actually gets published to NPM.
Here are the key stages:
Stage 1: Pre-Release Sanity Checks and Versioning
- What happens: Before any files are moved, the process ensures the project is in a good state. This involves running tests,
linting, and type-checking (npm run preflight). The version number in the root package.json and packages/cli/package.json
is updated to the new release version.
- Why: This guarantees that only high-quality, working code is released. Versioning is the first step to signify a new
release.
Stage 2: Building the Source Code
- What happens: The TypeScript source code in packages/core/src and packages/cli/src is compiled into JavaScript.
- File movement:
- packages/core/src/\*_/_.ts -> compiled to -> packages/core/dist/
- packages/cli/src/\*_/_.ts -> compiled to -> packages/cli/dist/
- Why: The TypeScript code written during development needs to be converted into plain JavaScript that can be run by
Node.js. The core package is built first as the cli package depends on it.
Stage 3: Assembling the Final Publishable Package
This is the most critical stage where files are moved and transformed into their final state for publishing. A temporary
`bundle` folder is created at the project root to house the final package contents.
1. The `package.json` is Transformed:
- What happens: The package.json from packages/cli/ is read, modified, and written into the root `bundle`/ directory.
- File movement: packages/cli/package.json -> (in-memory transformation) -> `bundle`/package.json
- Why: The final package.json must be different from the one used in development. Key changes include:
- Removing devDependencies.
- Removing workspace-specific "dependencies": { "@qwen-code/core": "workspace:\*" } and ensuring the core code is
bundled directly into the final JavaScript file.
- Ensuring the bin, main, and files fields point to the correct locations within the final package structure.
2. The JavaScript Bundle is Created:
- What happens: The built JavaScript from both packages/core/dist and packages/cli/dist are bundled into a single,
executable JavaScript file.
- File movement: packages/cli/dist/index.js + packages/core/dist/index.js -> (bundled by esbuild) -> `bundle`/gemini.js (or a
similar name).
- Why: This creates a single, optimized file that contains all the necessary application code. It simplifies the package
by removing the need for the core package to be a separate dependency on NPM, as its code is now included directly.
3. Static and Supporting Files are Copied:
- What happens: Essential files that are not part of the source code but are required for the package to work correctly
or be well-described are copied into the `bundle` directory.
- File movement:
- README.md -> `bundle`/README.md
- LICENSE -> `bundle`/LICENSE
- packages/cli/src/utils/\*.sb (sandbox profiles) -> `bundle`/
- Why:
- The README.md and LICENSE are standard files that should be included in any NPM package.
- The sandbox profiles (.sb files) are critical runtime assets required for the CLI's sandboxing feature to
function. They must be located next to the final executable.
Stage 4: Publishing to NPM
- What happens: The npm publish command is run from inside the root `bundle` directory.
- Why: By running npm publish from within the `bundle` directory, only the files we carefully assembled in Stage 3 are uploaded
to the NPM registry. This prevents any source code, test files, or development configurations from being accidentally
published, resulting in a clean and minimal package for users.
Summary of File Flow
```mermaid
graph TD
subgraph "Source Files"
A["packages/core/src/*.ts<br/>packages/cli/src/*.ts"]
B["packages/cli/package.json"]
C["README.md<br/>LICENSE<br/>packages/cli/src/utils/*.sb"]
end
subgraph "Process"
D(Build)
E(Transform)
F(Assemble)
G(Publish)
end
subgraph "Artifacts"
H["Bundled JS"]
I["Final package.json"]
J["bundle/"]
end
subgraph "Destination"
K["NPM Registry"]
end
A --> D --> H
B --> E --> I
C --> F
H --> F
I --> F
F --> J
J --> G --> K
```
This process ensures that the final published artifact is a purpose-built, clean, and efficient representation of the
project, rather than a direct copy of the development workspace.
## NPM Workspaces
This project uses [NPM Workspaces](https://docs.npmjs.com/cli/v10/using-npm/workspaces) to manage the packages within this monorepo. This simplifies development by allowing us to manage dependencies and run scripts across multiple packages from the root of the project.
### How it Works
The root `package.json` file defines the workspaces for this project:
```json
{
"workspaces": ["packages/*"]
}
```
This tells NPM that any folder inside the `packages` directory is a separate package that should be managed as part of the workspace.
### Benefits of Workspaces
- **Simplified Dependency Management**: Running `npm install` from the root of the project will install all dependencies for all packages in the workspace and link them together. This means you don't need to run `npm install` in each package's directory.
- **Automatic Linking**: Packages within the workspace can depend on each other. When you run `npm install`, NPM will automatically create symlinks between the packages. This means that when you make changes to one package, the changes are immediately available to other packages that depend on it.
- **Simplified Script Execution**: You can run scripts in any package from the root of the project using the `--workspace` flag. For example, to run the `build` script in the `cli` package, you can run `npm run build --workspace @qwen-code/qwen-code`.

369
docs/developers/development/telemetry.md Normal file
View File

@@ -0,0 +1,369 @@
# Observability with OpenTelemetry
Learn how to enable and setup OpenTelemetry for Qwen Code.
- [Observability with OpenTelemetry](#observability-with-opentelemetry)
- [Key Benefits](#key-benefits)
- [OpenTelemetry Integration](#opentelemetry-integration)
- [Configuration](#configuration)
- [Google Cloud Telemetry](#google-cloud-telemetry)
- [Prerequisites](#prerequisites)
- [Direct Export (Recommended)](#direct-export-recommended)
- [Collector-Based Export (Advanced)](#collector-based-export-advanced)
- [Local Telemetry](#local-telemetry)
- [File-based Output (Recommended)](#file-based-output-recommended)
- [Collector-Based Export (Advanced)](#collector-based-export-advanced-1)
- [Logs and Metrics](#logs-and-metrics)
- [Logs](#logs)
- [Metrics](#metrics)
## Key Benefits
- **🔍 Usage Analytics**: Understand interaction patterns and feature adoption
across your team
- **⚡ Performance Monitoring**: Track response times, token consumption, and
resource utilization
- **🐛 Real-time Debugging**: Identify bottlenecks, failures, and error patterns
as they occur
- **📊 Workflow Optimization**: Make informed decisions to improve
configurations and processes
- **🏢 Enterprise Governance**: Monitor usage across teams, track costs, ensure
compliance, and integrate with existing monitoring infrastructure
## OpenTelemetry Integration
Built on **[OpenTelemetry]** — the vendor-neutral, industry-standard
observability framework — Qwen Code's observability system provides:
- **Universal Compatibility**: Export to any OpenTelemetry backend (Google
Cloud, Jaeger, Prometheus, Datadog, etc.)
- **Standardized Data**: Use consistent formats and collection methods across
your toolchain
- **Future-Proof Integration**: Connect with existing and future observability
infrastructure
- **No Vendor Lock-in**: Switch between backends without changing your
instrumentation
[OpenTelemetry]: https://opentelemetry.io/
## Configuration
All telemetry behavior is controlled through your `.qwen/settings.json` file.
These settings can be overridden by environment variables or CLI flags.
| Setting | Environment Variable | CLI Flag | Description | Values | Default |
| -------------- | -------------------------------- | -------------------------------------------------------- | ------------------------------------------------- | ----------------- | ----------------------- |
| `enabled` | `GEMINI_TELEMETRY_ENABLED` | `--telemetry` / `--no-telemetry` | Enable or disable telemetry | `true`/`false` | `false` |
| `target` | `GEMINI_TELEMETRY_TARGET` | `--telemetry-target <local\|gcp>` | Where to send telemetry data | `"gcp"`/`"local"` | `"local"` |
| `otlpEndpoint` | `GEMINI_TELEMETRY_OTLP_ENDPOINT` | `--telemetry-otlp-endpoint <URL>` | OTLP collector endpoint | URL string | `http://localhost:4317` |
| `otlpProtocol` | `GEMINI_TELEMETRY_OTLP_PROTOCOL` | `--telemetry-otlp-protocol <grpc\|http>` | OTLP transport protocol | `"grpc"`/`"http"` | `"grpc"` |
| `outfile` | `GEMINI_TELEMETRY_OUTFILE` | `--telemetry-outfile <path>` | Save telemetry to file (overrides `otlpEndpoint`) | file path | - |
| `logPrompts` | `GEMINI_TELEMETRY_LOG_PROMPTS` | `--telemetry-log-prompts` / `--no-telemetry-log-prompts` | Include prompts in telemetry logs | `true`/`false` | `true` |
| `useCollector` | `GEMINI_TELEMETRY_USE_COLLECTOR` | - | Use external OTLP collector (advanced) | `true`/`false` | `false` |
**Note on boolean environment variables:** For the boolean settings (`enabled`,
`logPrompts`, `useCollector`), setting the corresponding environment variable to
`true` or `1` will enable the feature. Any other value will disable it.
For detailed information about all configuration options, see the
[Configuration Guide](./cli/configuration.md).
## Google Cloud Telemetry
### Prerequisites
Before using either method below, complete these steps:
1. Set your Google Cloud project ID:
- For telemetry in a separate project from inference:
```bash
export OTLP_GOOGLE_CLOUD_PROJECT="your-telemetry-project-id"
```
- For telemetry in the same project as inference:
```bash
export GOOGLE_CLOUD_PROJECT="your-project-id"
```
2. Authenticate with Google Cloud:
- If using a user account:
```bash
gcloud auth application-default login
```
- If using a service account:
```bash
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/service-account.json"
```
3. Make sure your account or service account has these IAM roles:
- Cloud Trace Agent
- Monitoring Metric Writer
- Logs Writer
4. Enable the required Google Cloud APIs (if not already enabled):
```bash
gcloud services enable \
cloudtrace.googleapis.com \
monitoring.googleapis.com \
logging.googleapis.com \
--project="$OTLP_GOOGLE_CLOUD_PROJECT"
```
### Direct Export (Recommended)
Sends telemetry directly to Google Cloud services. No collector needed.
1. Enable telemetry in your `.qwen/settings.json`:
```json
{
"telemetry": {
"enabled": true,
"target": "gcp"
}
}
```
2. Run Qwen Code and send prompts.
3. View logs and metrics:
- Open the Google Cloud Console in your browser after sending prompts:
- Logs: https://console.cloud.google.com/logs/
- Metrics: https://console.cloud.google.com/monitoring/metrics-explorer
- Traces: https://console.cloud.google.com/traces/list
### Collector-Based Export (Advanced)
For custom processing, filtering, or routing, use an OpenTelemetry collector to
forward data to Google Cloud.
1. Configure your `.qwen/settings.json`:
```json
{
"telemetry": {
"enabled": true,
"target": "gcp",
"useCollector": true
}
}
```
2. Run the automation script:
```bash
npm run telemetry -- --target=gcp
```
This will:
- Start a local OTEL collector that forwards to Google Cloud
- Configure your workspace
- Provide links to view traces, metrics, and logs in Google Cloud Console
- Save collector logs to `~/.qwen/tmp/<projectHash>/otel/collector-gcp.log`
- Stop collector on exit (e.g. `Ctrl+C`)
3. Run Qwen Code and send prompts.
4. View logs and metrics:
- Open the Google Cloud Console in your browser after sending prompts:
- Logs: https://console.cloud.google.com/logs/
- Metrics: https://console.cloud.google.com/monitoring/metrics-explorer
- Traces: https://console.cloud.google.com/traces/list
- Open `~/.qwen/tmp/<projectHash>/otel/collector-gcp.log` to view local
collector logs.
## Local Telemetry
For local development and debugging, you can capture telemetry data locally:
### File-based Output (Recommended)
1. Enable telemetry in your `.qwen/settings.json`:
```json
{
"telemetry": {
"enabled": true,
"target": "local",
"otlpEndpoint": "",
"outfile": ".qwen/telemetry.log"
}
}
```
2. Run Qwen Code and send prompts.
3. View logs and metrics in the specified file (e.g., `.qwen/telemetry.log`).
### Collector-Based Export (Advanced)
1. Run the automation script:
```bash
npm run telemetry -- --target=local
```
This will:
- Download and start Jaeger and OTEL collector
- Configure your workspace for local telemetry
- Provide a Jaeger UI at http://localhost:16686
- Save logs/metrics to `~/.qwen/tmp/<projectHash>/otel/collector.log`
- Stop collector on exit (e.g. `Ctrl+C`)
2. Run Qwen Code and send prompts.
3. View traces at http://localhost:16686 and logs/metrics in the collector log
file.
## Logs and Metrics
The following section describes the structure of logs and metrics generated for
Qwen Code.
- A `sessionId` is included as a common attribute on all logs and metrics.
### Logs
Logs are timestamped records of specific events. The following events are logged for Qwen Code:
- `qwen-code.config`: This event occurs once at startup with the CLI's configuration.
- **Attributes**:
- `model` (string)
- `embedding_model` (string)
- `sandbox_enabled` (boolean)
- `core_tools_enabled` (string)
- `approval_mode` (string)
- `api_key_enabled` (boolean)
- `vertex_ai_enabled` (boolean)
- `code_assist_enabled` (boolean)
- `log_prompts_enabled` (boolean)
- `file_filtering_respect_git_ignore` (boolean)
- `debug_mode` (boolean)
- `mcp_servers` (string)
- `output_format` (string: "text" or "json")
- `qwen-code.user_prompt`: This event occurs when a user submits a prompt.
- **Attributes**:
- `prompt_length` (int)
- `prompt_id` (string)
- `prompt` (string, this attribute is excluded if `log_prompts_enabled` is
configured to be `false`)
- `auth_type` (string)
- `qwen-code.tool_call`: This event occurs for each function call.
- **Attributes**:
- `function_name`
- `function_args`
- `duration_ms`
- `success` (boolean)
- `decision` (string: "accept", "reject", "auto_accept", or "modify", if
applicable)
- `error` (if applicable)
- `error_type` (if applicable)
- `content_length` (int, if applicable)
- `metadata` (if applicable, dictionary of string -> any)
- `qwen-code.file_operation`: This event occurs for each file operation.
- **Attributes**:
- `tool_name` (string)
- `operation` (string: "create", "read", "update")
- `lines` (int, if applicable)
- `mimetype` (string, if applicable)
- `extension` (string, if applicable)
- `programming_language` (string, if applicable)
- `diff_stat` (json string, if applicable): A JSON string with the following members:
- `ai_added_lines` (int)
- `ai_removed_lines` (int)
- `user_added_lines` (int)
- `user_removed_lines` (int)
- `qwen-code.api_request`: This event occurs when making a request to Qwen API.
- **Attributes**:
- `model`
- `request_text` (if applicable)
- `qwen-code.api_error`: This event occurs if the API request fails.
- **Attributes**:
- `model`
- `error`
- `error_type`
- `status_code`
- `duration_ms`
- `auth_type`
- `qwen-code.api_response`: This event occurs upon receiving a response from Qwen API.
- **Attributes**:
- `model`
- `status_code`
- `duration_ms`
- `error` (optional)
- `input_token_count`
- `output_token_count`
- `cached_content_token_count`
- `thoughts_token_count`
- `tool_token_count`
- `response_text` (if applicable)
- `auth_type`
- `qwen-code.tool_output_truncated`: This event occurs when the output of a tool call is too large and gets truncated.
- **Attributes**:
- `tool_name` (string)
- `original_content_length` (int)
- `truncated_content_length` (int)
- `threshold` (int)
- `lines` (int)
- `prompt_id` (string)
- `qwen-code.malformed_json_response`: This event occurs when a `generateJson` response from Qwen API cannot be parsed as a json.
- **Attributes**:
- `model`
- `qwen-code.flash_fallback`: This event occurs when Qwen Code switches to flash as fallback.
- **Attributes**:
- `auth_type`
- `qwen-code.slash_command`: This event occurs when a user executes a slash command.
- **Attributes**:
- `command` (string)
- `subcommand` (string, if applicable)
- `qwen-code.extension_enable`: This event occurs when an extension is enabled
- `qwen-code.extension_install`: This event occurs when an extension is installed
- **Attributes**:
- `extension_name` (string)
- `extension_version` (string)
- `extension_source` (string)
- `status` (string)
- `qwen-code.extension_uninstall`: This event occurs when an extension is uninstalled
### Metrics
Metrics are numerical measurements of behavior over time. The following metrics are collected for Qwen Code (metric names remain `qwen-code.*` for compatibility):
- `qwen-code.session.count` (Counter, Int): Incremented once per CLI startup.
- `qwen-code.tool.call.count` (Counter, Int): Counts tool calls.
- **Attributes**:
- `function_name`
- `success` (boolean)
- `decision` (string: "accept", "reject", or "modify", if applicable)
- `tool_type` (string: "mcp", or "native", if applicable)
- `qwen-code.tool.call.latency` (Histogram, ms): Measures tool call latency.
- **Attributes**:
- `function_name`
- `decision` (string: "accept", "reject", or "modify", if applicable)
- `qwen-code.api.request.count` (Counter, Int): Counts all API requests.
- **Attributes**:
- `model`
- `status_code`
- `error_type` (if applicable)
- `qwen-code.api.request.latency` (Histogram, ms): Measures API request latency.
- **Attributes**:
- `model`
- `qwen-code.token.usage` (Counter, Int): Counts the number of tokens used.
- **Attributes**:
- `model`
- `type` (string: "input", "output", "thought", "cache", or "tool")
- `qwen-code.file.operation.count` (Counter, Int): Counts file operations.
- **Attributes**:
- `operation` (string: "create", "read", "update"): The type of file operation.
- `lines` (Int, if applicable): Number of lines in the file.
- `mimetype` (string, if applicable): Mimetype of the file.
- `extension` (string, if applicable): File extension of the file.
- `model_added_lines` (Int, if applicable): Number of lines added/changed by the model.
- `model_removed_lines` (Int, if applicable): Number of lines removed/changed by the model.
- `user_added_lines` (Int, if applicable): Number of lines added/changed by user in AI proposed changes.
- `user_removed_lines` (Int, if applicable): Number of lines removed/changed by user in AI proposed changes.
- `programming_language` (string, if applicable): The programming language of the file.
- `qwen-code.chat_compression` (Counter, Int): Counts chat compression operations
- **Attributes**:
- `tokens_before`: (Int): Number of tokens in context prior to compression
- `tokens_after`: (Int): Number of tokens in context after compression