---
layout: single
title:  "Qwen (通义千问)"
date:   2023-12-25 08:00:00 +0800
categories: [AI 与大模型, 操作系统]
tags: [QWen, FastChat, MacBookProM2Max]
---

- [Qwen](https://github.com/QwenLM/Qwen/blob/main/README_CN.md)

## 快速开始

### 克隆代码
```shell
git clone https://github.com/QwenLM/Qwen.git
cd Qwen
```

### 创建虚拟环境
```shell
python -m venv env
source env/bin/activate
```

### 安装依赖
```shell
pip install -r requirements.txt
```

### 创建大模型链接
```shell
mkdir Qwen
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-14B-Chat Qwen/Qwen-14B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B Qwen/Qwen-1_8B
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B-Chat Qwen/Qwen-1_8B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat
```

### 聊天

- 命令行聊天
```shell
python cli_demo.py
```

- Web 聊天
```shell
python web_demo.py
```


## FastChat

### 克隆代码
```shell
git clone https://github.com/lm-sys/FastChat
cd FastChat
```

### 创建虚拟环境
```shell
python -m venv env
source env/bin/activate
```

### 安装
```shell
pip install --upgrade pip
pip install -e ".[model_worker,webui]"
```

### 创建大模型链接
```shell
mkdir Qwen
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-14B-Chat Qwen/Qwen-14B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B Qwen/Qwen-1_8B
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B-Chat Qwen/Qwen-1_8B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat
```

### 运行
- Controller
```shell
python -m fastchat.serve.controller
```

- Model Worker
```shell
python -m fastchat.serve.model_worker \
    --model-path Qwen/Qwen-1_8B-Chat --port 21002 \
    --worker-address http://localhost:21002 \
    --device mps
```

- OpenAI API Server
```shell
python -m fastchat.serve.openai_api_server --port 8000
```

- Web Server
```shell
python -m fastchat.serve.gradio_web_server --host 0.0.0.0 --port 8001
```

### 测试 OpenAI API
```shell
curl http://127.0.0.1:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Qwen-1_8B-Chat",
        "prompt": "你是谁？",
        "temperature": 0.9
    }'|jq
```

**使用 Web 聊天的时候出现乱码，感觉 ChatML 格式的问题。**

## 参考资料
- [Qwen](https://github.com/QwenLM/Qwen/blob/main/README_CN.md)
- [系统指令 (System Prompts)](https://github.com/QwenLM/Qwen/blob/main/examples/system_prompt.md)
- [Tokenization](https://github.com/QwenLM/Qwen/blob/main/tokenization_note_zh.md)
- [ReAct Prompting 示例](https://github.com/QwenLM/Qwen/blob/main/examples/react_prompt.md)
- [FAQ](https://github.com/QwenLM/Qwen/blob/main/FAQ_zh.md)
- [LLM/阿里：通义千问QWen-7b与Qwen-7B-Chat](https://zhuanlan.zhihu.com/p/647873194)