目录

快速开始

克隆代码

git clone https://github.com/QwenLM/Qwen.git
cd Qwen

创建虚拟环境

python -m venv env
source env/bin/activate

安装依赖

pip install -r requirements.txt

创建大模型链接

mkdir Qwen
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-14B-Chat Qwen/Qwen-14B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B Qwen/Qwen-1_8B
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B-Chat Qwen/Qwen-1_8B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat

聊天

  • 命令行聊天 
    python cli_demo.py
  • Web 聊天 
    python web_demo.py

FastChat

克隆代码

git clone https://github.com/lm-sys/FastChat
cd FastChat

创建虚拟环境

python -m venv env
source env/bin/activate

安装

pip install --upgrade pip
pip install -e ".[model_worker,webui]"

创建大模型链接

mkdir Qwen
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-14B-Chat Qwen/Qwen-14B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B Qwen/Qwen-1_8B
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-1_8B-Chat Qwen/Qwen-1_8B-Chat
ln -s /Users/junjian/HuggingFace/Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat

运行

  • Controller 
    python -m fastchat.serve.controller
  • Model Worker 
    python -m fastchat.serve.model_worker \
  --model-path Qwen/Qwen-1_8B-Chat --port 21002 \
  --worker-address http://localhost:21002 \
  --device mps
  • OpenAI API Server 
    python -m fastchat.serve.openai_api_server --port 8000
  • Web Server 
    python -m fastchat.serve.gradio_web_server --host 0.0.0.0 --port 8001

测试 OpenAI API

curl http://127.0.0.1:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Qwen-1_8B-Chat",
        "prompt": "你是谁？",
        "temperature": 0.9
    }'|jq

使用 Web 聊天的时候出现乱码，感觉 ChatML 格式的问题。

参考资料