---
layout: single
title:  "GPT4All"
date:   2023-12-03 08:00:00 +0800
categories: [AI 与大模型, 操作系统]
tags: [GPT4All, ChatGPT, OpenAI]
---

## 下载 [GPT4All 客户端](https://gpt4all.io/index.html)（macOS）

## 下载模型
![](/images/2023/GPT4All/Download-Model.png)

## 聊天
![](/images/2023/GPT4All/Chat.png)

## 基于目录构建本地文档集合
![](/images/2023/GPT4All/Local-Document-Collections.png)

## 本地服务
1. 启用 API 服务器
![](/images/2023/GPT4All/Enable-API-Server.png)

2. 打开服务聊天窗口
![](/images/2023/GPT4All/Server-Chat.png)

3. 查看本地下载的模型
```shell
ll /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/*.gguf
```
```
-rw-r--r--@ 1 junjian  staff    44M 12  3 10:30 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/all-MiniLM-L6-v2-f16.gguf
-rw-r--r--@ 1 junjian  staff   1.3G 12  3 12:53 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/incomplete-nous-hermes-llama2-13b.Q4_0.gguf
-rw-r--r--@ 1 junjian  staff   3.8G 12  3 10:09 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/mistral-7b-openorca.Q4_0.gguf
-rw-r--r--@ 1 junjian  staff   3.6G 12  3 11:10 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/orca-2-7b.Q4_0.gguf
```

```shell
# find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name \*.gguf
find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name '*.gguf'
```
```
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//all-MiniLM-L6-v2-f16.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//incomplete-nous-hermes-llama2-13b.Q4_0.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//orca-2-7b.Q4_0.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//mistral-7b-openorca.Q4_0.gguf
```

显示模型名称列表

```shell
find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name '*.gguf' -size +1G -exec basename -s .gguf {} \; | sed 's/\..*//g' | xargs -n 1 printf "%s\n"
```
```
orca-2-7b
nous-hermes-llama2-13b
mistral-7b-openorca
```

4. 使用 OpenAI API 调用本地服务
```py
import openai

openai.api_base = "http://localhost:4891/v1"
openai.api_key = "EMPTY"

# prompt = "你是谁？"
prompt = "每月补卡次数"

model = "mistral-7b-openorca"

response = openai.Completion.create(
    model=model,
    prompt=prompt,
    max_tokens=50,
    temperature=0.28,
    top_p=0.95,
    n=1,
    echo=True,
    stream=False
)

print(response)
print(response.choices[0].text)
```
```
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "references": [],
      "text": "\u4f60\u662f\u8c01\uff1f\n\u6211\u662f OpenAI \u7684\u8bed\u8a00\u6a21\u578b\uff0cGPT-3\u3002\u6211\u80fd\u591f\u7406\u89e3\u3001\u751f\u6210\u548c\u56de\u7b54\u5404\u79cd\u81ea\u7136\u8bed\u8a00\u95ee\u9898\u3002"
    }
  ],
  "created": 1701576280,
  "id": "foobarbaz",
  "model": "Mistral OpenOrca",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 38,
    "prompt_tokens": 16,
    "total_tokens": 54
  }
}
你是谁？
我是 OpenAI 的语言模型，GPT-3。我能够理解、生成和回答各种自然语言问题。
```

5. 调用 API 服务后的聊天窗口
![](/images/2023/GPT4All/Server-Chat1.png)

6. 选择本地文档集合
![](/images/2023/GPT4All/Local-Documents.png)

7. 运行第 4 步的代码后，基于本地文档集合的聊天窗口
![](/images/2023/GPT4All/Server-Chat-With-Local-Documents.png)

## 参考资料
* [gpt4all](https://github.com/nomic-ai/gpt4all)
* [GPT4All Chat UI](https://docs.gpt4all.io/gpt4all_chat.html)
* [LangChain GPT4All](https://python.langchain.com/docs/integrations/llms/gpt4all.html)