目录

下载 GPT4All 客户端（macOS）

下载模型

聊天

基于目录构建本地文档集合

本地服务

  1. 启用 API 服务器

  2. 打开服务聊天窗口

  3. 查看本地下载的模型

    ll /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/*.gguf

    -rw-r--r--@ 1 junjian  staff    44M 12  3 10:30 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/all-MiniLM-L6-v2-f16.gguf
-rw-r--r--@ 1 junjian  staff   1.3G 12  3 12:53 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/incomplete-nous-hermes-llama2-13b.Q4_0.gguf
-rw-r--r--@ 1 junjian  staff   3.8G 12  3 10:09 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/mistral-7b-openorca.Q4_0.gguf
-rw-r--r--@ 1 junjian  staff   3.6G 12  3 11:10 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/orca-2-7b.Q4_0.gguf

# find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name \*.gguf
find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name '*.gguf'

/Users/junjian/Library/Application Support/nomic.ai/GPT4All//all-MiniLM-L6-v2-f16.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//incomplete-nous-hermes-llama2-13b.Q4_0.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//orca-2-7b.Q4_0.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//mistral-7b-openorca.Q4_0.gguf

显示模型名称列表

find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name '*.gguf' -size +1G -exec basename -s .gguf {} \; | sed 's/\..*//g' | xargs -n 1 printf "%s\n"

orca-2-7b
nous-hermes-llama2-13b
mistral-7b-openorca
  1. 使用 OpenAI API 调用本地服务 ```py import openai

openai.api_base = “http://localhost:4891/v1” openai.api_key = “EMPTY”

prompt = “你是谁？”

prompt = “每月补卡次数”

model = “mistral-7b-openorca”

response = openai.Completion.create( model=model, prompt=prompt, max_tokens=50, temperature=0.28, top_p=0.95, n=1, echo=True, stream=False )

print(response) print(response.choices[0].text)

{ “choices”: [ { “finish_reason”: “stop”, “index”: 0, “logprobs”: null, “references”: [], “text”: “\u4f60\u662f\u8c01\uff1f\n\u6211\u662f OpenAI \u7684\u8bed\u8a00\u6a21\u578b\uff0cGPT-3\u3002\u6211\u80fd\u591f\u7406\u89e3\u3001\u751f\u6210\u548c\u56de\u7b54\u5404\u79cd\u81ea\u7136\u8bed\u8a00\u95ee\u9898\u3002” } ], “created”: 1701576280, “id”: “foobarbaz”, “model”: “Mistral OpenOrca”, “object”: “text_completion”, “usage”: { “completion_tokens”: 38, “prompt_tokens”: 16, “total_tokens”: 54 } } 你是谁？ 我是 OpenAI 的语言模型，GPT-3。我能够理解、生成和回答各种自然语言问题。 ```

  1. 调用 API 服务后的聊天窗口

  2. 选择本地文档集合

  3. 运行第 4 步的代码后，基于本地文档集合的聊天窗口

参考资料