← 返回首页
GPT4All

GPT4All

下载 GPT4All 客户端（macOS）

下载模型

聊天

基于目录构建本地文档集合

本地服务

  1. 启用 API 服务器

  2. 打开服务聊天窗口

  3. 查看本地下载的模型

ll /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/*.gguf

-rw-r--r--@ 1 junjian  staff    44M 12  3 10:30 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/all-MiniLM-L6-v2-f16.gguf
-rw-r--r--@ 1 junjian  staff   1.3G 12  3 12:53 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/incomplete-nous-hermes-llama2-13b.Q4_0.gguf
-rw-r--r--@ 1 junjian  staff   3.8G 12  3 10:09 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/mistral-7b-openorca.Q4_0.gguf
-rw-r--r--@ 1 junjian  staff   3.6G 12  3 11:10 /Users/junjian/Library/Application Support/nomic.ai/GPT4All/orca-2-7b.Q4_0.gguf

# find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name \*.gguf
find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name '*.gguf'

/Users/junjian/Library/Application Support/nomic.ai/GPT4All//all-MiniLM-L6-v2-f16.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//incomplete-nous-hermes-llama2-13b.Q4_0.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//orca-2-7b.Q4_0.gguf
/Users/junjian/Library/Application Support/nomic.ai/GPT4All//mistral-7b-openorca.Q4_0.gguf

显示模型名称列表

find /Users/junjian/Library/Application\ Support/nomic.ai/GPT4All/ -name '*.gguf' -size +1G -exec basename -s .gguf {} \; | sed 's/\..*//g' | xargs -n 1 printf "%s\n"

orca-2-7b
nous-hermes-llama2-13b
mistral-7b-openorca
  1. 使用 OpenAI API 调用本地服务
import openai

openai.api_base = "http://localhost:4891/v1"
openai.api_key = "EMPTY"

# prompt = "你是谁？"
prompt = "每月补卡次数"

model = "mistral-7b-openorca"

response = openai.Completion.create(
    model=model,
    prompt=prompt,
    max_tokens=50,
    temperature=0.28,
    top_p=0.95,
    n=1,
    echo=True,
    stream=False
)

print(response)
print(response.choices[0].text)

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "references": [],
      "text": "\u4f60\u662f\u8c01\uff1f\n\u6211\u662f OpenAI \u7684\u8bed\u8a00\u6a21\u578b\uff0cGPT-3\u3002\u6211\u80fd\u591f\u7406\u89e3\u3001\u751f\u6210\u548c\u56de\u7b54\u5404\u79cd\u81ea\u7136\u8bed\u8a00\u95ee\u9898\u3002"
    }
  ],
  "created": 1701576280,
  "id": "foobarbaz",
  "model": "Mistral OpenOrca",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 38,
    "prompt_tokens": 16,
    "total_tokens": 54
  }
}
你是谁？
我是 OpenAI 的语言模型，GPT-3。我能够理解、生成和回答各种自然语言问题。

  1. 调用 API 服务后的聊天窗口

  2. 选择本地文档集合

  3. 运行第 4 步的代码后，基于本地文档集合的聊天窗口

参考资料