1 篇文章带有标签 “proxy-server”

LiteLLM Proxy Server (LLM Gateway)

• 💥 LiteLLM Proxy Server (LLM Gateway)
• Hosted LiteLLM Proxy
• Proxy Config.yaml

安装

pip install 'litellm[proxy]'

编辑配置文件：config.yaml

model_list:
  - model_name: qwen-coder
    litellm_params:
      model: ollama/qwen2.5-coder:7b
  - model_name: bge-m3
    litellm_params:
      model: ollama/bge-m3
  - model_name: llava
    litellm_params:
      model: ollama/llava:7b
      api_base: "http://localhost:11434"
      # api_base: http://127.0.0.1:11434/v1 # ❌ 500 Internal Server Error
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4-32k
// ...

命令部署 # 集成 Langfuse LANGFUSE_PUBLIC_KEY=pk-lf-fd5d8fb