通过Shepherd Model Gateway路由代理LLM流量
Use Shepherd Model Gateway to route OpenAI, Anthropic, Responses API, embeddings, and MCP tool traffic across self-hosted and cloud model backends with cache-aware policies and observability.
Prerequisites
SMG binary, Docker image, Python package, or Rust install; one or more model worker endpoints; agent runtime configured to call the gateway; optional Prometheus/OpenTelemetry stack
Installation
Use the upstream install or setup path that matches your environment:
- docker pull lightseekorg/smg:latest
- pip install smg
- cargo install smg
Requirements and caveats from upstream:
- <a href="https://github.com/1991513ccie-png/skills"><img src="https://img.shields.io/badge/ghcr.io-lightseekorg%2Fsmg-blue?logo=docker" alt="Docker"></a>
- | High Availability | Mesh networking with SWIM protocol for multi-node deployments |
Basic usage or getting-started notes:
- Install — pick your preferred method:
- Run — point SMG at your inference workers:
- smg launch --worker-urls http://localhost:8000
- Source: https://github.com/1991513ccie-png/skills
- Extracted from upstream docs: https://raw.githubusercontent.com/lightseekorg/smg/HEAD/README.md
Documentation
- https://lightseekorg.github.io/smg
