Model serving
Run a deployed model behind an API that applications can call.
When to consider it
- When applications need online access to a hosted model
Considerations
- Endpoint support depends on model capability; capacity and access controls need design.
Connections
- In areaComposition & Runtime
- Supports functionServe model inference
- Used byvLLM
Sources
Last reviewed 2026-09-30