Local model serving
Run a model on a user-controlled machine and expose a local inference endpoint.
When to consider it
- When an application needs model responses from a local runtime
Considerations
- Available models, performance, and memory needs depend on the machine and exact artifact.
Connections
- In areaComposition & Runtime
- Supports functionServe model inference
- Used byOllama local runtime
Sources
Last reviewed 2026-09-30