Serve model inference
Expose a model through a request endpoint for application use.
Desired outcome
Responses from a deployed model through a stable service interface.
Explore from the outcome
Where this function can lead
These documented connections are starting points for selection, not a ranked list.
Ways to do it
Methods that can help achieve this outcome.
- Local model serving
Run a model on a user-controlled machine and expose a local inference endpoint.
Link evidence: Ollama API introduction
- Model serving
Run a deployed model behind an API that applications can call.
Link evidence: vLLM online serving
Tools and services
Software or services that can put a method to work.
- Ollama local runtime
Locally installed Ollama runtime for downloading and serving supported models.
Link evidence: Ollama API introduction
- vLLM
Inference engine with an HTTP server for supported model capabilities.
Link evidence: vLLM online serving
Typical inputs
- Model artifact, serving configuration, and requests
Typical outputs
- Model responses or embeddings, according to model capability
Sources
Last reviewed 2026-09-30