The niche world of deploying LLMs for inference can be confusing. Countless developer hours go into cross-referencing Hugging Face docs, technical reports, and vLLM or SGLang docs, all in pursuit of the sweet spot of package versions and flags that gets you to a running inference server.
At Overshoot, we know all too well that the distance between Hugging Face model weights and a deployed inference server can be minutes, but it can just as easily be days. For multimodal LLMs, the surface for surprises is even larger.
Following our survey of all relevant open source vision language models, we want to give the developer community reproducible guides to deploy every vision language model, with snippets that are tested and reproducible. This collection of guides is something like a Bestiary for taming wild models into deployed inference endpoints. Since Bestiary is barely pronounceable, we call it the Inference Field Guide.
We will be updating both our survey and our Field Guide as we onboard and test more models. We will expand model coverage to include image models and we will enrich our guides with benchmarking results and evaluations. We welcome your requests and suggestions, and we hope this proves a valuable resource for the vision AI community.
Inference Field Guide
Below are the deployment guides currently available. Each guide includes complete setup instructions, deployment commands, troubleshooting tips, and performance benchmarks.
Qwen3.5 family
Qwen3-VL family
- Qwen3-VL-2B-Instruct
- Qwen3-VL-4B-Instruct
- Qwen3-VL-8B-Instruct
- Qwen3-VL-30B-A3B-Instruct
- Qwen3-VL-32B-Instruct
Qwen2.5-VL family
- Qwen2.5-VL-3B-Instruct
- Qwen2.5-VL-7B-Instruct
- Qwen2.5-VL-72B-Instruct
InternVL3.5 family
- InternVL3.5-1B
- InternVL3.5-4B
- InternVL3.5-8B
- InternVL3.5-14B
InternVL3 family
- InternVL3-1B
- InternVL3-2B
- InternVL3-8B
Molmo2 family
GLM family
- GLM-4.6V-Flash
Kimi-VL family
- Kimi-VL-A3B-Instruct
Keye-VL family
- Keye-VL-8B-Preview
- Keye-VL-1.5-8B
Tarsier2 family
- Tarsier2-Recap-7B