CogVLM2 on live video streams
CogVLM2 · Zhipu AI · VLM · Open-weight
CogVLM2 is Zhipu AI's vision-language model built on Llama 3 8B with a deep visual-expert architecture, the predecessor line to Zhipu's GLM-V series. CogVLM2 is not currently in Overshoot's live model catalog. When a model like this is available through the API, you publish a camera or screen share over WebRTC, reference frames with an ovs:// URL, and stream the answer back through an OpenAI-compatible chat-completions request.
CogVLM2 is not currently in the live Overshoot model catalog (verified against the live model catalog on 2026-07-14). Availability changes over time - query GET /v1beta/models for the current list.
- Developer
- Zhipu AI
- Parameters
- 19B, built on Llama 3 8B
- Context window
- 8K tokens
- License
- CogVLM2 license (free for most commercial use)
- Released
- May 2024
- Inputs
- Text, images, video frames
- Overshoot availability
- Not in live catalogas of 2026-07-14
What CogVLM2 is good at
CogVLM2 pairs a Llama 3 8B language backbone with a deep visual-expert module that injects image features at multiple layers rather than only at the input, which gives it strong grounding on GUI and document layouts. That heritage carries a video variant, CogVLM2-Video, built for clip-level understanding.
The model is a natural fit for tasks that need to read and locate elements on a screen or document: it can identify buttons, fields, and text blocks precisely enough to drive downstream automation, and it does so at a moderate 19B parameter count.
- GUI element grounding for screen-reading automation
- Document layout understanding and text localization
- Clip-level video description through the CogVLM2-Video variant
CogVLM2 and the Overshoot API
CogVLM2 is not currently in Overshoot's live model catalog; GET /v1beta/models is the authoritative list. If a grounding-focused model like this were available through the API, the workflow would be the standard one: create a Stream, publish a LiveKit video track, then send a chat-completions request whose image_url or video_url is an ovs:// reference. Overshoot retains 600 seconds of frame history, so a request can anchor the latest frame, an exact timestamp, or a recent segment, which suits GUI-automation loops that repeatedly check the current screen state.
On Overshoot's hosted path, open-weight models stream responses token by token over SSE with sub-second time to first token, and reusing thread_id across repeated queries against the same stream keeps hitting the prompt cache, which matters for tight automation loops.
How it fits ahead of Zhipu’s GLM-V line
CogVLM2 is the predecessor to Zhipu’s newer GLM-4.5V, which improves on it with a mixture-of-experts architecture, a longer 64K context window, and broader benchmark coverage. Teams that specifically need CogVLM2’s deep visual-expert grounding behavior, its established GUI-automation heritage, or its more permissive commercial license terms may still prefer it over moving to GLM-4.5V.
Frequently asked questions
Can CogVLM2 analyze live video?
CogVLM2 supports image and clip-level video understanding, but it is not currently in Overshoot's live model catalog. When a model like this is available through the API, your application references the latest frame or a recent segment of a live WebRTC stream with an ovs:// URL inside a standard chat-completions request; check GET /v1beta/models for what is live.
Is CogVLM2 open source?
CogVLM2 ships under its own CogVLM2 license, which is free for most commercial use with some conditions. Open availability like that is what allows a model to be served on a low-latency hosted path, though CogVLM2 itself is not currently in Overshoot's live catalog.
How fast is CogVLM2 on Overshoot?
CogVLM2 is not currently in Overshoot's live model catalog, so no Overshoot latency figure applies. Models on Overshoot's hosted path typically answer in about 200ms, and at 19B parameters a model of CogVLM2's size would keep both time to first token and total response time low. GET /v1beta/models lists what is currently live.
What is CogVLM2 best used for?
It suits tasks built around precise GUI and document grounding, such as automation that needs to locate buttons, fields, or text blocks on a live screen share, and clip-level video description through its CogVLM2-Video variant.