Industrial Computer Vision: A Technical Guide for Engineers and Operations Teams

Industrial computer vision uses cameras and image models to inspect equipment, materials, products, and work processes in operating environments. A production system includes much more than a model: optics, lighting, transport, sampling, prompts, decision policy, observability, and a safe response path all determine whether it works. Overshoot supplies cloud vision-language model inference over live and recorded visual context. It returns answers in 200ms, while the application remains responsible for triggering requests and using their results.

What industrial computer vision systems do

Traditional industrial vision often answers a narrow question such as whether a part is present, whether a dimension falls inside tolerance, or how many objects crossed a line. These tasks suit calibrated rules, classical image processing, or a detector trained for fixed classes. A vision-language model, or VLM, handles instructions expressed in language and can describe relationships, visible state, and short sequences. It is useful when the question changes, the visual categories have a long tail, or an operator needs an explanation with the decision.

The distinction matters. A VLM response is probabilistic and should not become a safety interlock. Keep emergency stops, machine guarding, deterministic measurements, and hard real-time controls in certified local systems. A VLM can support operator review, summarize an exception, classify visible conditions, or provide context after another sensor raises an event. Choose the narrowest tool that satisfies the requirement.

Core architecture and data flow

Start at the image source. A camera and lens produce frames under controlled or variable lighting. A publisher encodes those frames and sends a video track over WebRTC through LiveKit. The publisher can run in a browser, native application, or server process. Overshoot creates a Stream through the /v1beta API and returns the LiveKit room information and publish token. The application publishes to that room, then addresses visual context with an ovs:// reference in an OpenAI-compatible chat-completion request.

The public API does not accept RTSP, RTMP, or ONVIF as direct ingest protocols. An external gateway may connect to an RTSP camera, decode or transcode the source, and publish a WebRTC video track. That gateway is part of the customer architecture. It owns camera credentials, reconnect behavior, codec compatibility, buffering, and any site-specific network rules.

1. Create a Stream in the closest available region, us-west1 or us-central1.
2. Publish the browser, native, or server video track to LiveKit.
3. Renew the 300-second Stream lease with keepalive while the source remains active.
4. Trigger a query from an operator action, timer, or external application event.
5. Reference the latest frame or a segment inside the 600-second retained history.
6. Parse the SSE response, validate its shape, and apply the site decision policy.
7. Delete the Stream when the publisher is finished.

Connect a live camera feed

A browser publisher can obtain a camera track with getUserMedia. Native and server publishers use the corresponding LiveKit SDK and a suitable video source. For an existing industrial camera network, place the gateway close enough to the source to avoid unnecessary decode, encode, and network delay. Measure the full capture-to-first-token path because model latency alone excludes camera exposure, encoding, uplink, and application processing.

Create a Stream through the v1beta API

curl -sS https://api.overshoot.ai/v1beta/streams \
  -X POST \
  -H "Authorization: Bearer $OVERSHOOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}'

The response contains the Stream identity and LiveKit publishing credentials. Treat the publish token as a credential. Keepalive returns a fresh token, which the publisher supplies to room.updateToken(...). A Stream moves forward to an ended state and cannot resume after it ends. Create a new Stream after lease expiry or explicit deletion.

Select frames and recent video precisely

The ovs:// scheme is a media reference parsed by Overshoot. It is not a fetchable web URL. For an image query, select frame_index=-1 for the latest frame, a specific frame index, a timestamp, or a negative offset from live. For video, define start and end anchors and optionally set max_fps. The default segment sampling cap is 1.0 FPS. Higher sampling rates increase the number of frames and usually increase visual tokens, preprocessing work, and inference time.

Use the shortest interval that contains the evidence. A jam investigation may need the ten seconds before a stoppage. A label check may need one sharp frame. The retained Stream history lasts 600 seconds, so archive required evidence in the application storage system before that window expires. Overshoot does not replace a plant historian or video management system.

Reference a recent Stream segment in a chat completion

{
  "model": "google/gemma-4-26B-A4B-it",
  "stream": true,
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "Describe the visible state change. If evidence is unclear, say unclear."},
      {"type": "video_url", "video_url": {
        "url": "ovs://streams/STREAM_ID?start_offset_ms=-8000&end_offset_ms=0&max_fps=1.0"
      }}
    ]
  }]
}

Design prompts and outputs for operations

An operational prompt should define the observation, allowed labels, uncertainty behavior, and output format. Avoid asking for hidden causes when the pixels only show effects. For example, request the visible belt state and obstructions instead of asking why a motor stopped. Include an explicit unclear state for occlusion, glare, or missing views. If application code consumes the answer, request a small JSON object and validate every field before use.

Keep the prompt and model id under version control. A wording change creates a new system behavior and needs evaluation. Use a stable thread_id when prompt caching fits the interaction, and log cache observability returned in the response. Do not assume a model update preserves task behavior. Pin the chosen model id and repeat the evaluation before changing it.

Evaluate accuracy without inventing claims

Overshoot does not publish an accuracy percentage for a customer-specific industrial task. Accuracy depends on the model, prompt, camera position, compression, lighting, product mix, and definition of a correct answer. Build a labeled evaluation set from the real operating envelope. Include normal production, rare failures, maintenance states, startup and shutdown, shift lighting, motion blur, occlusion, dirty lenses, and camera interruptions.

Score false positives and false negatives separately because their operational costs differ. Also score abstentions, malformed outputs, and time to operator resolution. Split evaluation clips by event or production run so near-duplicate frames do not leak across test sets. Review disagreement cases with operations staff who understand the process. The goal is a measured decision boundary for one workflow, not a universal model score.

• Freeze the model id, prompt version, sampling rule, and camera configuration for each baseline.
• Record the Stream anchor and source event so a reviewer can recover the exact evidence.
• Report confidence through observed error rates and abstentions, not model prose.
• Set separate acceptance thresholds for advisory output and automated application actions.
• Re-run the set after camera, codec, prompt, model, or line changes.

Plan latency, reliability, and network behavior

Overshoot answers in 200ms, but an industrial latency budget must include the complete path. Track camera exposure, frame availability, encoding, gateway buffering, uplink, Stream ingestion, request scheduling, time to first token, response completion, validation, and user interface or actuator delay. SSE exposes the response incrementally and makes time to first token observable. It does not remove upstream capture or network costs.

Test from the actual facility to both available regions. The closest geographic region may still take a poor network route. Exercise packet loss, publisher restarts, expired leases, keepalive failures, and API errors. Define whether the workflow queues, degrades to manual review, or disables itself during an outage. Overshoot is cloud inference and has no edge or offline runtime, so disconnected operation requires a separate local design.

Integrate safely with plant systems

The public product does not provide PLC, SCADA, MES, or webhook integrations. Your application can call the API and then communicate with approved plant systems through an integration layer that enforces identity, schema, rate, and policy. Keep network zones explicit. Store API credentials on a server or secured gateway and rotate them through the normal secret-management process.

Separate observation from control. A useful pattern writes the VLM result to an application queue, checks schema and policy, and presents uncertain cases to an operator. If a downstream system accepts an automatic action, constrain the action set and make it reversible where possible. Log request time, model id, prompt version, media anchor, parsed output, policy result, and final action. This evidence supports incident review without pretending the model supplied a deterministic measurement.

A practical pilot plan

1. Choose one visible, frequent, non-safety workflow that already receives human review.
2. Define the decision, evidence window, error costs, latency budget, and fallback before implementation.
3. Collect representative clips and label them with operations staff.
4. Build the WebRTC publisher and query path, including keepalive and cleanup.
5. Run offline evaluation first, then shadow live operation without changing the process.
6. Compare errors and review time against the current workflow.
7. Add constrained actions only after the shadow results meet written acceptance criteria.

A pilot should end with a decision: deploy this bounded workflow, revise it, or stop. Avoid expanding scope to rescue a weak result. Camera improvements, a narrower label set, or a different trigger often matter more than a longer prompt. Document the unsupported conditions so operators know when to disregard the output.

Summary

• Use VLMs for flexible visual questions and explanations; retain deterministic systems for safety and fixed metrology.
• Publish live video over WebRTC through LiveKit; bridge RTSP cameras with an external gateway.
• Budget the full path around the 200ms answer and test both available regions from the facility.
• Evaluate the exact prompt, model, camera, and sampling policy on representative footage.
• Keep integration, control policy, evidence storage, and outage behavior in the application architecture.

Build the first bounded inspection path: follow the Overshoot quickstart, then use the industrial computer vision use case to frame a pilot with operations and safety owners.