Computer Vision and Vision-Language Models

At the core of ORN's intelligence stack lies the integration of Computer Vision (CV) Models and Vision-Language Models (VLMs). These two layers work in tandem; CV models operate at the pixel level, extracting precise spatial and temporal details, while VLMs interpret those outputs into structured, human-readable representations that capture meaning, intent, and compliance. Together, they enable ORN to move beyond simple detection and labeling, achieving true semantic understanding and performance evaluation of user-submitted videos.

Computer Vision Models

Computer Vision Models form the perception backbone of ORN. They specialize in identifying and localizing objects, tracking hand and body poses, and segmenting actions across frames. CV models confirm the egocentric perspective of a recording by analyzing motion parallax, head stabilization patterns, and hand visibility, ensuring that the submission reflects authentic, first-person capture.

Within each task, CV models detect and follow the key entities involved such as dishes, utensils, tools, or sports equipment and track how users interact with them over time. They also perform temporal segmentation, dividing each video into start, action, and completion phases to confirm that the task was executed fully and continuously.

This pixel-level perception generates the raw data for higher-order reasoning. By providing frame-accurate detections, bounding boxes, and motion trajectories, CV models create a structured visual layer that defines what physically happened in each video - the essential groundwork for contextual interpretation.

Vision-Language Models

Building upon this foundation, Vision-Language Models (VLMs) serve as ORN's interpretive layer. VLMs align visual embeddings from CV outputs with natural language, enabling the system to describe and reason about actions in human-like terms.

Rather than simply detecting that a knife and apple are present, VLMs can articulate:

“The user picks up a knife, slices the apple into pieces, and places them into a bowl.”

This translation from visual to linguistic space allows ORN to assess sequence, intent, and outcome - determining not only what happened but also how well it was performed.

VLMs enable ORN to perform Skill Scoring with a semantic dimension. By generating narratives of user actions and comparing them against expert demonstration datasets, ORN can evaluate whether each step was performed correctly, in the right order, and with expected precision. For example, in a sushi-making task, CV models capture rolling motion and object use, while VLMs evaluate if the process aligns with expert-level execution: Was the roll tight? Were steps sequentially correct? Was the result consistent with professional standards?

Synergy and Application

The synergy between CV and VLMs transforms ORN from a labeling tool into an understanding engine.

• CV models classify and measure physical actions, forming the “eyes and structure.”

• VLMs interpret intent, logic, and compliance, serving as the “language and reasoning.”

Together, they produce data that is both quantitatively precise and semantically rich — yielding robotics training datasets that combine low-level action labels (e.g., movement, objects, and timings) with high-level contextual understanding (e.g., task success, technique quality, and user intent).

Broader Impact

By fusing CV and VLMs, ORN achieves a dual goal: scalable automation and interpretive intelligence. Human reviewers no longer need to manually watch full clips; they can instead validate AI-generated labels and summaries, significantly reducing review overhead.

For downstream consumers such as robotics and embodied AI companies, this combination provides robotics policy training-ready datasets that pair detailed visual annotations with linguistic narratives. These outputs deliver both perceptual fidelity (how things look and move) and behavioral context (why they occur), offering a richer training signal for robotic learning.

Through the integration of Computer Vision and Vision-Language Models, ORN transforms raw human recordings into structured knowledge - teaching machines to not only see and describe human activity, but to understand and learn from it.

All thresholds, parameters, and detection methods described are subject to continuous refinement as technology advances and as the requirements of the ecosystem evolve.