TOWARD INTERACTIVE STREAMING

ALIVE: Making Streaming Video Models Interactive

Jinzhao Li1,3,*Yuan Xu2,3,*Jinxuan Wu4Yukun Wang1Yijia Lei1Hanyu Ji2Yinuo Chen1Zining Huang1Chi Zhang1Hang Ye2Yiwu Zhong2Honglei Yan3Panwang Pan3Yizhou Wang2Yin Li5Miao Liu1,†

* Equal contribution † Corresponding author

1 Tsinghua University
2 Peking University
3 ByteDance
4 Beihang University
5 University of Wisconsin–Madison

THE NEXT STEP

Toward interactive streaming.

Streaming models can answer questions and monitor events. ALIVE goes further: coordinating changing requests and seeking the feedback needed to proceed.

01Proactive MonitoringRespond when the moment is right.
02Streaming Task CoordinationKeep up with evolving requests.
03Feedback-Seeking InteractionSeek the evidence needed to answer.
Three capabilities for interaction along a continuous stream.
Read the abstract

Streaming video LLMs can continuously observe the world, yet often handle user requests in isolation. We introduce ALIVE, an interactive streaming video model supporting proactive monitoring, streaming task coordination, and feedback-seeking interaction. We construct ALIVE-SFT-288K and ALIVE-RL-3K for two-stage training. Contrastive supervised fine-tuning teaches when and how to respond, while reinforcement learning with Behavior-Aligned Credit Assignment combines trajectory-level and local advantages to refine decisions, responses, and silence across multi-turn interactions. We also introduce FSI-Bench to evaluate feedback seeking and subsequent task completion. ALIVE achieves the highest overall scores among the compared models on four public benchmarks and FSI-Bench, demonstrating strong streaming understanding and interactive capabilities.

LEARNING TO INTERACT

Decide. Respond. Know when to wait.

Two stages teach a single model to choose an appropriate strategy, answer accurately, and remain silent when a response is unwarranted.

STAGE 01

Contrastive SFT

Similar visual observations can call for different behaviors. Controlled contrasts teach the role of available evidence, active requests, and user feedback.

Respond R↔Remain silent S
Action-strategy contrastsResponse-gate contrasts
STAGE 02

RL with Behavior-Aligned Credit Assignment

Trajectory-level and local advantages refine behavior across multi-turn interactions, aligning credit with the requirements of each interaction.

Trajectory-level+Behavior-aligned local
Strategy selectionTask responseCritical silence
Contrastive supervision establishes interaction behaviors; reinforcement learning refines them on model-generated trajectories.

DATA & EVALUATION

Built around sustained interaction.

ALIVE-SFT-288K288K

Contrastive supervision

288,436 interaction examples for supervised fine-tuning.

ALIVE-RL-3K3K

Behavior refinement

3,523 interaction examples for reinforcement learning.

FSI-BENCH1,235

Feedback-seeking episodes

Four tasks test strategy selection and subsequent task completion.

Data construction and distributions across monitoring, task coordination, and feedback seeking.

FIVE BENCHMARKS

Strong understanding. Strong interaction.

The highest overall scores among the compared models on four public benchmarks and FSI-Bench.

ALIVEJoyAI-VL-InteractionAURAVideoChat3
General streaming video understandingStreaming interaction
View scores and evaluation metrics
Overall scores reported in the ALIVE paper. Higher is better.
ModelOVO-BenchStreamingBenchProactiveVideoQAIPIBenchFSI-Bench
ALIVE68.976.051.640.352.8
JoyAI-VL-Interaction60.667.428.820.217.4
AURA65.373.132.021.68.3
VideoChat358.868.737.612.05.9

OVO-Bench, StreamingBench and IPIBench: overall accuracy (%). ProactiveVideoQA: PAUC at ω = 0.5. FSI-Bench: four-task macro accuracy (%) under the Explicit Decision Protocol (EDP). Bars share a zero baseline and a 0–80 scale. These are the paper's reported comparisons, not a live leaderboard.

ALIVE

Making streaming video models interactive.

Continuously perceive. Sustain meaningful interaction.

ALIVE overview

Open PDF ↗
ALIVE overview

Scroll to inspect the figure. Press Esc to close.