How Far Are Video Models from True Multimodal Reasoning?

Apr 1, 2026·
X. Zhang
,
J. Wei
,
Y. Wang
,
J. Tan
,
Y. Li
,
Y. Zhang
,
Z. Chen
,
D. Zhang
Dezhi Yu
Dezhi Yu
,
W. Xu
,
S. Jiang
,
Z. Liu
· 0 min read
Abstract
A study of the gap between current video models and true multimodal reasoning, evaluating whether models integrate visual, temporal, and language signals beyond superficial correlations.
Type
Publication
Preprint; ECCV 2026 submission context
publication
Dezhi Yu
Authors
Senior ML Engineer

I am a research-oriented machine learning systems engineer working on foundation model infrastructure, alignment, and evaluation. My work focuses on building efficient, reliable systems for large language models while studying the algorithms and data choices that make these models more useful, controllable, and cost-effective in real applications.

At TikTok, my recent work centers on Model-as-a-Service platforms and high-performance LLM inference. I develop serving infrastructure with vLLM and SGLang across model runtime integration, scheduling and continuous batching, KV-cache and memory management, distributed execution, observability, and reliability. This systems work is closely connected to my research on distributed disaggregated inference, preference optimization, instruction-tuning data selection, multimodal evaluation, and retrieval-augmented biomedical summarization.

My broader research spans reinforcement learning for robotics, healthcare sequence modeling, privacy-preserving machine learning, and motion planning. I am especially interested in model-system co-design: how model architecture, inference algorithms, data curation, hardware utilization, scheduling, and distributed runtimes interact. My goal is to advance frontier AI systems that are faster to experiment with, more rigorous to evaluate, and dependable enough to serve at scale.