Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing

Sen Liang, Cong Wang, Zhentao Yu, Fengbin Guan, Zhengguang Zhou, Teng Hu, Youliang Zhang, Yuan Zhou, Xin Li, Qinglin Lu, Zhibo Chen (* equal contribution)

Project

• We introduce Goku, a large-scale dataset featuring 2 million high-quality video editing pairs that extends task boundaries to multi-task and structural manipulations, along with Goku-Edit (an MLLM-based dual-branch model) and Goku-Bench for comprehensive evaluation.

HarmoView: Harmonizing Multi-View Constraints for Identity-Consistent Video Generation

Cong Wang, Zhentao Yu, Hongmei Wang, Weicong Liang, Zixiang Zhou, Zilin Yang, Jiarong Ou, Rui Chen, Yezhou Liu, Shiyu An, Yue Lu, Yuan Zhou, Qinglin Lu

Project

• We propose HarmoView, a robust framework for identity-consistent video generation that effectively integrates multi-view cues via three architectural refinements—Multi-level Feature Injection, learnable proxy tokens, and Jump-RoPE—together with a Progressive View Curriculum to enable stable multi-view training on top of a pre-trained T2V model.

SpongeBob: Sync-Aware Harmonious Audio-Visual Generative Editing

Sen Liang, Cong Wang, Fengbin Guan, Zhentao Yu, Yiting Lu, Yuanzhi Wang, Yuan Zhou, Xin Li, Zhibo Chen (* equal contribution)

Project

• SpongeBob is the first end-to-end audio-visual joint editing framework built on a dual-stream DiT, featuring a Sync-Aware Editing Mechanism for temporal/spatial alignment and a Context-Aware Module for consistency with preserved content, delivering harmonious video-audio co-editing.

SVG-Head: Hybrid Surface-Volumetric Gaussians for High-Fidelity Head Reconstruction and Real-Time Editing

He-Yi Sun, Cong Wang, Tian-Xing Xu, Jingwei Huang, Di Kang, Chunchao Guo, Song-Hai Zhang

Project

• By proposing surf-GS for modeling global appearance as a texture image and vol-GS for high-fidelity rendering of non-Lambertian regions, SVG-Head supports not only high-quality renderings, but also real-time, fine-grained texture editing.

Fine-Grained 3D Gaussian Head Avatars Modeling from Static Captures via Joint Reconstruction and Registration

Yuan Sun, Xuan Wang, Cong Wang, WeiLi Zhang, Yanbo Fan, Yu Guo, Fei Wang

Paper

• We formulate 3D Gaussian head avatar modeling from static captures as a joint reconstruction and registration problem, optimizing a prior-based and a prior-free set of Gaussians and binding them via non-rigid ICP, producing fine-grained, animatable avatars at high resolutions.

MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing

Cong Wang, Di Kang, He-Yi Sun, Shen-Han Qian, Zi-Xuan Wang, Linchao Bao, Song-Hai Zhang

Project

• By proposing the use of more appropriate representations for different parts of the human head and the corresponding hybrid rendering methods, we produce high-quality 3D human head avatar, and for the first time, support human head editing (i.e., texture editing and hair alteration).

Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatar

Cong Wang, Di Kang, Yan-Pei Cao, Linchao Bao, Ying Shan, Song-Hai Zhang

Project

• We propose a new surface-guided neural point representation and the corresponding rendering acceleration method to greatly improve the rendering quality of eyes and mouth interiors in 3D digital human head modeling.

LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inference

Cong Wang, Yu-Ping Wang, Dinesh Manocha

Project

• We propose to use an NN Sampler for more certain and reasonable MPI plane locations and introduce a reprojection loss to facilitate sampler learning. Additionally, we propose the block self-attention mechanism to enhance the network’s inference ability for the occluded regions.

MotionHint: Self-Supervised Monocular Visual Odometry with Motion Constraints

Cong Wang, Yu-Ping Wang, Dinesh Manocha

Project

• We introduce the motion prior of camera carriers (mainly unmanned vehicles) via a pre-trained PPNet, so that the predicted camera poses can be corrected according to the motion prior, yielding the state of art results.