Enhancing Video Generation with Explicit 3D Guidance for Accurate Motion Simulation

Add to your list(s) Download to your calendar using vCal

• fy277@cam.ac.uk
• Thursday 29 February 2024, 14:00-15:00
• SS03 - William Gates Building.

If you have a question about this talk, please contact Rafal Mantiuk.

Video generation is currently a highly popular task. However, common video generation models rely solely on large-scale data training to implicitly simulate motion, which can sometimes result in inaccuracies. To address this problem, we propose utilizing explicit 3D guidance to guide the video generation process. Given a single image and the user’s motion demands, we model the object and motion in 3D, allowing for rendering from various viewpoints. Subsequently, we inject the motion prior from these rendered views into the video generation model in producing high-quality videos with the desired motion.

Zoom link: https://cam-ac-uk.zoom.us/j/89792055791?pwd=OTYyQXdnUXZMcnU4VmZWL2p4dUhZdz09

This talk is part of the Rainbow Group Seminars series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• Interested Talks
• Rainbow Group Seminars
• SS03 - William Gates Building
• School of Technology
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.