Degas: Detailed Expressions on Full-Body Gaussian Avatars

DEGAS (Detailed Expressions on Full-Body Gaussian Avatars) is a novel modeling method that utilizes 3D Gaussian Splatting (3DGS) to create full-body avatars capable of exhibiting rich facial expressions. Despite advancements in neural rendering for lifelike avatars, the integration of detailed expressions into full-body models has been underexplored. DEGAS addresses this gap by employing a conditional variational autoencoder trained on multiview videos, which captures both body motion and facial expressions. Unlike traditional methods that rely on 3D Morphable Models (3DMMs), DEGAS uses an expression latent space derived from 2D portrait images, effectively linking 2D talking faces with 3D avatars. The resulting avatars can be reenacted to produce photorealistic images with nuanced facial expressions. The method's effectiveness is demonstrated through experiments on existing datasets and a newly introduced DREAMS Avatar Dataset, which includes multi-view captures of six subjects performing various expressions and motions. Additionally, an audio-driven extension of the method is proposed, leveraging 2D talking faces to enhance interactivity in AI agents.

- DEGAS is the first method to model full-body avatars with detailed facial expressions using 3D Gaussian Splatting.

- The approach integrates body motion and facial expressions through a conditional variational autoencoder.

- It utilizes an expression latent space based on 2D images, bridging 2D and 3D avatar technologies.

- The DREAMS Avatar Dataset features multi-view captures of subjects performing standard and freestyle motions.

- An audio-driven extension of DEGAS opens new avenues for interactive AI applications.