New AI Model Syncs Speech and Gestures for More Human-Like Digital Avatars
Researchers have developed Gelina, an AI system that generates speech and hand gestures together rather than separately, producing more natural and synchronized video avatars. The breakthrough matters for companies building customer service bots, virtual presenters, and video translation tools—applications that currently suffer from awkward, misaligned gestures that undermine user trust.
Originaltitel: Gelina: Unified Speech and Gesture Synthesis Via Interleaved Token Prediction
Human communication is multimodal, with speech and gestures tightly coupled, yet most computational methods for generating speech and gestures synthesize them sequentially, weakening synchrony and prosody alignment. We introduce Gelina, a unified framework that jointly synthesizes speech and co-speech gestures from text using interleaved token sequences in a discrete autoregressive backbone, with modality-specific decoders. Gelina supports multi-speaker and multi-style cloning and enables gesture-only synthesis from speech inputs. Subjective and objective evaluations demonstrate competitive speech quality and improved gesture generation over unimodal baselines.