Pacific Graphics 2026

Flexible Motion Generation from Language and Style References

Kai Weixian Lan1,4, Bodie Criswell2,4, Briana Fedkiw3,4, Zhan Zhang1,4, Joseph Teran1,4, Daniel Holden4

1University of California, Davis • 2University of Utah • 3Brown University • 4Epic Games

arXiv Video Code — coming soon

FlexMoGen generates long, high-quality human motion that follows a time-varying text prompt while reflecting the style of one or more reference clips — learned without any style labels.

FlexMoGen teaser: three content prompts and three style references produce one coherent stylized motion
FlexMoGen generates long, stylized human motions conditioned on time-varying text prompts and style reference clips. Given three sequential content prompts (“Jump with both feet”, “Side hop”, “Kick legs”) and corresponding style references (bottom row), our framework synthesizes a single temporally coherent motion sequence (top row) that faithfully reflects both the intended action and the target style within each segment. Color coding indicates the correspondence between style references, content prompts, and generated frames.

Abstract

We introduce FlexMoGen, a novel framework for flexible human motion synthesis conditioned on both natural language descriptions and motion style references. Text prompts are effective at defining semantic content, but they are often limited in capturing fine-grained style details such as timing, limb articulation, and expressive dynamics. A style example clip supplements the text by conveying these nuanced motion characteristics directly, enabling the model to preserve high-level intent while reproducing the desired stylistic traits. Given a text prompt and a style example clip, FlexMoGen generates high-quality motions that preserve semantic content while faithfully reflecting the target style, offering users greater control over the animation generation process. Unlike prior methods that rely on discrete style labels and do not generalize to long or multi-style generation, FlexMoGen learns a variational style encoder without style supervision and supports long, time-varying, multi-style synthesis. Our framework jointly pre-trains the style encoder and a text-to-motion latent diffusion model within a unified architecture, modulating motion style through a lightweight adaptation module. It integrates an efficient relative positional encoding scheme and is trained on both stylized and non-stylized datasets, enabling strong generalization to unseen text–style combinations. Experiments show that FlexMoGen achieves the best balance between content fidelity and style reflection.

Video

Narrated overview with all results animated.

How It Works

FlexMoGen framework: a style adaptation module injects style into a frozen text-to-motion diffusion backbone
Overview of our style-adapted text-to-motion framework. The input content motion xc is encoded by a pretrained motion encoder, and Gaussian noise is added to its latent embedding. The text-to-motion (T2M) model (top right) consists of 7 Transformer encoder layers that take motion, text, and time embeddings as inputs. Each self-attention block incorporates relative positional encoding into the key and value projections. During style finetuning, the T2M backbone is frozen, and only the weight matrices WsK and WsV in the Style Adaptation Module (SAM) are trained. SAM encodes a style example xs and injects its outputs as additive biases to the K and V vectors in the attention layers, guiding the network to preserve the semantics of xc while reflecting the style of xs.
1
Style Adaptation Module (SAM)
Rather than duplicating the backbone as in ControlNet-style approaches, or coupling style with the text tokens as in LoRA-based ones, SAM adds two lightweight learnable matrices that project the style embedding into key and value biases inside each attention layer. The T2M backbone stays frozen, so style conditioning costs very little and drops into a pretrained architecture unchanged.
2
Label-free variational style encoder
The style encoder is pretrained with a motion reconstruction objective and no style supervision, so style comes from example clips rather than a fixed label vocabulary. It embeds five body parts — legs, spine, and left and right arms — independently, which makes the style code a concatenation of per-part components rather than one monolithic vector.
Style encoder producing per-body-part style embeddings
Overview of style encoder pretraining. The encoder processes five body parts (legs, spine, and left/right arms) to produce per-part style embeddings. During training, style embeddings are sampled from the latent distribution and concatenated with the root trajectory for motion reconstruction through a decoder. At inference, only the encoder is used to extract the mean latent style code.
3
Relative and global positional encoding
Combining a local relative positional encoding with a single-channel global encoding that marks sequence boundaries gives fine-grained temporal control: styles can be assigned to intervals of the output, and generation extends well past the maximum training length.
Relative and global positional encoding schemes
Positional encoding schemes. Top: relative position encoding (RPE), where each frame attends to its neighbors within a local window. Bottom: global position encoding (GPE), a single-channel signal marking sequence boundaries with smooth transitions between 1s and 0s.

Results — Single Style

Comparisons against T2M + MotionPuzzle, SMooDi, and LoRA-MDM. Every style reference is held out from training. Click any clip to enlarge.

“Side hop to left” — Zombie style

Arms raised forward with a stiff body. Only our model hops to the left in a straight line with arms raised in front.

Style referenceZombie

LoRA-MDM

T2M + MotionPuzzle

SMooDi

FlexMoGenOurs

“Side hop to left” — Chicken style

Crouched, bent-knee posture with arms tucked in.

Style referenceChicken

LoRA-MDM

T2M + MotionPuzzle

SMooDi

FlexMoGenOurs

“Backpedaling” — Dinosaur style

Bent arms raised in front of the chest, bent torso and knees. Only our model backpedals in the correct direction without erratic movements.

Style referenceDinosaur

LoRA-MDM

T2M + MotionPuzzle

SMooDi

FlexMoGenOurs

“Spinning clockwise” — ArmsAboveHead style

Only our model spins clockwise with raised arms without erratic artifacts such as unintended translation.

Style referenceArmsAboveHead

LoRA-MDM

T2M + MotionPuzzle

SMooDi

FlexMoGenOurs

“Standing long jump” — Chicken style

Only our model produces a clean jump without visible artifacts such as ground penetration or unnecessary extra steps.

Style referenceChicken

LoRA-MDM

T2M + MotionPuzzle

SMooDi

FlexMoGenOurs

Results — Time-Varying Style

A single generated sequence in which the requested style changes partway through. Baselines tend to keep producing the first style after the switch was requested.

“Walking forward then stopping and turning left”

Styles transition Dinosaur → Chicken → Aeroplane. Only our model captures the intended style transitions while performing every action.

Style referencesDinosaur → Chicken → Aeroplane

T2M + MotionPuzzle

SMooDi

FlexMoGenOurs

“Jump with both feet, then side hop, then kick legs”

Simultaneous action and style transitions over 360 frames, with styles Chicken → Penguin → BeatChest applied in sequence. BeatChest is out-of-distribution: it is unseen during training.

Style referencesChicken → Penguin → BeatChest

FlexMoGenOurs

Results — Body-Part Style Mixing

Because the style encoder embeds the five body parts independently, per-part components can be taken from different reference clips and assembled into a single style code before injection. No single reference provides the resulting combination.

Three style references

Text prompt: “Walking forward then turning left”. The legs come from Chicken, both arms from Aeroplane, and the spine from Zombie.

LegsChicken

Both armsAeroplane

SpineZombie

FlexMoGenAll three at once

Four style references, asymmetric arms

Text prompt: “Walking forward then turning left”. The two arms are drawn from two different clips, so the generated motion is asymmetric between them.

Left armRaisedLeftArm

Right armArmsBehindBack

SpineZombie

LegsLegsApart

FlexMoGenAll four at once

Results — Style References Outside 100STYLE

Text prompt: “walking clockwise in a circle”. This reference comes from an internal capture of upper-body and prop-handling motions, entirely outside the style encoder’s pretraining distribution. Transfer in this setting is hit or miss rather than uniformly degraded, and the failures are graceful, in that the content prompt keeps being followed and what is lost is the stylistic character of the reference. The full set of references spanning that range is examined in the supplementary material.

Holding a soda can — transfers successfully

The left arm carried bent against the chest appears in our output, while the circular path asked for by the prompt is preserved.

Style referenceHolding a soda can

FlexMoGenOurs

BibTeX

If you find this work useful, please consider citing:

@article{lan2026flexmogen,
  title   = {Flexible Motion Generation from Language and Style References},
  author  = {Lan, Kai Weixian and Criswell, Bodie and Fedkiw, Briana and
             Zhang, Zhan and Teran, Joseph and Holden, Daniel},
  year    = {2026},
  note    = {Pacific Graphics 2026},
  eprint  = {2609.08032},
  archivePrefix = {arXiv},
  url     = {https://arxiv.org/abs/2609.08032}
}

Acknowledgements

The contributions of Kai Weixian Lan, Bodie Criswell, and Briana Fedkiw to this work were made during internships at Epic Games.