伴隨語音的手勢是人類溝通的重要組成部分，能提供補充口語內容的視覺線索，並提升具身互動的自然度。隨著虛擬代理人與數位化身日益普及，根據語音自動生成身體動作，對化身動畫、人機互動及數位溝通等應用日益重要。本論文透過建置華語語音—動作資料集，以及開發區域特定的生成架構，探討音訊驅動的全身骨架動作生成。 本研究根據四位演員的對話與獨白錄製資料，建置一套同步的華語多模態動作捕捉資料集。經處理後的資料集包含約 79.2 小時的同步語音、全身骨架動作與臉部混合形狀序列。全身動作以 BVH 格式表示，並採用與 Mixamo 相容的骨架結構。專門設計的前處理流程用於對齊各項錄製模態、移除不適用的片段、修正旋轉不連續問題，並將關節旋轉轉換為適合模型訓練的連續六維表示。 本研究進一步提出一套兩階段的音訊驅動全身動作生成架構。在第一階段，分別使用向量量化變分自編碼器，學習上半身、雙手與下半身的離散動作表示。在第二階段，音訊條件式遮罩生成 Transformer 根據語音音訊、說話者身分與情緒，預測各身體區域的動作詞元。預測出的詞元再由預先訓練的區域解碼器進行解碼，並組合為連續的全身動作序列。透過將動作生成表述為區域離散詞元預測問題，此架構旨在因應語音與合理身體動作之間的一對多關係。 本研究使用 L1 多樣性、節拍一致性與 Fréchet 手勢距離，將所提出的架構與在相同資料集上重新訓練的 CaMN 進行比較。情緒條件式模型取得最高的 L1 多樣性分數 8.0294，以及最低的 Fréchet 手勢距離 0.3721；相較之下，CaMN 的對應分數分別為 6.7137 與 0.7630。然而，該模型的節拍一致性分數為 0.5249，仍低於 CaMN 的 0.6272，顯示其語音與動作之間的節拍層級對齊較弱。整體而言，實驗結果顯示，結合區域特定的離散動作表示與音訊條件式遮罩生成建模，具有生成多樣化全身動作的可行性，且其在學習特徵空間中的分布更接近真實動作的分布。;Co-speech gestures are an integral component of human communication， providing visual cues that complement spoken language and enhance the naturalness of embodied interaction. As virtual agents and digital avatars become increasingly prevalent， automatically generating body motion from speech has become relevant to applications such as avatar animation， human–computer interaction， and digital communication. This thesis investigates audio-driven full-body skeletal motion generation through the construction of a Mandarin speech–motion dataset and the development of a region-specific generative framework. A synchronized Mandarin multimodal motion-capture dataset was constructed from conversation and self-talk recordings involving four actors. The processed dataset contains approximately 79.2 hours of synchronized speech， full-body skeletal motion， and facial blendshape sequences. The full-body motion is represented in BVH format using a Mixamo-compatible skeleton structure. A dedicated preprocessing pipeline aligned the recorded modalities， removed unsuitable segments， corrected rotational discontinuities， and transformed joint rotations into a continuous 6D representation suitable for model training. A two-stage framework was developed for audio-driven full-body motion generation. In Stage 1， separate vector-quantized variational autoencoders learn discrete motion representations for the upper body， hands， and lower body. In Stage 2， an audio-conditioned masked generative Transformer predicts region-specific motion tokens from speech audio， speaker identity， and emotion. The predicted tokens are decoded by the pretrained regional decoders and assembled into continuous full-body motion sequences. By formulating motion generation as regional discrete token prediction， the framework is designed to accommodate the one-to-many relationship between speech and plausible body motion. The proposed framework was evaluated against CaMN， retrained on the same dataset， using L1 Diversity， Beat Consistency， and Fréchet Gesture Distance. The emotion-conditioned model achieved the highest L1 Diversity score of 8.0294 and the lowest Fréchet Gesture Distance of 0.3721， compared with 6.7137 and 0.7630 for CaMN， respectively. However， its Beat Consistency score of 0.5249 remained lower than the CaMN score of 0.6272， indicating weaker beat-level alignment between speech and motion. Overall， the results demonstrate the feasibility of combining region-specific discrete motion representations with audio-conditioned masked generative modeling to produce varied full-body motion whose distribution more closely resembles ground-truth motion in the learned feature space.
