近年來，隨著智慧製造、人機協作與Physical AI之快速發展，傳統依賴固定程式與示教盒之機械手臂控制方式，已難以充分應對複雜、多樣且具高彈性需求之操作任務。本研究提出一套立體視覺方法能捕捉人體動作姿態，並應用於人機安全協作機制與手勢教導，接著將教導收集之資料進行機械手臂模仿學習訓練，與動作策略能勝任協作環境中的複雜任務。首先建構立體視覺辨識人體姿態之方法，進一步建立多人三維人體姿態檢測系統，透過相機陣列校正、多人匹配演算法與三維人體骨架重建，建立人機安全協作機制，用以提升機械手臂於多人環境下操作之安全性。接著，本研究進一步發展基於立體視覺之手勢教導方法，分別建構穿戴標記式與無標記式兩種教導系統。穿戴標記式教導系統利用紅外線動作捕捉系統量測手部姿態資訊，並搭配手勢辨識完成機械手臂末端位移、姿態角與夾爪控制，無標記式教導系統則透過多相機影像、MediaPipe Hand手部關鍵點偵測、三維重建，在無須穿戴標記物之條件下實現直覺式手勢教導，展現較高之應用彈性。 在動作策略學習方面，本研究以前述示範教導系統所蒐集之影像、機械手臂狀態與對應控制動作為基礎，建立模仿學習資料集，並採用Action Chunking with Transformers(ACT)作為核心學習架構。該模型結合深度殘差卷積神經網路、Transformer、條件式變分自編碼器、動作分塊與時步融合機制，以提升機械手臂對連續操作任務之動作預測能力，並降低單步預測造成之誤差累積。最後，本研究以機械手臂夾取任務驗證所建立之動作策略，結果顯示本方法可有效學習示範資料中的操作特徵，並完成對應之夾取操作。整體而言，本系統建立了一套由立體視覺之動作捕捉方法，應用於人機協作安全感知、直覺式教導到模仿學習動作策略之完整流程，可提升人機協作環境中的安全性與操作彈性，亦可作為未來發展智慧製造與自主機器人操作系統之重要基礎。;In recent years， with the rapid development of smart manufacturing， human–robot collaboration， and Physical AI， conventional robotic arm control methods that rely on fixed programs and teach pendants have become increasingly inadequate for complex， diverse， and highly flexible manipulation tasks. This study proposes a stereo vision-based method for capturing human motion and posture， which is further applied to human–robot safety mechanisms and gesture-based teaching. The demonstration data collected through the teaching process are subsequently used for robotic imitation learning， enabling the learned action policy to perform complex tasks in collaborative environments. First， a stereo vision-based human pose recognition method is developed and extended to a multi-person three-dimensional human pose detection system. Through camera-array calibration， multi-person matching algorithms， and 3D human skeleton reconstruction， a human–robot safety collaboration mechanism is established to enhance the operational safety of robotic arms in multi-person environments. Next， this study develops a stereo vision-based gesture teaching method and constructs both wearable marker-based and markerless teaching systems. The wearable marker-based system utilizes an infrared motion capture system to measure hand pose information and combines gesture recognition to control the end-effector position， orientation， and gripper of the robotic arm. In contrast， the markerless teaching system integrates multi-camera images， MediaPipe Hands for hand landmark detection， and 3D reconstruction to achieve intuitive gesture-based teaching without requiring the user to wear physical markers， thereby providing greater flexibility for practical applications. For action policy learning， the images， robotic arm states， and corresponding control actions collected from the aforementioned demonstration teaching systems are used to construct an imitation learning dataset. Action Chunking with Transformers (ACT) is adopted as the core learning framework. The model integrates a deep residual convolutional neural network， Transformer architecture， conditional variational autoencoder， action chunking， and temporal aggregation mechanisms to improve motion prediction for continuous manipulation tasks while reducing error accumulation caused by step-by-step prediction. Finally， robotic grasping tasks are conducted to validate the learned action policy. The experimental results demonstrate that the proposed approach can effectively learn manipulation characteristics from demonstration data and successfully perform the corresponding grasping tasks. Overall， this study establishes a complete framework that begins with stereo vision-based motion capture， extends to safety perception for human–robot collaboration and intuitive gesture-based teaching， and ultimately enables imitation learning-based action policy acquisition. The proposed framework enhances both safety and operational flexibility in human–robot collaborative environments and provides a foundation for the future development of smart manufacturing and autonomous robotic manipulation systems.
