在混合實境（Mixed Reality， MR）互動中，使用者或大型語言模型可能 提出自然語言形式的候選動作。然而，語意上合理的動作不一定適合在 目前物件實例上執行，其可行性仍取決於目標部件、關節運動類型與當 前關節位置。若系統僅依賴語意判斷，可能錯誤接受與目前物件狀態或 結構不相容的動作。因此，本研究提出一個狀態、部件與關節感知的動 作適用性驗證方法，作為高階動作提出與後續執行之間的執行前驗證 層。 本研究將自然語言候選動作與多部件關節物件脈絡建模為二元動作適 用性判斷問題。物件脈絡被表示為由物件、部件、關節與關節位置節 點所組成的關聯圖，並以目標部件指示特徵標示操作目標。候選動作 使用 Sentence-BERT 編碼，結構化脈絡則由關聯圖卷積網路（Relational Graph Convolutional Network， R-GCN）處理。圖表示與動作表示經融合 後，輸出動作適用性分數，並依據驗證集所選定的門檻轉換為接受或拒 絕決策。 本研究以 PartNet-Mobility 建立包含 13，990 筆樣本的多部件動作適用性 基準資料集，並採用物件實例互斥的訓練、驗證與測試切分。實驗結果 顯示，所提出的 R-GCN 模型達到 0.9201 的 F1 分數與 0.0555 的錯誤接 受率（False Acceptance Rate， FAR），明顯優於平坦 Sentence-BERT 與多 層感知器基線。進一步分析顯示，連續關節數值資訊有助於辨識狀態衝 突，而保留物件、部件、關節與關節位置之間的結構關係，則是提升多 部件動作驗證效能的主要因素。;In mixed-reality (MR) interaction， users or large language models may propose candidate actions in natural language. However， a semantically plausible action may not be suitable for a specific object instance because its executability de- pends on the designated target part， supported joint motion， and current articu- lation state. Relying only on semantic plausibility may therefore cause an MR system to accept actions that are incompatible with the current object context. This thesis proposes a state-， part-， and articulation-aware action-suitability ver- ification method as a pre-execution layer between high-level action proposal and downstream execution. The proposed framework formulates the verification task as binary classifi- cation over a natural-language candidate action and a structured multi-part articulated-object context. Each object context is represented as a relational graph containing object， part， joint， and joint-position nodes， together with a target-part indicator. Candidate actions are encoded using Sentence-BERT， while the structured context is processed using a relational graph convolutional network (R-GCN). The graph and action representations are fused to produce an action-suitability score， which is converted into an accept-or-reject decision using a threshold selected on the validation set. A PartNet-Mobility-derived benchmark containing 13，990 context-action sam- ples is constructed with object-instance-disjoint training， validation， and test splits. Experimental results show that the proposed R-GCN achieves an F1 score of 0.9201 and a false acceptance rate (FAR) of 0.0555， substantially out- performing the flat Sentence-BERT and MLP baseline. Further analysis shows that continuous articulation features are important for detecting state conflicts， while preserving explicit object–part–joint–position relationships provides the largest improvement in multi-part action verification.
