用於自然人機互動的可客製化手勢辨識系統設計;Customizable Gesture Recognition System for Natural Human-Machine Interactions

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/95586

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/95586

Title:	用於自然人機互動的可客製化手勢辨識系統設計;Customizable Gesture Recognition System for Natural Human-Machine Interactions
Authors:	維亞彥;Velez, Axel Yann
Contributors:	資訊工程學系
Keywords:	電腦視覺;辨識;手勢;Computer Vision;Recognition;Gestures
Date:	2024-07-24
Issue Date:	2024-10-09 17:04:38 (UTC+8)
Publisher:	國立中央大學
Abstract:	本論文設計了一個手勢識別系統，可實現更自然的人機互動。該系統基於美國手語 (ASL) 字母的識別，並且追踪用戶手部的動作。它可以辨識靜態手勢(ASL 字母)、複合手勢(ASL 字母序列)以及動態手勢(ASL 字母與手部動作相結合)。我們還設計了對每個手勢的多種動作支持，並提供使用者反饋。本系統的一個特色是允許用戶彈性添加自定義手勢和修改現有手勢，通過簡單地將手勢與特定手部動作相結合或定義靜態手勢序列。本研究首先介紹了開發此可定制手勢識別系統的動機，並確定了缺乏靈活性的現有系統所面臨的挑戰。隨後，詳細描述了系統的設計和實現，採用了包括卷積神經網絡 (CNNs)、長短期記憶網絡 (LSTMs) 及動態時間規整 (DTW) 在內的先進機器學習技術。這些技術被整合成一個階層式模組化的系統架構，能夠分類並辨識靜態、複合和動態手勢。系統實作階段包括資料的收集，以及其預處理，包括手部標誌並轉化為可用於訓練模型的資料集。評估階段採用各種指標（包括準確度、召回率和 F1 分數）來驗證系統所達到的高精度和強健性。最後，我們探討了手勢的個人化和不同的人機互動方式，驗證了系統的易用性及其在真實世界中的應用潛力。;This thesis presents a customizable hand gesture recognition system designed for natural human machine interactions. This system is based on the recognition of ASL (American Sign Language) letters as well as the tracking of the user’s hand movement. It can detect static signs (single ASL letter), composed gestures (sequence of ASL letters) and dynamic gestures (ASL letter combined with hand movement path). It is also designed to handle the various actions associated to them and providing feedback to the user. One of the key features of this system is its flexibility, allowing the user to add more gestures and easily modify existing ones by associating a sign and a movement path, or defining a sequence of static signs. The study begins with the motivations for developing a customizable gesture recognition system and outlines the challenges of existing systems that lack adaptability. It then details the design and implementation of the system, which leverages advanced machine learning techniques, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and Dynamic Time Warping (DTW). These techniques are integrated into a modular framework, capable of distinguishing and recognizing the static, composed, and dynamic gestures. The implementation phase covers the data collection and creation, and the preprocessing pipeline, including the extraction of the hand landmarks and their transformation into usable data for training our models. The evaluation phase demonstrates the system’s high accuracy and robustness across various metrics, including accuracy, loss and F1-score. Finally, the gesture customization and the different human-machine interactions are addressed, demonstrating the ease of use of the system and its real-world applications.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	22	View/Open

社群 sharing

Loading...