用於自然人機互動的可客製化手勢辨識系統設計;Customizable Gesture Recognition System for Natural Human-Machine Interactions

NCU Institutional Repository > 資訊電機學院 > 資訊工程研究所 > 博碩士論文 > Item 987654321/95586

jsp.display-item.identifier=請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/95586

题名:	用於自然人機互動的可客製化手勢辨識系統設計;Customizable Gesture Recognition System for Natural Human-Machine Interactions
作者:	維亞彥;Velez, Axel Yann
贡献者:	資訊工程學系
关键词:	電腦視覺;辨識;手勢;Computer Vision;Recognition;Gestures
日期:	2024-07-24
上传时间:	2024-10-09 17:04:38 (UTC+8)
出版者:	國立中央大學
摘要:	本論文設計了一個手勢識別系統，可實現更自然的人機互動。該系統基於美國手語 (ASL) 字母的識別，並且追踪用戶手部的動作。它可以辨識靜態手勢(ASL 字母)、複合手勢(ASL 字母序列)以及動態手勢(ASL 字母與手部動作相結合)。我們還設計了對每個手勢的多種動作支持，並提供使用者反饋。本系統的一個特色是允許用戶彈性添加自定義手勢和修改現有手勢，通過簡單地將手勢與特定手部動作相結合或定義靜態手勢序列。本研究首先介紹了開發此可定制手勢識別系統的動機，並確定了缺乏靈活性的現有系統所面臨的挑戰。隨後，詳細描述了系統的設計和實現，採用了包括卷積神經網絡 (CNNs)、長短期記憶網絡 (LSTMs) 及動態時間規整 (DTW) 在內的先進機器學習技術。這些技術被整合成一個階層式模組化的系統架構，能夠分類並辨識靜態、複合和動態手勢。系統實作階段包括資料的收集，以及其預處理，包括手部標誌並轉化為可用於訓練模型的資料集。評估階段採用各種指標（包括準確度、召回率和 F1 分數）來驗證系統所達到的高精度和強健性。最後，我們探討了手勢的個人化和不同的人機互動方式，驗證了系統的易用性及其在真實世界中的應用潛力。;This thesis presents a customizable hand gesture recognition system designed for natural human machine interactions. This system is based on the recognition of ASL (American Sign Language) letters as well as the tracking of the user’s hand movement. It can detect static signs (single ASL letter), composed gestures (sequence of ASL letters) and dynamic gestures (ASL letter combined with hand movement path). It is also designed to handle the various actions associated to them and providing feedback to the user. One of the key features of this system is its flexibility, allowing the user to add more gestures and easily modify existing ones by associating a sign and a movement path, or defining a sequence of static signs. The study begins with the motivations for developing a customizable gesture recognition system and outlines the challenges of existing systems that lack adaptability. It then details the design and implementation of the system, which leverages advanced machine learning techniques, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and Dynamic Time Warping (DTW). These techniques are integrated into a modular framework, capable of distinguishing and recognizing the static, composed, and dynamic gestures. The implementation phase covers the data collection and creation, and the preprocessing pipeline, including the extraction of the hand landmarks and their transformation into usable data for training our models. The evaluation phase demonstrates the system’s high accuracy and robustness across various metrics, including accuracy, loss and F1-score. Finally, the gesture customization and the different human-machine interactions are addressed, demonstrating the ease of use of the system and its real-world applications.
显示于类别:	[資訊工程研究所] 博碩士論文

文件中的档案:

档案	描述	大小	格式	浏览次数
index.html		0Kb	HTML	22	检视/开启

在NCUIR中所有的数据项都受到原著作权保护.

社群 sharing

数据加载中.....