基於序列至序列模型之 FMCW雷達估計人體姿勢

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：109

、訪客IP：3.16.66.206

姓名

鄭瑀萱(Yu-Hsuan Cheng) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

基於序列至序列模型之 FMCW雷達估計人體姿勢
(Human Pose Estimation Using FMCW Radar Based on Sequence-to-Sequence Models)

相關論文

★ 以非監督式表徵分離學習之邊緣運算裝置低延遲樂曲中人聲轉換架構

★ 基於多層次注意力機制之單目相機語意場景補全技術

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 ( 永不開放)

摘要(中)

人體姿勢估計應用在多個場景，包含自動駕駛、交通監控、患者監控、動作辨識、跌倒偵測等，有助於預防性。
與傳統的感測器相比，毫米波雷達更適用於不同的環境，因為使用者不
需要佩戴著裝置，且在低光源或是天氣惡劣的情況下，傳統感測器的性能會下降。毫米波雷達不會捕捉到使用者的面部，因此更具有隱私權保護及安全。毫米波雷達相較於光達，成本較低也更好取得。
本論文使用毫米波雷達生成三維點雲(x, y, z)，並基於序列至序列
(Sequence to sequence)模型估計人體姿勢。首先會經由體素化預處理點雲數據，並將10幀體素數據累加輸入到系統架構中，預測出 25個骨架關節點的體素索引，最後體素索引會根據體素化過程中使用的體素字典轉換回真實的三維世界座標。預測出來的結果再和 Ground Truth使用平均絕對誤差(MAE)做比較，目標是最小化誤差值。本實驗在編碼器加入了 self-attention相較於基準 (Baseline)，準確度提升了 5%，參數量減少 10M。

摘要(英)

Human pose estimation is applicable in various scenarios, including autonomous driving, traffic monitoring, patient monitoring, action recognition, and fall detection, contributing to preventive measures. Compared to traditional sensors, mmWave radar is more suitable for different environments as users do not need to wear devices, and its performance is less affected by low lighting or adverse weather conditions. Additionally, mmWave radar does not capture users′ facial features, providing privacy protection and security. It is also more cost-effective and accessible compared to lidar sensors.
In this paper, we utilize millimeter-wave radar to generate three-dimensional point clouds (x, y, z) and estimate human poses using a sequence-to-sequence model. Initially, the point cloud data is preprocessed through voxelization, and a sliding time window accumulates 10 frames of voxelized data as input to the system architecture. The model predicts voxel indices for 25 skeletal joints. Finally, the voxel indices are converted back to real-world 3D coordinates using the voxel dictionary employed during voxelization. The predicted results are compared with the ground truth using the Mean Absolute Error (MAE) metric, aiming to minimize the error. In the experiment, we introduce self-attention in the encoder, resulting in a 5% improvement in accuracy compared to the baseline, while reducing the parameter count by 10M.

關鍵字(中)

★ 毫米波雷達
★ 點雲
★ 體素化
★ 姿勢估計
★ 序列至序列模型

關鍵字(英)

★ mmWave radars
★ Point cloud
★ voxelization
★ Human pose estimation
★ Sequence-to-sequence model

論文目次

摘要 vi
Abstract vii
誌謝 viii
目錄 ix
圖目錄 xii
表目錄 xiv
第一章緒論 1
1-1 研究背景 1
1-2 研究動機與目的 2
1-3 論文架構 3
第二章毫米波雷達及深度相機相關介紹 4
2-1毫米波雷達 4
2-1-1 Range-FFT 6
2-1-2 Doppler-FFT 8
2-1-3 Angle FFT 10
2-1-4 毫米波雷達之三維點雲 11
2-2 深度相機 13
2-2-1硬體規格 14
2-2-2人體骨架追蹤 15
2-3毫米波雷達之文獻回顧 17
第三章 Sequence to Sequence 18
3-1 遞迴神經網路 18
3-1-1 遞迴神經網路介紹 19
3-1-2 長短期記憶網路介紹 20
3-1-3 門控循環單元 22
3-2 Sequence-to-Sequence Learning 24
3-2-1 Encoder-Decoder Model 25
3-2-2 Sequence-to-Sequence with Attention Mechanism 26
第四章提出之架構 28
4-1資料收集及資料前處理 28
4-2 序列到序列架構 30
4-3 損失函數 32
第五章實驗結果與分析討論 33
5-1 實驗環境與數據集介紹 33
5-2 評估方法 34
5-3 實驗結果與比較分析 35
第六章結論與未來展望 37
參考文獻 38

參考文獻

[1] TI毫米波雷達來源
https://www.ti.com/video/library.html
[2] Kinect v2深度相機來源
https://learn.microsoft.com/zh-cn/windows/apps/design/devices/kinect-for-windows
[3] R. Weimar et al., “Time-of-flight techniques for the investigation of kinetic energy distributions of ions and neutrals desorbed by core excitations”, Surface science, 2000
[4] J. Shotton et al., "Real-time human pose recognition in parts from single depth images", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1297-1304, 2011.
[5] A. D. Singh, S. S. Sandha, L. Garcia and M. Srivastava, "RadHAR: Human activity recognition from point clouds generated through a millimeter-wave radar", Proc. 3rd ACM Workshop Millimeter-Wave Netw. Sens. Syst., pp. 51-56, 2019.
[6] P. Zhao et al., "mID: Tracking and identifying people with millimeter wave radar", Proc. DCOSS, pp. 33-40, 2019.
[7] R. Zhang and S. Cao, "Real-time human motion behavior detection via CNN using mmWave radar", IEEE Sensors Lett., vol. 3, no. 2, pp. 1-4, Feb. 2019.
[8] A. Sengupta, F. Jin, R. Zhang and S. Cao, "mm-Pose: Real-time human skeletal posture estimation using mmWave radars and CNNs", IEEE Sensors J., vol. 20, no. 17, pp. 10032-10044, Sep. 2020.
[9] A. Sengupta and S. Cao, "mmPose-NLP: A natural language processing approach to precise skeletal pose estimation using mmWave radars", IEEE Trans. Neural Netw. Learn. Syst., Mar. 2022.
[10] A. Sherstinsky, "Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network", arXiv:1808.03314, 2018.
[11] J. Chung, C. Gulcehre, K. Cho and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling", arXiv:1412.3555, 2014.
[12] D. Bahdanau, K. Cho and Y. Bengio, "Neural machine translation by jointly learning to align and translate", International Conference on Learning Representations, 2015.
[13] I. Sutskever, O. Vinyals and Q. Le, "Sequence to sequence learning with neural networks", Proc. 27th Int. Conf. Neural Inf. Process. Syst., pp. 3104-3112, 2014.
[14] Y. Wu et al., "Google′s neural machine translation system: Bridging the gap between human and machine translation", arXiv:1609.08144, 2016.
[15] A. Vaswani et al., "Attention is all you need", Proc. Adv. Neural Inf. Process. Syst., pp. 5998-6008, 2017.

指導教授

張寶基陳永芳(Pao-Chi Chang Yung-Fang Chen)

審核日期

2023-8-15

推文