| 摘要: | 本研究使用Double Deep Q Network( Double DQN) 及Q-Learning演算法，訓練無人自駕車的自動駕駛與自動停車模式。其中，自駕車的多項數據為演算法輸入的特徵變數，包括雷達、汽車位置、汽車速度等，輸出則為各個行動的Q值估計。由於在無人自駕車中，不同情境下所需的狀態數量並不相同，因此本研究將道路行駛及正向停車區分為兩種模式：分別為自動駕駛模式及自動停車模式。
在自動駕駛模式的訓練中，本研究使用Double DQN在約9000個回合時得到了最佳的訓練結果，使得汽車行駛得較快速且順暢。而在自動停車模式的訓練中，本研究使用Double DQN訓練自駕車代理人，其訓練環境則是從停車場門口到停車位完成正向(head-in)停車，可是效果不佳，因此，自駕車代理人改採多重模式(Multi-mode)進行訓練：從停車場門口行駛到停車位附近使用自動駕駛模式，並在汽車到達停車位附近時切換為自動停車模式。從停車場門口到停車位附近的訓練使用Double DQN，在約9800個回合達到最佳結果;而從停車位附近停進車位的訓練中，本研究使用了Q-Learning在約3500個回合即達到了最佳的訓練結果。;The present study employees algorithms of Double Deep Q Network ( Double DQN) and Q-Learning for training self-driving car agents in driving and parking modes, with the input features form data of the car (e.g., radar, car position, speed, etc.), and the estimation of Q value for each action as the output.Under different modes, the state spaces would be quite different from each other; hence, in the present study, it aims to adopt two certain situations, i.e., the driving mode as well as the parking mode for investigation.
Trained by Double DQN, the self-driving mode got the best result with about 9000 episodes. Meanwhile, in the parking situation, Double DQN was applied at first training the car to drive from the entrance of the parking lot into the parking space, but the performance was poor. Therefore, the car agents could use muti-mode training for the self-parking situation: first, use self-driving mode (with Double DQN) from the entrance of the parking lot to the position near the parking space, and then the car was trained to park into the parking space with a self-parking mode by Q-Learning. Accordingly, for searching the parking-space situation, the best result was achieved with about 9800 episodes with Double DQN. Then the car was trained to park into the parking space with Q-Learning, with the best of 3500-episode training. |
