本論文提出一套應用於PTZ 雲台相機之無人機視覺追蹤控制系 統，以解決無人機目標在影像畫面中移動時，相機需即時調整方 位角與俯仰角以維持目標位於畫面中心附近的問題。系統透過影 像偵測模型取得無人機目標框與中心座標，並根據目標中心相對 於影像中心之像素偏移量、目標移動變化與相機角度狀態，建立 強化學習控制器所需之狀態資訊，使PTZ 相機能依據目前追蹤狀 態輸出對應之控制命令。 在控制模型設計上，本研究以Advantage Actor-Critic（A2C） 作為強化學習控制架構，並進一步結合雙向長短期記憶網路 （Bidirectional Long Short-Term Memory， BiLSTM），提出BiLSTMA2C 控制模型。A2C 模型可根據目前狀態動態選擇控制動作，而 BiLSTM-A2C 則利用連續時間序列中的目標位置變化，使模型能 觀察過去一段時間內的運動趨勢，以提升面對速度變化與不規則 軌跡時的控制穩定性。此外，為降低實際追蹤過程中短時間漏檢 與誤檢造成的影響，本研究亦加入偵測結果修正與軌跡補償機制， 以維持控制器輸入資訊的連續性與合理性。 為驗證所提出方法之有效性，本研究設計多種模擬場景進行 測試，包含理想飛行路徑與恆定速度、速度變化、軌跡擾動，以 及追蹤演算法漏檢與誤檢情境，並與傳統像素誤差控制方法及 DQN 方法進行比較。模擬結果顯示，A2C 與BiLSTM-A2C 相較於 比較方法能有效降低相機角度誤差與目標像素偏移誤差。其中， BiLSTM-A2C 在多數情境下具有較佳的追蹤穩定性，顯示時間序 列特徵有助於強化學習控制器掌握目標運動變化。 最後，本研究建構實際PTZ 無人機視覺追蹤系統進行實測驗 證。實測結果顯示，強化學習控制器能根據影像目標偏移與相機 狀態動態調整控制命令，使無人機目標較穩定地維持於畫面中 心附近。相較於傳統像素誤差控制方法與DQN，BiLSTM-A2C 於 RMSE、MAE 等指標中取得較佳結果，證明所提出之強化學習控 制方法可有效應用於PTZ 雲台無人機視覺追蹤任務，並具備實際 系統應用之可行性。;This thesis proposes a UAV visual tracking control system for a PTZ camera， aiming to address the problem of adjusting the pan and tilt angles in real time when a UAV target moves in the image frame. The system uses an image detection model to obtain the UAV bounding box and target center coordinates. Based on the pixel offset between the target center and the image center， the target motion variation， and the current camera angle state， the state information required by the reinforcement learning controller is constructed. The PTZ camera can then output corresponding control commands according to the current tracking state. In the control model design， this study adopts Advantage Actor-Critic (A2C) as the main reinforcement learning control framework and further combines it with a Bidirectional Long Short-Term Memory (BiLSTM) network to develop a BiLSTM-A2C control model. The A2C model can dynamically select control actions according to the current state， while the BiLSTM-A2C model utilizes the target position variations within a continuous time sequence. This allows the model to observe the motion trend over a period of time and improve control stability under speed variations and irregular trajectories. In addition， to reduce the influence of short-term missed detections and false detections during practical tracking， a detection correction and trajectory compensation mechanism is introduced to maintain the continuity and reliability of the controller input. To verify the effectiveness of the proposed method， several simulation scenarios are designed， including an ideal flight path with constant speed， speed variation， trajectory disturbance， and missed or false detection conditions. The proposed methods are compared with the Pixel Offset Method and the DQN method. The simulation results show that A2C and BiLSTM-A2C can effectively reduce the camera angle error and the target pixel offset error compared with the baseline methods. Among them， BiLSTM-A2C achieves better tracking stability in most scenarios， indicating that temporal features are helpful for the reinforcement learning controller to capture target motion variations. Finally， a real PTZ UAV visual tracking system is implemented for experimental validation. The real-world test results show that the reinforcement learning controller can dynamically adjust the control commands according to the image target offset and camera state， allowing the UAV target to remain closer to the image center. Compared with the Pixel Offset Method and DQN， BiLSTM-A2C achieves better results in terms of RMSE， MAE. These results demonstrate that the proposed reinforcement learning control method can be effectively applied to PTZ camera-based UAV visual tracking tasks and has practical feasibility for real-time system applications.
