隨著智慧製造的發展，生產系統逐漸朝向即時監控、動態決策與高效率資源配置 演進。然而，在具有多產品、多配方（Recipe）、平行機群、交期限制與改機需求的即時派工環境中，子批派工是一項複雜的生產排程問題。 本研究提出一套結合非監督式學習與多代理人系統的深度強化學習階層式即時派 工方法。首先，依據各配方的特徵，利用K-Means將具有相似生產負荷的配方分群；接著，由上層協調者整合各群集的工作負荷、急單需求與機群狀態，將可用機群分配給適當的群集。下層則為每個群集配置獨立的近端策略最佳化（PPO）代理人，分別處理具有不同特徵的配方子批，例如由特定代理人負責總加工工時較長的配方。此設計可透過協調者與下層代理人的分工縮小個別代理人的動作空間，並提升其對不同生產負荷特性的適應能力。 本研究以降低子批延遲時間與改機次數為主要目標，並使用LinearThompsonSam pling（LinTS）與 Linear Upper Confidence Bound（LinUCB）作為比較基準。實驗結果顯示，在最佳情況下，LinTS與多代理人系統PPO均可使所有子批於交期內完成。然而，LinTS 的改機次數約為多代理人系統PPO的4.64倍，LinUCB的改機次數則約為多代理人系統PPO的2.36倍，顯示多代理人系統PPO在維持交期表現的同時，能更有效降低改機需求並改善生產效率。;With the development of smart manufacturing， production systems are moving toward real-time monitoring， dynamic decision-making， and efficient resource allocation. However， in a real-time dispatching environment with multiple products， multiple recipes， parallel machine groups， due-date limits， and setup requirements， schedule dispatching is a complex production scheduling problem. This study proposes a hierarchical real-time dispatching method that combines unsupervised learning， a multi-agent system， and deep reinforcement learning. First， K-Means is used to group recipes with similar production loads based on their features. Next， an upper-level coordinator combines information about the workload of each cluster， urgent demand， and machine-group status， and then assigns available machine groups to suitable clusters. At the lower level， each cluster is assigned an independent Proximal Policy Optimization (PPO) agent to handle schedules with different recipe features. For example， a specific agent may be responsible for recipes with longer total processing times. Through the division of work between the coordinator and the lower-level agents， this design reduces the action space of each agent and improves its ability to adapt to different production-load conditions. The main objectives of this study are to reduce schedule tardiness and the number of setups. Linear Thompson Sampling (LinTS) and Linear Upper Confidence Bound (LinUCB) are used as baseline methods. The experimental results show that， in the best case， both LinTS and the multi-agent PPO system can complete all schedules before their due dates. However， the number of setups produced by LinTS is about 4.64 times that of the multi-agent PPO system， while the number produced by LinUCB is about 2.36 times that of the multi-agent PPO system. These results show that the multi-agent PPO system can maintain good due-date performance while reducing setup needs more effectively and improving production efficiency.
