DSpace community: 工業管理研究所

以深度強化學習優化無人自取店包裹配送之末端路徑規劃;以深度強化學習優化無人自取店包裹配送之末端路徑規劃

title: 以深度強化學習優化無人自取店包裹配送之末端路徑規劃;以深度強化學習優化無人自取店包裹配送之末端路徑規劃 abstract: 隨著電子商務快速成長與消費者對交付彈性及服務品質要求提升，末端配送逐漸成為物流系統中成本壓力較高且營運限制複雜的環節。在無人自取點配送情境下，路徑規劃不僅需考量配送成本，也必須同時納入取貨點容量限制、初始占用量、節點服務急迫性與配送後容量風險等因素。若配送決策僅以行駛距離最小化為目標，可能忽略部分取貨點因容量不足、接近滿載或服務優先性較高所產生的實際營運風險。本研究以無人自取點之包裹配送為研究情境，探討單日、靜態需求下之末端路徑規劃問題。系統設定為單一物流中心、單一車輛與單趟配送作業，並假設每日需求、指定取貨點、取貨點最大容量、初始占用量與節點急迫性皆於配送前已知。本研究不考慮配送途中回倉補貨、即時需求新增、包裹重新分派與多趟配送情形，並將問題建構為一個序列決策問題，使車輛能依據當前位置、剩餘車輛容量、取貨點剩餘容量、未配送需求與服務急迫性等資訊，逐步選擇下一個配送節點。本研究採用深度強化學習方法（Deep Reinforcement Learning， DRL），並利用近端策略優化（Proximal Policy Optimization， PPO）作為訓練架構。模型透過狀態設計描述配送系統當前資訊，並透過動作遮罩機制排除不符合容量限制之不可行配送節點，以降低模型產生無效決策的可能性。獎勵函數則同時考量配送完成獎勵、行駛距離懲罰、服務急迫性加權與近滿風險懲罰，使代理人能在配送成本、服務優先性與容量風險之間進行權衡，逐步學習符合問題限制的配送決策策略。本研究透過電腦模擬實驗，比較所提出之 DRL 方法與傳統啟發式方法在不同問題規模下之表現，並以總行駛距離、配送完成度、容量可行性、服務急迫性與整體路徑品質等指標進行評估。研究結果可用以檢視 DRL 方法應用於無人自取點末端配送路徑規劃問題時的適用性、穩定性與限制，並提供未來在取貨點交付模式下進行配送決策設計與方法改良之參考。;With the rapid growth of e-commerce and the increasing demand for flexible delivery services， last-mile delivery has become one of the most costly and operationally complex parts of logistics systems. In the context of unmanned self-pickup points， routing decisions should consider not only transportation cost， but also pickup point capacity， initial occupancy， service urgency， and capacity risk after delivery. This study investigates a single-day， static-demand last-mile routing problem for parcel delivery to unmanned self-pickup points. The system consists of one depot， one vehicle， and a single delivery trip. Customer demand， assigned pickup points， maximum pickup point capacity， initial occupancy， and service urgency are assumed to be known before delivery. The problem is formulated as a sequential decision-making problem， in which the vehicle selects the next pickup point according to the current system state. To solve this problem， this study adopts a Deep Reinforcement Learning (DRL) approach and uses Proximal Policy Optimization (PPO) as the training framework. The proposed model applies an action masking mechanism to exclude infeasible pickup points that violate capacity constraints. The reward function considers delivery completion， travel distance penalty， urgency-weighted service， and near-capacity risk penalty， enabling the agent to balance routing cost， service priority， and capacity feasibility. Computational experiments are conducted to compare the proposed DRL method with traditional heuristic methods under different problem scales. Performance is evaluated using total travel distance， delivery completion， capacity feasibility， service urgency， and overall route quality. The results provide insights into the applicability， stability， and limitations of DRL for unmanned self-pickup point delivery routing problems.

在資訊不對稱下產能保留合約搭配可轉讓機制之決策分析;A Decision Analysis for Capacity Reservation Contracts with Transfer Mechanism under Information Asymmetry

title: 在資訊不對稱下產能保留合約搭配可轉讓機制之決策分析;A Decision Analysis for Capacity Reservation Contracts with Transfer Mechanism under Information Asymmetry abstract: 本研究探討需求不確定下，產能保留合約結合轉讓機制之決策問題。在高科技產業、半導體供應鏈及其他資本密集型製造環境中，上游產能建置成本高、擴充期長，且短期內不易調整；相對地，下游需求高度不確定，因而使供需協調成為重要議題。產能保留合約使買方可於需求實現前先支付保留費以取得未來使用產能之權利，並於實際使用時再支付執行費。然而，需求實現後，各買方實際需求與原先保留量可能不一致，因而導致部分買方產能不足、部分買方持有剩餘產能，形成缺貨與閒置並存之資源錯置現象。本文建立單期、單一供應商與兩位買方之產能保留合約模型，比較無轉讓與允許轉讓兩種制度安排，並進一步探討需求不確定下之契約設計問題。研究首先在資訊對稱情境下建立基準模型，分析供應商保留價格、轉讓分成比例與買方最適預留量之決策結果；接著透過敏感度分析，探討需求機率、保留價格、轉讓分成比例與需求差異等參數變動對買方利潤、供應商利潤與系統總利潤之影響；最後延伸至資訊不對稱情境，分析供應商面對高、低需求型買方時之契約設計與利潤變化。數值結果顯示，當買方需求實現出現錯配時，允許轉讓可使未使用產能重新配置至高需求買方，降低產能閒置與缺貨損失，進而提升供應商利潤與系統總利潤。此外，在資訊不對稱下，供應商可透過差異化契約設計引導不同需求型買方選擇適合之產能保留決策。整體而言，本文將產能保留合約、轉讓機制與資訊不對稱整合於同一分析架構中，可作為需求不確定下供應鏈產能配置與契約設計之決策依據。;In this study， the decision-making problem of capacity reservation contracts combined with a transfer mechanism under demand uncertainty is examined. In high-tech industries， semiconductor supply chains， and other capital-intensive manufacturing environments， upstream capacity requires high investment costs， long expansion lead times， and limited short-term flexibility， while downstream demand is highly uncertain. A capacity reservation contract allows buyers to pay a reservation fee before demand realization to secure future capacity and pay an execution fee when the capacity is used. However， after demand is realized， actual demand may differ from the reserved quantity， causing capacity shortages for some buyers and unused capacity for others. A single-period model with one supplier and two buyers is established to compare the no-transfer and transfer mechanisms. Under information symmetry， the supplier’s reservation price， transfer-sharing ratio， and buyers’ optimal reservation quantities are evaluated. Sensitivity analysis is then conducted to examine how demand probability， reservation price， transfer-sharing ratio， and demand differences affect buyer profit， supplier profit， and total system profit. The model is further extended to an information asymmetry setting with high demand and low-demand buyer types. Numerical results show that the transfer mechanism reallocates unused capacity to the buyer with higher realized demand， thereby reducing idle capacity and shortage losses and improving supplier profit and total system profit. Under information asymmetry， differentiated contract design can guide buyers with different demand types toward appropriate reservation decisions. Overall， capacity reservation contracts， transfer mechanisms， and information asymmetry are integrated into a unified framework for supply chain capacity allocation and contract design under demand uncertainty.

Solving FJSP with due date related objectives using dual attention reinforcement learning;Solving FJSP with due date related objectives using dual attention reinforcement learning

title: Solving FJSP with due date related objectives using dual attention reinforcement learning;Solving FJSP with due date related objectives using dual attention reinforcement learning abstract: 本研究探討在柔性作業車間排程問題（Flexible Job-shop Scheduling Problem， FJSP）下，應用深度強化學習（Deep Reinforcement Learning， DRL）優化 total tardiness 的方法。不同於過去多數研究聚焦於 makespan，total tardiness 的目標函數具有有界於零、且在數值特性上與 makespan 不同等特點，使得為 makespan 設計的狀態特徵與訓練配置未必能直接適用。本研究以 Wang 等人（2024）提出的雙重注意力網路（Dual Attention Network， DAN）為基礎，將其目標由 makespan 改為 total tardiness，並系統地檢驗四個層面的設計。在特徵層面，於原框架的手工特徵之外引入 Normalized Operation Due Date、Slack per Operation 與 Estimated Tardiness 等交期特徵，以及新設計的前瞻特徵 Projected Slack Ratio，並檢驗這些特徵對 total tardiness 的效果;同時將隨機閘門(stochastic gates)嵌入式特徵選擇方法改編至強化學習目標，檢驗完整三十個手工特徵中可移除的冗餘程度。在架構層面，檢驗殘差連接對此淺層注意力網路的作用及其機制。在測試策略層面，檢驗取樣次數作為測試階段計算預算對 total tardiness 的影響。在訓練層面，以標準的訓練診斷量檢驗訓練的穩定性，並據以檢驗是否需要額外的學習率排程或梯度裁剪。實驗在兩種問題規模（10×5 與 15×10）上進行，並將所提出的排程器與優先派工法則( priority dispatching rules )及數學規劃時限上界進行比較。;This study investigates the use of Deep Reinforcement Learning (DRL) to optimize total tardiness in the Flexible Job-shop Scheduling Problem (FJSP). Unlike most prior work that minimizes makespan， the total-tardiness objective is bounded below at zero and differs in numerical character from makespan， so the state features and training configuration designed for makespan do not necessarily transfer directly. Building on the Dual Attention Network (DAN) of Wang et al. (2024)， this study changes its objective from makespan to total tardiness and systematically examines four aspects of the design. At the feature level， it adds due-date features — normalized operation due date， slack per operation， and estimated tardiness — together with a newly designed forward-looking feature， the projected slack ratio， to the original handcrafted features and examines their effect on total tardiness; it also examines， through an embedded stochastic-gate selection method adapted to the reinforcement-learning objective， the extent to which the full set of thirty handcrafted features can be reduced. At the architectural level， it examines the role of residual connections in this shallow attention network and the mechanism behind it. At the test-strategy level， it examines the effect of the number of sampling iterations， viewed as a test-time compute budget， on total tardiness. At the training level， it uses standard training diagnostics to examine training stability and， based on these， whether additional learning-rate scheduling or gradient clipping is needed. Experiments are conducted on two problem sizes (10×5 and 15×10)， and the proposed scheduler is compared against priority dispatching rules and a time-limited mathematical-programming upper bound.

Flexible job shop scheduling with batching and time window when minimizing makespan and total number of tardy jobs;Flexible job shop scheduling with batching and time window when minimizing makespan and total number of tardy jobs

title: Flexible job shop scheduling with batching and time window when minimizing makespan and total number of tardy jobs;Flexible job shop scheduling with batching and time window when minimizing makespan and total number of tardy jobs abstract: 本研究探討半導體擴散製程中的彈性零工式排程問題，同時考量平行批次處理與時間窗約束，目標為極小化最大完工時間與總逾期工件數。本研究提出四項方法。第一，延伸 Knopp et al. (2017) 之批次無感分離弧線圖，參考 Sheen and Liao (2007) 之區間加權弧線設計，新增時間窗屬性於弧線上，使圖模型能同時表達批次決策與時間窗限制。第二，延伸 Dauzère-Pérès and Paulli (1997) 之整合鄰域結構，提出同時驗證無環性與時間窗約束之可行性檢查。第三，延伸 Tsai (2024) 之下界計算，將時間窗約束納入路徑分析，並將逾期指標從層級調整至工件層級，提出複合節省值以評估候選移動對雙目標之影響。第四，將上述方法整合至仿生式 NSGA-II 框架。交配運算子採用位置對齊之雙向量染色體編碼，延伸 Pezzella et al. (2008) 與 Tsai (2024)，將工序排序與機台指派綁定為不可分割之基因單位，以保護批次結構。變異運算子以導向式局部搜尋取代隨機擾動，系統性地選擇最具改善潛力之移動。;This study investigates the Flexible Job-shop Scheduling Problem with parallel batching and time window constraints in semiconductor diffusion processes， with the objective of simultaneously minimizing the makespan and the total number of tardy jobs. Four methodologies are proposed. First， the batch-oblivious conjunctive graph of Knopp et al. (2017) is extended by incorporating a time window arc attribute， adopting the interval-weighted arc design of Sheen and Liao (2007)， enabling the graph to represent both batching decisions and time window constraints. Second， the integrated neighborhood structure of Dauzère-Pérès and Paulli (1997) is adapted with a feasibility check that simultaneously verifies acyclicity and time window constraint satisfaction. Third， the lower bounds of Tsai (2024) are extended to incorporate time window constraints and adapted from the stage-out level to the job level， and a composite saving value is proposed to assess the impact of candidate moves on both objectives. Fourth， these components are integrated into a memetic NSGA-II framework. The crossover operator employs a position-aligned two-vector chromosome encoding， extending Pezzella et al. (2008) and Tsai (2024)， where the operation sequence and machine assignment are coupled as an indivisible gene to preserve batching configurations. The mutation operator replaces random perturbation with a guided local search that systematically selects moves with the highest improvement potential.