DSpace collection: 博碩士論文

用於大型語言模型推論之兼具通訊效率與安全性的拆分式雲邊協作;Toward Communication-Efficient and Secure Split Edge-Cloud Collaboration for Large Language Model Inference

title: 用於大型語言模型推論之兼具通訊效率與安全性的拆分式雲邊協作;Toward Communication-Efficient and Secure Split Edge-Cloud Collaboration for Large Language Model Inference abstract: 大型語言模型（Large Language Model， LLM）已逐漸成為行動裝置與邊緣智慧服務的重要核心。然而，純雲端部署容易造成資料外送與延遲問題，純端側部署則受限於算力、記憶體與模型能力。拆分式雲邊協作（Split-based Edge Cloud Collaboration）將同一個模型切分為端側前段與雲端後段，並以中間表徵取代原始輸入進行傳輸，提供了兼顧模型能力與資料最小化的研究方向。然而，此架構仍面臨兩項主要挑戰。第一，Prefill 階段需要傳輸大量中間表徵（Activation），容易增加第一個 Token 產生時間（Time-To-First-Token， TTFT）。第二，中間表徵雖非原始文字，但仍可能洩漏個人可識別資訊（Personally Identifiable Information， PII）。為解決上述拆分式雲邊協作的瓶頸，本研究針對通訊效率與安全性提出兩項設計。通訊方面，本研究透過中間表徵壓縮降低跨邊界傳輸成本，在主要設定下可將每個 token 的傳輸量降低約 85.7%，並在長提示情境中降低約 65.6% 的 TTFT，同時保持接近未壓縮模型的生成品質。安全性方面，本研究進一步分析壓縮後表徵仍可能被線性探測器讀出敏感資訊，因此加入LEACE隱私防禦機制以降低與回答無關的特定概念。實驗結果顯示，該方法可將最差層的敏感資訊探測準確率由約 90.5% 降至 68.5%。整體而言，本研究展示拆分式雲邊協作在 LLM 推論中透過壓縮來達成通訊效率的提升，與透過防禦機制增加安全性，並提出一個可實作的系統設計方向，使其更接近實際部署需求。;Large Language Models (LLMs) are becoming important in mobile and edge intelligence services. However， cloud-only deployment may cause privacy and latency problems， while on-device deployment is limited by computation， memory， and model size. This study investigates split edge-cloud collaboration， where one LLM is divided into an edge-side front part and a cloud-side back part. This design sends intermediate activations instead of raw text， but it still faces two main challenges: high communication cost and possible privacy leakage. To address these problems， this study improves both communication efficiency and activation security. For communication， activation compression reduces the transmitted data per token by about 85.7% and reduces Time-To-First-Token (TTFT) by about 65.6% in a long-prompt setting， while keeping generation quality close to the uncompressed baseline. For security， this study shows that compressed activations may still leak Personally Identifiable Information (PII)， so a privacy refresh mechanism is added to reduce answer-irrelevant sensitive signals. Experimental results show that the worst-layer probing accuracy is reduced from about 90.5% to 68.5%. Overall， this study analyzes the communication and security problems of split edge-cloud collaboration for LLM inference and provides a practical design for more deployable split LLM systems.

基於新適應器無回放領域增量學習於細粒度農業航空影像辨識;Exemplar-Free Domain Incremental Learning with v-Adapter for Fine-Grained Agricultural Aerial Image Recognition

title: 基於新適應器無回放領域增量學習於細粒度農業航空影像辨識;Exemplar-Free Domain Incremental Learning with v-Adapter for Fine-Grained Agricultural Aerial Image Recognition abstract: 隨著航空攝影技術的發展，農業影像辨識已成為精準農業的核心。然而，在連續觀測作物生長週期的實務應用中，模型常面臨「領域增量學習 (Domain Incremental Learning， DIL)」的情境。農作物具備高度的「細粒度 (Fine-grained)」特性，例如不同作物在特定生長期 (如玉米與花生的收割期) 具有極高的類間相似度，且同一作物在不同階段亦存在巨大的類內變異度。傳統的微調 (Fine-tuning) 方法在學習新階段特徵時，極易產生災難性遺忘 (Catastrophic Forgetting)；而基於歷史資料回放 (Replay) 的技術則受限於龐大影像的儲存成本與實務限制；完整重新訓練 (Retrain) 則會導致時間成本呈非線性增長，無法滿足農業系統即時部署的需求。為解決上述挑戰，本研究建立一套基於側掛式適應器 (ν-Adapter) 的「無回放 (Exemplar-Free)」DIL 框架，應用於細粒度農業航空影像辨識。本框架透過參數隔離機制，僅需更新極少量的適應器參數 (約 0.13M 至 0.44M)，即可完整保留舊有任務知識，從根本上避免災難性遺忘。此外，為克服農業影像的細粒度混淆問題，本研究結合了監督式對比損失 (SupCon Loss) 與焦點損失 (Focal Loss) 之雙重損失函數，有效在潛在特徵空間中解耦高度相似的作物特徵，並強化模型對困難樣本的鑑別力。實驗採用農業部提供之高解析度 (15 cm/pixel) DMC III 多光譜 (Red-Green-BlueNear-Infrared， RGB-NIR) 航空影像資料集進行驗證。結果顯示，本研究所提之 ν-Adapter 在經歷連續增量任務後，能維持高達 0.9505 之平均 F1-Score，且增量效能落差與相對效能落差僅為 1.40%，顯著優於現有的尖端技術 (State-of-the-Art， SOTA) 基準模型 (如 Model Surgery (MOS)、Selecting the Optimal Parameters is All You Need (SOYO) 等)。同時，相較於完整重新訓練，本方法依適應器維度不同可縮短約 61.99% 至 65.87% 的訓練耗時。本研究成功兼顧了高精準度、抗遺忘能力與實務部署之高時效性，為智慧農業之持續性學習提供了一套極具潛力的解決方案。;Aerial image recognition is essential for precision agriculture. However， continuous crop monitoring requires models to handle Domain Incremental Learning (DIL) scenarios. Agricultural images are highly fine-grained， exhibiting severe inter-class similarity (e.g.， corn and peanuts at harvest) and massive intra-class variance across growth stages. Traditional finetuning suffers from catastrophic forgetting， replay-based methods face storage constraints， and full retraining incurs unacceptable time costs for rapid deployment. To address these challenges， this study develops an exemplar-free DIL framework based on a lightweight side adapter (ν-Adapter). Through parameter isolation， it updates only 0.13M to 0.44M parameters， retaining previous knowledge and mitigating catastrophic forgetting from the root. Furthermore， a dual loss function combining Supervised Contrastive Loss (SupCon Loss) and Focal Loss is integrated to decouple highly similar crop features in the latent space and enhance discrimination on hard examples. Experiments on the DMC III high-resolution multispectral dataset (RGB-NIR) demonstrate that the ν-Adapter achieves an outstanding average F1-Score of 0.9505 across sequential tasks. With a minimal incremental learning gap measure rate of 1.40%， it significantly outperforms state-of-the-art baselines (e.g.， MOS， SOYO). Additionally， depending on the adapter dimension， it reduces training time by 61.99% to 65.87% compared to full retraining. This framework successfully balances high accuracy， anti-forgetting capability， and time efficiency， offering a robust solution for continual learning in smart agriculture.

結合系統操作軌跡錄製與操作手冊語意解析的 GUI 模型標註與生成技術;結合系統操作軌跡錄製與操作手冊語意解析的 GUI 模型標註與生成技術

title: 結合系統操作軌跡錄製與操作手冊語意解析的 GUI 模型標註與生成技術;結合系統操作軌跡錄製與操作手冊語意解析的 GUI 模型標註與生成技術 abstract: Computer-Using Agent（CUA）讓使用者可以用自然語言要求AI 觀察畫面並操作圖形使用者介面（Graphical User Interface， GUI）；這類代理人通常仰賴生成式AI 與大型語言模型進行畫面理解、工具選擇與動作生成。若每個步驟都由模型在執行時即時判斷，路徑與動作就可能受到模型輸出的不確定性影響。對製造、醫療與受管制企業後台等高風險場域而言，重點不只是能否完成任務，而是能否先知道哪些操作路徑已被實際錄製、可被接受，且缺少知識時必須停止。這需要一份能描述畫面、操作意圖、前置條件與可走路徑的GUI 操作知識。本研究提出GUI Model Learning，作為自動化執行之前的GUI 操作知識形成方法。系統以操作手冊或SOP 建立GUI Model 初版，並透過錄製保存熟練使用者的實際操作。錄製資料中的螢幕畫面、OCR 與UI 元件定位結果、鍵盤滑鼠事件，會被整理成候選GUI Model。多次錄製可逐步補上穩定路徑、不同使用者的可接受做法，以及應停止的受阻情境；相較於傳統RPA 需要為多條完整路徑分別錄製或維護腳本，本研究將錄製結果整理為可組合的模型單位。這些資料可先用於知識傳承、教育訓練與流程觀察，當資料要成為正式模型或交給後續自動化使用時，再由審核工程師確認、退回或接受。 GUI Model 不只保存成功路徑，也保存「何時不應繼續」的判斷。例如前置條件不足、權限限制或資料驗證錯誤，都可能代表後續自動化應停止，而不是嘗試其他未知操作。這使GUI Model Learning 的重點從單次腳本重放，轉為建立一份可供人與系統共同使用的GUI 操作知識。受控工業檢測流程個案顯示，日常錄製能形成候選畫面情境、操作意圖、流程路徑與操作步驟；後續自動化元件只讀取已接受GUI Model 產生 RPA 腳本，並在缺少起點、路徑或步驟時停止。比較分析指出， RPA 適合固定執行，CUA 適合依畫面推理，但兩者若要用於高風險GUI 流程，都需要一層由人類示範與實際錄製形成的本地操作知識作為邊界。本研究補上的，正是從本地使用者操作經驗建立GUI Model 的方法，使以提示詞驅動的不確定操作目標能轉為沿已知路徑產生腳本的受控自動化流程。;Computer-Using Agents (CUAs) allow users to request graphical user interface operations in natural language. These agents often rely on generative AI and large language models for screen interpretation， tool selection， and action generation; if every step is decided at runtime， the resulting operation path can remain uncertain. In manufacturing， medical， and regulated enterprise settings， the central issue is not only whether an agent can complete a task， but whether the system knows which operation paths have been demonstrated， can be accepted， and must stop when knowledge is missing. This requires GUI operation knowledge that describes screens， operation intents， preconditions， and executable paths. This thesis proposes GUI Model Learning as a method for forming GUI operation knowledge before automation is executed. The system builds an initial GUI Model from a manual or SOP and records skilled users’ actual operations. Screen content， OCR and UI-element localization results， keyboard and mouse events are organized into a candidate GUI Model. Repeated demonstrations gradually add stable paths， acceptable variants across users， and blocked situations where execution should stop. Unlike traditional RPA， which often requires separate recording or maintenance for multiple complete paths， this work organizes demonstrations into composable model units. Candidate updates are reviewed by a human reviewer only when they are promoted into a formal model or used by downstream automation. A GUI Model stores not only successful paths， but also decisions about when execution should not continue. Missing prerequisites， permission limits， and validation errors may indicate that downstream automation should stop instead of trying unknown actions. GUI Model Learning therefore shifts the focus from replaying a single script to building GUI operation knowledge that both humans and automation systems can use. A controlled industrial inspection case study shows that daily recordings can form candidate screen contexts， operation intents， process paths， and action steps. Downstream automation components generate RPA scripts only from the accepted GUI Model; when the model lacks a start state， route， or action steps， the system stops instead of guessing. The comparison indicates that RPA is suitable for fixed execution and CUAs are suitable for screen-based task reasoning， but high-risk GUI workflows need a local operation-knowledge layer before either approach can be used responsibly. This thesis contributes that layer: a method for building a reviewable GUI Model from local demonstrations， so that prompt-driven uncertain operation goals can be transformed into controlled automation scripts along accepted paths.

結合 AI 與 Venom 技術的 Legacy UI Transformation：一種無需程式開發的介面改造框架;結合 AI 與 Venom 技術的 Legacy UI Transformation：一種無需程式開發的介面改造框架

title: 結合 AI 與 Venom 技術的 Legacy UI Transformation：一種無需程式開發的介面改造框架;結合 AI 與 Venom 技術的 Legacy UI Transformation：一種無需程式開發的介面改造框架 abstract: 現代軟體使用情境快速變動，使用者與組織對介面流程、資訊呈現與操作效率的需求也持續改變。然而，許多既有應用程式因原始碼不可取得、擴充機制不足、重構成本過高或替代方案導入困難，使使用者即使面臨介面不符需求的情況，仍必須繼續使用原系統。傳統透過 API、外掛、重新開發、機器人流程自動化或等待原廠改善的方式，往往受限於技術門檻、組織成本與軟體本身是否支援擴充，難以回應非開發者在日常工作中對介面重排、資訊遮蔽、操作集中與互動簡化的需求。本研究關注的核心問題，是使用者在「既有介面不符合需求」與「修改介面成本過高」之間所面臨的兩難。當使用者缺乏原始碼、原廠擴充機制或程式開發能力時，介面調整需求往往難以轉化為可立即使用且可維護的結果。為回應此問題，本研究提出 UI Transformer，作為結合 Venom 技術與大型語言模型輔助的 Legacy UI Transformation 框架，目標是在不修改既有桌面應用程式、不依賴原廠外掛機制，且不要求使用者撰寫程式的前提下，提供一套可操作、可驗證且可保存的介面改造流程。系統利用 Venom 的非侵入式介面擴充能力，在既有應用程式上方建立可操作的新介面層，並將原本偏向開發者撰寫擴充程式的流程，轉化為使用者可透過圖形化操作與自然語言指令完成的介面改造流程。本研究的主要貢獻在於：第一，提出一套面向既有桌面應用程式的無需程式開發介面改造流程，降低使用者在無原始碼情境下調整介面的門檻；第二，設計可驗證的中介資料模型，使介面生成、原介面對應、互動同步與持久化能以統一格式管理；第三，整合 GUI 與大型語言模型工具調用，使自然語言操作能在明確邊界內執行，並與手動編輯保持一致；第四，整理並實作非侵入式介面改造框架所需的穩定性機制，使改造結果能被檢查、保存與重新開啟。論文並以互動成本、介面改動成本與組織交付週期作為評估面向，說明 UI Transformer 對既有應用程式介面改善的適用範圍。;Modern software usage scenarios evolve rapidly， and users increasingly need interfaces that better match their workflows， information structures， and operational habits. However， many existing applications remain difficult to modify because their source code is unavailable， extension mechanisms are limited， or redevelopment and migration are too costly. Conventional approaches such as vendor feedback， API-based extension， plug-in development， robotic process automation， or full redevelopment often depend on technical expertise， organizational resources， and explicit support from the original software， making them insufficient for everyday interface refactoring needs of non-developers. This thesis addresses the dilemma between interfaces that no longer fit users′ needs and the high cost of modifying existing applications. When users lack source code， vendor-supported extension mechanisms， or programming expertise， interface adjustment needs are difficult to turn into usable and maintainable results. In response， this thesis proposes UI Transformer， a programming-free Legacy UI Transformation framework that combines Venom technology with LLM-based assistance. The framework provides an operable， verifiable， and persistent transformation workflow under three constraints: no modification of the existing desktop application， no reliance on vendor-provided Plug-In mechanisms， and no programming requirement for users. Built on Venom′s Non-Invasive Interface Extension capability， UI Transformer creates an editable interface layer above an existing application and turns a developer-oriented extension process into an End-User workflow that combines graphical editing with natural-language instructions. The contributions of this work are fourfold. First， it proposes a programming-free workflow for transforming existing desktop application interfaces under no-source-code constraints. Second， it designs a verifiable intermediate data model that unifies interface generation， mapping to the original application， interaction synchronization， and persistence. Third， it integrates GUI editing with LLM Tool Invocation so that natural-language operations remain bounded and consistent with manual edits. Fourth， it implements stability mechanisms that allow transformation results to be inspected， saved， and reopened. The thesis further uses interaction cost， interface-change cost， and organizational delivery lead time as evaluation dimensions to clarify the applicability of UI Transformer for improving existing application interfaces.