基於免訓練預評估模組之微型神經網路架構搜尋;Micro Neural Architecture Search Based on Training-Free Pre-evaluate Module

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/90051

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/90051

Title:	基於免訓練預評估模組之微型神經網路架構搜尋;Micro Neural Architecture Search Based on Training-Free Pre-evaluate Module
Authors:	劉冠宇;Liu, Kuan-Yu
Contributors:	資訊工程學系
Keywords:	神經網路架構搜尋;免訓練指標;微型機器學習;Neural Architecture Search;Training-Free Metrics;TinyML
Date:	2022-09-26
Issue Date:	2022-10-04 12:09:13 (UTC+8)
Publisher:	國立中央大學
Abstract:	近年物聯網裝置的迅速普及與神經網路理論與應用的加速發展，在邊緣裝置上部署神經網路已成為重要議題，其中，TinyML(Tiny Machine Learning)領域涵蓋了在微控制器上發展神經網路應用的理論與技術，也發展出許多領域試圖解決微控制器上的資源限制的問題，在微控制器規模的神經網路架構設計方面，神經網路架構搜尋(Neural Architecture Search)成為有效方法之一，但搜尋所花費的成本一直是神經網路架構搜尋方法難以克服的問題，近年隨著理論的發展催化了人們對神經網路架構的認識，其中神經正切核(Neural Tangent Kernel)與線性區域相關理論的結合，將其作為免訓練指標應用在神經網路架構搜尋方法上，顯著降低了搜尋的成本，也成為近年理論與方法結合的重要發展領域之一。本論文結合神經正切核理論與線性區域的相關理論，應用在微型神經網路架構搜尋方法上，提出了基於免訓練預評估模組之微型神經網路架構搜尋方法，獲得高效且適合在微控制器的平台資源限制下的網路架構。採用了基於免訓練預評估模組在搜尋演算法中降低整體搜尋架構的成本，能縮短搜尋高效網路架構的時間，建構自動且高效的神經網路架構設計方法。 ;In recent years, with the rapid popularization of IoT devices and the accelerated development of neural network theory and applications, the deployment of neural networks on edge devices has become an important topic. Among them, the field of TinyML (Tiny Machine Learning) covers the theory and technology of the development of neural networks on microcontrollers. Many theories and technologies have also been developed to try to solve the problem of resource constraints on microcontrollers. NAS (Neural Architecture Search) has become one of the effective methods, but the cost of searching has always been an insurmountable problem for the methods. In recent years, the development of theory catalyzes people to understand the neural network architecture. Among them, the combination of the NTK (Neural Tangent Kernel) and the related theory of the linear region is applied to the training-free metrics neural architecture search method. It significantly reduces the search cost, and has also become one of the important development areas of the combination of theory and methods in recent years. This paper combines the neural tangent kernel theory and the related theory of linear region, and applies it to the micro neural network architecture search method, and proposes a Micro Neural Architecture Search Based on Training-Free Pre-evaluate Module to obtain the network architecture under the platform resource constraints. The use of training-free pre-evaluation modules in the search algorithm reduces the cost of the search method, shortens the time to search for an efficient network structure, and builds an automatic and effective neural network structure design method.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	79	View/Open

社群 sharing

Loading...