一種可同時更新神經網路各層網路參數的新技術— 採用關聯式學習及管路化機制;Enabling simultaneous parameter updates in different layers for a neural network —using associated learning and pipeline

NCU Institutional Repository > 資訊電機學院 > 資訊工程研究所 > 博碩士論文 > Item 987654321/93152

jsp.display-item.identifier=請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/93152

题名:	一種可同時更新神經網路各層網路參數的新技術— 採用關聯式學習及管路化機制;Enabling simultaneous parameter updates in different layers for a neural network —using associated learning and pipeline
作者:	林廷翰;Lin, Ting-Han
贡献者:	資訊工程學系
关键词:	倒傳遞;反向鎖定;關聯式學習;平行化訓練;模型平行化;back-propagation;backward locking;associated learning;parallel training;model parallelism
日期:	2023-07-20
上传时间:	2024-09-19 16:44:40 (UTC+8)
出版者:	國立中央大學
摘要:	倒傳遞 (Back-propagation, BP) 廣泛運用於今日的深度學習演算法，然而它仍存在反向鎖定的問題導致模型訓練效率不佳。許多研究嘗試解決反向鎖定問題，而關聯式學習 (Associated Learning, AL) 便是其中一種模型架構。雖然關聯式學習理論上可透過管線化來增加訓練的效率，但原論文並未實現管線化，本論文補足這個部份，並透過大量實驗及效能分析工具(profiler) 觀察關聯式學習管線化後的實際行為。本論文亦和過去使用倒傳遞訓練之模型做比較，探討各自的優勢與限制，並討論關聯式學習未來的研究方向。;Back-propagation (BP) is widely utilized in deep learning algorithms, but it suffers from the issue of backward locking, resulting in inefficient model training. Various research efforts have been made to address this problem, and one promising solution is Associated Learning (AL). In theory, AL has the potential to enhance training efficiency through pipelining. However, the original proposal lacks the implementation of the pipeline. In this thesis, we bridge this gap by implementing the pipeline mechanism and conducting experiments on multiple GPUs. By leveraging profiling tools, we analyze the behavior of AL after pipelining. We compare models trained using back-propagation and pipelined AL to examine their respective advantages and limitations. Moreover, we discuss potential future research directions for Associated Learning.
显示于类别:	[資訊工程研究所] 博碩士論文

文件中的档案:

档案	描述	大小	格式	浏览次数
index.html		0Kb	HTML	13	检视/开启

在NCUIR中所有的数据项都受到原著作权保护.

社群 sharing

数据加载中.....