| 摘要: | 近年來深度學習 (deep learning) 技術快速發展，應用於影像的分類、辨識、偵測、與分割等多領域，效果皆取得突破性的進步；但現實生活中許多應用場域異常樣本的取得非常困難，因此無法有充足的資料進行監督式的深度學習訓練，為了解決這樣的問題，只使用正常影像進行訓練的半監督式異常偵測 (anomaly detection) 之深度學習開始發展。
印刷電路板 (printed circuit board, PCB) 上電子元件引腳焊接時的異物、焊錫短路、與元件上的損壞，人工檢測困難，同時伴隨著異常樣本不易取得的問題，因此我們提出了半監督式異常偵測深度學習系統，偵測電子元件是否異常，並同時標示出可能異常的區塊。
我們的異常偵測系統改自於Skip-GANomaly與DFR異常偵測網路，根據應用需求，考慮是否能使用異常資訊參與訓練提升效果，我們提出了兩個網路，分別是只使用正常影像參與訓練的GANomaly-like DA網路，與可加入少量異常資訊提升效果的GAN DFR light網路。GANomaly-like DA主要的改進部份有：i. 移除Skip-GANomaly生成器跳接，使生成器重建的影像更能區別出正常與異常；ii. 在生成器中加入注意力模組，使網路關注重要特徵；iii. 損失函數調整；iv. 提出自適應調整生成器與判別器訓練次數方法，減緩訓練時生成器與判別器失衡問題，使生成對抗網路訓練能夠更穩定。GAN DFR light主要改進部份有：i. DFR加入判別器，以生成對抗網路方式進行訓練；ii. 修改DFR生成器架構中的卷積與縮減通道數，加快運算速度並維持效果。
在實驗中，我們主要以印刷電路板電子元件引腳間的影像進行訓練及驗證；正常影像共有46,651張，異常影像共有211張，我們將正常影像分為訓練集4,665張，驗證集41,986張，異常影像全為驗證集。以未改進前的Skip-GANomaly偵測異常，得到驗證樣本的篩除率87.57%，召回率86.73%，AUCROC 0.96；經過架構調整、訓練方法改進、自適應調整訓練次數、與異常分數計算比重調整後，最終GANomaly-like DA偵測異常的驗證樣本之篩除率提升至99.98%，召回率提升至100%，AUCROC提升至1.0。而使用GAN DFR light利用部份異常影像輔助異常偵測，可將驗證樣本的篩除率提升至100%，召回率提升至100%，AUCROC 提升至1.0。
針對首次提出的自適應調整訓練次數方法，我們還額外蒐集了積體電路外表損壞之影像進行實驗；正常影像共6,498張，異常影像共163張，正常影像分為訓練集3,248張，驗證集3,250張，異常影像全為驗證集。透過比較訓練中的對抗損失，可發現針對不同資料集，加入自適應調整訓練次數方法後，只要藉由適當的參數調整，就能使生成對抗網路的訓練更加穩定，最終得到驗證樣本的篩除率99.75%，召回率100%，AUCROC 0.999。
;In recent years, the rapid development of deep learning methods has been applied to various fields such as image classification, recognition, object detection and segmentation, and gets the breakthrough progress. However, it is difficult to obtain sufficient abnormal samples in some applications in real life for supervised deep learning methods. In order to solve this problem, semi-supervised deep learning anomaly detection methods were developed.
The damage of electronic components on the printed circuit board (PCB) is difficult to detect manually, and is also accompanied by the problem that abnormal samples are not easy to obtain. Therefore, we propose to use an semi-supervised deep learning anomaly detection method to detect whether the electronic components are abnormal and mark the abnormal area.
Our methods are modified from Skip-GANomaly and DFR. According to using anomaly information to participate in training or not, we propose GANomaly-like DA and GAN DFR light. The main improvements of GANomaly-like DA are: i. Remove skip-connections between encoder and decoder, so that the images generated by the generator can be better distinguished between normal and abnormal, ii. Add an attention module to the generator, so that the network can pay attention to the features that are more important, iii. Adjust loss function, iv. Adaptively adjust the frequency of training between the generator and the discriminator to make the training of GAN more stable. The main improvements of GAN DFR light are: i. Add a discriminator to DFR, ii. Modify the convolutional layers and reduce the number of channels in the generator in DFR to reduce the execution time and maintain the effect.
In the experiment, we used images of the gap between the pins of electronic components on the printed circuit board for training and validation. After the structure adjustment, the training methods improvement, adaptively adjusting the frequency of training between the generator and the discriminator and the adjustment of the abnormal score calculation, GANomaly-like DA increased the specificity from 87.57% to 99.98%, the recall from 86.73% to 100% and the AUCROC score from 0.96 to 1.0. By using some abnormal images to participate in training, GAN DFR light increased the specificity from 87.57% to 100%, the recall from 86.73% to 100%, and the AUCROC score from 0.96 to 1.0.
For the first proposed method of adaptively adjusting the frequency of training between the generator and the discriminator, we additionally collected images of damaged IC for experiments. By comparing the adversarial loss during training, we confirmed that for different data sets, adaptively adjusting the frequency of training between the generator and the discriminator can make the training of GAN more stable, and finally we obtained the results of the specificity of 99.75%, the recall of 100%, and the AUCROC score of 0.999. |
