| 摘要: | 深度學習 (deep learning) 技術已廣泛應用在各項領域中；近年來許
多行業都引入此項技術來提升作業上的效率及精密度，特別是與影像結
合應用的部份。在深度學習的訓練策略中，最為理想的方式就是監督式學
習，但現實生活中資料集的正負樣本的比例經常不平衡，或是資料量不充
足，導致監督式學習的效果不佳。早期人們會使用影像翻轉、旋轉等資料
擴增的方式來填補資料不充足的問題，但是此作法容易造就出真實情況
中不會發生或不合理的資料集，反而會誤導網路模型的學習方向。為了讓
資料擴增能夠更貼近現實資料集的狀況，生成模型 (generative model) 將會是一項重要的技術之一。
在瑕疵檢測任務當中，經常發生瑕疵樣本的資料量不足，所以通常網
路模型會使用半監督式或非監督式學習的方法來進行訓練。因此我們將
針對如何轉移瑕疵樣本進行相關研究，提出了瑕疵影像轉移的系統，在轉
移瑕疵影像時，可以針對影像中的瑕疵部位進行調整。
我們的網路模型改自於本實驗室優化過的 pix2pix 條件式生成對抗網
路，為了更好掌控轉移影像的瑕疵部位，我們重新定義輸入端的條件向
量，此條件向量第一維度的數值將會影響到瑕疵部位顏色亮暗的變化。我
們所提出的瑕疵轉移網路 (defect reproducing GAN, DRPGAN)，主要的改進有：i. 為訓練階段條件向量的第一維度設計瑕疵區域亮暗變化演算法，在測試階段可以透過此條件向量第一維度的數值去掌控影像中瑕疵區域
的亮暗變化；ii. 為測試階段條件向量的第一維度設計預設值演算法。
實驗中我們主要使用鍵盤按鍵的影像來進行訓練與測試，鍵盤按鍵
的影像共有 218 組影像，每組影像擁有非瑕疵影像、瑕疵影像與瑕疵遮
罩，資料集裡的非瑕疵影像均透過人工影像編輯修復瑕疵影像所得。為了
維持轉移影像整體的品質，我們的資料集不區分訓練集與測試集，將所有
資料一同進行訓練，在測試階段僅專注在瑕疵區域上的變化。在鍵盤按鍵
的影像當中，我們可以透過調整條件向量第一維度的數值達到控制影像
中瑕疵區域的亮暗變化，再搭配瑕疵遮罩來提供位置與形狀，將期望的瑕
疵轉移到非瑕疵影像上。
針對自行設計條件向量第一維度的數值演算法，我們額外蒐集了不
同類型的影像資料集進行訓練與測試。新類型資料集擁有 301 組影像，
每組影像擁有非瑕疵影像、瑕疵影像、瑕疵遮罩，其非瑕疵影像是與瑕疵
影像極其相似的背景。在測試結果中，我們可以觀察到不管是新資料集還
是舊資料集，條件向量第一維度數值的調整皆能夠明確的調整瑕疵區域
的亮暗變化。
最後，我們將這些瑕疵轉移影像提供給現有的辨識器 EfficientNet-b0
作為訓練樣本，讓辨識器在測試階段時，能夠將七至八成的真實瑕疵樣本
進行成功的分類，證實我們瑕疵轉移網路所轉移的瑕疵影像，可以作為瑕
疵影像資料擴增的來源之一。
;“Deep learning” is widely used in various application fields. In recent years, many industries have introduced this technology to improve efficiency and accuracy in operations, especially in the part of combining with images. In deep learning training strategies, the most ideal approach is supervised learning. However, in real life, the ratio of positive and negative samples in datasets is often unbalanced or the amount of data is insufficient, resulting in poor performance of supervised learning. In the early days, people used image flipping and rotation to expand data to solve the problem of insufficient data. However, this method is easy to create data sets that will not occur or are unreasonable in real situations, which will mislead the learning direction of network models. In order to make data expansion more close to the situation of real data sets, generative models will be one of the important technologies.
In the task of defect detection, it often happens that there is not enough data for defective samples. Therefore, network models usually use semi supervised or unsupervised learning methods for training. Therefore, we will focus on how to reproduce defective samples and propose a system for reproducing defective images. When reproducing images, we can adjust the defective parts of images.
Our network model is adapted from the optimized pix2pix conditional generative adversarial network in our laboratory. In order to better control the defective parts of the transferred image, we redefine the input conditional vector. The value of the first dimension of this conditional vector will affect the change in brightness of the defective part. The main improvements of our proposed defect transfer network (defect reproducing GAN, DRPGAN) are: i. The design of the brightness change algorithm for the first dimension of the conditional vector during the training phase, which can control the brightness change of the defective area in the image through the value of the first dimension of this conditional vector during the testing phase; ii. The design of a default value algorithm for the first dimension of the conditional vector during the testing phase.
In our experiment, we mainly use keyboard image to train and test. There are 218 groups of keyboard images in total. Each group has non-defective images, defective images and defective masks. Non-defective images in dataset are obtained by manual image editing and repairing defective images. In order to maintain the overall quality of reproduced images, our dataset does not distinguish between training set and test set. All data are trained together and only focus on changes in defective areas during testing phase. In keyboard images, we can adjust the numerical value of the first dimension of condition vector to control brightness change of defective areas in images and use defect masks to provide location and shape to transfer expected defects to non defective images.
For self-designed numerical value algorithm for condition vector’s first dimension, we also collected different types of image datasets for training and testing. The new type dataset has 301 groups of images with non-defective images similar to defective ones as background. In test results, we can observe that whether it is a new dataset or an old one, adjusting numerical value of first dimension can clearly adjust brightness change of defective areas.
Finally, we provide these reproduced defect images to existing recognizer EfficientNet-b0 as training samples so that recognizer can successfully classify 70-80% real defect samples during testing phase. This confirms that defect images reproduced by our defect transfer network can be used as one source for expanding defect image datasets. |
