根據場域實際開發自動光學瑕疵檢測系統的經驗，瑕疵資料收集困難且標記耗時，經常成為系統開發上的時間瓶頸。為了盡早獲得足夠的資料開始訓練模型，資料擴增的手法是不可或缺的。然而，目前常見的擴增方法：基礎擴增方法（平移、旋轉等）以及大部分使用深度學習的擴增方法並未考慮這些擴增出來的影像是否真實存在，直接使用這些方法可能會產生不符合場域知識的資料。因此，我們希望能開發出一個生產高品質影像、且能使用場域知識進行控制的影像擴增系統。 我們準備了像素標記的生成指引，以此將場域知識加入系統中。由於少量訓練資料的限制，直接使用生成指引影像、套用經典GAN模型Pix2Pix的結果並不理想。在廣泛閱讀文獻後我們獲得啟發，提出分層式影像生成系統（GLC），將使用的電子零件影像分為三層三個元件，融合Rule-based與GAN，從元件庫中提取各層元件、按照領域知識Rule-based影像組合器組合，最後以GAN-based的深度學習修改器進行各層元件邊界處修正。 實驗結果顯示，與Baseline模型Pix2Pix相比，GLC在訓練深度學習模型時可減少79%的訓練資料，且生成出來的影像品質較Pix2Pix更為真實，使用了GLC擴增的分類器與沒擴增時相比，Error rate相對改善率更可達97%。 ;From the actual factory experience of Automatic Optical Inspection System (AOI System) development, we know that defect data is hard to collect and need a lot of time to label, often makes project behind schedule. If we want to obtain enough data for training in the early stage of project, data augmentation technique is necessary. However, common data augmentation method like standard augmentation (flip, shift) and most of deep learning generative architecture do not consider that if the product data really exist in real world. Directly implement these method may produce data which do not meet domain knowledge. Therefore, we intend to develop a data augmentation system that can generate high quality images and products can be controlled with domain knowledge. Pixel-wise generative guide are prepared to provide domain knowledge for generation process. Due to limited data, the implementation result of famous GAN model Pix2Pix (use generative guide image as input condition) was unfavorable. After paper researching, we get inspiration and propose Generative Layer Combiner (GLC). A generative system which combine rule-based method and GAN. GLC will separate each image in the dataset into three layers, three components. Three components from component libraries will be combined by rule-based image combiner which follows domain knowledge. After rule-based combination, boundary image of each layer will be refined by GAN-based deep learning modifier. From the experiment results, we can see that compare to baseline model Pix2Pix, GLC can produce higher quality images and reduce 79% real data requirement when training deep learning model. Moreover, compare to no augmentation CNN classifier, CNN classifier training with GLC augmentation’s performance improve by reduce error rate with relative IMP of 97%.
