| 摘要: | 現今沉浸式媒體發展越來越流行，包括 VR、MR等，這些應用的資料源都須經過影像拼接，影像拼接的成果將直接影響使用者觀看體驗，因此拼接影像的品質評估可使拼接時能更有效的獲取拼接效能。關於採用深度學習的拼接全景影像品質評估，現有的公開資料集中沒有大型人工標註數據集，蒐集資料也需要很高的成本，且無參考畫面拼接影像品質評估 (Blind Stitched Image Quality Assessment, BSIQA) 較為符合實際應用。因此，本論文提出以弱監督式學習 (weakly supervised learning) 進行失真偵測 (distortion detection)，其可在資料量少的狀況下取得更多拼接全景影像的拼接失真特徵，以提升整體品質評估效能。此外，針對拼接場景影像以場景資料集作為品質評估下游任務的預訓練資料集，增加拼接場景影像特徵抽取能力。最後，本論文也進行模型壓縮，在品質評估效能提升的同時，將網路以人工設計方式進行壓縮並重新訓練，使得網路模型可以在嵌入式系統中進行即時的品質評估運算。本論文所提之模型壓縮後方案與現有最好方案 DLNR-SIQA 比較，於 ISIQA 資料集的 Spearman 排序相關係數 (Spearman Rank Order Correlation Coefficient, SROCC) ，比 DLNR-SIQA 高 0.0226，Pearson 相關係數 (Pearson Linear Correlation Coefficient, PLCC) 高 0.0149，正規化均方根誤差 (Normalized Root Mean Square Error, NRMSE) 低 0.2566，因此在時間複雜度及評估準確度皆優於現有其他方案。;Nowadays, the immersive media is more and more popular, including VR, MR, etc. The data sources of these applications must be image stitched. Image stitching result will directly affect the user’s viewing experiment. Therefore, the quality assessment of stitched image can make stitching performance effectively to obtain. About Nowadays, the immersive media is more and more popular, including VR, MR, etc. The data sources of these applications must be applied image stitching. Since the quality of stitched images directly affects users’ viewing experiences, the quality assessment of stitched image can contribute to the performance of image stitching. Regarding the stitched panoramic image quality assessment using deep learning, there are no public large-scale human annotated datasets. Data collection also requires high costs. In practical, Blind Stitched Image Quality Assessment (BSIQA) is more suitable for real-world applications. Hence, this thesis proposes a distortion detector using weakly supervised learning. It can obtain more stitching features of stitched panoramic images with a small amount of training samples, and it can improve the overall performance of quality assessment. In addition, for the stitched scene images, the scene dataset is used as the pretraining dataset for the downstream task of quality assessment, which can improve extracted features of stitched scene images. Finally, this thesis also performs model compression. The model is manually designed to be compressed and retrained while the accuracy of image quality assessment is kept. Accordingly, real-time quality assessment in embedded systems can be achieved. Compared with the state-of-the-art scheme DLNR-SIQA on the ISIQA dataset, the proposed scheme outperforms DLNR-SIQA on the Spearman Rank Order Correlation Coefficient (SROCC) by 0.0226, Pearson Linear Correlation Coefficient (PLCC) by 0.0149, and the Normalized Root Mean Square Error (NRMSE) by -0.2566. To sum up, the time complexity and evaluation accuracy of the proposed scheme are better than those of existing schemes. |
