在深度學習訓練中，Dropout 和 DropConnect 是常被用來解決過度 擬合的正則化技術，Dropout 和 DropConnect 藉由在訓練過程中以一個 固定機率隨機地捨棄神經元及該神經元前後的連結，使得每個神經元彼 此之間不會過度依賴其他神經元，進而提高模型泛化的能力。 本文提出了一種新模型 Gradient DropConnect，它利用每個權重和 偏差的梯度以確定它們在訓練期間的下降捨棄機率。我們進行了一連串 的實驗以驗證這種方法可以有效地緩解過度擬合。;Dropout and DropConnect are regularization techniques often used to address the overfitting issue in deep learning. Dropout and DropConnect randomly discard neurons or links with a fixed probability during training so that each neuron does not depend too much on other neurons, thereby improving the model’s generalization ability. This paper proposes a new model, Gradient DropConnect, which leverages the gradient of each weight and bias to determine their dropping probabilities during training. We conducted thorough experiments to validate that such an approach can effectively mitigate overfitting.
