利用輔助語句與BERT模型偵測詞彙的上下位關係;Hypernym and Hyponym Detection Based on Auxiliary Sentences and the BERT Model

NCUIR > College of Electrical Engineering & Computer Science > Software Engineer > Electronic Thesis & Dissertation > Item 987654321/86306

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/86306

Title:	利用輔助語句與BERT模型偵測詞彙的上下位關係;Hypernym and Hyponym Detection Based on Auxiliary Sentences and the BERT Model
Authors:	曾莊;Tseng, Chuang
Contributors:	軟體工程研究所
Keywords:	詞向量;BERT 語言模型;微調;上下關係
Date:	2021-08-10
Issue Date:	2021-12-07 12:29:53 (UTC+8)
Publisher:	國立中央大學
Abstract:	詞向量模型是一種利用文本的上下文關係產生詞彙相應之向量的技術。通常，我們可利用詞向量間的餘弦相似度來計算兩個詞彙間的相關程度。然而，我們卻難以利用詞向量來偵測兩個詞彙是否具備上位詞-下位詞的關係。另外，由於上下關係是一種不對稱的語義關係，即使給定一對具備上位詞-下位詞關係詞彙，我們也難以採用一般對稱的距離量度來決定何者為上位詞、何者為下位詞。本論文提出一個基於 BERT 預訓練語言模型搭配額外建構的輔助語句來判斷一對詞彙的上下關係，任務共分兩階段。階段一：判斷詞對是否具有上下關係。若階段一的結果為真，則進入階段二：判斷何者為上位詞，何者為下位詞。經過實驗，我們發現兩種建構輔助語句的方式：BERT+Q 和 BERT+Q+PosNeg 能有效地利用詞向量判斷階段一及階段二的任務。;The word embedding model is a technique that utilizes contextual words to generate a vector for each word, which is called word embedding. Usually, we can use the cosine similarity between a pair of word embeddings to calculate the relevance score between the two words. However, it is diffi cult to use word embeddings to detect the hypernym-hyponym relationship between two words. In addition, being an asymmetric semantic relation ship, even when given a pair of vocabularies with a hypernym-hyponym relationship, it is challenging to apply general distance measures, which are often symmetric, to determine which is the hypernym and which is the hyponym. This thesis proposes a model based on a BERT pre-trained model with auxiliary sentences to determine the hypernym-hyponym relationship of a pair of words. The entire process is consisted of two tasks. First, when given a pair of words, the model determines whether the word pair has a hypernym-hyponym relationship. Then, if the result is true, the model pro ceeds to the second task: distinguishing the hypernym and the hyponym. Experimental results show that two approaches to construct auxiliary sen tences, BERT+Q and BERT+Q+PosNeg, can effectively accomplish both tasks.
Appears in Collections:	[Software Engineer] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	227	View/Open

社群 sharing

Loading...