想請教關於 Semantic-based graph 與 Syntactic-based graph 前處理的問題

[Annrison]

您好，不好意思打擾了，最近在嘗試用您的code建構語意和句法依存的GCN，想請問生成 mr_stan.pkl 和 mr_semantic_0.05.pkl 這兩個檔案的句子，有經過什麼樣的前處理呢？例如說像textGCN裡面會把數字轉成 "NUM"，連結轉成 "url "這種。

因為訓練的資料集較小，模型遲遲無法收斂，在想會不會是因為前處理不同的關係，或者可能會有其他的原因嗎？

def clean_str(string):
    """
    String cleaning
    """
    string = string.lower()
    string = re.sub(r"\n", " ", string)
    string = re.sub(r"\s{2,}", " ", string)
    string = re.sub(r""", " ", string)
    string = re.sub(r"(http://)?www\.[^ ]+", " _url_ ", string)
    string = re.sub(r"[^a-z0-9$\'_]", " ", string)
    string = re.sub(r"_{2,}", "_", string)
    string = re.sub(r"\'s", " \'s", string)
    string = re.sub(r"\'ve", " \'ve", string)
    string = re.sub(r"n\'t", " n\'t", string)
    string = re.sub(r"\'re", " \'re", string)
    string = re.sub(r"\'d", " \'d", string)
    string = re.sub(r"\'m", " \'m", string)
    string = re.sub(r"\'ll", " \'ll", string)
    string = re.sub(r"\$+", " $ ", string)
    string = re.sub(r"(?<=[a-z])I", " I", string)
    string = re.sub(r"(?<= )[0-9]+(?= )", "NUM", string)
    string = re.sub(r"(?<= )[0-9]+$", "NUM", string)
    string = re.sub(r"^[0-9]+(?= )", "NUM", string)
    string = re.sub(r"\s{2,}", " ", string)
    return string.strip()

[Annrison]

另外也想請問 BERT_based_similarity.py 這份檔案的問題

1. 字詞配對的問題
   想問在dependency graph 裡面，兩個字都會有 two order 的配對，為什麼在語意graph裡面只有一個順序呢？

2. 向量和字詞對應的問題
   裡面字詞 split_s 和向量 each互相對應，但bert有時候會將一個詞切開成兩段以上，讓兩邊長度會對不起來，如embedding切成 'em', '##bed', '##ding' 的情況，想請問這是刻意設計的嗎，還是這種誤差可以忽略不計呢？

for s in sentences:
    pbar.update(count)
    encoded_input = tokenizer(s, return_tensors='pt').to(device)
    output = model(**encoded_input)
    aa = torch.squeeze(output.last_hidden_state,dim=0)
    each = aa[1:-1,:]
    split_s = s.split()
    for word in range(0,len(split_s)):
      for another in range(0,len(split_s)):
        w1 = split_s[word]
        w2 = split_s[another]
        score = torch.cosine_similarity(each[word].view(1,-1),each[another].view(1,-1),dim=1)
        key = ''+w1+','+w2
        if key not in total_set.keys():
          total_set[key] = 1
        else:
          total_set[key]+=1
        if score.data>similarity_threshold:
          if key not in valid_set.keys():
            valid_set[key] = 1
          else:
            valid_set[key] += 1
    count +=1