GPT2からLM作成

[abePclWaseda]

What?

Why?

See also

[abePclWaseda]

• /egs2/librispeech_100/asr1/conf/tuning/train_asr_conformer_lr2e-3_warmup15k_amp_nondeterministic.yamlのmodel_confの部分の引数が、/espnet2/asr/espnet_model.pyに対応する。
• sym_sos: str = "<sos/eos>", sym_eos: str = "<sos/eos>",となっているが、GPT2の最初と最後を示すトークンは、どちらも<|endoftext|>
• 本来なら(丁寧なコードなら)model_confの部分の引数にsym_sos: <|endoftext|> sym_eos: <|endoftext|>を追加する必要があるが、今回は奇跡的に、vocab_size - 1のものを使用するという風になっていて、<|endoftext|>が指定されていた（笑）

[abePclWaseda]

Shallow Fusionなし

RESULTS

Environments

• date: Tue Oct 22 21:32:20 JST 2024
• python version: 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:47:35) [GCC 12.3.0]
• espnet version: espnet 202402
• pytorch version: pytorch 1.13.1
• Git hash: c3e3de659293976124a3c19eb94d9b207f485b16
  • Commit date: Fri Oct 18 23:32:44 2024 +0900

exp/asr_train_asr_conformer_lr2e-3_warmup15k_amp_nondeterministic_raw_en_hugging_face_openai-community-gpt2_sp

WER

dataset	Snt	Wrd	Corr	Sub	Del	Ins	Err	S.Err
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_clean	2703	54402	94.2	5.5	0.4	1.1	6.9	56.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_other	2864	50948	84.3	14.2	1.5	2.2	17.9	81.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_clean	2620	52576	94.0	5.5	0.5	1.1	7.0	57.9
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_other	2939	52343	84.3	14.1	1.6	2.1	17.8	81.0

CER

dataset	Snt	Wrd	Corr	Sub	Del	Ins	Err	S.Err
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_clean	2703	288456	98.1	1.2	0.7	0.9	2.8	56.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_other	2864	265951	93.4	4.1	2.5	2.1	8.7	81.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_clean	2620	281530	98.1	1.1	0.8	0.9	2.7	57.9
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_other	2939	272758	93.4	4.0	2.6	1.9	8.5	81.0

TER

dataset	Snt	Wrd	Corr	Sub	Del	Ins	Err	S.Err

[abePclWaseda]

shallow fusionあり（成功っぽい）

RESULTS

Environments

• date: Thu Oct 24 15:29:02 JST 2024
• python version: 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:47:35) [GCC 12.3.0]
• espnet version: espnet 202402
• pytorch version: pytorch 1.13.1
• Git hash: e54a7f26b452d7f621ba71b2e4b575c8a38e7737
  • Commit date: Wed Oct 23 14:25:22 2024 +0900

exp/asr_train_asr_conformer_lr2e-3_warmup15k_amp_nondeterministic_raw_en_hugging_face_openai-community-gpt2_sp

WER

dataset	Snt	Wrd	Corr	Sub	Del	Ins	Err	S.Err
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_clean	2703	54402	94.7	5.0	0.4	1.0	6.3	53.9
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_other	2864	50948	85.6	13.0	1.4	2.1	16.5	78.5
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_clean	2620	52576	94.6	5.0	0.4	1.0	6.4	55.0
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_other	2939	52343	85.4	13.1	1.5	2.0	16.6	78.7
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/dev_clean	2703	54402	94.2	5.5	0.4	1.1	6.9	56.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/dev_other	2864	50948	84.3	14.2	1.5	2.2	17.9	81.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/test_clean	2620	52576	94.0	5.5	0.5	1.1	7.0	57.9
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/test_other	2939	52343	84.3	14.1	1.6	2.1	17.8	81.0

CER

dataset	Snt	Wrd	Corr	Sub	Del	Ins	Err	S.Err
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_clean	2703	288456	98.2	1.1	0.7	0.8	2.6	53.9
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/dev_other	2864	265951	93.8	3.8	2.3	1.9	8.1	78.5
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_clean	2620	281530	98.3	1.0	0.7	0.8	2.5	55.0
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc.ave/test_other	2939	272758	93.8	3.7	2.5	1.8	8.0	78.7
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/dev_clean	2703	288456	98.1	1.2	0.7	0.9	2.8	56.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/dev_other	2864	265951	93.4	4.1	2.5	2.1	8.7	81.4
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/test_clean	2620	281530	98.1	1.1	0.8	0.9	2.7	57.9
decode_asr_lm_lm_train_transformer_gpt2_en_hugging_face_valid.loss.ave_asr_model_valid.acc_1022.ave/test_other	2939	272758	93.4	4.0	2.6	1.9	8.5	81.0

TER

dataset	Snt	Wrd	Corr	Sub	Del	Ins	Err	S.Err