Multilingual Model and non-MLM Model Fine-tuning

[kimwongyuda]

Hi, thanks for the amazing works, SPLADE.

I have some questions when fine-tuning SPLADE.

First, it works well when fine-tuning bert model using MSMARCO data.
However, if the pre-trained model is changed to multilingual-bert model, it doesn't work.

Second, when i fine-tune deberta-v3 model that is non-MLM model such as ELECTRA instead of bert model, it doesn't work either.

Do you have any expected problems with these two questions?