Question of inference

[huangjia2019]

Hi mkshing, you mention "In the inference phase, you need to input ids to the model by using model.forward() so that you cannot use model.generate() attribute. ". Possible to elaborate more on this part? I dont really know the difference of :

1. using model.forward(input_ids)
2. using model(input_ids)

Thanks for help

[mkshing]

Hi! I believe there’s actually no difference because model() redirects to model.forward() eventually.