Hi, a little bit confuse about your code, please give me some help.

[tbwxmu]

I noticed that they use the "Save and Load Checkpoints" to synchronize all models in different process in the PyTorch tutorial https://pytorch.org/tutorials/intermediate/ddp_tutorial.html

So, I want to know if there are some implicit synchronization mechanisms in your distributed_tutorial code.

[LearnedVector]

hello @tbwxmu did you figure this out?

[linminhtoo]

Anyone has a solution for this?