About the Data

[waynechu1021]

Thanks for your work!
I have a question about the data pipeline. In the Nav-COT-38K you released, I noticed that the image source for the data uses MP3D skybox images. As far as I know, this is inconsistent with the images rendered on the path. The usual practice is to render the corresponding observation images in the simulator based on the action sequence of the ground truth. I'm wondering if there are any implementation errors in this. In addition, I observed that your data includes Val-Unseen splits, but this wasn't mentioned in your article. Could this potentially lead to data leakage? Please correct me if I'm wrong.

[Believeht029]

Thanks for the thoughtful questions about Nav-COT-38K. We do start from the MP3D skybox renders, but the pipeline slices them into the same 60° perspective views the simulator would produce at each waypoint. The heading/elevation metadata in the action trace drives the sampling, so the images align with the ground-truth trajectory even though we begin from the panoramic source—there isn’t a mismatch with the executed path.
Regarding the Val-Unseen split: it’s packaged with the release purely for evaluation parity with prior work. None of those episodes are used in training or prompt generation, and our paper’s reported numbers come only from the standard train/test settings.
Please let me know if you spot anything else or want to see the sampling script.

[waynechu1021]

Thank you for your kind answer; I understand your point. However, I checked some examples in the dataset, and they don't match the actual rendered images in the simulator at all (for example the sample with episode_id = 1 in R2R train split). These skybox images don't seem to reflect continuous environmental changes. Furthermore, I noticed you seem to be using ground truth reference waypoints in R2R/RxR to calculate actions. This seems to simplify the task setup and is somewhat similar to navigation in previous discrete environment settings. I'm wondering if this conflicts with models like NaVILA that are trained on atomic actions?