Questiona about pretraining

[ZhenyuSun-Walker]

Hello Sir, thanks for your impressive work. I wonder how to correctly deal with the pretrain.py, since I met the problem shown as below:

Original Traceback (most recent call last):
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/jaxtyping/_decorator.py", line 522, in wrapped_fn
    return wrapped_fn_impl(args, kwargs, bound, memos)
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/jaxtyping/_decorator.py", line 449, in wrapped_fn_impl
    out = fn(*args, **kwargs)
  File "/home/sunzhenyu/Projects/SGDreamer/flowmap/flowmap/dataset/validation_wrapper.py", line 29, in __getitem__
    return next(self.dataset_iterator)
  File "/home/sunzhenyu/Projects/SGDreamer/flowmap/flowmap/dataset/dataset_merged.py", line 10, in __iter__
    remaining = [len(dataset) for dataset in self.datasets]
  File "/home/sunzhenyu/Projects/SGDreamer/flowmap/flowmap/dataset/dataset_merged.py", line 10, in <listcomp>
    remaining = [len(dataset) for dataset in self.datasets]
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/jaxtyping/_decorator.py", line 522, in wrapped_fn
    return wrapped_fn_impl(args, kwargs, bound, memos)
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/jaxtyping/_decorator.py", line 449, in wrapped_fn_impl
    out = fn(*args, **kwargs)
  File "/home/sunzhenyu/Projects/SGDreamer/flowmap/flowmap/dataset/dataset_re10k.py", line 176, in __len__
    return len(self.index.keys())
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/jaxtyping/_decorator.py", line 522, in wrapped_fn
    return wrapped_fn_impl(args, kwargs, bound, memos)
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/site-packages/jaxtyping/_decorator.py", line 449, in wrapped_fn_impl
    out = fn(*args, **kwargs)
  File "/home/sunzhenyu/Projects/SGDreamer/flowmap/flowmap/dataset/dataset_re10k.py", line 164, in index
    with (root / data_stage / "index.json").open("r") as f:
  File "/home/sunzhenyu/anaconda3/envs/SGDreamer/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: 'datasets/re10k/test/index.json'

How to fix the problem?

Besides, I dound that in the flowmap/config/pretrain.yaml, the setting of dataset is like

But what is the acid datasets?

Aniticipating for your earliest reply!