KeyError: 'ref_junction_map'

[abhiagwl4262]

I get the following error when I try to train on wireframe dataset.
Wireframe doesn't have this key.
python -m sold2.experiment --mode train --dataset_config sold2/config/wireframe_dataset.yaml --model_config sold2/config/train_full_pipeline.yaml --exp_name sold2_full_v0.5.7 --pretrained --pretrained_path pretrained_model/ --checkpoint_name sold2_wireframe.tar

    junc_map = data["ref_junction_map"].cuda()
KeyError: 'ref_junction_map'

[rpautrat]

Hi, in your config file wireframe_dataset.yaml, did you make sure to have the 'return_type' set to 'paired_desc' and to set the 'gt_source_train' and 'gt_source_test' paths? These are necessary to be able to train the full pipeline, as mentioned in the steps of the ReadMe.

[abhiagwl4262]

@rpautrat I am sorry but there are so many terminologies in SOLDv2 work which is making it complex to adapt.

what is difference in "official" and "exported" dataset ?

Can I directly jump to step5 for training on my custom data ?

@rpautrat

step 1 is training model on synthetic which is not needed to do again because we already have trained model by you guys, am I correct ?
step 2 and 3 - Why should I create pseudo labels when I already have actual labels ?

[rpautrat]

You are right that Step 1 is not necessary if you use the pre-trained model.

The Wireframe dataset does have existing labels, this is what you would use if you would select the option gt_source_train= 'official'. But the point of SOLD2 is to not use these labels (which are biased towards wireframe lines only), and we instead propose to export a pseudo ground truth that does not have this bias. That's why Steps 2 and 3 are necessary.

I hope this is clearer now.

[abhiagwl4262]

I still get the same error when I run step 5 directly with gt_source_train="official" and gt_source_test="official" and also return_type: "paired_desc"

[abhiagwl4262]

When I try step 4 I get the following error -

  File "/home/ubuntu/abhishek/SOLD2/sold2/train.py", line 254, in train_single_epoch
    heatmap = data["heatmap"].cuda()
KeyError: 'heatmap'

I used the command -

python -m sold2.experiment --mode train --dataset_config sold2/config/wireframe_dataset.yaml --model_config sold2/config/train_detector.yaml --exp_name sold2_wireframe --pretrained --pretrained_path pretrained_model/ --checkpoint_name sold2_synthetic.tar

When I try step 2 I get the following error -

  File "/home/ubuntu/abhishek/SOLD2/sold2/export.py", line 238, in homography_adaptation
    images_warped = warp_perspective(input_images, H_tensor, (H, W),
TypeError: warp_perspective() got an unexpected keyword argument 'flags'

I used the command -
python -m sold2.experiment --exp_name wireframe_train --mode export --resume_path pretrained_model/ --model_config sold2/config/train_detector.yaml --dataset_config sold2/config/wireframe_dataset.yaml --checkpoint_name sold2_synthetic.tar --export_dataset_mode train --export_batch_size 4

[rpautrat]

Now that I remember, the code is actually not made to train on the official ground truth. We only used it to evaluate the line descriptors on these lines, but it is currently not compatible with training.

What you could do for step 4 would be to modify the return function of the wireframe dataloader

SOLD2/sold2/dataset/wireframe_dataset.py

Lines 548 to 559 in 3d7bcd6

	return {
	"image": to_tensor(image),
	"junctions": to_tensor(junctions).to(torch.float32)[0, ...],
	"junction_map": to_tensor(junction_map).to(torch.int),
	"line_map_pos": to_tensor(
	line_map_pos).to(torch.int32)[0, ...],
	"line_map_neg": to_tensor(
	line_map_neg).to(torch.int32)[0, ...],
	"heatmap_pos": to_tensor(heatmap_pos).to(torch.int32),
	"heatmap_neg": to_tensor(heatmap_neg).to(torch.int32),
	"valid_mask": to_tensor(valid_mask).to(torch.int32)
	}

If you modify the keys 'line_map_pos' to 'line_map' and 'heatmap_pos' to 'heatmap', it might work.

Step 5 with the 'paired_desc' option is not available at all currently. It would need additional changes to the dataloader.

But again, the goal of SOLD2 is to train without existing ground truth, so it doesn't make sense to use the official ground truth, which is not good for localization for example.

[rpautrat]

Regarding the error "TypeError: warp_perspective() got an unexpected keyword argument 'flags'", the issue is with your kornia version. We used an older one that had different arguments. Please consider using kornia==0.3.0 and it should solve it.

[abhiagwl4262]

Regarding the error "TypeError: warp_perspective() got an unexpected keyword argument 'flags'", the issue is with your kornia version. We used an older one that had different arguments. Please consider using kornia==0.3.0 and it should solve it.

I updated kornia==0.3.0 and Now it installed torch==1.5.0. Now its giving me error -

File "/home/ubuntu/anaconda3/envs/sold/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 78, in <module>
    @torch.jit._script_if_tracing
AttributeError: module 'torch.jit' has no attribute '_script_if_tracing'

Now I Installed torchvision==0.7.0 which installed torch 1.6.0. this setup is working now. Now I am able to run step 2

[abhiagwl4262]

@rpautrat Where do I give path to my custom data ?

[rpautrat]

Training on your own data is not supported for now, but several people did it by creating a new dataloader inspired by holicity_dataset.py.

More details in this issue: #19 (comment)