Training and Labeling for My Custom Dataset

[baokhanhtran]

Hello Rémi Pautrat
Because I am a beginner in the Deep Learning field, I am so sorry if I bother you. I am facing some challenge with training my dataset, leading to that I really need your help for my problem:
I have 10.000 RGB Images and 10.000 files containing keypoints which were detected by SuperPoint (x,y coordinates). My files are saved in “.txt” format. Now I have 10.000 IR Images which were aligned with 10.000 RGB Images. Now I want to using keypoints (on RGB images) with IR Images to train again for IR images:

1. Is it possible to train?
2. Do I need to follow all the steps?
3. What format should I change for my label file? Or can I use the “.txt” file?
4. How can I import my label file?
5. Do I need to resize my image when training?

Thank you so much for reading.

[rpautrat]

Hi, yes you can do your training on IR images with your existing keypoints. But you need to change a few things.

You need to convert your .txt files to the format we use in this repo: .npz files, one per image, which contains a dict {'points': np.ndarray}, where the ndarray is a num_keypoint x 2 array (in row-col convention, not x-y). These files must be present in a folder $EXPER_DIR/outputs/your-folder and later in your training config file you need to specify this dataset in the field data->labels.

Once this is done, you can just do step 6 to train directly the detector and descriptors.

And you may need to resize your images indeed. SuperPoint was trained on images 240x320. I recommend training on a similar size if possible. In that case, you need to also resize the keypoints accordingly.