Skip to content
/ SiamSeg Public

[Official Repo] SiamSeg: Self-Training with Contrastive Learning for Unsupervised Domain Adaptation Semantic Segmentation in Remote Sensing

License

Apache-2.0 license
12 stars 3 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

woldier/SiamSeg

BranchesTags

Repository files navigation

SiamSeg: Self-Training with Contrastive Learning for Unsupervised Domain Adaptation Semantic Segmentation in Remote Sensing


Framework
Prediction results of our proposed method.


SiamSeg
Contrastive Learning Cross-domain Different
Network Architectural and Cross-domain Image Different.

News:

  • [2024/10/17] ✨✨The arxiv paper is available.
  • [2024/10/13] 🔥🔥Update the code and scripts.

1. Creating Virtual Environment

Install the necessary dependencies:

torch=1.10.2,torchvision=0.11.3 and mmcv-full=1.5.0.

Recommended use of conda virtual environments

conda create -n SiamSeg python==3.8 -y
conda activate SiamSeg

pip command to install torch && torchvision && mmcv-full

pip install torch==1.10.2+cu111 torchvision==0.11.3+cu111 -f https://download.pytorch.org/whl/torch_stable.html 
pip install mmcv-full==1.5.0 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.10.0/index.html
pip install kornia matplotlib prettytable timm yapf==0.40.1

for CN user:

pip install torch==1.10.2+cu111 -f https://mirror.sjtu.edu.cn/pytorch-wheels/cu111/?mirror_intel_list
pip install torchvision==0.11.3+cu111 -f https://download.pytorch.org/whl/torch_stable.html 
pip install mmcv-full==1.5.0 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.10.0/index.html
pip install kornia matplotlib prettytable timm yapf==0.40.1

Installation of the reference document refer:

Torch and torchvision versions relationship.

Official Repo CSDN

Version relationship of mmcv and torch.

MMCV

2.Preparation of data sets

We selected Postsdam, Vaihingen and LoveDA as benchmark datasets and created train, val, test lists for researchers.

2.1 Download of datasets

ISPRS Potsdam

The Potsdam dataset is for urban semantic segmentation used in the 2D Semantic Labeling Contest - Potsdam.

The dataset can be requested at the challenge homepage. The '2_Ortho_RGB.zip', '3_Ortho_IRRG.zip' and '5_Labels_all_noBoundary.zip' are required.

ISPRS Vaihingen

The Vaihingen dataset is for urban semantic segmentation used in the 2D Semantic Labeling Contest - Vaihingen.

The dataset can be requested at the challenge homepage. The 'ISPRS_semantic_labeling_Vaihingen.zip' and 'ISPRS_semantic_labeling_Vaihingen_ground_truth_eroded_COMPLETE.zip' are required.

LoveDA

The data could be downloaded from Google Drive here.

Or it can be downloaded from zenodo, you should run the following command:

cd /{your_project_base_path}/SiamSeg/data/LoveDA

# Download Train.zip
wget https://zenodo.org/record/5706578/files/Train.zip
# Download Val.zip
wget https://zenodo.org/record/5706578/files/Val.zip
# Download Test.zip
wget https://zenodo.org/record/5706578/files/Test.zip

2.2 Data set preprocessing

Place the downloaded file in the corresponding path The format is as follows

SiamSeg/
├── data/
│   ├── LoveDA/
│   │   ├── Test.zip
│   │   ├── Train.zip
│   │   └── Val.zip
├── ├── Potsdam_IRRG_DA/
│   │   ├── 3_Ortho_IRRG.zip
│   │   └── 5_Labels_all_noBoundary.zip
├── ├── Potsdam_RGB_DA/
│   │   ├── 2_Ortho_RGB.zip
│   │   └── 5_Labels_all_noBoundary.zip
├── ├── Potsdam_IRRG_DA/
│   │   ├── ISPRS_semantic_labeling_Vaihingen.zip
│   │   └── ISPRS_semantic_labeling_Vaihingen_ground_truth_eroded_COMPLETE.zip
  • Potsdam
python tools/convert_datasets/potsdam.py data/Potsdam_IRRG/ --clip_size 512 --stride_size 512
python tools/convert_datasets/potsdam.py data/Potsdam_RGB/ --clip_size 512 --stride_size 512
  • Vaihingen
python tools/convert_datasets/vaihingen.py data/Vaihingen_IRRG/ --clip_size 512 --stride_size 256
  • LoveDA
cd data/LoveDA
unzip Train.zip, Val.zip, Test.zip

3.Training

3.1 Preparation of pre-trained models

mit_b5.pth : We provide a script mit2mmseg.py in the tools directory to convert the key of models from the official repo to MMSegmentation style.

python tools/model_converters/mit2mmseg.py ${PRETRAIN_PATH} ./pretrained

Or you can download it from google drive.

The structure of the file is as follows

SiamSeg/
├── pretrained/
│   ├── mit_b5.pth (needed)
│   └── ohter.pth  (option)

3.2 Potsdam IRRG to Vaihingen IRRG

tips

When using distributed training scripts under linux, you need to set the permissions of the training scripts due to the file permissions.

cd SiamSeg
chmod 777 ./tools/dist_train.sh
chmod 777 ./tools/dist_test.sh
# Potsdam IRRG to Vaihingen IRRG
# CUDA_VISIBLE_DEVICES Visible GPU ids are 0-3 Total four GPU processors
# PORT Sets the communication port of the master for distributed training.
# The last 4 indicates the total number of GPUs used
CUDA_VISIBLE_DEVICES=0,1,2,3  PORT=10985 \
 ./tools/dist_train.sh \
 configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_PotsdamIRRG_2_VaihingenIRRG.py 4

3.3 Potsdam RGB to Vaihingen IRRG

# Potsdam RGB to Vaihingen IRRG
# CUDA_VISIBLE_DEVICES Visible GPU ids are 0-3 Total four GPU processors
# PORT Sets the communication port of the master for distributed training.
# The last 4 indicates the total number of GPUs used
CUDA_VISIBLE_DEVICES=0,1,2,3  PORT=10985 \
 ./tools/dist_train.sh \
 configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_PotsdamRGB_2_VaihingenIRRG.py 4

3.4 Vaihingen IRRG to Potsdam IRRG

# Potsdam IRRG to Vaihingen IRRG
# CUDA_VISIBLE_DEVICES Visible GPU ids are 0-3 Total four GPU processors
# PORT Sets the communication port of the master for distributed training.
# The last 4 indicates the total number of GPUs used
CUDA_VISIBLE_DEVICES=0,1,2,3  PORT=10985 \
 ./tools/dist_train.sh \
 configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_VaihingenIRRG_2_PotsdamIRRG.py 4

3.5 Vaihingen IRRG to Potsdam RGB

# Potsdam IRRG to Vaihingen IRRG
# CUDA_VISIBLE_DEVICES Visible GPU ids are 0-3 Total four GPU processors
# PORT Sets the communication port of the master for distributed training.
# The last 4 indicates the total number of GPUs used
CUDA_VISIBLE_DEVICES=0,1,2,3  PORT=10985 \
 ./tools/dist_train.sh \
 configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_VaihingenIRRG_2_PotsdamRGB.py 4

3.6 LoveDA Rural to Urban

# Potsdam IRRG to Vaihingen IRRG
# CUDA_VISIBLE_DEVICES Visible GPU ids are 0-3 Total four GPU processors
# PORT Sets the communication port of the master for distributed training.
# The last 4 indicates the total number of GPUs used
CUDA_VISIBLE_DEVICES=0,1,2,3  PORT=10985 \
 ./tools/dist_train.sh \
 configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_Rural_2_Urban.py 4

4.Testing

4.1 Potsdam IRRG to Vaihingen IRRG

# for dist test
CUDA_VISIBLE_DEVICES=4,5,6,7  PORT=10985 \
  sh tools/dist_test.sh \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_PotsdamRGB_2_VaihingenIRRG.py  \
  {beast_model_path}  4 --eval mIoU mFscore

# for predict label save  
# launcher must set to 'none'
# opacity between 0 and 1
PYTHONPATH=$(pwd):$PYTHONPATH  CUDA_VISIBLE_DEVICES=0 \
  python tools/test.py \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_PotsdamRGB_2_VaihingenIRRG.py  \
  {beast_model_path} --eval mIoU mFscore --launcher none  --opacity 1.0

4.2 Potsdam RGB to Vaihingen IRRG

# for dist test
CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=10985 \
  sh tools/dist_test.sh \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_PotsdamRGB_2_VaihingenIRRG.py  \
  {beast_model_path}  4 --eval mIoU mFscore

# for predict label save  
# launcher must set to 'none'
# opacity between 0 and 1
PYTHONPATH=$(pwd):$PYTHONPATH  CUDA_VISIBLE_DEVICES=0 \
  python tools/test.py \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_PotsdamRGB_2_VaihingenIRRG.py  \
  {beast_model_path} --eval mIoU mFscore --launcher none  --opacity 1.0

4.3 Vaihingen IRRG to Potsdam IRRG

# for dist test
CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=10985 \
  sh tools/dist_test.sh \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_VaihingenIRRG_2_PotsdamIRRG.py  \
  {beast_model_path}  4 --eval mIoU mFscore
  
# for predict label save  
# launcher must set to 'none'
# opacity between 0 and 1
PYTHONPATH=$(pwd):$PYTHONPATH  CUDA_VISIBLE_DEVICES=0 \
  python tools/test.py \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_VaihingenIRRG_2_PotsdamIRRG.py  \
  {beast_model_path} --eval mIoU mFscore --launcher none  --opacity 1.0

4.4 Vaihingen IRRG to Potsdam RGB

# for dist test
CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=10985 \
  sh tools/dist_test.sh \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_VaihingenIRRG_2_PotsdamRGB.py  \
  {beast_model_path}  4 --eval mIoU mFscore
# for predict label save

# for predict label save  
# launcher must set to 'none'
# opacity between 0 and 1
PYTHONPATH=$(pwd):$PYTHONPATH  CUDA_VISIBLE_DEVICES=0 \
  python tools/test.py \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_VaihingenIRRG_2_PotsdamRGB.py  \
  {beast_model_path} --eval mIoU mFscore --launcher none  --opacity 1.0

4.5 LoveDA Rural to Urban

# for dist test
CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=10985 \
  sh tools/dist_test.sh \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_Rural_2_Urban.py  \
  {beast_model_path}  4 --eval mIoU mFscore

# for predict label save  
# launcher must set to 'none'
# opacity between 0 and 1
PYTHONPATH=$(pwd):$PYTHONPATH  CUDA_VISIBLE_DEVICES=0 \
  python tools/test.py \
  configs/SiamSeg/siamseg_daformer_sepaspp_mitb5_512x512_40k_Rural_2_Urban.py  \
  {beast_model_path} --eval mIoU mFscore --launcher none  --opacity 1.0

References

Many thanks to their excellent works

About

[Official Repo] SiamSeg: Self-Training with Contrastive Learning for Unsupervised Domain Adaptation Semantic Segmentation in Remote Sensing

Resources

Readme

License

Apache-2.0 license
Activity

Stars

12 stars

Watchers

1 watching

Forks

3 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages