Final Project

Work by: Aditya R

Goal of the Project

The aim of the project is to improve the val_iou_score of a given model that performs image segmentation to analyze the surface of the moon, by tuning it's architecture and hyperparameters.

Dataset Used

For this project, the Artificial Lunar Landscape dataset has been used.
Link to the Kaggle Dataset

🤞Attempts

Preliminary Attempt

No Changes made to any parameters!

Model History:

val_iou_score Trend:

---

Attempt 1

Noticed that the original shape of the image is 480 x 720, but the original notebook was resizing the image to 256 x 256. By reducing the size of the image, many features in the image are lost. This is why I changed the height and width of the image to the original image's height and width.
Parameters Updated:

1. Image Height = 480
2. Image Width = 720

Model History:

val_iou_score Trend:

Notes: In only 5 epochs, it seems to have lower score than the original notebook.

---

Attempt 2

Since the number of epochs is too low, tried increasing the number of epochs to 10 to check for improvement.

Parameters Updated:

1. Epochs = 10

Model History:

val_iou_score Trend:

Notes: Increasing the epochs have shown improvements in the model's val_iou_score.

---

Attempt 3

Decreasing the batch size to 8 in order to have the model train on more images on each epoch. Increased the number of epochs to 20 to check the effect.
Parameters Updated:

1. Batch Size = 8
2. Epochs = 20

Model History:

val_iou_score Trend:

Notes: Decreasing the batch size does create a noticeable improvement in the model's val_iou_score.

---

Attempt 4

Transfer learning using VGG16 model: Since the traditional method of manually creating tensorflow layers and building a UNet seems to result in lower val_iou_scores, we can try incorporate Trasfer learning using the VGG16 pretrained model.
Since the VGG16 model accepts only square images as input, input image shape was changed to (480, 480, 3).
Parameters Updated:

1. Batch Size = 8
2. Epochs = 20
3. Image Height = 480
4. Image Width = 480
5. Activation: 'softmax'
6. Loss:'categorical_crossentropy'
7. Optimizer: Adam
8. encoder_weights: imagenet

Model History:

val_iou_score Trend:

Notes:
Transfer learning has definitely improved the val_iou_score of the model, but the score fluctuates a lot.

---

Attempt 5

Adjusting the learning rate in order to lower the fluctuation in val_iou_score.
Parameters Updated:

1. Learning Rate = 5e-5
2. Epochs = 15

Model History:

val_iou_score Trend:

Notes: The model's val_iou_score seems to come down quickly after reaching a large value. This could be because we are training the model with too many images in each epoch.

---

Attempt 6

Adjusting the batch size to 16 to reduce overfitting.
Parameters Updated:

1. Batches = 16

Model History:

val_iou_score Trend:

Notes: Overfitting seems to have been reduced, but the val_iou_score of the model fluctuates a lot. This may be because the optimizer is not controlled by proper parameters.

---

Attempt 7

Changing the optimizer to RMSprop to try to control the fluctuations in the model's val_iou_score. Parameters Updated:

1. Optimizer = tf.keras.optimizers.RMSprop(lr)

Model History:

val_iou_score Trend:

Notes: The model's val_iou_score seems to get stable at 0.939 with higher epochs.

---

Attempt 8

Trying to decrease the number of epochs to check if model stabilizes at 0.939 val_iou_socre.
Parameters Updated:

1. Epochs = 10 Model History:
   

val_iou_score Trend:

Notes: The model's val_iou_score is stable even in lower epochs.

Final Conclusion

From this experiment we can conclude that by changing the following parameters, we can develop a segmentation model for lunar surface images with a val_iou_score of about 0.939:

1. Image Height: 480
2. Image Width: 480
3. Epochs: 10 / 15 / 20
4. Batch Size: 16
5. Learning Rate: 5e-5
6. Incorporating Trasfer learning with VGG16 Pretrained model.
7. Activation: 'softmax'
8. Loss:'categorical_crossentropy'
9. Optimizer: Root Mean Squared Propogation
10. encoder_weights: imagenet