Are class percentages are maintained in train, val and test datasets?

[DivyenduDutta]

Currently train, val and test split is monolithic. See data_preprocess.py -> make_train_val_test_split()

If the split is so that class percentages are not maintained then thats not a good way to split the dataset.

First, we need to check if class percentages are maintained in the splits or not?

If not, then split the dataset into train, val and test so that the class percentages match those of the original full dataset.

[DivyenduDutta]

#11

Even though we did not explicitly try to ensure that the class percentages sync up when splitting into train, val, test datasets, it still matches up with the overall dataset. I think this is a result of the dataset being large enough.

See Analysis_And_Thoughts.md for the class percentage values of the full dataset, train, val and test datasets