Label Errors in ImageNet-O Eval Set

[vishaal27]

Hey, while going through some of the images in the ImageNet-O evaluation dataset, I found quite a few mislabeled images. For example:

Here the soap dispenser, manhole cover, abacus, metal nail and abacus are mislabeled.

Here the umbrella, brocolli, punching bag and binoculars are mislabeled

If my understanding is correct from reading the original paper, ImageNet-O was created not for classification, but for OOD detection. Hence, I wonder if this difference in intent is causing these issues, and hence it is not really a suitable dataset for classification.

Please let me know your thoughts. Thanks!