ImageNet 21k based filtered dataset

[isidentical]

Image-based filtering. We select a subset of examples whose visual content overlaps with ImageNet
classes. After applying English language (fasttext) and caption length filtering, we cluster the
image embeddings extracted by the OpenAI ViT-L/14 model for each image into 100K groups using
Faiss [ 75]. We then find the nearest neighbor group for every ImageNet training example, and keep
examples belonging to these groups. We apply this procedure using either ImageNet-21K (14M
images) or ImageNet-1K (1.2M images), forming two subsets.

In the paper, regarding the composition of "Image filters", it mentions that either ImageNet-21K or ImageNet-1K can be used. Looking into the code however, especially for the Datacomp 1B, it looks like only IN1K is used. Is there a version of the Datacomp 1B with IN21K?

[sagadre]

Hi @isidentical, thanks for the questions! In our scaling experiments we scaled both the IN1k and IN21k strategies up to the large pool (filtering 1.28B samples). Looking at Table 27 in the paper and comparing rows Image-based clustering (ImageNet1k) and Image-based clustering (ImageNet21k), we noticed average performance of 0.481 vs. 0.471. Hence, we only scaled up the IN1k strategy to the xlarge pool (filtering 12.8B samples). Unfortunately we don't have a IN21k version of DataComp 1B on hand