All submissions fail after uploading CodaLab competition bundle to CodaBench

[gstrag]

Hello,

I'm having a hard time fixing this issue. There are no output messages apart from this "Execution Time Limit exceeded. Limit was 600 seconds" in the stderr of the Scoring Logs.

I've looked at the existing issues but I couldn't find anything like this. I can provide more information about the specific competition if needed.

Any help would be greatly appreciated. Thanks.

[gstrag]

Actually, I'm trying to move two competitions, and the same issue happens for both.

[ihsaan-ullah]

Hi @gstrag

Can you please share your bundle and competition link with us. If you don't want to share them here you can send these in an email to info@codabench.org mentioning the issue number in the email

[gstrag]

Hi @ihsaan-ullah I just sent an email as requested. Thank you!

[Didayolo]

Hi @gstrag, have you received an answer to your email? I can't find it in my mailbox.

[gstrag]

Hi @Didayolo

I haven't received any answer. I sent the email again to info@codabench.org. Thank you

[ihsaan-ullah]

Hi @gstrag sorry for the delay.

It looks like the metadata file in your scoring program is problematic.

Here is what you have in you metadata file

command: python $program/auc.py $input $output $hidden
description: Example competition evaluation program.

After changing the command to the following I managed to not get Time out error but some other errors that you can fix easily.

command: python3 auc.py $input $output

---

Note: I am not sure if the problem comes from python version i.e. using python3 instead of python fixes the problem OR it comes from how the file is executed i.e. auc.py vs $program/auc.py ---- TO BE INVESTIGATED.

We may also want to run some tests on other CodaLab bundles if they have this problem or not and compare with this bundle.