Results are here. Forked from OpenAI's repo.
To generate the completions (after pip installing the requirements), run:
mkdir results
python run.py
Then to evaluate the completion results, run
pip3 install -e .
python -m human_eval.evaluate_functional_correctness results/name_of_results_file