Version of lm_eval

[LeoYorkson]

hello， nice work! I want to know what's the version of lm-eval you use cause the latest version doesn't support python 3.9+. Hope your kind reply~

[pixeli99]

Hi, thanks for following. I was using 0.4.3 at the time.

However, evaluation and training are relatively independent. You can also create a new environment to use the latest lm eval.

:)

[LeoYorkson]

Thank you for your reply. I still couldn't reproduce your results, however :(
I think something wrong in the version of package and model I used. Could you please provide sufficient info about the details.
By the way, I found that the results between LISA and you paper are quite different. eg, MT-Bench, GSM8K,WinoGrande, yours seems better. However, my reproduction were much lower. What causes this, implementation or the package version.
Finally, even I set batch size =1 and activate layers = 2, the memory usage of LISA still achived 44G+.
I'm new in this area so I just use the scripts you author gave and I failed to figure out the reason :(
Anticipating your kind reply~