On WizardCoder and MetaMath models

[ghostchoir]

Hi,

Thank you for sharing your codebase. It helps me with developing a QAT method a lot!

However, I'm having trouble reproducing the results on WizardCoder and MetaMath models.

For WizardCoder-7b, it seems there are only 2 available repos for now:
https://huggingface.co/TheBloke/WizardCoder-Python-7B-V1.0-GGUF
https://huggingface.co/TheBloke/WizardCoder-Python-7B-V1.0-GPTQ

For MetaMath-7b, I was able to find multiple versions:
https://huggingface.co/meta-math/MetaMath-7B-V1.0
https://huggingface.co/meta-math/MetaMath-Llemma-7B
https://huggingface.co/meta-math/MetaMath-Mistral-7B

Could you clarify which versions are used in the experiments?

Many thanks for considering my request.

[DD-DuDa]

Thank you very much for your interest in our work.

The models we used at the time, WizardCoder and MetaMath, were available in various versions on Huggingface.

However, unfortunately, it seems that they have been removed by the authors for some unknown reason. You might want to consider using other models in the same field, such as CodeLlama.

[mustavikhan05]

Hi,

I'm running experiments using the gpt-2 models to test your hypothesis about architecture matching.

However, I continue to get an error in the training process.

The model path I pass in the bash script is the gpt-large model. My student model is gpt-medium so i hard code the path to the student model into train.py. The error I continue to encounter is the following:

RuntimeError: linalg.vector_norm: Expected a floating point or complex tensor as input. Got Long

Could you clarify where this might be originating from?