Getting output of "ValleyLlamaModel" nan which causes loss:0.0, learning_rate:0.0

[gauravsh0812]

Hi there!

I am working with my custom dataset that has images and videos along with their corresponding json file. When I am trying to train them using LLama-7B weights, all I am getting {'loss': 0.0, 'learning_rate': 0.0, 'epoch': 0.01}.
Can anyone please help me with this?
FYI, my dataset looks like this: (the value of "human" conversation is -- question follows by \n image. I am not able to write here. It is not showing the image part hence mentioning it here!)

[
{
"id": 0,
"image": "aon_213_60sec_part1.jpg",
"conversations": [
{
"from": "human",
"value": " question...\n "
},
{
"from": "gpt",
"value": " answer ..."
}
]
},
...
...
...
]