Skip to content

Color inversion, black image and nan in loss after ~20 epochs #6

@victor-felicitas

Description

@victor-felicitas

I've tried to train the model on original summer2winter_yosemite dataset.
After ~20 epochs all sample images turned completely black, and all all loss parameters turned to nan.
However, the model continued to run for 30 more epochs regularly saving checkpoints until I stopped it.

I've also used another, my own dataset, and it ran correctly for 70 epochs at least, unfortunately the only result I had was color inversion of images.
Any advice on changing training parameters (I used default)?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions