Skip to content

Fix wiggle lines in TB when experiment is resumed #22

@fvisin

Description

@fvisin

The model checkpoints and tensorboard events are not saved with the same frequency. When the model is reloaded and training resumes this causes the main loop to write new events to the event file that have the same global step (x-coordinate) as some previous event. As a result the TB graph looks weird. This can be fixed by means of using SessionLog messages as suggested in the documentation.

I don't have time to work on this right now, but I'd be happy to review a PR.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions