What it the best hyper-parameter setting?

[NoOneUST]

Hello, should I use what kind of hyper-paramter for the first try? For example, learning rate, AdamW or Madgrad?

[lessw2020]

Hi @NoOneUST,
I would recommend just starting with no modifications - i.e. just run Ranger as it's designed to try and establish smart defaults.
After that, you can start adjusting, and would recommend learning rate as probably the first thing to adjust esp based on the initial training results.
Hope that helps!

[wassname]

I'll just add one example I've found. For timeseries, this repo uses a subset of ranger21 https://github.com/jdb78/pytorch-forecasting/blob/master/pytorch_forecasting/optim.py.

Make of it what you will ;p