(Feature) Add the ability to use other optimizers and LRScheduler

[PonteIneptique]

See #76

[emanjavacas]

Hey. The changes to config are cluttering a bit the code. We should group lr-related parameters. I could take care of that. But I am more concerned with the inclusion of this new dependency for the optimizers. CosineAnnealing is already implemented in pytorch, so what's all that additional stuff contributing? We should strive to keep things as simple as possible and only bring in new deps and functionality if it's really needed, that's why I am asking.

[PonteIneptique]

The new dependency is mostly here to bring other optimizers. I tested Ranger but there are many more. It did not yield better results though. But it provides space for researching on this front. Does that answer ?
I agree on grouping LR Related parameters, I decided to not add a sub-dictionary because I was not sure which way you'd like to go. :)

[emanjavacas]

Alright. Then I'd keep that outside of main, since it's not clear whether it's useful or not.

…

On Sun, Dec 6, 2020 at 8:28 PM Thibault Clérice ***@***.***> wrote: The new dependency is mostly here to bring other optimizers. I tested Ranger but there are many more. It did not yield better results though. But it provides space for researching on this front. Does that answer ? I agree on grouping LR Related parameters, I decided to not add a sub-dictionary because I was not sure which way you'd like to go. :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPIPI4RTZUTVUFI6LBIW6LSTPLOHANCNFSM4UPO3VSQ> .

-- Enrique Manjavacas

[PonteIneptique]

No problem, there is also nothing urgent here (as everything is touching only the Trainer, not the deployed model).
I am currently running same kind of test on a bigger corpus. Can we decided how how the parameters should be shown until then ?

[PonteIneptique]

Closed in favor of #79