Skip to content

it is conceptually mistake to use log loss for performance comparison  #252

@Sandy4321

Description

@Sandy4321

Describe the bug
it is conceptually mistake to use log loss for performance comparison
since
1
lightgbm as all gmb models
( maybe except https://stanfordmlgroup.github.io/projects/ngboost/ NGBoost: Natural Gradient Boosting for Probabilistic Prediction
Tony Duan*, Anand Avati*, Daisy Yi Ding, Sanjay Basu, Andrew Ng, Alejandro Schuler )
are not good in prediction probabilities
2
then same should be carefully tested for deep NN

3
may you change provement that your code is performing , for example
https://github.com/manujosephv/pytorch_tabular/blob/main/examples/PyTorch%20Tabular%20with%20Bank%20Marketing%20Dataset.ipynb
to compare actual confusion matrices pls
I did some attempt
preds_GATE_Full = tabular_model.predict(test)
but what is the way to calculate ground truth predictions from your models
image

4
where is description of used algorithms , especially for categorical data values preprocessing ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions