Query about training results

[Durrells]

Hi! Thank you very much for your excellent work and generous code sharing.

I am running your code and trying to reproduce the results of your paper.

In the run_mnist.ipynb notebook, when I run the code to train the model, the [test/ood] AUROC remains around 50. Do you have any suggestions on this issue?

I didn't modify most of the code and just commented out the line

plot_gan(epoch, model, p_batch, Q0[:16], gen, maxiter=config.maxiter, **train_kwargs)

Here are some of the results:
Epoch=24 LR=0.0006578
100%|██████████| 47/47 [06:48<00:00,  8.69s/it, FP=0.00%, GN_Qt=0.603, P=3.756, Q0=-8.280, Qt=-1.314, R=1.00%, lipschitz_ratio=1.642, loss=-0.018]
24/24 [==============================] - 6s 265ms/step
4/4 [==============================] - 1s 234ms/step
36/36 [==============================] - 10s 268ms/step
Train examples DescribeResult(nobs=6016, minmax=(1.0218256, 5.828541), mean=3.8103046, variance=0.6504586325222913, skewness=-0.4136563609162037, kurtosis=-0.11905740571011902)
Test examples DescribeResult(nobs=892, minmax=(1.4499859, 5.792158), mean=3.813585, variance=0.5596361719531778, skewness=-0.27269367098282, kurtosis=-0.2819219690014747)
OOD examples DescribeResult(nobs=9108, minmax=(-0.023437515, 6.628998), mean=3.880484, variance=1.299644826991872, skewness=0.14530889035233022, kurtosis=-0.48042250906903483)
[train/ood] Avg Dist=-0.070 T=6.629 Acc=60.22%
[train/ood] AUROC =49.567
[test /ood] Avg Dist =-0.06689906120300293 T=6.629 Acc=91.08%
[test /ood] AUROC =49.439
Epoch=25 LR=0.0006435
100%|██████████| 47/47 [06:47<00:00,  8.67s/it, FP=0.03%, GN_Qt=0.585, P=5.180, Q0=-7.722, Qt=-1.162, R=1.00%, lipschitz_ratio=1.597, loss=-0.001]
24/24 [==============================] - 6s 264ms/step
4/4 [==============================] - 1s 233ms/step
36/36 [==============================] - 10s 267ms/step
Train examples DescribeResult(nobs=6016, minmax=(1.6921498, 5.955354), mean=4.2076674, variance=0.4955580529825745, skewness=-0.47829623738328975, kurtosis=-0.019798706490049334)
Test examples DescribeResult(nobs=892, minmax=(2.1150906, 5.875563), mean=4.2115965, variance=0.42422112698774417, skewness=-0.3225641967050908, kurtosis=-0.2273849390289704)
OOD examples DescribeResult(nobs=9108, minmax=(0.9116055, 6.60164), mean=4.287896, variance=0.9838932968199129, skewness=0.055762059084495164, kurtosis=-0.446357031111134)
[train/ood] Avg Dist=-0.080 T=6.602 Acc=60.22%
[train/ood] AUROC =48.594
[test /ood] Avg Dist =-0.07629966735839844 T=6.602 Acc=91.08%
[test /ood] AUROC =48.417
Epoch=26 LR=0.0006292
100%|██████████| 47/47 [06:42<00:00,  8.57s/it, FP=0.00%, GN_Qt=0.590, P=4.211, Q0=-8.329, Qt=-1.318, R=1.00%, lipschitz_ratio=1.603, loss=-0.023]
24/24 [==============================] - 6s 269ms/step
4/4 [==============================] - 1s 234ms/step
36/36 [==============================] - 10s 269ms/step
Train examples DescribeResult(nobs=6016, minmax=(0.820788, 5.470547), mean=3.5400138, variance=0.603637361724676, skewness=-0.44938489407989984, kurtosis=-0.07952032619838567)
Test examples DescribeResult(nobs=892, minmax=(1.249703, 5.3880486), mean=3.5439668, variance=0.5181393888261584, skewness=-0.30554993974965317, kurtosis=-0.27075616964335314)
OOD examples DescribeResult(nobs=9108, minmax=(-0.16365284, 6.181051), mean=3.6074412, variance=1.2093181372395694, skewness=0.10241750543799147, kurtosis=-0.4865591246523189)
[train/ood] Avg Dist=-0.067 T=6.181 Acc=60.22%
[train/ood] AUROC =49.412
[test /ood] Avg Dist =-0.06347441673278809 T=6.181 Acc=91.08%
[test /ood] AUROC =49.282

Thank you again for your sharing, and I look forward to your reply.