BetaCalibration not passing Scikit-learn check_estimator

[perellonieto]

It would be a good idea to make the BetaCalibration model fully compatible with Scikit-learn.

Although it may be already the case, it would be a good idea to be sure that it passes all the checks recomended by Scikit-learn in Rolling your own estimator.

In this guide, one of the requirements is that the estimator passes some checks in the form:

from sklearn.utils.estimator_checks import check_estimator
from sklearn.svm import LinearSVC
check_estimator(LinearSVC)  # passes

If we try with our calibration method it fails

from sklearn.utils.estimator_checks import check_estimator
from betacal import BetaCalibration
check_estimator(BetaCalibration)

raises

ValueError: bad input shape (20, 5)

However, this is a known issue with the check, that only expects estimators with multiple dimensional inputs. This is not the case for BetaCalibration, as well as the IsotonicCalibration or SigmoidCalibration that is already in Scikit-learn. In one of this cases, the check fails in the same manner.

There are some oppen issues concerning this problem and proposing to add some tags to the check (eg. accepts only one dimensional inputs) see scikit-learn/scikit-learn#6554

This means that we need to use the same tricks that Scikit-learn used to check the other calibration methods.