Support Partial Model Availability During Prediction by Adjusting VT Hat

[dashertr]

Currently, the predict2() method assumes that all models used during training are also available during prediction. This causes dimension mismatch errors when some models are missing from the prediction property (for example, predicting "ChRad" after training on "BE"). Since VT hat is learned from the full set of training models, its dimensions must be aligned with the set of models present at prediction time. If models are missing, the corresponding columns of VT hat and the associated parts of the input matrix must be excluded to ensure consistency.

[turtuledunnad]

I just check the predict code. The problem that we discussed is only a problem when we calculate the combined forecasts using exactly entries from Vt_hat - but we did not, we transform the principle components weights into models weights and then using matrix multiplication to find the combined forecasts. In short, as long as we are using the same models for training and testing (which you should expect this to happen, otherwise our information from training data will be useless), it does not matter if you train for BE and then test for ChRad, as long as you are using the same model.

[dashertr]

I just check the predict code. The problem that we discussed is only a problem when we calculate the combined forecasts using exactly entries from Vt_hat - but we did not, we transform the principle components weights into models weights and then using matrix multiplication to find the combined forecasts. In short, as long as we are using the same models for training and testing (which you should expect this to happen, otherwise our information from training data will be useless), it does not matter if you train for BE and then test for ChRad, as long as you are using the same model.

The problem comes up when you train on something like BE with 10 models, for example, which means the SVD and the samples are set up based on those 10 models. Then, if you try to predict on something like ChRad that only has 7 models available, the matrix multiplications in rndm_m_random_calculator break because the dimensions of betas and Vt_hat no longer match.

Currently, in bmc-class there is a reduced vt hat being used in the predict2 method, but that only solves half of the problem. The samples (betas) still reflect the full model space, so their shape and meaning don’t match the reduced Vt_hat anymore, which causes the matrix multiplication for model_weights_random to break.

There is a workflow set up on test.ipynb in the bmc-class branch, where you can see the problem clearly.

[dashertr]

I just check the predict code. The problem that we discussed is only a problem when we calculate the combined forecasts using exactly entries from Vt_hat - but we did not, we transform the principle components weights into models weights and then using matrix multiplication to find the combined forecasts. In short, as long as we are using the same models for training and testing (which you should expect this to happen, otherwise our information from training data will be useless), it does not matter if you train for BE and then test for ChRad, as long as you are using the same model.

The problem comes up when you train on something like BE with 10 models, for example, which means the SVD and the samples are set up based on those 10 models. Then, if you try to predict on something like ChRad that only has 7 models available, the matrix multiplications in rndm_m_random_calculator break because the dimensions of betas and Vt_hat no longer match.

Currently, in bmc-class there is a reduced vt hat being used in the predict2 method, but that only solves half of the problem. The samples (betas) still reflect the full model space, so their shape and meaning don’t match the reduced Vt_hat anymore, which causes the matrix multiplication for model_weights_random to break.

There is a workflow set up on test.ipynb in the bmc-class branch, where you can see the problem clearly.

I have created a new branch for this problem and implemented a predict3 method where both Vt_hat and samples are sliced to reflect the new model space. However, I am not sure if this is mathematically ok or not. The results from the evaluation in test.ipynb seem good. Would you mind taking a look at the math?