Predictor does not always accept the selected schema

[marcellodebernardi]

Describe the bug
We've occasionally observed that the input_schema chosen by the agent is not actually accepted by the predictor. It's unclear at this stage how this happens, given that part of the agent flow includes validating the predictor works with an input sample matching the selected input schema.

To Reproduce
Unsure yet, to be investigated. At a high level:

1. Build a model
2. Model doesn't accept inputs matching the stated schema when running inference

Expected behavior
The model must accept inputs conforming with the stated schema.

Additional context
This issue was discovered by a user testing the Plexe AI platform. We need to reproduce this locally using the agent builder library.

[DebjyotiRay]

there prolly is a mismatch betwen when the input schema gets registered during model building and when the actual schema implemented in the predictor code.

also, during validation the predictor code[tested with sample inputs, which come from object registry] is successful, as retrieved from the get_solution_schemas.

Please correct my thought process, but maybe there's a race-condition in the numbering of events: the registration and retrieval of schemas.