diff --git a/TEST_RUN_DETAILS.md b/TEST_RUN_DETAILS.md new file mode 100644 index 000000000..47b716fe6 --- /dev/null +++ b/TEST_RUN_DETAILS.md @@ -0,0 +1,188 @@ +So i have been trying to run the tests with all different configurations possible. Attaching all the results below: + +### With `sklearn and production` +- No test fails, 1 pass +```bash +C:\Users\ASUS\Documents\work\opensource\openml-python>pytest -m "sklearn and production" +======================================= test session starts ======================================= +platform win32 -- Python 3.14.0, pytest-9.0.2, pluggy-1.6.0 +rootdir: C:\Users\ASUS\Documents\work\opensource\openml-python +configfile: pyproject.toml +testpaths: tests +plugins: anyio-4.12.0, flaky-3.8.1, asyncio-1.3.0, cov-7.0.0, mock-3.15.1, rerunfailures-16.1, timeout-2.4.0, xdist-3.8.0, requests-mock-1.12.1 +asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function +collected 358 items / 354 deselected / 4 selected + +tests\test_flows\test_flow_functions.py sss. [100%] + +========================== 1 passed, 3 skipped, 354 deselected in 2.88s =========================== +``` + +### With `sklearn and not production` +- 41 fails 12 passes +```bash +============================================== short test summary info ============================================== +FAILED tests/test_runs/test_run.py::TestRun::test_offline_and_online_run_identical - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_publish_with_local_loaded_flow - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_to_from_filesystem_no_model - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_to_from_filesystem_search - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_to_from_filesystem_vanilla - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test__run_exists - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test__run_task_get_arffcontent - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_check_erronous_sklearn_flow_fails - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_delete_run - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/6 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_initialize_cv_from_run - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_initialize_model_from_run - Exception: OpenMLServerException('Input value does not match allowed values in foreign column. - problematic inpu... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_learning_curve_task_1 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_learning_curve_task_2 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_local_run_metric_score - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_local_run_swapped_parameter_order_flow - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_local_run_swapped_parameter_order_model - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/100 returned code 274: ... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_column_transformer_pipeline - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_gridsearch - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_linear_regression - KeyError: 'oml:error' +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_logistic_regression - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_maskedarrays - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_pipeline_dummy_pipeline - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_randomsearch - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_flow_on_task_downloaded_flow - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_on_dataset_with_missing_labels_array - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_on_dataset_with_missing_labels_dataframe - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_regression_on_classif_task - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/44 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id_1 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id_1_after_load - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id_after_load - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::test__run_task_get_arffcontent_2 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[2-None-0] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[-1-None-0] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[1-None-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[1-sequential-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[1-threading-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[-1-threading-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_exisiting_setup_exists_2 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_existing_setup_exists_1 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_existing_setup_exists_3 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +============= 41 failed, 12 passed, 3 skipped, 302 deselected, 4 warnings, 1 rerun in 446.43s (0:07:26) ============= +``` + +### Running all of them together +- 87 fails 229 passes +```bash +============================================== short test summary info ============================================== +FAILED tests/test_datasets/test_dataset.py::test_add_remove_ontology_to_dataset - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/feature/ontology/add returned co... +FAILED tests/test_datasets/test_dataset.py::test_add_same_ontology_multiple_features - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/feature/ontology/add returned co... +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test__get_dataset_features - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test__get_dataset_qualities - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/qualities/2 returned code 364: D... +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test__getarff_path_dataset_arff - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_data_edit_cannot_edit_critical_field_if_dataset_has_task - ValueError: TIMEOUT: Failed to fetch uploaded dataset - 1094 +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_data_edit_critical_field - ValueError: TIMEOUT: Failed to fetch uploaded dataset - 1095 +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_dataset_by_name - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_dataset_cache_format_feather - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_dataset_cache_format_pickle - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_dataset_lazy_all_functions - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_dataset_sparse - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_dataset_uint8_dtype - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_get_online_dataset_arff - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::TestOpenMLDataset::test_publish_dataset - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::test_list_datasets_by_number_features - assert 8 <= 0 +FAILED tests/test_datasets/test_dataset_functions.py::test_list_datasets_by_number_missing_values - assert 5 <= 1 +FAILED tests/test_datasets/test_dataset_functions.py::test_list_datasets_combined_filters - assert 1 <= 0 +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[True-True-True] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[True-True-False] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/qualities/1 returned code 364: D... +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[True-False-True] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[True-False-False] - KeyError: 'oml:error' +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[False-True-True] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[False-True-False] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/qualities/1 returned code 364: D... +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[False-False-True] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_datasets/test_dataset_functions.py::test_get_dataset_lazy_behavior[False-False-False] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_runs/test_run.py::TestRun::test_offline_and_online_run_identical - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_publish_with_local_loaded_flow - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_tagging - AssertionError: Test server state is incorrect +FAILED tests/test_runs/test_run.py::TestRun::test_to_from_filesystem_no_model - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_to_from_filesystem_search - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run.py::TestRun::test_to_from_filesystem_vanilla - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test__run_exists - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test__run_task_get_arffcontent - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_check_erronous_sklearn_flow_fails - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_delete_run - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/6 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_format_prediction_classification_incomplete_probabilities - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_format_prediction_classification_no_probabilities - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_format_prediction_task_learning_curve_sample_not_set - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_format_prediction_task_without_classlabels_set - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_initialize_cv_from_run - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_initialize_model_from_run - Exception: OpenMLServerException('Input value does not match allowed values in foreign column. - problematic inpu... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_learning_curve_task_1 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_learning_curve_task_2 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_local_run_metric_score - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_local_run_swapped_parameter_order_flow - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_local_run_swapped_parameter_order_model - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/100 returned code 274: ... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_column_transformer_pipeline - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_gridsearch - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_linear_regression - KeyError: 'oml:error' +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_logistic_regression - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_maskedarrays - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_pipeline_dummy_pipeline - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_and_upload_randomsearch - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_flow_on_task_downloaded_flow - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_on_dataset_with_missing_labels_array - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_on_dataset_with_missing_labels_dataframe - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_regression_on_classif_task - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/44 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id_1 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id_1_after_load - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::TestRun::test_run_with_illegal_flow_id_after_load - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_runs/test_run_functions.py::test__run_task_get_arffcontent_2 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[2-None-0] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[-1-None-0] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[1-None-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[1-sequential-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[1-threading-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_runs/test_run_functions.py::test_joblib_backends[-1-threading-10] - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/2 returned code 274: No... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_exisiting_setup_exists_2 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_existing_setup_exists_1 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_existing_setup_exists_3 - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_setups/test_setup_functions.py::TestSetupFunctions::test_setuplist_offset - assert 0 == 10 +FAILED tests/test_study/test_study_functions.py::TestStudyFunctions::test_publish_study - assert 0 == 10 +FAILED tests/test_study/test_study_functions.py::TestStudyFunctions::test_study_attach_illegal - assert 0 == 10 +FAILED tests/test_tasks/test_classification_task.py::OpenMLClassificationTaskTest::test_class_labels - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_classification_task.py::OpenMLClassificationTaskTest::test_download_task - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_classification_task.py::OpenMLClassificationTaskTest::test_get_X_and_Y - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_classification_task.py::test_get_X_and_Y - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_learning_curve_task.py::OpenMLLearningCurveTaskTest::test_class_labels - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_learning_curve_task.py::OpenMLLearningCurveTaskTest::test_download_task - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_learning_curve_task.py::OpenMLLearningCurveTaskTest::test_get_X_and_Y - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/20 returned code 274: N... +FAILED tests/test_tasks/test_regression_task.py::OpenMLRegressionTaskTest::test_get_X_and_Y - KeyError: 'oml:error' +FAILED tests/test_tasks/test_task_functions.py::TestTask::test_download_split - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_tasks/test_task_functions.py::TestTask::test_get_task - KeyError: 'oml:error' +FAILED tests/test_tasks/test_task_functions.py::TestTask::test_get_task_lazy - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +FAILED tests/test_tasks/test_task_functions.py::TestTask::test_get_task_with_cache - openml.exceptions.OpenMLServerException: https://test.openml.org/api/v1/xml/data/features/1 returned code 274: No... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_get_task_with_cache - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_clustering_task - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks_by_tag - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks_by_type - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks_empty - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks_length - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks_paginate - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_list_tasks_per_type_paginate - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::TestTask::test_removal_upon_download_failure - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::test_delete_task_not_owned - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::test_delete_task_with_run - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::test_delete_success - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_functions.py::test_delete_unknown_task - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_methods.py::OpenMLTaskMethodsTest::test_get_train_and_test_split_indices - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_tasks/test_task_methods.py::OpenMLTaskMethodsTest::test_tagging - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_for_tasks - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_with_multiple_batches - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_for_datasets - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_for_flows - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_for_setups - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_for_runs - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_for_evaluations - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_list_all_few_results_available - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test__create_cache_directory - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +ERROR tests/test_utils/test_utils.py::test_correct_test_server_download_state - AssertionError: Expected path C:\Users\ASUS\Documents\work\opensource\openml-python\tests\files\org\openml\test\t... +============= 87 failed, 229 passed, 16 skipped, 23 warnings, 27 errors, 5 rerun in 3462.30s (0:57:42) ============== +``` \ No newline at end of file