Skip to content

fix: update config for CLMS global collections #1985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jlahovnik wants to merge 4 commits into
base: develop
Choose a base branch
from
Open

fix: update config for CLMS global collections #1985

jlahovnik wants to merge 4 commits into from

Conversation

@jlahovnik
Copy link
Collaborator

@jlahovnik jlahovnik commented Jan 8, 2026

The configuration on wekeo for the CLMS global collections has changed: Instead of having one dataset per collection, there is one dataset for all collections which can be filtered by productType and productIdentifier to get the data for a specific collection.
Note that productIdentifier can be used as a filter but also for searching by id.

@jlahovnik jlahovnik self-assigned this Jan 8, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026 •
edited
Loading

Test Results

    4 files  ±0      4 suites  ±0   2m 54s ⏱️ -2s
  670 tests ±0    669 ✅ ±0  1 💤 ±0  0 ❌ ±0 
2 722 runs  ±0  2 716 ✅ ±0  6 💤 ±0  0 ❌ ±0 

Results for commit 54e0fba. ± Comparison against base commit 7d72d64.

♻️ This comment has been updated with latest results.

@jlahovnik jlahovnik marked this pull request as draft January 8, 2026 10:01
@eodag-bot
Copy link
Collaborator

eodag-bot commented Jan 8, 2026 •
edited
Loading

badge

Code Coverage (Ubuntu)

Details 
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         247      10  95.95%   104-115, 629
config.py                                      303      24  92.08%   70-72, 75, 78, 81, 85, 89, 93-95, 583-585, 690-692, 711, 719, 749-754, 756
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/collection.py                              125       7  94.40%   173, 209, 212, 302, 305, 323, 326
api/core.py                                    739      50  93.23%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1146, 1283, 1326, 1339, 1406, 1420-1421, 1497-1502, 1514-1517, 1850, 2092, 2155-2156, 2185-2186
api/provider.py                                380      35  90.79%   186, 190-191, 316-337, 405, 520, 524-525, 530-533, 544, 620-630, 820-821, 869, 876, 907-910, 946-947, 955-956
api/search_result.py                           181      19  89.50%   104, 113, 120, 134, 178, 195, 292, 347-350, 420, 425-426, 460, 474, 497-498, 504
api/product/__init__.py                         18       2  88.89%   60, 62
api/product/_assets.py                          52       5  90.38%   97, 183, 191, 194-198
api/product/_product.py                        217      18  91.71%   177, 246-247, 265-266, 375, 404, 411, 518, 538, 561-564, 573-576, 622
api/product/metadata_mapping.py                787      53  93.27%   125-127, 220-225, 246, 304-305, 393, 414, 466-467, 504, 525-528, 551, 563-564, 605, 628, 658-663, 728-733, 745, 753, 985, 1160, 1169-1173, 1190-1195, 1328, 1351, 1360, 1382, 1387, 1439, 1511, 1532, 1558, 1572, 1597, 1643, 1712, 1784
api/product/drivers/__init__.py                 12       0  100.00%
api/product/drivers/base.py                     24       0  100.00%
api/product/drivers/generic.py                   7       0  100.00%
api/product/drivers/sentinel1.py                15       0  100.00%
api/product/drivers/sentinel2.py                15       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 22       4  81.82%   48, 55, 68-69
plugins/manager.py                             173      16  90.75%   102-107, 179, 201, 219-220, 232, 271-272, 372-375, 387-388
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                          101      10  90.10%   178-180, 228-229, 255-257, 288-289
plugins/apis/usgs.py                           182      25  86.26%   161, 267, 301, 343-345, 350, 378-379, 384, 414-421, 432-437, 459-465
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  22       4  81.82%   45, 58, 81, 95
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   153-156, 177-182
plugins/authentication/openid_connect.py       232      28  87.93%   91-92, 104-122, 169, 175-203, 211, 350-353, 379, 420
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                128       9  92.97%   180, 217, 288-289, 339-343
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 100-102, 109-111, 118
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   49-54, 67-113
plugins/crunch/filter_latest_tpl_name.py        31      20  35.48%   46-54, 64-95
plugins/crunch/filter_overlap.py                63      25  60.32%   62-65, 72-75, 81, 85, 89, 100-116, 131-157
plugins/crunch/filter_property.py               30       5  83.33%   55-60, 63-64
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        401      76  81.05%   270, 304, 353-356, 386-387, 395-399, 479-482, 522-524, 528, 559-560, 566-570, 601, 666-674, 738-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       280      35  87.50%   133, 162, 309-310, 368-369, 411, 415-426, 440, 517-521, 551, 586-587, 612-621, 680, 701, 723, 731, 765
plugins/download/http.py                       559      78  86.05%   229, 271-274, 336-339, 342, 349-354, 385-387, 404, 419, 479, 514, 528, 542, 550-554, 570-575, 586, 605, 642-645, 666, 676, 683, 703, 843, 875, 905-914, 950, 975-976, 995-1000, 1009, 1024-1026, 1030, 1033, 1048-1049, 1133, 1185, 1227-1228, 1240, 1250, 1300-1301, 1331, 1351, 1392-1394, 1448-1449
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         188      18  90.43%   107, 111, 135-141, 198-201, 294, 315, 439, 486, 508-511, 520
plugins/search/build_search_result.py          526      81  84.60%   254-255, 291, 294, 332, 335, 366-368, 611-622, 752, 776, 778, 845, 853-857, 878, 890, 916, 961, 985, 1019-1034, 1082, 1107, 1110, 1114, 1154-1174, 1215, 1242-1243, 1252-1261, 1325, 1340, 1346, 1365-1374, 1495-1496, 1540, 1549-1551, 1608, 1656-1666
plugins/search/cop_marine.py                   268      56  79.10%   57, 65-67, 77-78, 83, 88-89, 105, 107, 110, 176-177, 220, 236, 242, 246, 250, 261, 272-273, 281, 318-321, 327, 348, 352, 356, 360, 364-368, 374-377, 380-397, 414-417, 470-474, 479, 491
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 247-288
plugins/search/qssearch.py                     821      94  88.55%   415-416, 533-534, 557-558, 570-574, 789-795, 853, 949, 956, 1027, 1048, 1051-1052, 1070, 1079-1080, 1107, 1181, 1190, 1195-1212, 1221, 1236, 1245-1248, 1258, 1286, 1369, 1392, 1465-1466, 1472, 1562, 1669-1673, 1739, 1742, 1746-1747, 1768-1771, 1783, 1805-1817, 1824, 1859-1861, 1884-1890, 1897, 1951, 1974, 1979-1980, 1995, 2001, 2011, 2095, 2099, 2110, 2134, 2147, 2155-2165, 2202-2206
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      18  78.57%   99-127, 171-174, 187, 229
types/__init__.py                              162      44  72.84%   58, 62, 71-75, 86-98, 126-128, 135-140, 185, 204, 207, 245, 255-271, 276, 278, 300, 305, 313, 323
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                             84       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
utils/__init__.py                              596      45  92.45%   71, 168, 201-202, 211-237, 240, 255, 335-339, 414-418, 498, 538-539, 568, 943-946, 954-955, 997, 1016-1017, 1046, 1064-1065, 1177, 1244, 1408, 1497, 1512, 1521, 1533, 1708
utils/cache.py                                  22       0  100.00%
utils/dates.py                                  61       4  93.44%   75-77, 150
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             47       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    240      12  95.00%   200-203, 245, 263, 489, 537-538, 585, 660, 686
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
TOTAL                                         9600    1300  86.46%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 54e0fba

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@eodag-bot
Copy link
Collaborator

eodag-bot commented Jan 8, 2026 •
edited
Loading

badge

Code Coverage (Windows)

Details 
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         247      10  95.95%   104-115, 629
config.py                                      303      24  92.08%   70-72, 75, 78, 81, 85, 89, 93-95, 583-585, 690-692, 711, 719, 749-754, 756
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/collection.py                              125       7  94.40%   173, 209, 212, 302, 305, 323, 326
api/core.py                                    739      50  93.23%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1146, 1283, 1326, 1339, 1406, 1420-1421, 1497-1502, 1514-1517, 1850, 2092, 2155-2156, 2185-2186
api/provider.py                                380      35  90.79%   186, 190-191, 316-337, 405, 520, 524-525, 530-533, 544, 620-630, 820-821, 869, 876, 907-910, 946-947, 955-956
api/search_result.py                           181      19  89.50%   104, 113, 120, 134, 178, 195, 292, 347-350, 420, 425-426, 460, 474, 497-498, 504
api/product/__init__.py                         18       2  88.89%   60, 62
api/product/_assets.py                          52       5  90.38%   97, 183, 191, 194-198
api/product/_product.py                        217      18  91.71%   177, 246-247, 265-266, 375, 404, 411, 518, 538, 561-564, 573-576, 622
api/product/metadata_mapping.py                787      53  93.27%   125-127, 220-225, 246, 304-305, 393, 414, 466-467, 504, 525-528, 551, 563-564, 605, 628, 658-663, 728-733, 745, 753, 985, 1160, 1169-1173, 1190-1195, 1328, 1351, 1360, 1382, 1387, 1439, 1511, 1532, 1558, 1572, 1597, 1643, 1712, 1784
api/product/drivers/__init__.py                 12       0  100.00%
api/product/drivers/base.py                     24       0  100.00%
api/product/drivers/generic.py                   7       0  100.00%
api/product/drivers/sentinel1.py                15       0  100.00%
api/product/drivers/sentinel2.py                15       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 22       4  81.82%   48, 55, 68-69
plugins/manager.py                             173      16  90.75%   102-107, 179, 201, 219-220, 232, 271-272, 372-375, 387-388
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                          101      10  90.10%   178-180, 228-229, 255-257, 288-289
plugins/apis/usgs.py                           182      25  86.26%   161, 267, 301, 343-345, 350, 378-379, 384, 414-421, 432-437, 459-465
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  22       4  81.82%   45, 58, 81, 95
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   153-156, 177-182
plugins/authentication/openid_connect.py       232      28  87.93%   91-92, 104-122, 169, 175-203, 211, 350-353, 379, 420
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                128       9  92.97%   180, 217, 288-289, 339-343
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 100-102, 109-111, 118
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   49-54, 67-113
plugins/crunch/filter_latest_tpl_name.py        31      20  35.48%   46-54, 64-95
plugins/crunch/filter_overlap.py                63      25  60.32%   62-65, 72-75, 81, 85, 89, 100-116, 131-157
plugins/crunch/filter_property.py               30       5  83.33%   55-60, 63-64
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        401      76  81.05%   270, 304, 353-356, 386-387, 395-399, 479-482, 522-524, 528, 559-560, 566-570, 601, 666-674, 738-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       280      37  86.79%   133, 162, 229-231, 309-310, 368-369, 411, 415-426, 440, 517-521, 551, 586-587, 612-621, 680, 701, 723, 731, 765
plugins/download/http.py                       559      78  86.05%   229, 271-274, 336-339, 342, 349-354, 385-387, 404, 419, 479, 514, 528, 542, 550-554, 570-575, 586, 605, 642-645, 666, 676, 683, 703, 843, 875, 905-914, 950, 975-976, 995-1000, 1009, 1024-1026, 1030, 1033, 1048-1049, 1133, 1185, 1227-1228, 1240, 1250, 1300-1301, 1331, 1351, 1392-1394, 1448-1449
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         188      18  90.43%   107, 111, 135-141, 198-201, 294, 315, 439, 486, 508-511, 520
plugins/search/build_search_result.py          526      81  84.60%   254-255, 291, 294, 332, 335, 366-368, 611-622, 752, 776, 778, 845, 853-857, 878, 890, 916, 961, 985, 1019-1034, 1082, 1107, 1110, 1114, 1154-1174, 1215, 1242-1243, 1252-1261, 1325, 1340, 1346, 1365-1374, 1495-1496, 1540, 1549-1551, 1608, 1656-1666
plugins/search/cop_marine.py                   268      56  79.10%   57, 65-67, 77-78, 83, 88-89, 105, 107, 110, 176-177, 220, 236, 242, 246, 250, 261, 272-273, 281, 318-321, 327, 348, 352, 356, 360, 364-368, 374-377, 380-397, 414-417, 470-474, 479, 491
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 247-288
plugins/search/qssearch.py                     821      94  88.55%   415-416, 533-534, 557-558, 570-574, 789-795, 853, 949, 956, 1027, 1048, 1051-1052, 1070, 1079-1080, 1107, 1181, 1190, 1195-1212, 1221, 1236, 1245-1248, 1258, 1286, 1369, 1392, 1465-1466, 1472, 1562, 1669-1673, 1739, 1742, 1746-1747, 1768-1771, 1783, 1805-1817, 1824, 1859-1861, 1884-1890, 1897, 1951, 1974, 1979-1980, 1995, 2001, 2011, 2095, 2099, 2110, 2134, 2147, 2155-2165, 2202-2206
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      18  78.57%   99-127, 171-174, 187, 229
types/__init__.py                              162      44  72.84%   58, 62, 71-75, 86-98, 126-128, 135-140, 185, 204, 207, 245, 255-271, 276, 278, 300, 305, 313, 323
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                             84       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
utils/__init__.py                              596      45  92.45%   71, 168, 201-202, 211-237, 240, 255, 335-339, 414-418, 498, 538-539, 568, 943-946, 954-955, 997, 1016-1017, 1046, 1064-1065, 1177, 1244, 1408, 1497, 1512, 1521, 1533, 1708
utils/cache.py                                  22       0  100.00%
utils/dates.py                                  61       4  93.44%   75-77, 150
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             47       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    240      12  95.00%   200-203, 245, 263, 489, 537-538, 585, 660, 686
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
TOTAL                                         9600    1302  86.44%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 54e0fba

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@jlahovnik
Copy link
Collaborator Author

jlahovnik commented Jan 8, 2026

currently there is an issue with collection CLMS_GLO_DMP_333M - the search result contains incorrect items. It seems to be an issue on provider side -> waiting for feedback

@jlahovnik
Copy link
Collaborator Author

jlahovnik commented Jan 9, 2026

currently there is an issue with collection CLMS_GLO_DMP_333M - the search result contains incorrect items. It seems to be an issue on provider side -> waiting for feedback

workaround implemented

@jlahovnik jlahovnik marked this pull request as ready for review January 9, 2026 11:27
@jlahovnik jlahovnik requested a review from sbrunato January 9, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sbrunato sbrunato Awaiting requested review from sbrunato

At least 1 approving review is required to merge this pull request.

Assignees

@jlahovnik jlahovnik

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jlahovnik @eodag-bot