IEBH-459: Refactor dataset download logic and improve error handling

[vmoshynskyi]

• Fix situations when the dataset cannot be downloaded

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
__init__.py	0	0	100%
pilotcli.py	0	31	0%	7, 9–10, 12–16, 19–23, 25–30, 33–41, 44–46
commands
__init__.py	0	0	100%
container_registry.py	0	47	0%	7, 9–12, 15–16, 18, 21–26, 29–35, 38–46, 49–55, 58–67
dataset.py	0	90	0%	7–8, 10–15, 18–19, 21, 24–25, 33–35, 44–49, 51–59, 61–71, 74–78, 87–91, 93–102, 104–114, 117–120, 128–138, 140–141
entry_point.py	0	71	0%	7, 9–10, 12–29, 32–34, 37–39, 42–46, 49–53, 56–60, 63–66, 69–71, 74–77, 80–97
file.py	0	75	62%	38, 136, 139, 142, 161–162, 164–167, 193, 195, 268–275, 277, 279, 292–298, 300, 327–334, 336, 370–376, 378–383, 385–397, 399–402, 404–406, 408–410
project.py	0	40	0%	7–8, 10–13, 16–17, 19, 22–25, 33, 41, 50–54, 56–64, 66–76
use_config.py	0	13	0%	7, 9–11, 14–15, 17, 20–22, 29–31
user.py	0	30	0%	7, 9–15, 18–19, 21, 24–30, 32, 34, 37–38, 40, 43–44, 53–57
configs
__init__.py	0	0	100%
app_config.py	0	0	100%
user_config.py	0	13	80%	51, 60, 64, 68, 72, 76, 80, 84, 88, 92, 96, 100, 104
models
__init__.py	0	0	100%
convert_type.py	0	8	0%	7, 9, 12–17
service_meta_class.py	0	1	80%	11
singleton.py	0	0	100%
upload_form.py	0	28	60%	12, 27, 31, 35, 39, 43, 47, 51, 55, 59, 63, 67, 71, 75, 79, 83, 87, 91, 95, 99, 127, 140–146
resources
__init__.py	0	0	100%
custom_error.py	0	0	100%
custom_help.py	0	0	100%
services
__init__.py	0	0	100%
services/container_registry_manager
__init__.py	0	0	100%
container_registry_manager.py	0	131	0%	7, 9, 11–16, 19–22, 24–28, 30–35, 37–46, 48–62, 64–66, 68–84, 86–88, 90–101, 103–105, 107–112, 127–138, 142–145, 147–164, 166, 168–171
services/crypto
__init__.py	0	0	100%
crypto.py	0	19	40%	36, 43–48, 58–60, 67–73, 75, 77
services/dataset_manager
__init__.py	0	0	100%
dataset_detail.py	0	23	70%	44, 46, 63, 70–75, 77–79, 81–90, 92
dataset_download.py	0	126	0%	7–10, 12–13, 15–22, 24, 27–35, 37–40, 45–52, 54–55, 57–60, 65–71, 73, 75–83, 85–87, 89–97, 99–102, 104–105, 107–113, 115–117, 119–120, 128–131, 133–140, 142–143, 145–147, 149–158, 160, 162–169, 171
dataset_list.py	0	6	84%	41, 47–48, 50–52
model.py	0	8	0%	7, 10–14, 16–17
services/file_manager
__init__.py	0	0	100%
file_lineage.py	0	7	41%	15–16, 25, 28–30, 32
file_list.py	0	49	26%	27–31, 33–36, 39, 47–62, 65–67, 70–77, 79–90
file_manifests.py	0	95	25%	21–26, 41–45, 49–50, 53–61, 63, 67–70, 73–77, 79, 83–84, 87–89, 93–94, 97–104, 106, 109–117, 122–128, 133–138, 141–147, 149–151, 154–160, 162–164, 166–169
file_tag.py	0	39	31%	25, 29–33, 35, 38–50, 52–54, 56–57, 61–63, 66–70, 72–75, 77–78
services/file_manager/file_download
__init__.py	0	0	100%
download_client.py	0	189	17%	31–43, 46–55, 58–59, 61–62, 65, 67–68, 72–75, 82, 87–90, 92–100, 102, 104, 108–113, 115, 118–119, 122–128, 130–131, 133–135, 138–147, 150–155, 159–168, 176–178, 180–187, 189–193, 198–208, 210–211, 214–215, 220–221, 225–230, 234–235, 238–240, 242–253, 257–260, 263–267, 269–271, 273–275, 277–278, 280, 284–286, 288–301, 303
model.py	0	1	87%	17
services/file_manager/file_upload
__init__.py	0	0	100%
exception.py	0	1	75%	12
file_upload.py	0	59	59%	32–39, 73, 96, 124–125, 129, 131, 133–135, 137, 139, 151–157, 159–164, 166, 168–173, 177, 179–180, 182–183, 185–193, 195–196, 198, 250–251, 255
models.py	0	5	91%	23, 41, 140–142
upload_client.py	0	87	54%	99–102, 164–166, 179, 181–186, 188–192, 195, 197–209, 211, 254, 257, 261–266, 270–275, 277, 281, 283, 285, 287, 305, 330–331, 336, 340–343, 347, 363–365, 367–369, 378, 383–384, 386, 388–389, 391–394, 396, 410–416, 426
upload_validator.py	0	26	50%	29–31, 34–36, 40–44, 47–56, 59–60, 65, 67, 69
services/logger_services
__init__.py	0	0	100%
log_functions.py	0	0	100%
services/output_manager
__init__.py	0	0	100%
error_handler.py	0	8	91%	88–90, 102–104, 109, 118
help_page.py	0	10	85%	29–30, 38–39, 48–49, 61–62, 102–103
message_handler.py	0	65	58%	19–24, 29, 34, 39, 49, 54–56, 61, 66, 71, 76, 89, 94, 101, 106, 111, 115, 131, 146, 156, 166, 185, 189, 193, 207–218, 229–234, 236–242, 253, 257–258, 262–263, 265–266, 270, 274, 278
services/project_manager
__init__.py	0	0	100%
project.py	0	6	83%	40, 46–47, 49–51
services/user_authentication
__init__.py	0	0	100%
decorator.py	0	13	67%	31, 34, 51–55, 57, 61–64, 66
token_manager.py	0	21	57%	27, 30–32, 35, 38–39, 42–43, 63, 67, 71–72, 78–79, 81–84, 86–87
user_login_logout.py	0	43	64%	40–43, 113, 117–123, 125–127, 131–132, 136–143, 145–147, 151–152, 159–164, 166–168, 172–175
user_set_config.py	0	9	47%	17–20, 24–26, 28–29
utils
__init__.py	0	0	100%
aggregated.py	0	47	49%	24, 47, 63–74, 76–78, 84–85, 87, 91–94, 96–97, 111–118, 120–121, 125–132, 134–136
TOTAL	2785	1540	44%

[Copilot]

Pull request overview

This PR refactors the dataset download logic to improve error handling and prevent download failures. The changes introduce exponential backoff retry logic with configurable timeouts, better HTTP error handling using raise_for_status(), and standardized filename generation. The version is bumped from 2.2.7 to 2.2.8, and the package name is updated from "app" to "cli" to better reflect its purpose.

Key changes:

• Enhanced error handling with proper HTTP exception catching and status code checks
• Implemented retry mechanism with exponential backoff (up to 15 retries with 1-5 second intervals) for download preparation
• Simplified filename generation logic by removing the generate_download_url() method and generating filenames directly based on dataset code, version, and timestamp

Reviewed changes

Copilot reviewed 4 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
pyproject.toml	Updated package name to "cli" and bumped version to 2.2.8
app/services/output_manager/message_handler.py	Updated log message from "Current dataset version" to "Looking for dataset version" for clarity
app/services/dataset_manager/dataset_download.py	Major refactoring: improved error handling with HTTPError exceptions, added retry logic with exponential backoff, simplified URL generation, and standardized filename creation
app/resources/custom_help.py	Updated version to 2.2.8 and revised changelog description
.gitignore	Uncommented .python-version to ignore Python version files

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.