Rebase #48

Abhinavpv28 · 2026-01-09T05:09:52Z

…ts To C Implementation (#35)

Created a comprehensive L2 test suite with 6 test modules covering error handling, normal uploads, retry logic, security, resource management, and upload strategies Updated build scripts and CI/CD workflow to compile with L2_TEST_ENABLED flag and execute the new test suite

This feature file contains scenarios for normal log uploads, large log file handling, and MD5 checksum verification for the uploadSTBLogs service.

…ts To C Implementation (#35) * Created a comprehensive L2 test suite with 6 test modules covering error handling, normal uploads, retry logic, security, resource management, and upload strategies Updated build scripts and CI/CD workflow to compile with L2_TEST_ENABLED flag and execute the new test suite This feature file contains scenarios for normal log uploads, large log file handling, and MD5 checksum verification for the uploadSTBLogs service.

Copilot

Pull request overview

This pull request adds a comprehensive L2 (Level 2) test suite for the uploadSTBLogs service, consisting of 6 test modules covering error handling, normal uploads, retry logic, security, resource management, and upload strategies. The changes include conditionally disabling sleep() calls in the reboot strategy implementation when testing is enabled, and updates to build scripts and CI/CD workflows to support the new test infrastructure.

Key Changes

Conditional compilation of sleep() calls in strategy_reboot.c using L2_TEST_ENABLED flag to speed up testing
New test infrastructure with Python-based test suites and Gherkin feature files for behavior-driven testing
Build configuration updates to include L2_TEST_ENABLED flag and reference the correct branch

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 82 comments.

Show a summary per file

File	Description
uploadstblogs/src/strategy_reboot.c	Wraps sleep() calls with #ifndef L2_TEST_ENABLED to allow faster test execution
test/run_uploadstblogs_l2.sh	Test runner script that executes all 6 L2 test suites with proper environment setup
test/functional-tests/tests/uploadstblogs_helper.py	Helper functions for test execution including log parsing, process management, and file operations
test/functional-tests/tests/test_uploadstblogs_*.py	Six comprehensive test modules covering different aspects of the uploadSTBLogs service
test/functional-tests/features/*.feature	Gherkin feature files describing test scenarios in behavior-driven format
cov_build.sh	Updated to use feature/upload_L2 branch and add L2_TEST_ENABLED compiler flag
.github/workflows/L2-tests.yml	Updated CI/CD workflow to execute the new L2 test suite

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-09T05:18:59Z

test/functional-tests/tests/uploadstblogs_helper.py

+    result = subprocess.run(f"grep '{marker}' /opt/logs/telemetry*.log 2>/dev/null", 
+                          shell=True, capture_output=True, text=True)


Using shell=True with subprocess.run poses a security risk when the marker parameter contains unsanitized input. The marker parameter is directly interpolated into a shell command. Consider using a list of arguments without shell=True or properly sanitizing the marker parameter.

Copilot · 2026-01-09T05:18:59Z

test/functional-tests/tests/uploadstblogs_helper.py

+    result = subprocess.run(f"grep -c '{pattern}' {log_file} 2>/dev/null || echo 0", 
+                          shell=True, capture_output=True, text=True)
+    return int(result.stdout.strip())


Using shell=True with subprocess.run poses a security risk when the pattern parameter contains unsanitized input. The pattern parameter is directly interpolated into a grep command. Consider validating/sanitizing the pattern parameter or using a Python-based search instead of grep.

Suggested change

result = subprocess.run(f"grep -c '{pattern}' {log_file} 2>/dev/null || echo 0",

shell=True, capture_output=True, text=True)

return int(result.stdout.strip())

try:

count = 0

with open(log_file, "r", encoding="utf-8", errors="ignore") as f:

for line in f:

if pattern in line:

count += 1

return count

except (FileNotFoundError, PermissionError, OSError):

return 0

Copilot · 2026-01-09T05:18:59Z

test/functional-tests/tests/test_uploadstblogs_normal_upload.py

+        # Check for archive creation logs
+        archive_logs = grep_uploadstb_logs_regex(r"Archive created successfully")
+        # Process should complete successfully
+        assert len(archive_logs) > 0, "Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"


The assertion message contains an f-string that is not properly formatted. The message should use an f-string prefix to include the variable values, i.e., f"Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"

Suggested change

assert len(archive_logs) > 0, "Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"

assert len(archive_logs) > 0, f"Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"

Copilot · 2026-01-09T05:19:00Z

test/functional-tests/features/uploadstblogs_retry_logic.feature

@@ -0,0 +1,70 @@
+####################################################################################
+# If not stated otherwise in this file or this component's Licenses


The comment header has a typo. It should say "Licenses.txt file" instead of just "Licenses file". This inconsistency appears across multiple files in this PR and should be corrected for consistency with standard RDK copyright headers.

Suggested change

# If not stated otherwise in this file or this component's Licenses

# If not stated otherwise in this file or this component's Licenses.txt file

Copilot · 2026-01-09T05:19:00Z

cov_build.sh

 cd telemetry
 cp include/*.h /usr/local/include
-sh  build_inside_container.sh 
+sh  build_inside_container.sh


This line has an unnecessary trailing space after the shell command. The space should be removed for cleaner code.

Suggested change

sh build_inside_container.sh

sh build_inside_container.sh

Copilot · 2026-01-09T05:19:19Z

test/functional-tests/tests/test_uploadstblogs_normal_upload.py

+"""
+
+import pytest
+import time


Import of 'time' is not used.

Copilot · 2026-01-09T05:19:19Z

test/functional-tests/tests/test_uploadstblogs_security.py

+"""
+
+import pytest
+import time


Import of 'time' is not used.

Copilot · 2026-01-09T05:19:19Z

test/functional-tests/tests/uploadstblogs_helper.py

+import os
+import time
+import re
+import json


Import of 'json' is not used.

Copilot · 2026-01-09T05:19:19Z

test/functional-tests/tests/uploadstblogs_helper.py

+    """Get file size in bytes"""
+    try:
+        return os.path.getsize(filepath)
+    except:


Except block directly handles BaseException.

Copilot · 2026-01-09T05:19:20Z

test/run_uploadstblogs_l2.sh

+cd /usr/common_utilities
+sed -i '/file_upload\.sslverify/s/= 1;/= 0;/' uploadutils/mtls_upload.c
+sed -i 's/\(ret_code = setCommonCurlOpt(curl, s3url, NULL, \)true\()\)/\1false\2/g' uploadutils/uploadUtil.c
+sed -i '/if (auth) {/,/}/s/^/\/\/ /' uploadutils/uploadUtil.c


The test runner script directly patches /usr/common_utilities/uploadutils/mtls_upload.c and uploadutils/uploadUtil.c to disable TLS certificate verification and bypass authentication by forcing file_upload.sslverify to 0, changing setCommonCurlOpt(..., true) to false, and commenting out the if (auth) { ... } block. If this script is run in any environment whose binaries are then used beyond isolated testing, it produces builds that accept unverified HTTPS servers and skip authentication, enabling man-in-the-middle attacks and unauthorized log uploads. Instead of source-level sed patches that weaken security controls, introduce test-specific configuration or compile-time flags that preserve proper certificate validation and authentication in all non-test builds.

…ts To C Implementation (#49) * Simplify some pieces of code over engineered by AI models. * Update run_uploadstblogs_l2.sh for L2 coverage --------- Co-authored-by: Abhinav P V <Abhinav_Valappil@comcast.com>

…ts To C Implementation (#51) * Remove logging of response URLs * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Shibu Kakkoth Vayalambron <shibu.kakkoth@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot · 2026-01-13T18:18:12Z

test/functional-tests/tests/test_uploadstblogs_error_handling.py

+        result = run_uploadstblogs()
+
+        # Check for configuration error detection
+        error_logs = grep_uploadstb_logs_regex(r"ERROR|error|fail.*properties|invalid")


Variable error_logs is not used.

Suggested change

error_logs = grep_uploadstb_logs_regex(r"ERROR|error|fail.*properties|invalid")

error_logs = grep_uploadstb_logs_regex(r"ERROR|error|fail.*properties|invalid")

assert len(error_logs) > 0, "Configuration error should be logged for corrupted device properties"

Copilot · 2026-01-13T18:18:13Z

test/functional-tests/tests/test_uploadstblogs_error_handling.py

+        result = run_uploadstblogs()
+
+        # Check for informative message
+        logs = grep_uploadstb_logs_regex(r"no.*file|empty|nothing to upload")


Variable logs is not used.

Suggested change

logs = grep_uploadstb_logs_regex(r"no.*file|empty|nothing to upload")

grep_uploadstb_logs_regex(r"no.*file|empty|nothing to upload")

Copilot · 2026-01-13T18:18:13Z

test/functional-tests/tests/test_uploadstblogs_error_handling.py

+        result = run_uploadstblogs()
+
+        # Should not attempt upload
+        upload_logs = grep_uploadstb_logs_regex(r"upload.*success|uploading|HTTP")


Variable upload_logs is not used.

Suggested change

upload_logs = grep_uploadstb_logs_regex(r"upload.*success|uploading|HTTP")

upload_logs = grep_uploadstb_logs_regex(r"upload.*success|uploading|HTTP")

assert len(upload_logs) == 0, "Should not log upload attempts without files"

…ts To C Implementation (#54) * Update cov_build.sh to enable the rdkcertselector feature in the common_utilities build configuration to support the migration of operation support log upload scripts to C implementation. The change is part of resolving L2 test failures.

Copilot

Pull request overview

Copilot reviewed 61 out of 62 changed files in this pull request and generated 29 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-21T19:05:58Z

test/functional-tests/tests/test_uploadstblogs_error_handling.py

+        result = run_uploadstblogs()
+
+        # Check for configuration error detection
+        error_logs = grep_uploadstb_logs_regex(r"ERROR|error|fail.*properties|invalid")


Copilot · 2026-01-21T19:05:58Z

test/functional-tests/tests/test_uploadstblogs_error_handling.py

+        result = run_uploadstblogs()
+
+        # Check for size-related logs
+        logs = grep_uploadstb_logs_regex(r"size|large|limit|truncate")


Copilot · 2026-01-21T19:05:58Z

test/functional-tests/tests/test_uploadstblogs_normal_upload.py

+        assert len(archive_logs) > 0, "Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"
+
+        upload_logs = grep_uploadstb_logs_regex(r"upload.*success|uploading|HTTP")
+        # Telemetry should be attempted
+        assert len(archive_logs) > 0, "Upload Process should complete and succeed"


Variable upload_logs is not used.

Suggested change

assert len(archive_logs) > 0, "Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"

upload_logs = grep_uploadstb_logs_regex(r"upload.*success|uploading|HTTP")

# Telemetry should be attempted

assert len(archive_logs) > 0, "Upload Process should complete and succeed"

assert len(archive_logs) > 0, (

f"Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"

)

upload_logs = grep_uploadstb_logs_regex(r"upload.*success|uploading|HTTP")

# Telemetry should be attempted

assert len(upload_logs) > 0, (

f"Upload process should complete and succeed. Found {len(upload_logs)} upload-related logs: {upload_logs}"

)

Copilot · 2026-01-21T19:05:59Z

test/functional-tests/tests/test_uploadstblogs_normal_upload.py

+        # Check for compression/archive activity
+        compression_logs = grep_uploadstb_logs_regex(r"compress|archive|tgz")


Variable compression_logs is not used.

Suggested change

# Check for compression/archive activity

compression_logs = grep_uploadstb_logs_regex(r"compress|archive|tgz")

Copilot · 2026-01-21T19:05:59Z

test/functional-tests/tests/test_uploadstblogs_error_handling.py

+        result = run_uploadstblogs()
+
+        # Check for no files detection
+        no_files_logs = grep_uploadstb_logs_regex(r"no.*file|empty|not found|no logs")


Variable no_files_logs is not used.

Copilot · 2026-01-21T19:06:04Z

test/functional-tests/tests/test_uploadstblogs_upload_strategies.py

+        result = run_uploadstblogs()
+
+        # Check for event publishing
+        event_logs = grep_uploadstb_logs_regex(r"event|publish|success")


Variable event_logs is not used.

Suggested change

event_logs = grep_uploadstb_logs_regex(r"event|publish|success")

grep_uploadstb_logs_regex(r"event|publish|success")

Copilot · 2026-01-21T19:06:05Z

test/functional-tests/tests/test_uploadstblogs_upload_strategies.py

+        result = run_uploadstblogs(args)
+
+        # Check strategy logging
+        logs = grep_uploadstb_logs_regex(r"strategy|STRAT_|upload.*type")


Variable logs is not used.

Suggested change

logs = grep_uploadstb_logs_regex(r"strategy|STRAT_|upload.*type")

grep_uploadstb_logs_regex(r"strategy|STRAT_|upload.*type")

Copilot · 2026-01-21T19:06:05Z

test/functional-tests/tests/uploadstblogs_helper.py

+    try:
+        subprocess.run(f"echo '' > {UPLOADSTB_LOG}", shell=True)
+        return True
+    except:


Except block directly handles BaseException.

Copilot · 2026-01-21T19:06:05Z

test/functional-tests/tests/uploadstblogs_helper.py

+        if os.path.exists(LOCK_FILE):
+            os.remove(LOCK_FILE)
+        return True
+    except:


Except block directly handles BaseException.

Copilot · 2026-01-21T19:06:05Z

test/functional-tests/tests/uploadstblogs_helper.py

+            for chunk in iter(lambda: f.read(4096), b""):
+                md5_hash.update(chunk)
+        return md5_hash.hexdigest()
+    except:


Except block directly handles BaseException.

Copilot AI review requested due to automatic review settings January 9, 2026 05:09

Abhinavpv28 requested a review from a team as a code owner January 9, 2026 05:09

Copilot started reviewing on behalf of Abhinavpv28 January 9, 2026 05:10 View session

Copilot AI reviewed Jan 9, 2026

View reviewed changes

Abhinavpv28 and others added 2 commits January 12, 2026 13:40

RDK-57502 - [RDKE] Migrate Operation Support Log Upload Related Scrip…

ffd0993

…ts To C Implementation (#49) * Simplify some pieces of code over engineered by AI models. * Update run_uploadstblogs_l2.sh for L2 coverage --------- Co-authored-by: Abhinav P V <Abhinav_Valappil@comcast.com>

Copilot AI review requested due to automatic review settings January 13, 2026 18:09

Copilot started reviewing on behalf of shibu-kv January 13, 2026 18:09 View session

Copilot AI reviewed Jan 13, 2026

View reviewed changes

Abhinavpv28 added 2 commits January 14, 2026 08:57

Enable breakpad support for analyzing any crashes

d68ca8c

Copilot AI review requested due to automatic review settings January 21, 2026 18:58

Copilot started reviewing on behalf of shibu-kv January 21, 2026 18:59 View session

Changelog updates for 2.0.0 release

6e7c40d

Copilot AI reviewed Jan 21, 2026

View reviewed changes

Merge branch 'release/2.0.0' into develop

8696fdc

		result = subprocess.run(f"grep '{marker}' /opt/logs/telemetry*.log 2>/dev/null",
		shell=True, capture_output=True, text=True)

-    result = subprocess.run(f"grep -c '{pattern}' {log_file} 2>/dev/null || echo 0",
-                          shell=True, capture_output=True, text=True)
-    return int(result.stdout.strip())
+    try:
+        count = 0
+        with open(log_file, "r", encoding="utf-8", errors="ignore") as f:
+            for line in f:
+                if pattern in line:
+                    count += 1
+        return count
+    except (FileNotFoundError, PermissionError, OSError):
+        return 0

	assert len(archive_logs) > 0, "Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"
	assert len(archive_logs) > 0, f"Archive process should complete. Found {len(archive_logs)} archive-related logs: {archive_logs}"

		@@ -0,0 +1,70 @@
		####################################################################################
		# If not stated otherwise in this file or this component's Licenses

	error_logs = grep_uploadstb_logs_regex(r"ERROR\|error\|fail.*properties\|invalid")
	error_logs = grep_uploadstb_logs_regex(r"ERROR\|error\|fail.*properties\|invalid")
	assert len(error_logs) > 0, "Configuration error should be logged for corrupted device properties"

	logs = grep_uploadstb_logs_regex(r"no.*file\|empty\|nothing to upload")
	grep_uploadstb_logs_regex(r"no.*file\|empty\|nothing to upload")

	upload_logs = grep_uploadstb_logs_regex(r"upload.*success\|uploading\|HTTP")
	upload_logs = grep_uploadstb_logs_regex(r"upload.*success\|uploading\|HTTP")
	assert len(upload_logs) == 0, "Should not log upload attempts without files"

	error_logs = grep_uploadstb_logs_regex(r"ERROR\|error\|fail.*properties\|invalid")
	grep_uploadstb_logs_regex(r"ERROR\|error\|fail.*properties\|invalid")

	logs = grep_uploadstb_logs_regex(r"size\|large\|limit\|truncate")
	grep_uploadstb_logs_regex(r"size\|large\|limit\|truncate")

		# Check for compression/archive activity
		compression_logs = grep_uploadstb_logs_regex(r"compress\|archive\|tgz")

	event_logs = grep_uploadstb_logs_regex(r"event\|publish\|success")
	grep_uploadstb_logs_regex(r"event\|publish\|success")

	logs = grep_uploadstb_logs_regex(r"strategy\|STRAT_\|upload.*type")
	grep_uploadstb_logs_regex(r"strategy\|STRAT_\|upload.*type")

Rebase #48

Are you sure you want to change the base?

Rebase #48

Uh oh!

Conversation

Abhinavpv28 commented Jan 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!