Add timed test suite wrapper macros for process_watchdog integration #520

parth21999 · 2026-01-21T00:39:28Z

Summary

Add TIMED_TEST_SUITE_INITIALIZE and TIMED_TEST_SUITE_CLEANUP macros in common/inc/c_pal/timed_test_suite.h
Provides automatic timeout protection for integration tests using process_watchdog
Watchdog init runs BEFORE user suite init code
Watchdog deinit runs AFTER user suite cleanup code
Add unit tests to verify fixture ordering

Dependencies

Requires Add variadic fixture support to TEST_SUITE_* macros c-testrunnerswitcher#271 for variadic fixture support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

parth21999 · 2026-01-21T00:50:34Z

/AzurePipelines run #Resolved

azure-pipelines · 2026-01-21T00:50:46Z

Azure Pipelines successfully started running 1 pipeline(s).

parth21999 · 2026-01-21T01:32:30Z

/AzurePipelines run #Resolved

azure-pipelines · 2026-01-21T01:32:46Z

Azure Pipelines successfully started running 1 pipeline(s).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

parth21999 · 2026-01-21T02:37:09Z

/AzurePipelines run #Resolved

azure-pipelines · 2026-01-21T02:37:23Z

Azure Pipelines successfully started running 1 pipeline(s).

parth21999 · 2026-01-21T02:41:29Z

/AzurePipelines run #Resolved

azure-pipelines · 2026-01-21T02:41:39Z

Azure Pipelines successfully started running 1 pipeline(s).

mattdurak · 2026-01-21T19:11:33Z

common/tests/CMakeLists.txt

    build_test_folder(thandle_ptr_ut)
    build_test_folder(real_thandle_log_context_handle_ut)
    build_test_folder(tqueue_ut)
+    add_subdirectory(timed_test_suite_ut)


add_subdirectory

Maybe should have a comment that it intentionally doesn't use c-testrunnerswitcher #Closed

Changed to use c-testrunnerswitcher.

mattdurak · 2026-01-21T19:12:03Z

common/tests/timed_test_suite_ut/main.c

+    if (timed_test_suite_ut_succeeded() != 0)
+    {
+        // Final validation of fixture ordering has failed
+        failedTests++;


can we Log something here? #Resolved

mattdurak · 2026-01-21T19:14:19Z

common/tests/CMakeLists.txt

+    add_subdirectory(timed_test_suite_ut)
 endif()

 if(${run_int_tests})


Consider an int test that demonstrates the timeout works. I guess since the process_watchdog will terminate, the test would probably need to launch that process and make sure it gets terminated within the timeout (+ delta)

mattdurak

dcristoloveanu · 2026-01-21T19:19:08Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+static int call_counter = 0;
+
+// Mock process_watchdog functions
+int process_watchdog_init(uint32_t timeout_ms)


process_watchdog_init

Wait, where is this used? #Resolved

This is called inside TIMED_TEST_SUITE_INITIALIZE macro.

Added specs and comment.

dcristoloveanu · 2026-01-21T19:19:19Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+    return 0;
+}
+
+void process_watchdog_deinit(void)


process_watchdog_deinit

And this one? where is it used? #Resolved

This is called inside TIMED_TEST_SUITE_CLEANUP macro.

Added specs and comment

dcristoloveanu · 2026-01-21T19:19:55Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+// This ensures the macros use our mock functions
+#include "c_pal/timed_test_suite.h"
+
+CTEST_BEGIN_TEST_SUITE(timed_test_suite_ut)


CTEST_BEGIN_TEST_SUITE

The mix of CTEST_ and TIMED_TEST_SUITE_INITIALIZE me does not likey

TIMED_TEST_SUITE_INITIALIZE/TIMED_TEST_SUITE_CLEANUP are the macros under test.

CTEST_BEGIN_TEST_SUITE/CTEST_END_TEST_SUITE are being used as the testing framework.

How else can we write these tests?

Changed to use BEGIN_TEST_SUITE/END_TEST_SUITE.

Does that make it better?

dcristoloveanu · 2026-01-21T19:21:47Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+    // Verify watchdog deinit was called AFTER user cleanup
+    if (watchdog_deinit_call_order <= user_cleanup_call_order)
+    {
+        (void)printf("ERROR: watchdog_deinit_call_order (%d) should be greater than user_cleanup_call_order (%d)\r\n",


printf

Aren't we in c-pal? Why do we need printf? #Resolved

Changing to use LogError.

dcristoloveanu · 2026-01-21T19:22:30Z

common/tests/timed_test_suite_ut/main.c

+{
+    size_t failedTests = 0;
+
+    (void)logger_init();


logger_init

Sooooo, if we fail we continue? #Resolved

Added check

dcristoloveanu · 2026-01-21T19:23:59Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+    }
+    else
+    {
+        result = 0;


This can be validated in the test suite cleanup and then we do not need the shoehorn-y timed_test_suite_ut_succeeded to be called from main.
I think it would make the test cleaner. #Resolved

dcristoloveanu

🕐

- Switch from CTEST_* macros to testrunnerswitcher macros for consistency - Add LogError when fixture ordering validation fails - Add detailed comment explaining mock function usage pattern - Convert from add_subdirectory to build_test_folder/build_test_artifacts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Replace printf with LogError in timed_test_suite_ut_succeeded() - Handle logger_init() failure properly in main.c - Only run tests if logger initialization succeeds Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Remove SRS_TIMED_TEST_SUITE_43_001 (default timeout constant) from requirements, code tags, and test tags - Add timeout value assertion in process_watchdog_init mock to verify the correct timeout_ms is passed - Add additional fixtures (additional_init_fixture, additional_cleanup_fixture) to verify variadic argument passing and fixture ordering - Add comprehensive fixture ordering assertions throughout Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

parth21999 · 2026-01-21T22:22:19Z

common/tests/timed_test_suite_ut/main.c

+
+    if (logger_init() != 0)
+    {
+        (void)printf("logger_init failed\r\n");


(void)printf("logger_init failed\r\n");

Use LogError

parth21999 · 2026-01-21T22:23:08Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+// Copyright (C) Microsoft Corporation. All rights reserved.
+
+#include <stdint.h>
+#include <inttypes.h>


Only need inttypes, dont need stdint also

parth21999 · 2026-01-21T22:32:40Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+// including timed_test_suite.h, we can track when they are called to verify fixture ordering.
+// Expected timeout value (must match TIMED_TEST_DEFAULT_TIMEOUT_MS from timed_test_suite.h)
+#define EXPECTED_TIMEOUT_MS 600000
+


Move the above the comment explaining the mock process_watchdog functions

azure-pipelines · 2026-01-21T22:40:01Z

No pipelines are associated with this pull request.

azure-pipelines · 2026-01-21T22:40:03Z

No pipelines are associated with this pull request.

azure-pipelines · 2026-01-21T22:40:08Z

No pipelines are associated with this pull request.

azure-pipelines · 2026-01-21T22:40:12Z

No pipelines are associated with this pull request.

parth21999 · 2026-01-21T22:41:39Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+        "user_init should be call #3");
+}
+
+/*Tests_SRS_TIMED_TEST_SUITE_43_006: [ TIMED_TEST_SUITE_CLEANUP shall call TEST_SUITE_CLEANUP with any additional fixtures passed via variadic arguments, followed by the watchdog deinit fixture as the last fixture. ]*/


Tests_SRS_TIMED_TEST_SUITE_43_006

Tests tag for the same requirement is placed multiple times. Place the tag only once.

parth21999 · 2026-01-21T22:43:28Z

common/tests/timed_test_suite_ut/timed_test_suite_ut.c

+#define EXPECTED_TIMEOUT_MS 600000
+
+/*Tests_SRS_TIMED_TEST_SUITE_43_002: [ TIMED_TEST_SUITE_INITIALIZE shall create a static fixture function that calls process_watchdog_init with timeout_ms. ]*/
+int process_watchdog_init(uint32_t timeout_ms)


process_watchdog_init

Add LogInfo in text fixtures, functions init/cleanup so that reading the test output clearly shows what happened

…leanup includes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add timed test suite wrapper macros for process_watchdog integration

ad00bbe

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

microsoft-github-policy-service bot requested a review from anporumb January 21, 2026 00:39

try with non-master c-testrunnerswitcher

4b86644

Update c-testrunnerswitcher to include variadic fixture support

8129d99

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

mattdurak reviewed Jan 21, 2026

View reviewed changes

mattdurak previously approved these changes Jan 21, 2026

View reviewed changes

dcristoloveanu reviewed Jan 21, 2026

View reviewed changes

dcristoloveanu requested changes Jan 21, 2026

View reviewed changes

parth21999 dismissed mattdurak’s stale review via 28e8234 January 21, 2026 21:31

parth21999 and others added 4 commits January 21, 2026 13:43

Replace printf with LogError and handle logger_init failure

64a8a77

- Replace printf with LogError in timed_test_suite_ut_succeeded() - Handle logger_init() failure properly in main.c - Only run tests if logger initialization succeeds Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Move fixture ordering validation into mock function

669a4e9

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add SRS requirements for timed_test_suite module

61ac87c

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

parth21999 commented Jan 21, 2026

View reviewed changes

Address PR review comments: add LogInfo, remove duplicate SRS tags, c…

a70b30e

…leanup includes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add timed test suite wrapper macros for process_watchdog integration #520

Are you sure you want to change the base?

Add timed test suite wrapper macros for process_watchdog integration #520

Conversation

parth21999 commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Dependencies

Uh oh!

parth21999 commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azure-pipelines bot commented Jan 21, 2026

Uh oh!

parth21999 commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azure-pipelines bot commented Jan 21, 2026

Uh oh!

parth21999 commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azure-pipelines bot commented Jan 21, 2026

Uh oh!

parth21999 commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azure-pipelines bot commented Jan 21, 2026

Uh oh!

mattdurak Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattdurak Jan 21, 2026 • edited by parth21999 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattdurak left a comment

Choose a reason for hiding this comment

Uh oh!

dcristoloveanu Jan 21, 2026 • edited by parth21999 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcristoloveanu Jan 21, 2026 • edited by parth21999 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcristoloveanu Jan 21, 2026 • edited by parth21999 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcristoloveanu Jan 21, 2026 • edited by parth21999 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

parth21999 commented Jan 21, 2026 •

edited

Loading

parth21999 commented Jan 21, 2026 •

edited

Loading

parth21999 commented Jan 21, 2026 •

edited

Loading

parth21999 commented Jan 21, 2026 •

edited

Loading

parth21999 commented Jan 21, 2026 •

edited

Loading

mattdurak Jan 21, 2026 •

edited

Loading

mattdurak Jan 21, 2026 •

edited by parth21999

Loading

dcristoloveanu Jan 21, 2026 •

edited by parth21999

Loading

dcristoloveanu Jan 21, 2026 •

edited by parth21999

Loading

dcristoloveanu Jan 21, 2026 •

edited by parth21999

Loading

dcristoloveanu Jan 21, 2026 •

edited by parth21999

Loading

dcristoloveanu Jan 21, 2026 •

edited by parth21999

Loading