How to get testcases properties?

[nicola-lunghi]

I have the following xml

<testsuite name="LatencyTestSuite/LatencyTest" tests="8" failures="8" disabled="0" skipped="0"
  errors="0" time="20.006" timestamp="2023-10-19T09:02:39.209">
  <testcase name="LatencyTest/LAT_64" value_param="64" file="test_latency.cpp" line="125"
    status="run" result="completed" time="2.5" timestamp="2023-10-19T09:02:39.209"
    classname="LatencyTestSuite/LatencyTest">
    <failure
      message="test_latency.cpp:146&#x0A;Expected equality of these values:&#x0A;  test_result.EdmaRttValid&#x0A;    Which is: 0&#x0A;  1"
      type=""><![CDATA[test_latency.cpp:146
Expected equality of these values:
  test_result.EdmaRttValid
    Which is: 0
  1]]></failure>
    <failure
      message="test_latency.cpp:147&#x0A;Expected: (test_result.EdmaRtt01) &gt; (0), actual: 0 vs 0"
      type=""><![CDATA[test_latency.cpp:147
Expected: (test_result.EdmaRtt01) > (0), actual: 0 vs 0]]></failure>
    <failure
      message="test_latency.cpp:148&#x0A;Expected: (test_result.EdmaRtt02) &gt; (0), actual: 0 vs 0"
      type=""><![CDATA[test_latency.cpp:148
Expected: (test_result.EdmaRtt02) > (0), actual: 0 vs 0]]></failure>
    <properties>
      <property name="TestType" value="LATENCY_EDMA" />
      <property name="PktSize" value="64" />
      <property name="EdmaRttTimeout" value="0" />
      <property name="EdmaRttValid" value="0" />
      <property name="EdmaRtt01" value="0" />
      <property name="EdmaRtt02" value="0" />
      <property name="dma_error_reg" value="0" />
    </properties>
  </testcase>
</testsuite>

I need to extract the test properties there's a way to do it?
thanks,
Nicola

[EnricoMi]

Have you tried this? https://github.com/weiwei/junitparser#handling-custom-xml-attributes

[EnricoMi]

Looks like TestSuite has properties(), which provides access to properties of the suite, but TestCase does not have this. Should be easy to access this to TestCase, or your own derivative of TestCase.

Which flavour of XML dialect are you having there? Which tool generates this?

[nicola-lunghi]

I'm generating it with gtest. In gtest you can attach extra properties to a test case and they are added to the junit file.

…

On Thu, 19 Oct 2023, 14:43 Enrico Minack, ***@***.***> wrote: Looks like TestSuite has properties(), which provides access to properties of the suite, but TestCase does not have this. Should be easy to access this to TestCase, or your own derivative of TestCase. Which flavour of XML dialect are you having there? Which tool generates this? — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGB6YTAOL2JA5TY3JSNQ3UDYAEVBDAVCNFSM6AAAAAA6HA5BDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZRGAYTSOJVGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nicola-lunghi]

The use case is that we need to keep track of some values during the tests and then if they succeeded graph it out the values

…

On Fri, 20 Oct 2023, 07:38 nick83ola, ***@***.***> wrote: I'm generating it with gtest. In gtest you can attach extra properties to a test case and they are added to the junit file. On Thu, 19 Oct 2023, 14:43 Enrico Minack, ***@***.***> wrote: > Looks like TestSuite has properties(), which provides access to > properties of the suite, but TestCase does not have this. Should be easy > to access this to TestCase, or your own derivative of TestCase. > > Which flavour of XML dialect are you having there? Which tool generates > this? > > — > Reply to this email directly, view it on GitHub > <#116 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AGB6YTAOL2JA5TY3JSNQ3UDYAEVBDAVCNFSM6AAAAAA6HA5BDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZRGAYTSOJVGE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[weiwei]

Hi @nicola-lunghi, please check the example at https://github.com/weiwei/junitparser#handling-xml-with-custom-element to see if it helps.

[ZioTino]

Hi,
pytest also allows for custom properties at testcase level.
The output format in xml is the same as the one above.
It would be nice to see the same feature valid for testsuite implemented for testcase.

[gilmatok]

Hello to everyone reading this, I found a solution for this problem.

As you know, you can add custom properties to JUnit XML using Pytest's record_property function. Here's an example from my conftest.py:

@pytest.fixture(autouse=True)
def record_vm_info(request, record_property):
    vm_name = request.config.getoption('--vm-name')
    record_property('vm_name', vm_name)
    vm_ip = request.config.getoption('--vm-ip')
    record_property('vm_ip', vm_ip)

The above fixture will add the vm_name and vm_ip properties to the output JUnit XML file for each test case. You can also use record_testsuite_property to add properties to each test suite.

The output JUnit XML file will contain these properties like so:

<?xml version="1.0" encoding="utf-8"?>
...
<testcase name="test_name">
<properties>
<property name="vm_name" value="example" />
<property name="vm_ip" value="127.0.0.1" />
</properties>
</testcase>

To extract these properties with junitparser, you need to create two custom element classes:

class PropertiesElement(Element):
    _tag = 'properties'

class PropertyElement(Element):
    _tag = 'property'
    name = Attr()
    value = Attr()

Now to iterate over the properties, you need to iterate over the custom elements of the test case using iterchildren like so:

for suite in suite:
    for case in suite:
        properties = case.child(PropertiesElement)
        for property in properties.iterchildren(PropertyElement):
            print(f'Name: {property.name}, Value: {property.value}')

If you're using record_testsuite_property, you simply need to do the same for the test suite level.

@weiwei Despite the workaround, since properties is a feature supported by many platforms, I suggest adding support for this in junitparser.

[gilmatok]

@nicola-lunghi @ZioTino Please see my solution above.

[weiwei]

@gilmatok If it's from pytest then we have a "flavor" for it in https://github.com/weiwei/junitparser/blob/master/junitparser/xunit2.py which is supposed to fully support pytest. I think it'll be fine to add it over there. Care to submit a PR?

[eap]

It looks like ctest has also added a <properties> section to testcase output in its junit-xml output format. This makes a ton of sense for ctest because ctests often use tags to categorize, select, and filter tests both at runtime and in cdash.

Anyways, I have created PR #171 to address this feature gap in junitparser, and I wanted to add this extra context here to clarify that it is now the case that both ctest and pytest support properties in individual TestCases.