Add assertions to the response

[tonybaloney]

Requirement

Add the number of assertions passed as a property of the test suite.

Proposed implementation

Add a hook function for pytest_assertion_pass to the NunitXML class, e.g.

    def pytest_assertion_pass(self, item, lineno, orig, expl): 
        ...

Use this to increment an instance property of assert_count.

When pytest_sessionfinish runs, add a assertions key to the stats dictionary and set the value as the assert_count value.

Then, modify NunitTestRun.test_suites() in nunit.py to set the assertions property to stats['assertions']. Currently it's hardcoded to None.

Test requirements

Test the following scenarios:

• Assertions is 0 when there are no assertions
• Assertions is 1 when there is one test case with 1 assertion
• Assertions is 3 when there is one test case with 1 assertion and another test case with 2 assertions

See https://github.com/tonybaloney/pytest-nunit/blob/master/pytest_nunit/nunit.py#L37-L39 and https://github.com/nunit/docs/wiki/Test-Result-XML-Format#assertions

Types already exist in the models classes.

[tonybaloney]

@adekanyetomie

[tonybaloney]

See https://docs.pytest.org/en/3.0.1/assert.html for details on how the assertions work

[tonybaloney]

After some exploration, the requirement to flush *.pyc* files after adding this feature is a big limitation.
Also, the assert hook only works for passed asserts. This would make for some confusing outputs. It's supposed to be total assertions (including, I assumed failed ones)

Fyi @asottile

[asottile]

yeah the new assertion hook is pretty limited -- probably why it's marked as experimental

I think we eventually need to get to a point where we version pytest's features in the pyc files (similar to how the abi is marked in the .so files for cpython) and automatically invalidate based on that

but until something like that exists the flushing needs to happen :/