txn/s for no of client > 1 is wrong

[scjoe]

If i have more than one client, (e.g. 4) i see that i can submit more new orders in same time, than if have only on client. For example.:

Result for one client on my machine (after 120s):
NEW_ORDER 1646 41954704.7615 39.23 txn/s

Result for four clients on my machine (after 120s):
Execution Results after 120 seconds
NEW_ORDER 4657 229757157.326 20.27 txn/s

Obviously 4657 is greater than 1646, but the calculated rate for the second run is smaller than the rate for the first. I think the way it is calculated is wrong. You cannot simply add the execution time in Results.stopTransaction(), because the client are executing in parallel. The right way would be to calculate tx/s per client and then add the results.

[scjoe]

As far as i understand the specification http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c_v5.11.0.pdf, it's much simpler:
The specification says in 5.4 'The reported MQTh is the total number of completed New -Order transactions (see Clause 5.1.2), ...
divided by the elapsed time of the interval. ....
In your test this simply means 'txn_cnt / duration'
Or am i getting something wrong?

[apavlo]

@scjoe I haven't worked on this code in almost a decade. Can you provide the line numbers for what you want me to look at?

Also, you are correct that the TPC-C spec says to report NewOrders per minute. Academic papers typically report the total throughput for all five txn types.

The TPC-C spec also says that there is supposed to be a wait time and terminal output, but nobody implements those parts either.

[scjoe]

In results.py line 120 the rate per second is calculated. The txn_time is the total of all measurements for a transaction type, e.g. 'new orders'. But because these transactions runs in parallel (at least with more than one client), the calculated rate is not the number of processed transactions per second.

[asya999]

Actually, technically it's correct, even if it's not useful. It's txn per second per thread so of course it'll be lower for multiple threads than for one thread.