Benchmarking: compare results over different runs

[chaoran-chen]

Add a diff subcommand to the evobench evaluation tool that takes two result datasets and prints percentage differences. Refinement needed.

[pflanze]

I now have an abstraction to represent the stats for all probes of a run, and a method to compare with another one.

Will also add a method to calculate stats across all benchmarking runs of the same version, too.

[pflanze]

This feature is implemented in two levels (or three when including level 0):

0 - "single"-run statistics: take a single log file and calculate the statistics for each probe. E.g. "evobench single a.log --excel output.xlsx". This has already been done before.

1 - "summary" statistics: take multiple log files of the same software version and calculate the statistics over one of the values of the statistics for the single run (from level 0). E.g. "evobench summary a.log b.log c.log --excel output.xlsx". This already necessitates a refactor to allow "higher level" statistics (statistics over statistics). This is done in 5827bdc

2 - "trend" statistics, which is what is described in this ticket: calculate the difference between two statistics from level 1. Using [ ] for grouping command line arguments: E.g. "evobench trend [ a.log b.log c.log ] [ d.log e.log ] --excel output.xlsx". This needs to be done still, but will probably not need new refactoring/features and thus will be done quickly.

[pflanze]

Ah, the reason that this was still open is the part 3, for which"This needs to be done still, but will probably not need new refactoring/features and thus will be done quickly." It's just low priority that why it's not done yet.