I checked other feature requests, but they seem to be asking something slightly different from what I am asking:
Specflow+ Runner supports executing test cases on different targets. The example provided would start the calculator, and approach it through different browsers. In the reporting as generated by Specflow+ Runner itself, the different targets are reported on in isolation.
Unfortunately, this information is not persisted to TestExecution.json: the results provided there seems to be limited to the last test run being executed. Through some file merging, you can get a report that describes multiple test runs of the same scenario, including a summary on the number of passed/failed tests.
But there, the analysis options end: the step results shown seem to always be the first instance of a scenario that failed (as written in TestExecution,json). This means that if tests fail, you can't actually see on what specific target the test has failed.
What I would love to see is:
- A summary of test results per target, rather than a "x failed, y passed" list.
- The ability to select any of the targets on which a scenario is run, after which the step results should reflect the results related to the run of that scenario on that specific target.
- A top level filter option where I can select the target, and have the entire tree of features / scenarios filtered in accordance to the selected target, using the same behavior as already exists for other filtering options.
Showing the first failed test run as a default seems like a sane choice when no selection is created.
Please sign in to leave a comment.