SpecRun / SpecFlow+ Runner retirement - Specflow reports replacement
Sad to learn about SpecRun+ retirement :( I see in the announcement that for reports replacement users are being directed to LivingDocs. Living Docs are great, but the statistics breakdown is more detailed in the specflow reports, and the summary failure links are useful too. A long time ago it used to be possible to generate the specfllow reports from an executable based on results from e.g. Nunit. Would it be possible to do the same using the TestExecution.json ?
David Galloway I'm sad too!
I have successfully implemented Runner in my project, each pipeline provides HTML report in the end and copies it into the blob storage where I have clean overview at all my test runs.
Now it breaks my whole work, I can't replace it with any other tool.
ExtentReports is not an option, Allure neither.
LivingDoc is nice but it does not display only the scenarios that did run and marks rerun scenarios which passed still as failed, what is not readable.
I'm still looking for some other solution.0
Here are my thoughts on what's most useful in the specflow reports and lacking in the living docs (which are excellent, but serve a slightly different purpose) :-
The most useful bits of the Specflow reports are at the top – you get
- overall stats at the test scenario level,
- the feature summary showing the breakdown of test results per feature.
- These 2 pieces of info really quickly help to identify new test failures (or even new test passes in the case that a new feature has been implemented)
- The Error Summary is useful for jumping straight to failed tests and also for quickly providing a sense of whether all tests failed for the same reason or if there are multiple causes
- I sometimes use table.CompareToInstance<T> and the display of the results for this in the specflow report are much better than in living docs
Please sign in to leave a comment.