Specflow + Living Docs reports dont match up?
Our net5 solution with
<PackageReference Include="SpecFlow.Plus.LivingDocPlugin" Version="3.9.57" />
<PackageReference Include="SpecRun.SpecFlow" Version="3.9.31" />
Configured to run in Azure Devops with Dotnet Test but the results are not the same:
From the console output of the dotnet Test
Attachments:
C:\windows\ServiceProfiles\NetworkService\AppData\Local\Temp\kvssf3wi.yjm\xxxxx Web API Tests_2022-01-07T161511.logC:\windows\ServiceProfiles\NetworkService\AppData\Local\Temp\kvssf3wi.yjm\TestResults_2022-01-07T162549.html
Failed! - Failed: 4, Passed: 140, Skipped: 0, Total: 144, Duration: 10 m 30 s - Tests.dll (net5.0)


-
Hey Antonio Chiong,
Can you provide some details of the tests that are showing as 'others' in your execution? Anything interesting in the logs? (maybe send them over to info@specflow.org if you can)
Is the same true when running the livingdoc CLI from your local machine, maybe you can try it? I would be interested to see the testexecution.json file generated by SpecFlow.Plus.LivingDocPlugin. If you try, it would be good for you to send this to me also.
Thanks, Jordan.
0 -
Hi Jordan,
I just run it locally and I was watching the testexecution.json and while running it seems to be collecting all the data but almost at the end from let's say from 30k file goes back to like 6k.
I email you the results of my local run
0 -
Hi Antonio Chiong,
If you are running tests in parallel, then you will need to generate multiple testexecution.json files and then squash them. Otherwise the results will be overwritten as far as I understand it.
please see this link for generating multiple json files per thread - https://docs.specflow.org/projects/specflow-livingdoc/en/latest/LivingDocGenerator/Setup-the-LivingDocPlugin.html?highlight=testexecution.json
and this link for merging the results - https://docs.specflow.org/projects/specflow-livingdoc/en/latest/Guides/Merging-Multiple-test-results.html
Please let me know if this does no solve your issue.
0 -
Yes it did thanks so much
0
Please sign in to leave a comment.
Comments
4 comments