TestExecution.json file is not getting updated with the actual results.
Answered
Hello,
We are doing a POC using specflow and livingdoc for publishing the test results, but unfortunately the TestExecution.json file is not getting updated with the actual test results. I am using a simple xunit assertion and deliberately failing the test case, when I do a 'dotnet test' command, it is showing 1 test failed, but when I view the same in living doc, it is showing as not executed.
-
How do you execute the livingdoc cli to generate the LivingDoc.html after each test run?
0 -
I am not generating the LivingDoc.html on my local machine, instead I am directly running a SpecFlowPlus task on azure DevOps and publishing it, but when I see the test results, it is marked as not executed.
0 -
0 -
0 -
Ha, probably your test task doesn't have "Continue on error" set.
Without that, the following tasks are never executed and so the LivingDoc can't be updated, because the task is not executed.
0 -
Continue on error is actually set :) Below is the screenshot, TestExecution.json is getting updated every time, but just that it is reporting as not executed, where I expect to see a pass/failure status.
0 -
Could you post a screenshot of the LivingDoc task also?
0 -
0 -
Ok, that looks good.
Could you also make a screenshot of the output from the livingdoc task?
0 -
Ok I think I am missing there, SpecFlowPlus@0 is the only task which I am using currently.
0 -
The SpecFlowPlus@0 is the livingdoc task I meant.
0 -
Is this what you are looking for?
0 -
0 -
Hi Andi, could you please assist on what we could be missing? I have a demo on this and based on that we have to take a decision:) thanks
0 -
Please change the "projectFilePath" to "nBIF.RSME/FeatureTests".
You need to have the project path and not the solution path there.
0 -
Hey Andreas, so I am having a similar problem where I am getting a Error: Cannot find the specified test-execution-json during my pipeline build.
0 -
I don’t use YAML and just use in the build step in azure devops, but I don’t understand how it is having trouble finding the path when locally my TestExecution.json is there
0 -
Hi Andreas Willich, for me the issue is not about finding the TestExecution.json path, it is actually getting published in specflow living doc section successfully, but within the report the execution status is showing as "Not Executed"/"Skipped", where I do expect to see the test step marked as failed, if you see the screenshots above which I have posted, in the build console it is reporting as failed but not in the specflow living doc.
0 -
Lakshman, SPL (Sudheendran Periyapatna)
The projectFilePath is part of the logic to match the execution result with the scenarios. If it is not correct, we can't match and so we don't display the result.
It has nothing to do with finding the TestExecution.json.
0 -
Hi Andreas Willich Thanks, on correcting the path it worked:)
0 -
Awesome! Have fun with it!
0
Please sign in to leave a comment.
Comments
21 comments