Azure DevOps SpecFlow+LivingDoc publish report from TestExecution.json
AnsweredHi
We are facing an issue generating and publishing the test result report in Azure DevOps.
Our current workflow is to build .net projects and run tests within a docker container. Therefore, we use the LivingDoc CLI tool to generate the TestExecution.json file. This is all good. We also export the file out of the container so that it is available in the Azure DevOps pipeline.
The last step is a SpecFlowPlus@0 task in Azure DevOps. This idea (maybe we got it wrong and it is not possible) is to use the already generated TestExecution.json, together with the feature folder to publish a report to the specflow hub.
Unfortunately, the task ends in error (TargetInvocationException and NullReferenceException). We have tried with different versions of dotnet core on the build agent but same error. The yaml task definition looks like the one below.
Any help would be greatly appreciated.
Thank you
Dom
- task: SpecFlowPlus@0
inputs:
generatorSource: 'FeatureFolder'
projectFilePath: '<path to the source feature folder>'
projectName: '<our project name>'
testExecutionJson: '<path of TestExecution.json>'
projectLanguage: 'en'
-
Can you post the livingdoc cli call that runs in your docker container? Because the TestExecution.json is generated by the LivingDoc SpecFlow plugin and not the cli tool. So I am a little bit unsure what is really happening.
0 -
Hi Andreas.
These are the steps we run inside the container.
TestExecution.json is generated (in fact, the html report can be generated in the same way) and then it is copied outside the container with a COPY command.
We can see the generated TestExecution.json file on the build agent, so it is available for the Azure DevOps pipeline task.
RUN dotnet add package SpecFlow.Plus.LivingDocPlugin
RUN dotnet build "<our project file>" -c Release -o /app/build
# run tests on docker build
RUN dotnet test
#SpecFlow LivingDoc generation
RUN dotnet tool install --tool-path ~/bin SpecFlow.Plus.LivingDoc.CLI
RUN ~/bin/livingdoc feature-folder "Features" --output /src/livingdoc/TestExecution.json0 -
Hi,
I'm facing a similar situation. Can you confirm it's possible to generate the TestExecution.json (inside the container where tests run) and then export it so that it's available in the Azure DevOps pipeline?
This way the json is not being generated by the plugin, but by the living doc cli inside the container. So I'm not sure this will work. At the moment I'm getting the same error described in the initial post (TargetInvocationException and NullReferenceException).
Thanks,
Daniel
0 -
Domenico Del Vecchio & Daniel Melao
The TestExecution.json is generated by the SpecFlow.Plus.LivingDocPlugin NuGet package. And only by that.
It is never generated by the LivingDoc CLI tool.
The TestExecution.json is always an input for the livingdoc CLI. The parameter for this is "-t". You can't use it as output.
Also the folder has to be the root folder of your project. Not the folder of your feature files. If this is not correct, we don't can match the test results to the correct scenario.
What you want is to generate a FeatureData.json, that can then be used in the Azure DevOps task.
The parameter for it is --output-type JSON. This json-file includes everything. You can copy it then out of the container.
0 -
Thanks Andreas. It is working now.
0 -
Awesome!
0
Please sign in to leave a comment.
Comments
6 comments