[SpecFlow+LivingDoc] SpecFlow.Plus.LivingDocPlugin support for multiple project execution in a single run
Answered
As of LivingDoc.html contains details of single Project(i.e. based on the project executed in last) while executing test cases distributed across multiple project using CLI.
One of the Situation:
- Create Solution .Net4.8 containing SpecFlow Project "Project1" with Feature files as "Feature1" and "Feature2"
- Add another SpecFlow Project "Project2" with Feature files as "Feature3" and "Feature4"
- Update csproj property to generate binaries of all the projects in common folder "..\bin\Debug"<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<OutputPath>..\bin\Debug\</OutputPath>
</PropertyGroup> - Run Test cases using NUnit Console Runner, with CLI command containing multiple dlls
nunit3-console Project1.dll Project2.dll --where "test==Project1.Features.Feature1 || test==Project2.Features.Feature3" - FeatureData.json created containing details related to Project2 only
-
Hi Ankit,
This is currently not supported as the data in the feature file is always overwritten with the latest run.
However, we would be interested to work on it if more users vote for the issue on the forum.
Cheers
0 -
I have the same problem
0 -
I have livingdoc being created in azure devops, and I have a visual studio project full of tests. Of course I want to run these tests using many different azure devops pipeline. Some run against different environments and some form integration tests in particular environments. To this end I want to list each run and results in the living doc. I think this is what this feature request is for? In one pipeline I have the Project name as "Run tests against int env" and in the other I have "Run tests against staging env", but only the latest one seems to appear.
3 -
We have a similar setup to you by the sounds of it Andrew .
We have a repo with a SpecFlow Project that gets built in a build pipeline. This single build artifact is then used by 3 separate stages within a single pipeline:
Each of these stages initiate its own Specflow+Living Doc task and save its own TestExecution.json to its own folder location (outside of the agent pool folder that gets overwritten every run):
... Making sure that the SpecFlow.json is also pointing to the same location.
I had hoped that this would be enough to see all three of these test runs to exist within the LivingDoc interface, however, still, even with all the TestExecution.Json files available, only the latest run is displayed in the GUI:
We would love to see this feature accommodated for out of the box.
I have since found this link, but still not seeing it work as I had hoped.
0 -
SpecFlow+ LivingDoc is limited to saving one doc per branch of a repository.
What you can do is use the LivingDoc CLI in your release pipeline to generate the living doc per stage. Be aware that you need to handle the resulting LivingDoc html on your own.
Please see the following which should help you work through this - https://docs.specflow.org/projects/specflow-livingdoc/en/latest/LivingDocGenerator/Generating-Documentation.html
0 -
I believe its this limitation we would like to see removed. We are allowed to specify a Project Name, so that should be enough to allow a different entry to be made. Most teams will have Test Automation code in a master branch of a repository and will want to run it against different environments with different test suites. It would be great if we could list these runs and results in the living doc. Not sure how the rest of the above post is supposed to help us with getting this all to work inside azure devops?
2 -
Does anyone know if the LivingDoc support for multiple projects is being considered as a feature request?
0 -
This is a bit of an oversight as it won't support mono repos with multiple projects. Is this a possibility in the future?
0 -
Paul Smith In the case of mono repos we are suggesting using the SpecFlow+ LivingDoc CLI directly and upload the generated HTML file to a webserver of you.
0 -
Hello Andreas,
Given I have a test project in repro A and an other in repro B.
When I create the build pipeline executing tests A and tests B.
Then, I need a way to report the aggregated A and B results into the Azure DevOps.
Given the Merging Multiple Test Results — SpecFlow+ LivingDoc documentation solution, would I need to merge the specflow results json into one file (in a share I guess?) and point the Azure DevOps extension at that aggregated json file?
0 -
Johnathan Seal you are correct what you need to do.
For the both repos you need to generate with the LivingDoc CLI and output format JSON two TestExecution.json. In a separate pipeline that takes both files and merge them together.
But it will only be visible in one repo in SpecFlow+LivingDoc for Azure DevOps.
0 -
This is very limiting.
We have exactly the same issue above, multiple pipelines for each branch and every pipeline overrides the previous specflow+livingdoc entries.
What is the use of the 'projectName' setting then if we can only have a project at the time?
I would expect the results be overridden by projectName and not the entire content.
I don't think the solution to use an external web server is a good one. We use ADO for all our needs (build, test, check results, etc.) SpecFlow has livingDoc to allow the same so why create a separate environment?
Is this something that can be done?
6
Please sign in to leave a comment.
Comments
12 comments