Generating unique TestExecution.json names on sequential multirun
CompletedHi,
We have a specflow project containing 7k+ scenario's.
We are migrating to LivingDocs for the documentation using the AzDo Extension.
We've segmented our testruns (using tag filtering but they are still running in sequence).
What we've found is that the TestExecution.json file is being overwritten in bin. This causes only the last testrun to be shown (using a single job to sequentially run the filtered test runs and after those runs we have the livingdocs task).
Looking at the documentation I see that it is possible to overwrite the default testexecution path to use a dynamic filename. Unfortunately I don't see how I can update this to represent a unique value in a sequential run. According to the documentation we have only 2 placeholders for the specflow.json file which we can use to make the filename more distinct. PID and TID. Since we don't run in parralel and we run the different runs in isolation (but still within the same job) this will work most of the time (issuing different PID for the 2nd run of the same process). However it is not guaranteed that it will be unique since windows reissues PIDs.
Is there a way to use the current datetime dynamically in the filename as a placeholder (since build and deploy steps are seperate and testrun is part of deploy)?
Furthermore if above is not feasible is there a way to achieve generating multiple unique testexecution.json for all segmented testruns without the need of manual PS or copy-rename tasks after every partial test execution?
current (partial) specflow.json:
"livingDocGenerator": {
"enabled": true,
"filePath": "TestExecution_{ProcessId}_{ThreadId}.json"
}
Frameworkversions:

-
PID and TID are the only placeholders at the moment. I will talk with the team if we can add the timestamp on short notice.
Until that, your only option is to be 100% sure that you move the files after each VSTest task.
1 -
In your upgrade now the latest Plugin version: 3.9.30
you can use the {Now} placeholder to include the timestamp in the generated json file names:
0 -
Thanks for the fix, we will upgrade soon and very the new feature.
0
Please sign in to leave a comment.
Comments
3 comments