disable parallel execution with specflow+ runner
hey
all my tests are running in parallel and for most of them its fine but only some of them are not capable of running in parallel
i have tried to disable the parallel execution with some methods without success (if this is impotent we are running the tests through Azure Devops CI )
what i have tried so far
creating partial classes with the same namespace and attribute NonParallelizable
namespace TestNameSpace
{
[NonParallelizable]
public partial class AutoGenFeatureClassName{ }
}
add the "addNonParallelizableMarkerForTags" to specflow.json (if feature file called Feature1.feature)
{
"generator" {
"addNonParallelizableMarkerForTags": [ "FeatureName1", "FeatureName2" ]
}
}
also tried some more phrasings like
{
"generator" {
"addNonParallelizableMarkerForTags": [ "FeatureName1Feature", "FeatureName2Feature" ]
}
}
and
{"generator" {
"addNonParallelizableMarkerForTags": [
{ "tag": "FeatureName1" },
{ "tag": "FeatureName2" }
]}}
please help me to understand how to disable those tests from running in parallel
(my last resort would be to change them all to be the same fetureName)
-
Hey Eliav Ran,
Have you included the .srprofile file in your project?
This is what controls the parallelisation attributes.
Please see https://docs.specflow.org/projects/specflow-runner/en/latest/Profile/Execution.html#execution and set 'testThreadCount' to 1 or remove the file.
please let me know.
0 -
thank you Jordan Western
just to clarify, i want to disable the parallel running only for some of the tests (features). all te rest can run in parallel
0 -
Ah, then I don't know a solution to this...
Can I ask why these specific tests can't be executed in parallel?
0 -
yes Jordan Western ,
this is not idle but we are testing process that include Stored Procedure that run on certain DB
all of those test are running on the same DB (dedicated for testing)they cant run in parallel because they are cleaning the db in between and inserting new data in the setup
so if they run together they will interrupt each other
0 -
Ok I see the problem. Are you cleaning the entire database or just adding/removing specific records?
1 -
all records that where added in the test
0 -
So the DB is not able to handle multiple calls concurrently?
0 -
i cant run multiple test scenarios in parallel because they are using the same tables for testing
lets say im given inserting in table grades information, then calling get avg, assert avg
in another feature given insert into table grades different setup then calling get all grades assert all customers
if both of the tests will access the same table and insert different data (suitable to each test ) they will fail because the expected result will be different (this is simplified example, our test case is more complicated )
0 -
Here are my suggestions (maybe they are not useful, but maybe they are!), as the runner can't execute some in parallel and some synchronously -
1. Tag these tests with something like '@ThreadUnsafe', have one test task in your CI/CD pipeline that executes the parallel tests excluding tests with this tag, and another task that targets your thread unsafe tests (you will need to do some config transformation in between to change the 'testThreadCount' property in the .srprofile) You can probably achieve the same thing with a .runsettings file, this might be cleaner. https://docs.specflow.org/projects/specflow-runner/en/latest/Profile/Profiles.html?highlight=runsettings#using-profiles
2. Place these tests in a separate project in the solution, this would make running the tests locally easier too as you would not have to mess around with any settings.
0
Please sign in to leave a comment.
Comments
9 comments