Changing from NUnit to SpecFlow Runner
I have a SpecFlow project that I created with NUnit as the test runner. I want to try out the SpecFlow Runner.
How can I change the project over? (The instructions in the documentation don't work. They are basically to uninstall the SpecFlow.NUnit package and install the SpecRun.SpecFlow package. When I do that, the test set showing in the Test Explorer is duplicated, and neither set will run.)
Any help is appreciated. Thanks.
-
Could you try to delete your obj and bin folder and try it again after you changed the NuGet packages? Perhaps there are some leftover files that are causing this behavior.
0 -
Sorry it took me so long to get back to you.
I did try what you suggested. The test set is no longer duplicated, but the remaining set will not run. The Outcomes line of the Group Summary page says "50 tests not run"
Disappointing. I had high hopes for SpecRunner.
0 -
Dave Hanna, did you sign up for a SpecFlow account? You need one to use the SpecFlow+ Runner (see https://specflow.org/blog/introducing-the-specflow-account/).
You have to look at the output pane in Visual Studio. There should be some text about it.
0 -
I did sign up for an account, but it's not asking for it (or I failed to notice if it asked for it) when I run.
I have another question - when you remove the SpecFlow.NUnit package, should you also remove the NUnit and NUnitAdapter packages, or leave them there? And if you do remove them, what do you do for your Assert statements, since they are defined in NUnit.Framework?
Thanks.
0 -
You need the latest version of the SpecFlow+ Runner (>= 3.4) that the activation works.
Personally, I recommend using FluentAssertions for writing asserts with all unit test runners. But you can still use the NUnit Asserts with the SpecFlow+ Runner. They work. What is not working are the warnings that NUnit introduces some versions ago
0 -
The SpecRun.SpecFlow package that is in the solution is 3.4.19. I added NUnit back in after removing SpecFlow.NUnit and rebuilt. This time I watched the Output window as I ran the test (or attempted to run them), and there were no messages beyon\d the build output.
I don't know what else to do.
0 -
You have to switch to the test output pane after running the tests.
0 -
Well, I have gone to the link, created an account and activated it (twice now), but it continues to say "No tests run (activation needed.)" I guess I'm just dead in the water.
0 -
Ah, ok. Interesting
Could you post the complete output? There should be some more information about what is happening.
I will create a ticket for this thread, so support can have a deeper look at it with you.
0 -
Andreas, there's one other thing you should know - It is possible that this may not be a SpecRun problem. I work at a highly regulated bank that is knee-deep in security, and it is more than possible that they are blocking access to the necessary server..I am able to reach the website for activation from my browser, but I don't know if that is the same service that the test sequence goes to, and even if it is, it may be open from the browser but blocked from other applications (like Visual Studio). This is all speculation and I don't have a way to know, but I don't want the team spending a lot of time chasing something that may not be solvable.
I'm sorry, but I've already reverted back to the NUnit version of the code that I was working on, so I can't capture any additional output from the test run.
0 -
Ah, then it could be that your corporate proxy is blocking the activation. In that case, I could have issued a legacy license key.
If you want to give the SpecFlow+ Runner a new spin in the future, please contact our support and ask for a license key, because of your corporate IT security.
0 -
Andreas,
Thank you so much for your assistance.
I did obtain license key, and succeeded in installing it.
I tried running one of my tests to see what would happen, and I get a report file and a log file. But they both show that no tests were run.
Unnamed project Test Execution Report
- Project: Unnamed project
- Configuration:
- Test Assemblies: C:\Users\...\RegressionTest\bin\Debug\netcoreapp3.1\RegressionTest.dll
- Start Time: 10/18/2020 9:21:14 AM
- Duration: 00:00:00.1272611
- Test Threads: 1
Result: tests executed with warnings
Success rate Tests Succeeded Failed Pending Ignored Skipped n/a 0 0 0 0 0 0 Test Timeline Summary
2020-10-18T09:21:14.7559064-05:00;Thread#12;Info;Activation check completed 2020-10-18T09:21:14.7560290-05:00;Thread#12;Info;Starting test run 2020-10-18T09:21:14.7912508-05:00;Thread#12;Info;Discovering target: Default 2020-10-18T09:21:14.7914945-05:00;Thread#12;Info;Discovering tests from assembly: C:\Users\...\RegressionTest\bin\Debug\netcoreapp3.1\RegressionTest.dll 2020-10-18T09:21:14.8886626-05:00;Thread#12;Info;Discovered tests from assembly: C:\Users\...\RegressionTest\bin\Debug\netcoreapp3.1\RegressionTest.dll 2020-10-18T09:21:14.8899869-05:00:Test Suite Dump 2020-10-18T09:21:14.8914544-05:00:#0: Unnamed project/ - 2020-10-18T09:21:14.8914884-05:00:#1: Default - 2020-10-18T09:21:14.8914924-05:00:#2: RegressionTest - 2020-10-18T09:21:14.8915208-05:00:End: Test Suite Dump 2020-10-18T09:21:14.9059134-05:00:Thread#0:Scheduler: get next test for thread 0 2020-10-18T09:21:14.9073456-05:00:Thread#:Scheduler: at the end of normal test queue 2020-10-18T09:21:14.9073745-05:00:Thread#0:Scheduler: enumerator for thread is at end 2020-10-18T09:21:14.9074527-05:00:Thread#0:Scheduler: No more tests for thread 2020-10-18T09:21:14.9142521-05:00:Thread#-1:Scheduler: get next test for thread -1 2020-10-18T09:21:14.9143097-05:00:Thread#:Scheduler: at the end of normal test queue 2020-10-18T09:21:14.9143129-05:00:Thread#-1:Scheduler: enumerator for thread is at end 2020-10-18T09:21:14.9156186-05:00;Thread#12;Info;test run finished 2020-10-18T09:21:14.9158703-05:00;Thread#12;Info;publishing test results 2020-10-18T09:21:14.9160997-05:00;Thread#12;Info;Not publishing results. 2020-10-18T09:21:14.9161992-05:00;Thread#12;Info;test results published 2020-10-18T09:21:14.9162995-05:00;Thread#12;Info;generating reports 2020-10-18T09:21:14.9999358-05:00;Thread#12;Info;Starting external report generation process 2020-10-18T09:21:18.8731262-05:00;Thread#12;Info;Successfully generated reports. 2020-10-18T09:21:18.8766230-05:00;Thread#12;Info;Successfully generated reports 2020-10-18T09:21:18.8841776-05:00:Total: 0 2020-10-18T09:21:18.8862659-05:00:Succeeded: 0 2020-10-18T09:21:18.8865812-05:00:Ignored: 0 2020-10-18T09:21:18.8867551-05:00:Pending: 0 2020-10-18T09:21:18.8874919-05:00:Skipped: 0 2020-10-18T09:21:18.8877003-05:00:Failed: 0 2020-10-18T09:21:18.8928186-05:00;VSTestExecutionThread;Info;Adding attachments to VSTest 2020-10-18T09:21:18.8928857-05:00;VSTestExecutionThread;Info;adding
I'm definitely getting closer, but I've still got something wrong.
Again, thanks for your help.
0 -
One more thing: After I posted the above response, I also tried removing the NUnit3.TestAdapter package.
(When I converted, I removed the SpecFlow.NUnit package, but left NUnit and NUnit3.TestAdapter packages. So, in this step I removed the TestAdapter.)
The results were the same as shown previously.
0 -
Which SpecFlow NuGet packages do you have in your project?
Is SpecRun.SpecFlow.3-4-0 in the list?
0 -
I have SpecRun.SpecFlow 3.4.19
and SpecFlow.Plus.LivingDocPlugin 3.4.167
0 -
Hmm, somehow the SpecFlow+ Runner can't find any scenarios in the assembly.
Could you share the project with us? That's absolutely strange what is happening to you.
0 -
I certainly would if I could. Unfortunately, my employer is hypersensitive about security, and there are designed protocols that keep us from moving code outside the corporate network. And if I found a way around it, I could lose my job.
I appreciate you efforts, and realize that without seeing the project, there's not much more you can do. Thanks for trying.
0 -
Understandable. A lot of our users can't share the projects.
You could try to compare your project with our examples (https://github.com/SpecFlowOSS/SpecFlow.Plus.Examples) to see if there are differences.
0 -
Andreas,
I apologize for the lengthy delay. It's been several months, and I've finally freed up enough bandwidth to be able to spend a few days on this.
I've taking my existing NUnit based solution, stripped out the SpecFlow.NUnit package, as well as the NUnit.TestAdapter package (but kept the NUnit framework package).
I've added the following NuGet packages:
Microsoft.NET.Test.Sdk
Microsoft.TestPlaftorm.TestHost
SpecFlow.Tools.MsBuild.Generation
SpecRun.Runner
SpecRun.SpecFlow
And the result builds, and will run tests (although there's something a little wonky in the test discovery, but I'll deal with that later). I get the test report html file in TestResults.
But the tests fail. The issue is that I have an appsettings.json at the root of the project, set with the CopyToOutput property as CopyAlways. But there is now a SpecRunRunner directory in the bin/Debug/netcore3.1 directory, and when InitConfig attempts to access the appsettings.json file, it's looking for it in bin/Debug/netcore3.1/SpecFunRunner/netcore3.1 (i.e., two directories below the directory where it was copied.
Since initializing the configuration is virtually the first thing any test does, I can't get any of the tests to run. I've tried a couple things to get it to find the appsettings file (changing the path where it's looking for it, and moving it into the SpecRunRunner directory ), and none of them worked.
I'm a little baffled as to why it's looking for a file that's called out in the code in the SpecRunRunner directory, since that is not where the dll's end up
Can you give me any hints? I can probably supply some additional information without violating security, so please feel free to ask.
Thanks for your patience and help.
0 -
I got a bit of inspiration, and changed the access to the app settings to use a full path constructed using Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "appsettings.json")
That seemed to work, but then it went on to the next problem, which is that, when it goes to open the WebDriver, it can't find chromedriver.exe, even though the directory shows that it's sitting right there in the execution directory (bin/Debug/netcoreapp3.1/), right alongside the DLLs constructed by the build.
I'm baffled as to how all these changes came about just from trying to the change to the SpecFlow+ Runner.
0 -
I found some code in one of the examples in the SpecFlow.Plus.Examples repository, specifically, the Selenium WebTest example.
There, rather than instantiating the ChromeDriver with a simple "new ChromeDriver()", which is what I had been doing, they used the ChromeDriverService, passing it the location of ChromeDriver.exe. So I used the same trick as I used with the appsetting.json file:
return new ChromeDriver(ChromeDriverService
.CreateDefaultService(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
, and everything ran.
0
Please sign in to leave a comment.
Comments
21 comments