Skip to main content

Changing from NUnit to SpecFlow Runner



  • Andreas Willich

    Could you try to delete your obj and bin folder and try it again after you changed the NuGet packages? Perhaps there are some leftover files that are causing this behavior.

  • Dave Hanna

    Sorry it took me so long to get back to you.

    I did try what you suggested. The test set is no longer duplicated, but the remaining set will not run.  The Outcomes line of the Group Summary page says "50 tests not run"


    Disappointing.  I had high hopes for SpecRunner.

  • Andreas Willich

    Dave Hanna, did you sign up for a SpecFlow account? You need one to use the SpecFlow+ Runner (see

    You have to look at the output pane in Visual Studio. There should be some text about it.

  • Dave Hanna

    I did sign up for an account, but it's not asking for it (or I failed to notice if it asked for it) when I run.


    I have another question - when you remove the SpecFlow.NUnit package, should you also remove the NUnit and NUnitAdapter packages, or leave them there?  And if you do remove them, what do you do for your Assert statements, since they are defined in NUnit.Framework?


  • Andreas Willich

    You need the latest version of the SpecFlow+ Runner (>= 3.4) that the activation works.

    Personally, I recommend using FluentAssertions for writing asserts with all unit test runners. But you can still use the NUnit Asserts with the SpecFlow+ Runner. They work. What is not working are the warnings that NUnit introduces some versions ago

  • Dave Hanna

    The SpecRun.SpecFlow package that is in the solution is 3.4.19.  I added NUnit back in after removing SpecFlow.NUnit and rebuilt.  This time I watched the Output window as I ran the test (or attempted to run them), and there were no messages beyon\d the build output.

    I don't know what else to do.


  • Andreas Willich

    You have to switch to the test output pane after running the tests.

  • Dave Hanna

    Well, I have gone to the link, created an account and activated it (twice now), but it continues to say "No tests run (activation needed.)"  I guess I'm just dead in the water.


  • Andreas Willich

    Ah, ok. Interesting

    Could you post the complete output? There should be some more information about what is happening.

    I will create a ticket for this thread, so support can have a deeper look at it with you.

  • Dave Hanna

    Andreas, there's one other thing you should know - It is possible that this may not be a SpecRun problem.   I work at a highly regulated bank that is knee-deep in security, and it is more than possible that they are blocking access to the necessary server..I am able to reach the website for activation from my browser, but I don't know if that is the same service that the test sequence goes to, and even if it is, it may be open from the browser but blocked from other applications (like Visual Studio).  This is all speculation and I don't have a way to know, but I don't want the team spending a lot of time chasing something that may not be solvable.


    I'm sorry, but I've already reverted back to the NUnit version of the code that I was working on, so I can't capture any additional output from the test run.

  • Andreas Willich

    Ah, then it could be that your corporate proxy is blocking the activation. In that case, I could have issued a legacy license key.

    If you want to give the SpecFlow+ Runner a new spin in the future, please contact our support and ask for a license key, because of your corporate IT security. 

  • david.hanna


    Thank you so much for your assistance.

    I did obtain license key, and succeeded in installing it.

    I tried running one of my tests to see what would happen, and I get a report file and a log file.  But they both show that no tests were run.

    Unnamed project Test Execution Report

    • Project: Unnamed project
    • Configuration:
    • Test Assemblies: C:\Users\...\RegressionTest\bin\Debug\netcoreapp3.1\RegressionTest.dll
    • Start Time: 10/18/2020 9:21:14 AM
    • Duration: 00:00:00.1272611
    • Test Threads: 1

    Result: tests executed with warnings

    Success rate Tests Succeeded Failed Pending Ignored Skipped
    n/a   0 0 0 0 0 0

    Test Timeline Summary



    2020-10-18T09:21:14.7559064-05:00;Thread#12;Info;Activation check completed
    2020-10-18T09:21:14.7560290-05:00;Thread#12;Info;Starting test run
    2020-10-18T09:21:14.7912508-05:00;Thread#12;Info;Discovering target: Default
    2020-10-18T09:21:14.7914945-05:00;Thread#12;Info;Discovering tests from assembly: C:\Users\...\RegressionTest\bin\Debug\netcoreapp3.1\RegressionTest.dll
    2020-10-18T09:21:14.8886626-05:00;Thread#12;Info;Discovered tests from assembly: C:\Users\...\RegressionTest\bin\Debug\netcoreapp3.1\RegressionTest.dll
    2020-10-18T09:21:14.8899869-05:00:Test Suite Dump
    2020-10-18T09:21:14.8914544-05:00:#0: Unnamed project/ - 
    2020-10-18T09:21:14.8914884-05:00:#1: Default - 
    2020-10-18T09:21:14.8914924-05:00:#2: RegressionTest - 
    2020-10-18T09:21:14.8915208-05:00:End: Test Suite Dump
    2020-10-18T09:21:14.9059134-05:00:Thread#0:Scheduler: get next test for thread 0
    2020-10-18T09:21:14.9073456-05:00:Thread#:Scheduler: at the end of normal test queue
    2020-10-18T09:21:14.9073745-05:00:Thread#0:Scheduler: enumerator for thread is at end
    2020-10-18T09:21:14.9074527-05:00:Thread#0:Scheduler: No more tests for thread
    2020-10-18T09:21:14.9142521-05:00:Thread#-1:Scheduler: get next test for thread -1
    2020-10-18T09:21:14.9143097-05:00:Thread#:Scheduler: at the end of normal test queue
    2020-10-18T09:21:14.9143129-05:00:Thread#-1:Scheduler: enumerator for thread is at end
    2020-10-18T09:21:14.9156186-05:00;Thread#12;Info;test run finished
    2020-10-18T09:21:14.9158703-05:00;Thread#12;Info;publishing test results
    2020-10-18T09:21:14.9160997-05:00;Thread#12;Info;Not publishing results.
    2020-10-18T09:21:14.9161992-05:00;Thread#12;Info;test results published
    2020-10-18T09:21:14.9162995-05:00;Thread#12;Info;generating reports
    2020-10-18T09:21:14.9999358-05:00;Thread#12;Info;Starting external report generation process
    2020-10-18T09:21:18.8731262-05:00;Thread#12;Info;Successfully generated reports.
    2020-10-18T09:21:18.8766230-05:00;Thread#12;Info;Successfully generated reports
    2020-10-18T09:21:18.8841776-05:00:Total: 0
    2020-10-18T09:21:18.8862659-05:00:Succeeded: 0
    2020-10-18T09:21:18.8865812-05:00:Ignored: 0
    2020-10-18T09:21:18.8867551-05:00:Pending: 0
    2020-10-18T09:21:18.8874919-05:00:Skipped: 0
    2020-10-18T09:21:18.8877003-05:00:Failed: 0
    2020-10-18T09:21:18.8928186-05:00;VSTestExecutionThread;Info;Adding attachments to VSTest

    I'm definitely getting closer, but I've still got something wrong.


    Again, thanks for your help.

  • david.hanna

    One more thing: After I posted the above response, I also tried removing the NUnit3.TestAdapter package.

    (When I converted, I removed the SpecFlow.NUnit package, but left NUnit and NUnit3.TestAdapter packages.  So, in this step I removed the TestAdapter.)


    The results were the same as shown previously.

  • Andreas Willich

    Which SpecFlow NuGet packages do you have in your project? 

    Is SpecRun.SpecFlow.3-4-0 in the list? 

  • david.hanna

    I have SpecRun.SpecFlow 3.4.19

    and SpecFlow.Plus.LivingDocPlugin 3.4.167

  • Andreas Willich

    Hmm, somehow the SpecFlow+ Runner can't find any scenarios in the assembly.

    Could you share the project with us? That's absolutely strange what is happening to you.

  • david.hanna

    I certainly would if I could.  Unfortunately, my employer is hypersensitive about security, and there are designed protocols that keep us from moving code outside the corporate network.  And if I found a way around it, I could lose my job.

    I appreciate you efforts, and realize that without seeing the project, there's not much more you can do.  Thanks for trying.

  • Andreas Willich

    Understandable. A lot of our users can't share the projects.

    You could try to compare your project with our examples ( to see if there are differences.

  • david.hanna


    I apologize for the lengthy delay.  It's been several months, and I've finally freed up enough bandwidth to be able to spend a few days on this.

    I've taking my existing NUnit based solution, stripped out the SpecFlow.NUnit package, as well as the NUnit.TestAdapter package (but kept the NUnit framework package).

    I've added the following NuGet packages:







    And the result builds, and will run tests (although there's something a little wonky in the test discovery, but I'll deal with that later). I get the test report html file in TestResults.

    But the tests fail.  The issue is that I have an appsettings.json at the root of the project, set with the CopyToOutput property as CopyAlways.  But there is now a SpecRunRunner directory in the bin/Debug/netcore3.1 directory, and when InitConfig attempts to access the appsettings.json file, it's looking for it in bin/Debug/netcore3.1/SpecFunRunner/netcore3.1 (i.e., two directories below the directory where it was copied.

    Since initializing the configuration is virtually the first thing any test does, I can't get any of the tests to run. I've tried a couple things to get it to find the appsettings file (changing the path where it's looking for it, and moving it into the SpecRunRunner directory ), and none of them worked.

    I'm a little baffled as to why it's looking for a file that's called out in the code in the SpecRunRunner directory, since that is not where the dll's end up

    Can you give me any hints?  I can probably supply some additional information without violating security, so please feel free to ask.

    Thanks for your patience and help.



  • david.hanna

    I got a bit of inspiration, and changed the access to the app settings to use a full path constructed using Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "appsettings.json")

    That seemed to work, but then it went on to the next problem, which is that, when it goes to open the WebDriver, it can't find chromedriver.exe, even though the directory shows that it's sitting right there in the execution directory (bin/Debug/netcoreapp3.1/), right alongside the DLLs constructed by the build.

    I'm baffled as to how all these changes came about just from trying to the change to the SpecFlow+ Runner.

  • david.hanna

    I found some code in one of the examples in the SpecFlow.Plus.Examples repository, specifically, the Selenium WebTest example.

    There, rather than instantiating the ChromeDriver with a simple "new ChromeDriver()", which is what I had been doing, they used the ChromeDriverService, passing it the location of ChromeDriver.exe.  So I used the same trick as I used with the appsetting.json file:


    return new ChromeDriver(ChromeDriverService


    , and everything ran.


Please sign in to leave a comment.

Powered by Zendesk