[SpecFlow] Analysing test duration

Robert Brown

June 17, 2021 00:17

I have some SpecFlow tests that provide test output like this:

Given my location is set to secret
-> done: Location.GivenMyLocationIsSetTo("secret") (0.0s)
And the date today is 03 Jun 2017 15:00
-> done: Date.GiveThatTheEffectiveDateIs(2017-06-03 3:00:0...) (0.0s)
And I have selected a client with the following properties
  --- table step argument ---
  | property                  | value      |
  | SomeAttr                  | someval    |
  | SomeAttr2                 | someval2   |
-> done: Client.GivenIHaveSelectedAClientWithTheFollowingProperties(<table>) (0.0s)
And I select a currency of CAD
-> done: Currency.WhenISetCurrency("CAD") (0.0s)
Then I see the following messages
  --- table step argument ---
  | classification | message       |
  | Class1         | Hello world   |
-> done: Message.ThenISeeTheFollowingMessages(<table>) (2.9s)

I have noticed that for all of my tests, the final step is the one that takes all the time (i.e. 2.9s in above - see the bottom line - all previous steps add up to zero).

I have many thousands of test cases and they take many hours to run, which has become a problem.

• Is this normal or to be expected?
• Does this duration metric include tear-down of the test harness itself (or is it purely the code-undertest)?
• Is there any tips on how I could go about analysing that duration and breaking it down further?

Note:

• I am not using Selenium
• The entire test takes 2.9s - 100% of which appears to be in that last step

1

• 

  Alexandra Horváth-Fung

  June 24, 2021 11:29

  Hi Robert Brown,

  The duration of the step shows how long that step is running. So basically it measures how long the binding method runs. If the step is really quick then it might display 0.
  We are not aware of any bugs around this, so most probably the Then step really takes long in the test suite. So you should analyze what happens in the Then step.

  0
• 

  Robert Brown

  June 29, 2021 01:28

  Thanks Alexandra,

  > you should analyze what happens in the Then step.

  Is there an way to have specflow report durations for method calls within the step?

  0
• 

  Viktor Nemes

  June 29, 2021 05:29 Edited

  No, Specflow is only measuring the execution of the whole step. To measure the duration of the method calls you need to use a profiler https://docs.microsoft.com/en-us/visualstudio/profiling/beginners-guide-to-performance-profiling?view=vs-2019 

  0

Please sign in to leave a comment.