[BeforeTestRun] to throw an exception that would fail the test run
Hey Guys,
I throw an exception in ```[BeforeTestRun]``` hook. This solution throws exception I need but it just skips the tests and marks build as successful.
How to mark the build as failed if this hook throws an exception?
[Binding]
public class Hooks
{
private static readonly Device _device = new Device();
[BeforeTestRun]
public static async Task Deploy()
{
Console.WriteLine("Deploying {0}", _device.Version);
await _device.Deploy();
if (!_device.IsVersionDeployed())
{
throw new Exception($"Device version: { _device.Version } was not reported as deployed");
}
}
}
}
-
Which unit test runner are you using and how do you execute it in your pipeline?
0 -
SpecFlowPlusRunner
0 -
I assume that you are doing it then with vstest.console.exe or dotnet.exe.
It could be that VSTest is somehow hiding the exception, because it didn't happened during a scenario.
Are you executing the scenarios in parallel or not?
If not, you could move this to a BeforeScenario hook and do only the deploy when you instantiate the device class.
0 -
Scenarios are sequential.
Exception does show in HTML report. It looks a bit ugly you know something went wrong. The Log file has the exception.
Moving it to BeforeScenario would be a workaround, but really I need it only once. Maybe a better way would be to run form of ```IsVersionDeployed``` before scenario? Of course it always be correct if it was correct at the first check.
I was also thinking about some harsh way of dealing with it, maybe something like:System.Environment.Exit(1)
Is BeforeFeature gonna behave like BeforeTestRun?
Log from CISuccessfully generated reports.Successfully generated reportsResult: test framework error: At least one test thread aborted.Total: 19Succeeded: 0Ignored: 0Pending: 0Skipped: 19Failed: 00 -
I don't understand what your code is doing in the first post, but it seems you have some sort of precondition for the test run. Is that precondition something which can be verified in a separate script which could be executed before the test run?
That way you can avoid starting the test run if the precondition isn't met.If such a script isn't possible, what if you move the logic to a beforescenario hook and cache the result in a way such that the check only occurs once and the result is reused for any subsequent scenarios.
0 -
Yeah, no.
System.Environment.Exit(1)Just gives more confusing error.
Sam Shackleton Test framework takes care of which version of AUT is being tested. In Pytest-BDD it would have been called test fixture.
I could write separate console app to do this. Since C# does have API for what I need to do. The throw would not be hidden by VSTest.
My expectation was that if something throws in the test setup hook, everything would nicely blow up and the run would fail.0 -
Sam Shackleton I will consider both options. It would be nice if SpecFlow provided caching mechanism, need to read the manual. In fact it needs to be the first test that is aware if it's looking at the right AUT.
0 -
Actually I probably shouldn't have said caching.
What about this:
[Binding]
public class Hooks
{
private static readonly Device _device = new Device();
private static bool _isVersionDeployed;
[BeforeTestRun]
public static async Task Deploy()
{
Console.WriteLine("Deploying {0}", _device.Version);
await _device.Deploy();
_isVersionDeployed = _device.IsVersionDeployed();
}
[BeforeScenario]
public void EnsureVersion()
{
if (!_isVersionDeployed)
{
throw new Exception($"Device version: { _device.Version } was not reported as deployed");
}
}
}0 -
I went with this. Thanks Sam Shackleton.
0
Please sign in to leave a comment.
Comments
9 comments