Specflow+ LivingDoc Test results generating as other instead of passing

Answered

December 21, 2020 19:54

Good afternoon,

I am using LivingDoc to generate documentation in AzDo to show the spec flow tests and I am running into an issue where the test results are not showing as passed nor failed but as others when the tests are passing in the pipeline. Has anyone been experiencing this?

Thanks in advance!

Comments

17 comments

Gerald Maboshe

December 29, 2020 12:39 Edited
Hey Andreas Willich I am also stuck on this. Any help?
0
Kiran Kumar Kalmekolan

December 31, 2020 16:50
Hi, I am facing the same issue.
0
Müller Mike

January 04, 2021 16:13
Hi, I'm having the same problem. Tests pass in the pipeline but the test results in LivingDoc are all shown as Others.
0
Östen Lundahl

January 04, 2021 16:23
Having the same problem.
0
Balazs Epresi

January 05, 2021 12:22 Edited
Good afternoon!

In this case we would need more information, so it would be great if all of you create a request/ticket and we can discuss the details there.

Thanks
0
Nils Lausberg

January 05, 2021 13:02
Not sure what is the exact configuration of the persons complaining about the issue in this thread, but believe it should be the same that I reported a few days earlier under https://support.specflow.org/hc/en-us/community/posts/360014167137-Specflow-Livingdoc-Azure-DevOps-plugin-Problems-with-uploading-test-results-with-Azure-Devops-Server-w-o-internet-access- In the comments you see some additional log entries I faced
0
Balazs Epresi

January 06, 2021 11:51 Edited
Hello,

Because I don't see any details of your projects, I can only add a generic information regarding this issue:

If you want to see the Test Execution results in AzDo, you have to provide the Test Project Root folder and not a subfolder (e.g. not the Features folder).

We are going to update our documentation and will try to provide some warnings when the configurations are not matching.

Nils Lausberg maybe this will solve your issue as well, please give it a try.
3
Östen Lundahl

January 06, 2021 16:45
Changing the root folder solved the issue for me.
1
Lynnox Davis

January 06, 2021 18:47
Balazs Epresi Thank you! That solved the problem.
1
Nils Lausberg

January 06, 2021 19:31
Balázs Epresi Yes, it also solves also my issue. So probably good to change documentation. Thank you
1
Kiran Kumar Kalmekolan

January 07, 2021 11:20
Balazs Epresi Thank you. Changing the root folder solved the issue for me too.
1
Gerald Maboshe

January 07, 2021 13:30
Changing feature folder to root works. Thanks!
1
Sudhindra Kumar

January 25, 2021 10:38
Balazs Epresi : Does this work for parallel execution using Specrun ?
0
Rishen Soekhoe

April 12, 2021 12:55
Hi, I'm facing this issue as well, though I am already providing the Test Project Root folder. Is there something else that could cause this issue?
0
icthomson7

March 18, 2022 15:15
Hello,

I'm facing a similar issue, I've switched to trying to use test assembly too but still facing this problem. I have 3 separate test projects in my solution and 2 of them show results fine, the other just shows as "Other".

I've tried digging in to see if there's any differences between the projects and setup, but can't see anything.

Anyone got any other ideas?
0

Eliott Morales

September 19, 2023 01:20 Edited

I have the same problem, report is generated but tests shown as Other. I can generate the report just fine on local. Azure is the problem. any suggestions? Thank you

- task: SpecFlowPlus@0
    inputs:
        generatorSource: 'TestAssembly'
        projectFilePath: '$(System.DefaultWorkingDirectory)'
        testAssemblyFilePath: '$(System.DefaultWorkingDirectory)\**\**\**\**\CRMTestProject.dll'
        testExecutionJson: '$(System.DefaultWorkingDirectory)\**\**\**\**\TestExecution.json'     
    continueOnError: true
    displayName: livingdoc

Eliott Morales

September 21, 2023 14:06

I have also tried this and same result. I have tried other configurations, The report gets generated but no test results displayed.

  - task: SpecFlowPlus@0
    inputs:
        generatorSource: 'TestAssembly'
        projectFilePath: '$(System.DefaultWorkingDirectory)\CRMTestProject'
        projectName: 'LivingDoc Reporter'
        testAssemblyFilePath: '$(System.DefaultWorkingDirectory)\CRMTestAutomation\bin\Debug\net6.0\CRMTestProject.dll'
        featureDataJsonFilePath: '$(System.DefaultWorkingDirectory)\CRMTestAutomation\bin\Debug\net6.0\TestExecution.json' 
        projectLanguage: 'en'   
        includeTestOutput: 'all'        
    continueOnError: true
    displayName: livingdoc1

Please sign in to leave a comment.

Specflow+ LivingDoc Test results generating as other instead of passing

Comments

Didn't find what you were looking for?