Skip to main content

Comments

9 comments

Date Votes
  • Official comment
    Mario Steiner
    Edited

    We are happy to announce the new SpecFlow Output API 🎉

    It is now possible to display texts and attachments in SpecFlow+ LivingDoc and in your IDE’s test explorer output window.

    Release notes: Releases - BDD framwork for NET (specflow.org)

    Get started: Output API — documentation (specflow.org)

    We are looking forward to your feedback and to further improve the Output API.

  • Gavrilovs, Jevgēnijs

    Screenshots help to realize the problem in tests much faster. Would be very nice to have this feature. 

    4
  • Andrew Knight

    Adding screenshots to LivingDoc for failed steps would be awesome. Currently, my team and I use an edited version of the SpecFlow+ report from SpecFlow+ Runner in which we capture and embed screenshots in the HTML. It makes a HUGE improvement in how we analyze and triage failures.

    As a hack, I tried to embed an HTML img tag with an image path in an exception message. However, when I generated the LivingDoc report, the exception message appeared as a failure for the correct step, but the HTML code was displayed as a literal string, not as embedded HTML. This is probably a good thing for security, but it means we cannot hack a screenshot into the report.

    Unfortunately, implementing this could be tricky. One way could be for LivingDoc to search for a specific type of error message format that has a path to a screenshot. Testers would need to make sure that (a) automation code captures screenshot files, (b) automation code puts the right message in the exception, and (c) screenshot paths link up correctly with the generated report. Another way could be for the SpecFlow framework to provide a custom exception type that includes a field for a screenshot path. Nevertheless, testers would still need to handle the three aforementioned cases.

    Ultimately, since screenshots are so valuable, I think it would be beneficial for SpecFlow to add some sort of mechanism to associate screenshots with steps. This idea can be broadened to other test artifacts like logs, REST API request dumps, downloaded files, etc. Essentially, SpecFlow could provide a way to associate artifacts with a step upon completion. Then, those artifacts could be listed or linked from the SpecFlow+ Runner report or the SpecFlow+ LivingDoc report.

    4
  • Mario Steiner
    Edited

    Update: In progress

    thanks, everyone for your interest and your feedback regarding the support of screenshots for failed scenarios/tests.

    We are currently trying to specify and implement the first version of the screenshot functionality.

    0
  • Vanessa Völk

    Thanks again for your interest and feedback so far. We would like to a closer look at this problem now, for which we appreciate your help. 

    It would be great, if all of you could answer some questions here, so we learn more about your needs.  

    Thanks, we're already excited for your insights!

     

    0
  • Karol Czechowski

    Hi Vanessa Völk, I just started using LivingDoc and I really like the way how it displays the scenarios from feature files. However, I really miss the feature with screenshots that Andrew Knight mentioned. What I did previously is HTML report generated by SpecRunner with screenshots attached but the biggest issue is that AzureDevops can't really display it properly and only way to get it is just download it. It's really painful so I am looking for some other solution. Another thing I didn't find in LivingDoc documentation is when I set up more than 1 testThreadCount in Execution settings in SpecRunner, Living Doc is not handling all the test results. Is there any solution for this?
    Thanks! 

    0
  • Andreas Willich

    Karol Czechowski We fixed the issue with the SpecFlow+ Runner and more than one Thread with the latest versions. You can now configure the generated file path to get a separate file for each thread (details: https://docs.specflow.org/projects/specflow-livingdoc/en/latest/LivingDocGenerator/Setup-the-LivingDocPlugin.html#placeholder-support-in-filepath) and then merge the multiple files together (details: https://docs.specflow.org/projects/specflow-livingdoc/en/latest/Guides/Merging-Multiple-test-results.html)

    1
  • Karol Czechowski
    Edited

    Andreas Willich Thanks for a quick response! I did those steps but one issue I face is that {ThreadId} value in filename is always the same ("2") and ProcessId names are unique so I can't pass it for merging to the AZDO filepath because wildcard doesn't work there. And I would like to have this not sensitive to the real number of generated JSON files.

    0
  • Vanessa Völk
    Update: after we received a lot of feedback to include screenshots (and other test output) into SpecFlow+ LivingDoc, we started working on a solution to support it.

    Therefore, we would like to get some early feedback about it and so we are making an interactive webinar next week. We will demo the current solution at the beginning and then open the mics for questions and discussion.
     
    The webinar takes place next Wednesday, May 12th at 4:00 pm CEST. 
    You can register here: 
    https://tricentis.zoom.us/webinar/register/WN_YGLN5RpZQVabM6_G2_FR1A
     
    We hope you are able to join and we can't wait to hear your feedback! 😊

    1

Please sign in to leave a comment.

Didn't find what you were looking for?

New post
Powered by Zendesk