Howto make multiple execution of a single test in a test execution in Xray

[Pål Skiftun Athammer]

Hi.

How can I add multiple test execution of the same test in a test execution? Or do i have to make multiple test execution for that test?

[Dave Mathijs]

Hi @Pål Skiftun Athammer welcome to the Atlassian Community!

A Test Run is part of a Test Execution.

See the documentation article on Test Runs for more info.

[Pål Skiftun Athammer]

So what i'm reading you can only have 1 test run pr test in each test execution. And have to make multiple test execution.

For me i have perform these tests 8 times.

So i have made test for each of the tests that needed to be tested. 

And then need to make 8 executions linked to my testplan for those tests.

Me problem is that I have to go back and forwards into those executions. because I want to run the 8 x test 1, then 8x test 2.

But since it is seperate execution it is a lot of back and forward on the different executions

[Stefan Salzl]

Hi @Pål Skiftun Athammer 

could you describe your use case in more details? especially: why would it be necessary to execute 1 and the same test case 8 times?

Best
Stefan

[JohnnyDoe]

@Stefan Salzl  I have same problem, i have cucumber test with parameters, i executed the test N times, each times with different parameters. e.g:

# Feature definition in Grehkin syntax
Feature: Is it Friday yet?
  Everybody wants to know when it's Friday

  Scenario Outline: DEMO-2555: Checking whether it is Friday
    Given today is "<day>"
    When I ask whether it's Friday yet
    Then I should be told "<resp>"
    Examples:
      | day    | resp |
      | Sunday | Nope |
      | Monday | Nope |
      | Friday | Yes  |

 This test is executed 3 times,, each time with different day- and i want to report the result of each test to xray. Do i really need to create separate test execution for each case? Or is there any way to differentiate each run and have it all in test runs ? Its one test with different parameters.

Now this is primitive example, but lets say i have test plan that contains 10 of cucumber tests. Each  release iteration i want to have new Test Execution that executes these tests . Each of these cucumber test has parameters like these and multiple Scenario outlines. Now i will have 1000 Test executions for each iteration? Thats not really wanted. Or is there some good practice to do so?

[Stefan Salzl]

Hi @JohnnyDoe 

to be honest I´m not quite sure if or how this works with cucumber tests as I´m not familliar with cucumber.

Nevertheless there is a similar functionality for manual tests as you could add datasets and refer to this dataset within your test steps.

This will later (in the test execution) create (so called) "iterations" within your test run for each of the options from your dataset.

Did you try to add and execute your test? What is/was the result? Would be curious too to see how this works with cucumber tests.

Best
Stefan

[JohnnyDoe]

@Stefan Salzl 

I am using api to import every data - would you mind navigating me to some example? The xray json format from docs literally does not work...

Also:

1)   I am sending result after each test execution, is there "merge" for test run ? E.g  create one if doesnt exist and update if it does? As if i execute test 3 times ( since there are 3 parameters) first time i want to craete test run with first step ( execution as u mentioned) and second and third time i want to just add step ( these are limitations of project since afaik each run of cucumber test with different paramete ris treated like different test run

2) If i add step, i dont see if it failed or passed. So it it kinda.. useless? Maybe i am just using wrong but i have trouble to see some working examples in docs.

Thanks.

[JohnnyDoe]

@Stefan Salzl i still fail to find any examples or guide how to do this using mix of adding dataset/steps and importing test results via REST API.

[Stefan Salzl]

Hi @JohnnyDoe 

unfortunately my knowledge in the area of cucumber tests in xray is not sufficient to guide you.

I would warmly recommend to get in touch with the vendor support team. They are eager to help and ask those deep questions.

Best
Stefan