Importing test results creates duplicate test

[srinathbhaskaran]

I am sorry if this is already answered but I could not find this question in the community.

Here is what we are tying to do:

1. Create a new test via app with test type as Automated[Generic] but entering Summary & Test Definition. We are entering the test definition as test script path('.' separated)+summary. Let's assume the test key for this test is MYPROJ-999.

2. We then create a Test Plan and include this test in the test plan.

3. We run the automated test and generate Junit xml report.xml

4. We post the result using REST API with POST method.

curl -H "Content-Type: multipart/form-data" -u user:pwd -F "file=@pcx-qm-report-stg-exp.xml" https://jira.my.com/rest/raven/1.0/import/execution/junit?projectKey=MYPROJ^&testPlanKey=MYPROJ-702400178

Expected Result:

The result for MYPROJ-999 should be updated with PASS/FAILED under the Test Plan. No new test should be created.

Actual Result:

The API is creating a new test under the Test Plan. The existing test MYPROJ-999 is not updated instead a new test created.

[José Domingues [Xray]]

Hi @srinathbhaskaran

To clarify some points, when you are importing JUnit execution results and you don't have the Tests for these results, previously created on your Jira (we try to find it using the Generic test Definition field), Xray will try to create the Tests. If you have required fields for the issue type Test, Xray is not able to create the Tests.

JUnit’s Test Cases are identified by the pair of attributes “classname” and “name” attributes. Test Cases are imported to Xray’s Generic Test issues, and the “classname” and “name” attributes are concatenated and mapped to the Generic Test Definition field of the Generic Test.
If a Test already exists with the same Generic Test Definition, then it is not created again.

Best Regards,

Team Xray

[Rishith Ramanand]

Hello @José Domingues [Xray] ,

Have one more question on the same lines. I have parameterized methods due to which now my test case will have 'n' entries with "name" attribute having brackets and the parameters in it. How to handle this kind of situation when importing JUnit results? Because the test remains the same just the parameters fed to the test differs

[Amit Rawat]

@José Domingues [Xray]  Can we get this logic modified and not just look for the Generic Test Definition field?

If you run Junit tests in parallel, then the Suite name is also added before the Class Name and Name fields.

And if I export a cucumber test case from my Xray Test Plan then there is no way I can match the test definition with the definition of my Junit Test Case.

Is there a way around if not a fix for this issue?

[Rogério Paiva - Xray Xporter]

Hi @Amit Rawat 

This is the way Xray works by design, so changing this behavior will affect thousands of customers and would cause a lot of stress.

So it is best to contact Xray Support (http://Xraysupport.getXray.app) to share your use case so that the support team can investigate an alternative solution.

Thank you.

Kind regards,
Rogerio Paiva [Xray Support Team]

[srinathbhaskaran]

I do not know if this helps but I have made further observations. So, as you all know, we can create test multiple ways as listed below.

#1 Create a new test using Jira UI by selecting test type as Automated[Generic] & then run cURL command to import results

[My observations]

• created test via jira ui
• imported the results
• no duplicate test is created
  
  

#2 Create a new test using REST API by setting test type as Automated[Generic] & then run cURL command to import results

[My observations]

• created test via REST API
• imported the results
• duplicate test is created
  
  

#3 Create a new test by cloning an automated test which got created as a results of importing results from REST API & then  run cURL command to import results

[My observations]

• created a new test by cloning an existing test which got created as a result of importing result. i have ensured the test definition field is populated as required.
• import the results for this new test
• no duplicate test is created

As far as I can tell, there is no difference between the test which I create via UI and the test which I am creating  via REST API. They look exactly the same. This is #2 scenario where it is creating duplicate test.

[srinathbhaskaran]

After lot of debugging, I found the root cause and it is so trivial but causes big issue. Not sure if this is a bug??

Anyways, here is the thing.

When we create a test of type Automated[Generic] using Jira app UI or the test gets created via import results API, the test definition field holds the value of classpath+'.'+classname value, but with a space in the beginning of the text.

But when I was creating a test via create issue api endpoint and I was populating the definition field, I was not giving a space at the beginning of classpath+'.'+classname value because I was not even aware that Jira/Xray adds a space in the beginning.

So, I tested by giving a space to the test definition field in the json payload and creating a test and then trying to import the results for the same test, I see that duplicate test is not created.

[Amit Rawat]

@srinathbhaskaran 

The test types I could see in XRay are Generic, Manual and Cucumber. How to create a test which you have mentioned as Automated[Generic]?

[srinathbhaskaran]

These are the options I see. My understanding is, you need to have these options visible on the UI before you can create the tests via REST API. 

@Amit Rawat