JIRA Dashboard/Analytics and Reporting

[Ajay Taneja]

Hi,

I have imported unit test result into JIRA using XRAY-REST API and everything is fine and working perfect. I was now wanting to get some Analytics of the imported results using JIRA Dashboard.

However, I have not been successful to get any analytics. For example, for a test plan which corresponds to a repository, I have imported the results of 1000 tests. Each of these tests have been executed several times. Now, I want to see analytics which shows the trend of [performance of each test – how to get such analytics using out-of-the-box JIRA functionality? I am sure it should be possible.

For a test-plan page in JIRA I have the status results of tests populated in a table which has the fields for the columns as: Status, Resolution, Key, Summary (name of the test), Test Runs (number of times the test has been executed) and Test status (latest for the last run). 

Can anyone help? @Stefan Salzl , it would be great to interact again if possible!

Thanks a lot!

CC: @Stefan Salzl 

[Stefan Salzl]

Hi @Ajay Taneja 

let‘s wrap this up to make sure we are on the same page:

you imported a result file that covers 1000 testcases —> which means there is a test execution issue created with a test run for each of those tests? is that correct?

You said the tests ran several times. does that mean you imported several result files which means you‘ve also got several test executions with test runs of the test cases (in xray test cases are issues of type test).

What exactly would you like to see as a result in your analyze? what exactly do you mean with „performance“?

Best
Stefan

[Ajay Taneja]

Hi @Stefan Salzl 

What I meant was:

1)There are 1000 test cases - that is:1000 unit tests - test_1, test_2, test_3..., test_n,..test_1000

2) The developer commits the code to GitLab test_5, test_6, ...fail

3) He fixes the bug

4) Commits, passes

5) Cycle repeats

Thus 1000 unit tests, over a period of time, each test is executed several times - some times for a bug fix, some times when a new functionality is added..

Thus, I want to see some chart/analytics that shows the performance (that is, when the test(s) had passed, failed) over the period time - some dashboarding. That is what I meant

[Stefan Salzl]

Ok got that.

So still if you import a test execution file xray will create a test issue for each of the 1000 unit tests in jira (where it will reference the test run in the test execution) - if not exist already. 

So....
1) you run your unit tests the first time (cycle1). when you import the first cycle (unit test run) result file a test execution and the test issues will be created (also with some failed runs of lets say test_5, test_6)

2) you fix the bugs - and re-run the unit tests (cycle2)

3) now you import the result file of cycle2 --> a new test execution issue is created for this execution referencing your unit tests to the test issues that were created in step 1 (and showing test runs and results of cycle2)

4) let´s say you repeat step 2 & 3 two more times so you got 4 test executions, each with 1000 test runs (each is an instance of a test case that gets executed)

You could then use the test runs report (mentioned in the post by @Nicolas Grossi ) and filter for eg. your testplan. This will then list all your test runs.

some dashboarding

some chart/analytics

is pretty unspecific. We can´t guess what exactly you are expecting.

If you´d like to see a certain test case evolving over time you would have to have a dashboard/timeline for each of your test cases showing the results it´s testRUNs over the time.

Best
Stefan

[Ajay Taneja]

Hi @Stefan Salzl , I am not able to locate Test Runs over time dashboard

[Stefan Salzl]

@Ajay Taneja 

ok. referring to your screenshots this differs from what I understood from your previous requirement: How did tests behave over a timespan. The screenshots don´t show any timeline either. sololy a list of test and the amount of executions.

That´s why I tried to thoroughly identify the specific requirement.

Best
Stefan

[Ajay Taneja]

Hi @Stefan Salzl , those screenshots indeed show a list of test and the amount of executions., that is fine and is useful - the way it is organized

I still have my original requirement: How did tests behave over a timespan, you talked about a dashboard/timeline for each of the test cases showing the results it´s testRUNs over the time.

How can I get a Dashboard for "testRUNs over the time"?

I tried using the Pie chart but it doesn't give me the right number of tests as per the test plan. Also, it is not clear what filter to select

See attached, For the same test plan id 1484, which has 4 tests executed 25,5,1,1 times respectively  I don't see how its calculated 16 tests and all are passed and set to status done. Yet it shows TODO status