Issue trend analysis - how do people make sense of customer tickets?

[Rob Hughes]

In joining my current B2B company as Product Manager I am struggling to draw product themes and trends out of our customer support tickets  that would help me and the team prioritise areas of focus. (approx 200 new tickets per week).

It's a historic Jira set up with coding best suited to issue resolution but not so much for ongoing product improvement / historic trend analysis.

In this context I find I am heavily reliant on textual analysis and it's only me doing this manually at present. So occasionally I dump a sample set into Excel and manually analyse the text and create my own themes.

This is quite fatiguing, and I expect some analytics tools or ML engines are out there that could help me, atleast with ad hoc sample analysis to help me get the right categorisations in place.

I am not aware that Jira does this natively nor have I seen any plugins.

Eventually I hope this analysis will help us to create the correct tags / field options in Jira to better manage going forward.

Does anyone else have this challenge and how do you solve? Thanks.

(EDIT - bonus points for ideas from people familiar with working within the constraints of GDPR!)

[Walter Buggenhout]

Hi @Rob Hughes and welcome to the Community!

I am not going to try to create the illusion that I will be able to solve all your challenges is a simple Q&A thread here, but I hope I can get you going in a certain direction.

From what you are saying, it seems that your service management practice currently lacks the structure you need to find areas of improvement. Obviously, reading through all the tickets after the facts is an insane effort and TG someone is willing to do it 😅.

But it would be much easier if categorising your tickets was just part of the process of handling tickets. I have just been engaged in a 1-year project with a customer in a similar situation. We first set up the service desk in such a way that all incoming requests were either incidents or service requests, using issue types to identify both.

Next, we spent some time (and refined as we moved along) what types of request or incidents were being reported. Using assets, we built a 3-level hierarchy of those requests (e.g. Incident > incident with the mobile app > log in issue). We used the workflow to require agents to categorise every ticket early on in the process and built reporting (with EazyBI) on top of that to analyse trends, as you mention.

Over the past year, that helped us identifying bottlenecks in the system, detecting customer complaints that could easily be avoided, conducting targeted experiments to solve structural issues and - in the end - get control of the overall service experience.

The first and most important change was identifying the categories that were relevant and categorising the tickets as soon as possible, in the tickets themselves.

Hope this helps!

[Rob Hughes]

thanks for your response Walter, your approach sounds very sound.
How did you go about this part of the process, what tools did you use to aid? - "Next, we spent some time (and refined as we moved along) what types of request or incidents were being reported."
That's the bit I'm looking out for any shortcuts on.

[Walter Buggenhout]

What types of incidents / requests is going to be specific to the products and/or services you provide to your customers, I would assume.

To give you a high level indication, the customer I'm referring to delivers both services, products and software. Under incidents, we categorised everything that could go wrong and grouped it roughly under:

- Level 1: incident (a defect or something that does not do what it is supposed to)

- Level 2: what is the incident related to / Level 3: more detail

      - e.g.: my product has a defect

            - e.g.: a warning light is flashing

            - e.g.: the product doesn.t work at all

      - e.g.: problem with my invoice

            - e.g. I did not receive last month's invoice

            - e.g. Invoice is not correct

      - e.g. there's an issue with my mobile app

            - e.g. I can't log in

            - e.g. a feature is not working

The service requests are more like questions people can ask to get more information about stuff, request for new features, ...

But, as I said, the list came to life by looking at the product and services offering that this customer had in place and by analysing what people were asking about. The categories are also never final, by the way. When necessary (e.g. a new service is being offered or a product is no longer available), we update the list.

In the beginning, we also offered the option "I have another question" to categorise things that did not match any of the predefined options. When we spotted trends there, that helped us define new and relevant categories as we went along.

It is and never has been the idea to build a complete hierarchy of all items that could possibly be reported. The idea was to get an overview on the most relevant issues people were reporting, so areas of improvement could be identified.

[Walter Buggenhout]

In terms of tools, it's pretty limited. The service desk is JSM premium, the categorisation is in Assets (part of the product), as it lets you set up multiple cascading custom fields to define the 3-level hierarchy.

For the reporting, we used EazyBI.