AI tools for ticket categorization and summarization

[Katie Nix]

Hi all! 

I have two use cases where I think an AI solution could help. We are open to paid 3rd party solutions!  

I've explored some of the built-in Atlassian Intelligence features and looked a bit on Marketplace, but not finding what I'm looking for yet.

1. Ticket categorization 

We have a huge backlog of feature requests and would like help categorizing them by "problem area". I can imagine using automation to pass ticket data one at a time out to an AI LLM (like the workflow described here) but I think a better solution would be trained on our own past tickets (i.e. we manually categorize 100 historical feature requests and the AI uses the history when categorizing new tickets). Does anyone know of a solution like this? 

2. Ticket summarization 

We have a project where developers and product managers answer questions about our product. There is often some back-and-forth on ticket comments until the question is fully answered. Then the ticket is handed off to our documentation team. It can take significant time for them to figure out what information from the ticket needs to be documented, and it would be helpful to have a quick summary to orient them as they pick up new tickets. The summary should consider not just the ticket fields, but also the comment history. Does anyone know of an app or integration that can smartly summarize, including ticket fields, comments, and questions?

[Rhys Diab _Rogue Wombat Software_]

Hi Katie,

A bit late, but if you're still looking for something to help summarise the content of comments, we've just released AI Issue Summariser. It will generate summaries of your issues, and takes into account long comment threads.

Full disclosure, I'm the creator of the app.

Kind Regards,

Rhys

[Elie Iskandar - Infosysta Apps]

Hi @Katie Nix ,

For your second use case, Smart AI for Jira can help!

Our app provides AI-powered ticket summarization, considering both ticket fields and comment history to generate clear and concise summaries. You can try it out and see how it streamlines your documentation process.

Additionally, Smart AI for Jira can also enhance your ticket descriptions, ensuring clarity and completeness before handoff. Plus, we offer an AI-powered reply feature that allows you to generate intelligent responses and add them as comments effortlessly.

Best,

Infosysta Team

[Payal _appbox_ai_]

Hi Katie,

Regarding your first question, while RAG or a trained LLM would yield superior results, you can also try compiling a list of relevant keywords mapped to their corresponding "valid" categories and include these examples directly in your prompts, which can improve the AI's performance in categorizing future issues. Also,  there isn't a marketplace application that currently supports local RAG. 

For your second inquiry, there are several AI-driven tools available to summarize issues. Our team at Appbox has also recently developed two marketplace applications that allow you to create custom prompts for generating ticket summaries, incorporating comments and other ticket fields, and performing actions that are AI-Driven.

GPT For Jira

AI Agents for Jira

I'm confident that these applications can address your use cases and offer additional functionalities to enhance your workflow. 

Thanks,

Team Appbox

[Ramachandran Radhakrishnan]

Hello Ms.Katie,

Just happened to see your this message. Surprisingly I just developed and implemented the same solution to one of my client (along with few other things) and would be happy to walk them through you (at no cost) over a zoom / gmeet.  Not sure if you have found a solution already but I am sure it's worth your time to have a look. I never thought this issue is faced by many using Jira.

Just drop a line  at rk@rfp.plus

[Clark Everson]

Hi @Katie Nix 

For number 1, it is possible to train AI on your own data but this actually wouldn't be an LLM for this type of AI. This would happen prior to even getting to an LLM, this is actually the foundation of AI. If your company has a data scientist they would be the best person to talk to, if not you may want to look at some consultants to help with this

For number 2 this is more what an LLM is about. If you were to use Jira Cloud's AI capabilities it should be able to do this from my understanding of how it works. However one thing to consider with AI it isn't just asking "summarize this", it needs to be prompted in a clear precise way for it to know what to look at.

So if I were to use Atlassians AI tool for instance and I was in a ticket with a lot of comments I would phrase it something like this :

1: 
You are a new team member onboarding into this work. Please review this ticket, including the fields and comments associated. Once reviewed please let me know when you are ready to provide a summary
AI responds: I am ready
You: Provide a summary of the most important data from a ticket that includes a brief overview of the problem statement, the acceptance and what the correspondence has been so far
Please format it as bullet points like
Problem statement
-Bullet 1
-Bullet 2
etc
Acceptance
-Bullet 1
Other information
-Bullet 1

etc

That is only one way, it depends on the model though

Once you make the templates for asking though they can generally be reused

Best,

Clark

[Katie Nix]

Thanks Clark!

Understood about #1, I mentioned LLM because the linked blog post used ChatGPT as an example. Anyway, we don't have the resources for me to pull a data scientist off of something else and do something custom. There are some apps out there for sentiment analysis, but I haven't found anything on the marketplace for categorization that can actually use our own data. Just wondering if anyone else has found something.

For #2 - can you point me toward what feature you're talking about that would do this? There is a "Summarize" button that released recently for ticket Comments which fulfills most of my use case but has to be triggered by the person looking at the ticket, pressing the button each time they want a summary. I don't see a way to trigger anything like what you're describing as part of a workflow, via native Atlassian tools. But there's a lot of new stuff that I haven't looked at.

[Clark Everson]

Hi @Katie Nix 

It would be Atlassian Intelligence to my knowledge: https://www.atlassian.com/platform/artificial-intelligence

For the first, LLMs are where you would ask it what it thinks, but you still do the work(think asking google). But the model on the backend would be what determines that, hence the training data. If there is a tool out there that does it, honestly hiring a contractor maybe cheaper for the one off model build. I would recommend going forward though building some automation rules in your projects that do the categorization off of different fields

Best,
Clark