Rovo Agent Not Following Prompt Instructions When Creating Jira Issues

[Kapuozzo Pitocco]

## Environment
- **Jira Cloud**: Free tier
- **Forge/Rovo**: Free tier development instance
- **Forge CLI Version**: 11.3.1
- **Node.js Version**: 22.15.0
- **Browser**: Chrome 135.0.7049.95 (Official Build) (64-bit)

## Issue Description
I've created a Jira Rovo agent designed to create bulk test issues without asking for user confirmation or issue details. According to my manifest.yml, the agent should:

1. Extract project key from user input (or ask for it if missing)
2. Extract number of issues to create (default 10)
3. Generate random summaries and descriptions automatically
4. Create all requested issues immediately without confirmation

**However**, despite explicit instructions in the prompt to never ask for details, the agent keeps asking users to provide or confirm issue details rather than generating them automatically.

## Expected Behavior
When a user says: "Create 2 issues where project key is GS"

The agent should:
1. Extract project key "GS"
2. Extract number of issues "2"
3. Automatically generate random summaries/descriptions
4. Create the issues immediately
5. Respond with only the list of created issue keys

## Actual Behavior
When I say: "Create 2 issues where project key is GS"

The agent responds with:
```
To create the issues, I'll need the following details for each issue:
1. **Issue 1:**
* Description:
* Issue Type:
* Summary:
2. **Issue 2:**
* Description:
* Issue Type:
* Summary:
Could you please provide these details?
```

OR sometimes it responds with:
```
Please review these details before I create one jira issue:
**Project Key:** GS
Please confirm if you would like to proceed.
```

## What I've Tried
1. Updated my prompt with stronger language explicitly stating NEVER to ask for confirmation
2. Added examples of correct behavior in the prompt
3. Fixed the API calls to use the proper route format
4. Made the action inputs more explicit
5. Enhanced error handling in the JavaScript code

## Code
### manifest.yml
```yaml
modules:
rovo:agent:
- key: jirabulkissuecreator-agent
name: JiraBulkIssueCreator
description: An agent for creating bulk test issues in Jira
prompt: |
# JiraBulkIssueCreator Agent

You are a specialized agent designed ONLY for creating bulk test issues in Jira projects.

## YOUR BEHAVIOR RULES (ABSOLUTE AND NON-NEGOTIABLE)
1. NEVER ask the user for issue details or confirmation
2. NEVER ask for summaries or descriptions
3. ALWAYS generate issue content yourself
4. ALWAYS create the requested number of issues immediately
5. ONLY ask for project key if it's missing

## WORKFLOW (FOLLOW EXACTLY)
1. Get project key from user (or ask ONLY for this if missing)
2. Extract number of issues (default: 10)
3. Extract theme if mentioned (otherwise use random themes)
4. For EACH issue:
- Generate a random summary
- Generate a random description
- Use "Story" as issue type unless specified otherwise
- IMMEDIATELY call create-jira-issue action
5. After creating ALL issues, respond with ONLY:
- "Created X issues for project Y:"
- List of issue keys

## EXAMPLES OF CORRECT BEHAVIOR

User: "Create 3 issues for project ABC"
You: "Creating 3 issues for project ABC..."
[ACTION: create-jira-issue with project=ABC] (3 times)
You: "Created 3 issues for project ABC: ABC-123, ABC-124, ABC-125"

User: "Create issues in my project"
You: "What's the project key?"
User: "DEF"
You: "Creating 10 issues for project DEF..."
[ACTION: create-jira-issue with project=DEF] (10 times)
You: "Created 10 issues for project DEF: DEF-45, DEF-46, ..."

## IMPORTANT REMINDERS
- You MUST generate ALL content WITHOUT user input
- You MUST create ALL issues immediately without confirmation
- The ONLY question you may ask is for project key if missing
- You MUST use the create-jira-issue action for EACH issue
conversationStarters:
- Create 5 test issues in my project ABC
- Generate some test tickets for DEF project
- Populate my XYZ project with test issues
actions:
- create-jira-issue
action:
- key: create-jira-issue
name: Create Jira issue
function: createJiraIssues
actionVerb: CREATE
description: Creates a single Jira issue in the specified project
inputs:
projectKey:
title: Project Key
type: string
required: true
description: The Jira project key (e.g., ABC, DEF, XYZ)
summary:
title: Summary
type: string
required: false
description: Issue summary/title
description:
title: Description
type: string
required: false
description: Detailed issue description
issuetype:
title: Issue Type
type: string
required: false
description: Type of issue (default is Story)
enum:
- Story
- Bug
- Task
- Epic
function:
- key: createJiraIssues
handler: index.createJiraIssues
app:
runtime:
name: nodejs22.x
id: ari:cloud:ecosystem::app/051c530d-7b4f-482f-af1d-27ce8112044b
permissions:
scopes:
- write:jira-work
```

### index.js
```javascript
// Enhanced index.js
import api, { route } from '@forge/api';

// Function to generate a random summary based on a theme
const generateSummary = (theme = '') => {
const themes = theme ? [theme] : ['Marketing', 'Development', 'Design', 'Support', 'Sales'];
const selectedTheme = theme || themes[Math.floor(Math.random() * themes.length)];

const summaries = {
'Marketing': [
`Create ${selectedTheme} campaign for Q2`,
`Design social media strategy for ${selectedTheme}`,
`Update website content for ${selectedTheme} initiative`,
`Plan ${selectedTheme} event for next month`,
`Develop email templates for ${selectedTheme} outreach`
],
'Development': [
`Implement new feature for ${selectedTheme} module`,
`Fix bug in ${selectedTheme} component`,
`Refactor ${selectedTheme} code for better performance`,
`Add unit tests for ${selectedTheme} functionality`,
`Update documentation for ${selectedTheme} API`
],
'Design': [
`Create mockups for ${selectedTheme} page`,
`Update UI elements in ${selectedTheme} section`,
`Design new icons for ${selectedTheme} feature`,
`Improve user flow in ${selectedTheme} process`,
`Redesign ${selectedTheme} dashboard layout`
],
'Support': [
`Investigate customer issue with ${selectedTheme}`,
`Update ${selectedTheme} knowledge base articles`,
`Create tutorial for ${selectedTheme} feature`,
`Improve ${selectedTheme} onboarding process`,
`Analyze feedback on ${selectedTheme} functionality`
],
'Sales': [
`Update ${selectedTheme} pricing strategy`,
`Create proposal template for ${selectedTheme} clients`,
`Research competitors in ${selectedTheme} market`,
`Develop presentation for ${selectedTheme} prospects`,
`Analyze ${selectedTheme} sales data from last quarter`
]
};

// If theme is provided but not in our list, create generic summaries
const themeSummaries = summaries[selectedTheme] || [
`Create new ${selectedTheme} initiative`,
`Update ${selectedTheme} documentation`,
`Improve ${selectedTheme} process`,
`Research ${selectedTheme} trends`,
`Develop ${selectedTheme} strategy`
];

return themeSummaries[Math.floor(Math.random() * themeSummaries.length)];
};

// Function to generate a random description
const generateDescription = (summary = '') => {
const descriptions = [
`This task involves creating comprehensive documentation and implementation plan. We need to ensure all stakeholders are aligned and have the necessary resources.`,
`We need to analyze current performance metrics and identify areas for improvement. This should result in a set of actionable recommendations.`,
`This requires coordination with multiple teams to ensure successful delivery. Please schedule a kickoff meeting to align on expectations and timeline.`,
`Research best practices and industry standards before implementation. Document findings and share with the team for review.`,
`This is a high priority item that needs to be completed before the end of the sprint. Focus on delivering an MVP that can be iterated upon.`
];

return `${summary} - ${descriptions[Math.floor(Math.random() * descriptions.length)]}`;
};

export const createJiraIssues = async (context) => {
console.log('Function called with context:', JSON.stringify(context));

// Extract parameters with more robust fallback strategy
let projectKey, providedSummary, providedDescription, issuetype;

// Try to extract from various possible locations in the context
try {
projectKey = context.projectKey ||
context.payload?.projectKey ||
context.parameters?.projectKey ||
(typeof context === 'string' ? JSON.parse(context).projectKey : null);

providedSummary = context.summary ||
context.payload?.summary ||
context.parameters?.summary;

providedDescription = context.description ||
context.payload?.description ||
context.parameters?.description;

issuetype = context.issuetype ||
context.payload?.issuetype ||
context.parameters?.issuetype ||
'Story';
} catch (e) {
console.warn('Error parsing context:', e);
// If JSON parsing fails, try to extract directly
if (typeof context === 'object') {
projectKey = context.projectKey;
}
}

console.log('Extracted parameters:', { projectKey, summary: providedSummary, description: providedDescription, issuetype });

// Validate required parameters
if (!projectKey) {
console.error('Missing required projectKey parameter');
throw new Error('Project key is required to create a Jira issue');
}

// Use provided values or generate them if not provided
const summary = providedSummary || generateSummary();
const description = providedDescription || generateDescription(summary);

console.log('Creating Jira issue with:', { projectKey, summary, issuetype });

try {
// Use route to define the Jira API endpoint
const jiraApiRoute = route`/rest/api/3/issue`;

const response = await api.asApp().requestJira(jiraApiRoute, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({
fields: {
project: { key: projectKey },
summary,
description: {
type: "doc",
version: 1,
content: [
{
type: "paragraph",
content: [
{
type: "text",
text: description
}
]
}
]
},
issuetype: { name: issuetype }
}
})
});

const data = await response.json();
if (!response.ok) {
console.error('Error creating Jira issue:', data);
throw new Error(`Failed to create Jira issue: ${data.errorMessages ? data.errorMessages.join(', ') : 'Unknown error'}`);
}
console.log('Jira issue created successfully:', data);
return {
issueKey: data.key,
success: true,
message: `Created issue ${data.key}`
};
} catch (error) {
console.error('Exception when creating Jira issue:', error);
throw error;
}
};
```

## Additional Information
The function for creating the issue works correctly when called directly. The API calls are now using the proper `route` format to comply with Forge's security model. The issue appears to be specifically with the agent not following the instructions in its prompt.

I've successfully deployed the app and there are no errors during deployment. The app is correctly installed in my Jira instance.

## Questions
1. Is there a way to verify that the agent is actually using the prompt I've defined?
2. Are there known limitations with Rovo agents and their ability to follow specific behavior instructions?
3. Could there be a caching issue with the agent prompt?
4. Is there a way to debug the agent's reasoning or see what prompt it's actually using?

Any guidance would be greatly appreciated, as I've exhausted the troubleshooting options I know of.

Thanks!

[Dr Valeri Colon _Connect Centric_ · Accepted solution]

Rovo agents may not always strictly follow prompt instructions due to underlying model behavior. There's currently no way to fully debug or view the prompt Rovo uses at runtime, and caching can sometimes cause older versions to persist. Try re-deploying the app and renaming the agent to force a refresh. Atlassian has not confirmed full support for deterministic agent behavior yet, so some variability is expected. Consider joining the developer community for more eyes on this.

[Kapuozzo Pitocco]

Hi Valeri, 

tnx for the reply. I also noticed issues with caching, reason why I uninstall/deploy/install at each new agent version, and I change the conversationStarters each time to make sure I'm on the new version (which is really painful tbh). 

I fact I'm not expecting complete predictability, as none of the underlying LLM provides the same, so there's no way for Atlassian to implement it. 

However, the agent predictably ignores everything I say in the prompt regarding function parameters and if there is any (whether it is required or not does not make any difference) it is requested from the user. if I remove them from the manifest, then they are completely ignored from the agent. I just do not get how the all thing is set up, undoubtedly being able to see what is actually passed to the underlying LLM would help a lot, but it is not given. 

Tnx for the suggestion to join the developer community, I'll do. 

Best, 

Giulio 

[Dr Valeri Colon _Connect Centric_]

@devpartisan I think this is another good question for your 'principles and practices for agent instructions (ie prompt engineering) in a GitHub repo.'

https://github.com/ibuchanan/forge-rovo-metaprompting/tree/main/prompts

[Dr Valeri Colon _Connect Centric_]

Thanks @Kapuozzo PitoccoGiulio. From my understanding you’re right — as of May 2025, Rovo agents interpret inputs in manifest.yml as structured prompts, which currently override or influence the LLM’s behavior, even if your agent prompt says not to ask for input. If a field is marked required or present in inputs, Rovo will try to get that from the user.

I don't think Atlassian has provided a way to fully override this with prompt logic alone. The best workaround (which you're already using) is to:

• Avoid defining inputs you don’t want the agent to request,

• Generate values directly in the action handler (index.js), and

• Use dynamic defaults or fallbacks in the code.

https://developer.atlassian.com/platform/forge/manifest-reference/modules/rovo-agent/