Story Points Actual vs Estimate in Retrospective: were they accurate (in NextGen)

[Lionel Wolberger]

Background: A well known best practice in Agile/Scrum is to examine story point sizing upon conclusion of a sprint. Since this is such a basic Agile metric, we expect Jira to support it. 

About examining story point sizing: An item, whether a User Story, or Epic, will have estimated story points at the time of creation. When completed it can have assigned its actual story points, being the time it actually took to complete the item. Agile stakeholders learn a lot when, after a sprint, they examine the delta between "actual" and "estimated." Were the items sized correctly? Why not?  Such comparison by the team leads, over time, to better estimation.

Question: What is Jira's suggested best practice for recording, in a NextGen Jira Story Issue item, the actual story points (on moving the Jira object to "done")? Are we expected to add a custom field for "Actual Story Points"? 

Disambiguation note: this question does NOT address the story point estimate field in NextGen that went from not Global to Global; it does not address Classic JIRA; it does not ask specifically about Rules Fields.

[Anat Kutner]

@Bill Sheboy Hey! I would love to hear a bit more about the automation rules you created to measure the actual SP, this really sounds like something I could use with my teams,

Thanks!

[Bill Sheboy]

Hi @Anat Kutner 

We did not build automation rules to measure actual story points.  Honestly, I am unclear what "actual story points" means, unless a team is equating them to time units only.

Instead we built rules to capture status-by-status cycle time, and then used that for better cycle time measures than the built-in reporting provides, and to measure the Age of WIP.  We are now using one of the lighter-weight marketplace apps to do the cycle time measures.

IMHO, Building/buying a lot of stuff to measure "actual story points" seems non-value adding, and a permanent fix to a temporary symptom.  It may be better to pause at each retrospective to quickly ask the team: "Which of these stories ended up different than we expected?  Why did that happen?"  Reviewing each variation may reveal improvements, eliminating the need to save "actual story points".

Kind regards,
Bill

[Bill Sheboy]

Hi @Lionel Wolberger 

To confirm my understanding, is your team re-sizing "done" work at the end of sprints in order to measure this delta you are asking about?

Thanks, and best regards,

Bill

[Lionel Wolberger]

Thanks for the opportunity to clarify. 

It's not actually resizing. It is assessment of the original Story Point Estimate, as compared with the ACTUAL execution effort. 

The flow
* Estimate story points in backlog (poker, fibonacci)
* Enter the item in the sprint
* Ticket moves to "Done"
* Estimate ACTUAL story point time (needs another field? Change the story point field as a resize?)
* In Retrospective, examine the delta

[Bill Sheboy]

Thanks for the additional information, @Lionel Wolberger 

How high is the team's WIP and how much throughput do they have?

When WIP is low (so less task-switching), one alternative to estimating the actual size would be to review cycle time, look for outliers, and discuss those for learning opportunities.  The built-in cycle time report can help.  Although, there are better ones available on the marketplace or you can build your own with automation rules.

[Lionel Wolberger]

Thanks @Bill Sheldon for the clarifying questions. Our WIP, cycle time are all estimates. We seek a real number to compare against, so we can assess how well our teams are estimating story points. 

I did some more research and found this, How To Track Amount of Time In Status - 

This will provide real data, as it will provide the time in status for each issue (story, task, etc). The thread above discusses it at length, and unfortunately, shows that there is no consensus or simple answer. 

-- the Timentify plug-in is free but has very few reviews. 

-- the Time In Status plug-in from the marketplace. "Identify bottlenecks in your process by reporting on how much time each issue spent on each status, each assignee or each group." Dana wrote in Nov-2019 that " it ... hasn't been that useful or helpful. " 
-- It also mentions this Time in Status plug-in which is cheaper. "Generate reports of how long issues stayed in the specific status."

But "time in status" is still not "actual effort", as tickets can sit in a status for a while before they are picked up and actual developer time is spent. 

[Bill Sheboy]

Hi @Lionel Wolberger   Thanks for the update on your progress.

I think that until you have time measures, adding the custom field for "actual size" would help you diagnose what is happening with sizing forecasts aligning to resulting effort.

Rather than buy measurement tools, we built our own tracking using the automation rules feature and custom fields.  This allows us to measure lead time, cycle times (as needed), actual WIP, age of WIP, etc.  This gives teams better control over what is actually being measured (rather than a vendor's interpretation of "lead time" and "cycle time").

And you are correct: measuring this way would be calendar time rather than effort time.  The calendar time is more relevant to our Kanban teams.  You can mitigate this by removing non-working hours and days for measures in your calculations.

[Zoryana Bohutska _SaaSJet_]

Hi @Lionel Wolberger 

At this discussion https://community.atlassian.com/t5/Jira-Questions/How-to-track-amount-of-time-spent-in-status/qaq-p/590835, Dana mentioned Time in Status by OBSS : " We looked at the Time in Status plugin by OBSS, but so far find that it runs on a 24-hour clock and hasn't been that useful or helpful. "

Time in Status for Jira Cloud by SaaSJet lets set multi-calendars to exclude non-working hours from the calculation. This app allows to monitor cycle and lead time by using status groping feature (adding time of few statuses to one column) us you have mentioned behind.

Another tool that can help you is Time Between Statuses.  It lets set start/pause/end statuses for calculation cycle and lead time.

Also, you can try the build-in solution from Jira - Control Chart. It shows the average cycle and lead time at the diagram view.

Hope you will find the best solution for your request

Regards

Zoryana

[Lionel Wolberger]

Thanks @Zoryana Bohutska _SaaSJet_ , I appreciate your sharing the vendor solutions here. They look really relevant and helpful. 

We are a Next gen project: do you know of a "Control Chart" solution in Next Gen? 

[Zoryana Bohutska _SaaSJet_]

Sorry, no, it is available only for scrum and kanban
projects, but in this year it will be released for nextgen https://community.atlassian.com/t5/Jira-Software-questions/Control-chart-for-next-gen-project/qaq-p/1342760

By the way, mentioned apps work with nextgen.

Regards

Zoryna