How do we help UAT testers know what to test?

[Dreamforce Attendee]

From this comment in our Atlassian Cookbook for Salesforce Development:

One of the problems we have been experiencing is getting our internal UAT resources on the same page with our development team.  We have testers that work as part of the dev team but we also have another group in the company that does final "UAT" testing before we release a package to our customers.  Do you have any suggestions for workflow and Issue mgmt to help us with this?  The biggest challenge we keep getting back from these users is "we aren't sure what to test."  Keep in mind we are a Salesforce ISV developer and we produce applications that are sold to other Salesforce.com customers.  So this last level of UAT testing involves non-technical people in other departments that have to create customer facing content like training material, release notes, FAQ's, marketing collateral, demo's etc.  This is also the last "line of defense" before a new package is released to our customers.  

[devpartisan]

From some of the comments in the OP, I get the distinct impression that your UAT group don't have much context for the product they are testing. As such, I can see 2 paths:

The enlightened path is to help that UAT group understand what is being built through some form of collaborative specification. Discuss the business objectives and features with this UAT team early so they have a chance to develop their own understanding about how to explore the application. Also, take a tip from Rocket Surgery Made Easy and focus your UAT team's efforts on a small number of the most important potential issues.

The more cynical approach is to remove the UAT step altogether. Without sufficient time to build an understanding of what has been built, that team will be unable to make much of that testing effort. It is unlikely to be worth the delay in getting the product into your customer's hands.

[Melanie Wright]

We use JIRA Capture to track the acceptance criteria for each feature - this is written up well here: http://blogs.atlassian.com/2011/08/test_sessions_with_bonfire/ 

I usually build out the test sessions during planning and development - when I have questions about how a new feature should behave, I document the answers as test sessions. These are then available to guide us through the UAT testing phase. 

Having these tracked in test sessions is great for us because we can communicate expectations with traceability and accountability – we can show which scenarios we are anticipating and that we did actually test them. 

See the documentation here: Working with Sessions.

image2014-10-27 11:23:50.png

image2014-10-27 11:28:3.png

image2014-10-27 11:23:14.png