How to do paging with SearchService that works againts a static result set ?

[Emre Toptancı _OBSS_]

I am using Jira's SearchService for reporting againts large number of issues (>200K issues). Returning this many issues from a search and loading them into memory is clearly bad for the system. (Most of the time ends up with an OutOfMemory Exception)

On the other hand, SearchProvider.search() supports paging but each page has to be retrieved with a separate search() so searching repeatedly for different pages may return inconsistent data if the underlying data changes between your searches.

Keeping in mind that memory is the bottleneck:

1. How can I do paging against a static (or let's say stable) result set?
2. If that is not possbile, how can I do a search that returns only issueID's (so resultset uses far less memory) and I will retrieve each issue individually later (so garbage collector will collect the objects that I am done with)

I am working with Jira 7.x server but I prefer a solution that works on Jira 6.x as well, if possible.

P.S. I found some community answers in the site but most of them does not match my need or they are old and the api seems to be deprecated.

[Anton Chemlev - Toolstrek - · Accepted solution]

Hi, 

Did you try smth low-level like this - https://docs.atlassian.com/software/jira/docs/api/7.3.3/com/atlassian/jira/issue/search/SearchProviderFactoryImpl.html ?

[Emre Toptancı _OBSS_]

Thanks Anton,

I didn't try that and it sure looks promising.

While searching for a solution, I actually came across some other answers pointing to this but the answers mostly emphasized this was moved out of the API and the best way to go was to use SearchService. So I saw it as a deprecated api.

Anyway, I will try to derive a solution from here but it is hard to find a sample code about this. Any directions that will lead to a code sample will be greatly appreciated.

[Emre Toptancı _OBSS_]

I implemented the SearchProvider approach and got very good results. It is poorly documented (as usual) so requires some trial and error but sure works.

Using this approach I made a search that only returns issueIDs (List<String>) which uses very little memory. Iterating through this list I retrieved each issue individually and let the issue reference go as soon as I was done. This made a HUGE positive impact on memory use because now GC can collect issue objects that are no longer necessary.

Here is the code sample to get the issueId list:

 SearchProvider searchProvider = ComponentAccessor.getComponent(SearchProvider.class);
 FieldHitCollector collector = new FieldHitCollector("issue_id");
 List<String> values = collector.getValues();