Need guidance on pagination of JIRA align APIs

[arnavane]

Greetings Team, I am working API integration for JIRA align to process EPIC and features information. I could see there is limitation of the max number of records in the response. I could not find total number of records and options to traverse through all the records. I had referred Top but that too not helping beyond 100. As we have skip option we can loop the API calls till we get empty response. Request your guidance if we can make the data fetch without limit (with privileged role) or other cleaner,  better approach to fetch all records (any bulk option by submitting request asynchronously and process further once the response is ready. Thank you 

When I tried https://cisco.jiraalign.com/rest/align/api/2/features?expand=true&$filter=parentId eq <epic id1> or parentId eq <epic id2> ...... I am seeing Cannot get features. The node count limit of '100' has been exceeded. To increase the limit, set the 'MaxNodeCount' property on EnableQueryAttribute or ODataValidationSettings. How to use skip or / and top in this scenario to paginate?

[Paula Hidalgo]

The number of records returned is 100 so when the return quantity is less than 100, you exit a while true loop. 

Not sure what script language you are using but here is generic python code (works with Python 3.9 or later) based on your description:

import requests
session = requests.Session()
bearer_token = "user:yourid|yourAPIToken"
session.headers.update({"Authorization":f"Bearer {bearer_token}"})
base_url = "https://company.jiraalign.com/rest/align/api/2/features"

params = {"expand" : True, "filter": "parentId eq <epic id1> or parentId eq <epic id2>", "$skip":0}

ja_info = []

while True:
    response = session.get(base_url, params=params)
    response.raise_for_status()
    page = response.json()
    ja_info += page # add the new page information to the existing array

    if len(page) !=100:
        break
    params ["$skip"] += 100

print(ja_info) # returns the JSON list

print(len(ja_info) # returns the number of records fetched

 If you want to save the output to a json file:

# add the import and the function to the top of the script
import json 
def create_json_file(fn_json, data_json):
    with open(fn_json, "w") as output_file:
        output_file.write(json.dumps(data_json, sort_keys=True, indent=4))
    return

# instead of the line print(ja_info)call the function create_json_file
create_json_file("filename.json",ja_info)