Java API search fails after some requests

[Andreas Malerz]

We develop and use a Desktop Application written in JAVA. The App will collect data from our JIRA system and create WORD reports. Collecting data from JIRA means: Running a small number (<20) of JQL requests per report.

We use two auth mechanisms: basic authentication (username & password) and the access tokens provided by JIRA.

The problem is pretty strange. We start creating the report, the first n (eg. 5) JQL requests are executed, the data is collected, everything ok. But the next request fails. As a reply we get a 401 response.

Our technical approach:

We use those libs from Atlassian (from our POM file), downloaded from your repo:

<repository>
  <id>AtlassianRepository</id>
  <name>AtlassianRepository</name>
  <url>https://packages.atlassian.com/mvn/maven-atlassian-external/</url>
</repository>

...

<dependency>
  <groupId>com.atlassian.jira</groupId>
  <artifactId>jira-rest-java-client-core</artifactId>
  <version>5.2.7</version>
</dependency>
<dependency>
  <groupId>io.atlassian.fugue</groupId>
  <artifactId>fugue</artifactId>
  <version>5.0.0</version>
</dependency>

In our code, we use the AsynchronousJiraRestClientFactory class as such:

authHandler = new BasicHttpAuthenticationHandler( username, password );
new AsynchronousJiraRestClientFactory().create( jiraUri, authHandler );

or:

authHandler = new OwnBearerAuthenticationHandler( accessToken );
new AsynchronousJiraRestClientFactory().create( jiraUri, authHandler );

the OwnBearerAuthenticationHandler implements as such:

@Override
public void configure( Builder builder ) {
    builder.setHeader( "Authorization", "Bearer " + apiToken );
}

After creating the JiraRestClient, we use it like this:

restClient.getSearchClient().searchJql(
        params.getJql(),
        params.getMaxResults(),
        params.getStartAt(),
        params.getFields()
).claim();

We use the same instance of the JiraRestClient for all requests.

In general this approach works. We are able to create small reports (one or two requests to JIRA), everything works like a charm. But if we try to generate bigger reports with 10 to 20 requests, it will sometimes fail.

From the log I can see that eg. the first 6 requests work, but the 7th fails and I get a 401 error including a html page with this error as error message (instead of a JSON answer).

If I try that a few times, only the first 2 or 3 requests will work, with the very same error for the next request. After some additional tries even the first request will fail. The error message will tell me that certain accesses are forbidden for "anonymous users".

All this happens after the connection of the client with credentials. The connection always works, but after that I see errors.

If I use the very same JQL statement on the web, everything works like a charm. In fact, all of our JQL statements are generated and tested on the JIRA website, with the same credentials.

In the log of JIRA we found (in a similar case) those entries:

2024-02-22 11:21:51,455+0100 https-jsse-nio-8443-exec-25 anonymous 681x30787x1 - <ip-addr> /rest/api/latest/search login : 'tho' tried to login but they do not have USE permission or weren't found. Deleting remember me cookie.
2024-02-22 11:21:51,533+0100 https-jsse-nio-8443-exec-25 anonymous 681x30787x1 - <ip-addr> /rest/api/latest/search The user 'tho' has FAILED authentication. Failure count equals 1
2024-02-22 11:21:51,548+0100 https-jsse-nio-8443-exec-25 anonymous 681x30787x1 - <ip-addr> /rest/api/latest/search login : 'tho' tried to login but they do not have USE permission or weren't found. Deleting remember me cookie.
2024-02-22 11:21:51,563+0100 https-jsse-nio-8443-exec-25 anonymous 681x30787x1 - <ip-addr> /rest/api/latest/search The user 'tho' has FAILED authentication. Failure count equals 2
2024-02-22 11:22:02,294+0100 https-jsse-nio-8443-exec-15 anonymous 682x30788x1 - <ip-addr> /rest/api/latest/search login : 'tho' tried to login but they do not have USE permission or weren't found. Deleting remember me cookie.
2024-02-22 11:22:02,310+0100 https-jsse-nio-8443-exec-15 anonymous 682x30788x1 - <ip-addr> /rest/api/latest/search The user 'tho' has FAILED authentication. Failure count equals 3
2024-02-22 11:22:02,310+0100 https-jsse-nio-8443-exec-15 anonymous 682x30788x1 - <ip-addr> /rest/api/latest/search HttpSession created [1cn893k]
2024-02-22 11:22:02,322+0100 https-jsse-nio-8443-exec-15 anonymous 682x30788x1 - <ip-addr> /rest/api/latest/search The user 'tho' is required to answer a CAPTCHA elevated security check. Failure count equals 4

User tho is a collegue of mine. She was able to create reports for a while, but at some point the system failed and we got those errors. She tried to authenticate with user/password. The password was definitely correct, as it worked for some time and then failed. The password works for the web login form.

Trivia:

We found out that it might help to login and logout on the website a few times (~5 times) in a row. After that the report generation might work again. It seems that there is some sort of fraud prevention which decides that there are too many requests. But if you login and logout a few times, the internal counter is resetted and you have some requests to go. (This is just an assumption.)

Questions:
Is the implementation approach correct?
Do you know any JIRA configurations which might lead to such behaviour? Is there anything we can do in our JIRA instance?
Is this a bug on the JIRA side? Is there any way to circumvent this?
Any other ideas and approaches we can try?

[Florian Bonniec]

Hi @Andreas Malerz 

You could removed CAPTCHA from the configuration. Also you could do a first call to get a session then use this session for the other calls. Authenticate at each call lead to the creation of a session for each call that could lead to performance issue with a high volume of calls.

https://developer.atlassian.com/server/jira/platform/basic-authentication/

https://developer.atlassian.com/server/jira/platform/security-overview/#cookie-based-authentication

Regards

[Andreas Malerz]

Hi Florian,

thanks for your answer.

In the past we used to get error messages indicating that a captcha has to be answered by the user. We can handle that and give our users a hint to do that if necessary. Currently we do not get those error messages. I guess this is because we use JIRA access tokens which are there to be used by applications. I guess if you use them, you would never get a captcha as this makes no sense in a scenario where an app is using a rest api.

Do you think it is helpfull to disable CAPTCHA in the configuration? It might turn any other limitation off, but I do not know.

Your second suggestion is already in place. We create the JiraRestClient instance (implemented by Atlassian) once and use this instance for all JQL requests. It seems that the Client is doing the authentication for each request anyways.

Some weeks ago we recreated the Client for each request and ran in the same error messages. I fixed that but with no luck so far.

Any other ideas?