Hi Team,
We have use case to crawl all spaces and pages from confluence site.
But we have encountered an issue where crawler account is not able to discover pages defined with restrictions. We are using Content API as mentioned below:-
Sample Endpoint: https://<SiteURL>/wiki/rest/api/content/search?expand=body.storage,restrictions.read.restrictions.user,restrictions.read.restrictions.group,space,ancestors,history,history.lastUpdated,history.contributors.publishers.users,children.attachment,metadata.labels&limit=25&start=0&cql=type in (page,blogpost) and space = <spaceName>. It gives empty response.
This crawler account (app authorized using OAuth2 with required scopes) has access to all spaces but not part of restrictions defined at page level.
Basically, we want crawler account to access all spaces and pages (even restricted ones).
Can you please help in answering below:
1. How crawler account can access to restricted pages?
2. Is there any other endpoint which can be used to get restricted pages by crawler account?
Welcome to the Atlassian Community!
No.
The whole point of restrictions is to stop people who should not see pages from seeing them.
You will need to grant your crawler access to the restricted pages if you want it to be able to see them.
As far as I know, Confluence Cloud admins can't view restricted spaces to which they have not been granted view permission. This is different than Confluence server.
However, as the admin you can still grant yourself view permission, and then scrape the space.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Online forums and learning are now in one easy-to-use experience.
By continuing, you accept the updated Community Terms of Use and acknowledge the Privacy Policy. Your public name, photo, and achievements may be publicly visible and available in search engines.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.