Get content and structure of confluence pages for sphinx

[Dirk Baumbach]

Hello,

we want to create a documentation for our software products with the following features:

• Sources for documentation content in
  • code (.cpp-files, .m-files, .py-files, ...)
  • in local source files(in Markup Language like Markdown, rst, xml, tex, ...)
  • confluence pages
• a modern look, with a mobile friendly layout
• accessible without having to have permissions for confluence spaces.
• easy to maintain for the actually developers. They shouldn't have to care for the documentation build, because it runs automatically and stable

The different content sources have the following reason: We develop about 5 technically independed products, but they all share the same download and installation routine for instance. Instead of writing the same text 5 times, we want to write it once, but it should appear in all 5 build documentations. We already use confluence for all sorts of things, so the idea is to write these texts there and somehow get them into the build product documentation.

Our customers like the readthedocs style, which is based on Sphinx. Sphinx can already do nearly all the things we want and we already have experience with it from previous projects. It works fine and we already have an automatic build-system for it.

The interesting question is how to integrate the confluence content automatically. The main question is how to GET the content to a local machine as a readable file (HTML, json, rst, markdown, whatever). For all further questions, like how to prepare the files for sphinx and so on are a piece of cake from there.

We already looked what we can do so far with our limited rights in our confluence space. There is an export-to-html functionality for whole confluence-spaces. This functionality will create a .zip-file, where the plain html content without much css formatting are stored as single html files. We can combine these html files with sphinx, so they can be integrated into the sphinx documentation with the same layout as the other content. This already looks fine to us, but it has several problems:

• The export-to-html function is a manual step, which cannot be automated in an obvious way
• If whole spaces are exported, the whole structure must be integrated in sphinx with much accuracy. The HTML files sphinx generates must have the same name as the html files confluence exports, otherwise the internal links will not work in the generated documentation.
• There are still some parts of the original confluence pages, we cannot get rid of easily. For instance each page with pictures will have an 'attachments' section at the end of the page. Also the breadcrumbs of the confluence page are displayed alltough the are redundand in the sphinx documentation.

I strongly believe it is possible to get the content of confluence pages to be used in a third party tool like sphinx. But we have a hard time to figure out how.

I already seen there is a REST api for confluence pages. From what i have seen this is way to powerful for our needs, because we get much more information as we need and we don't have experts on webprogramming and so on. In addition our company proxy settings are really a PITA here.

The Atlassian CLI seemes to be a promising thing, because it already provides a nice command window syntax which we can use in all our automation tools. But the documentation does not tell me cleary if it can do what we want here and how.

For instance there is the getContent function which gets some content. But from the text a can not guess what i get, when i would use this function.

Any suggestions here?

Sincerely,

Dirk Baumbach

[Michael Kuhl {Appfire}]

Hi @Dirk Baumbach using --outputFormat 999 with getContent will get you page metadata and page source.  You will have some parsing to do on the content, but it should be a good starting point.

Running this:

--action getContent --space recipes --id 109347329 --outputFormat 999

Gets you this (truncated for brevity) result:

 ----------------------------------------------

Data for content with id 109347329

Page id . . . . . . . . . . . : 109347329

Title . . . . . . . . . . . . : How to Create a Confluence Space Using the Confluence CLI

Space key . . . . . . . . . . : recipes

Space name  . . . . . . . . . : Recipes

Space home page id  . . . . . : 103612743

Space home page title . . . . : Recipes for Business Intelligence & Automation

Parent id . . . . . . . . . . : 103612743

Parent title  . . . . . . . . : Recipes for Business Intelligence & Automation

Creator . . . . . . . . . . . : Betsy

Created . . . . . . . . . . . : 6/3/15 1:11 PM

Modifier  . . . . . . . . . . : Betsy

Modified  . . . . . . . . . . : 6/4/15 11:35 AM

Version . . . . . . . . . . . : 8

Version message . . . . . . . :

Content type  . . . . . . . . : page

Status  . . . . . . . . . . . : current

Is home page  . . . . . . . . : No

Labels  . . . . . . . . . . . : basic_level confluence confluence-command-line-interface-add-on example howto recipe

Position  . . . . . . . . . . :

URL . . . . . . . . . . . . . : https://bobswift.atlassian.net/wiki/spaces/recipes/pages/109347329/How+to+Create+a+Confluence+Space+Using+the+Confluence+CLI

Source  . . . . . . . . . . . : <ac:structured-macro ac:macro-id="f25e7a95-41ce-4a59-b899-98ec2fcae572" ac:name="div" ac:schema-version="1"><ac:parameter ac:name="id">ProductTOC</ac:parameter><ac:rich-text-body><h3>On this page</h3><p><ac:structured-macro ac:macro-id="d6c981e9-da70-4361-90bc-30f27ec30387" ac:name="toc" ac:schema-version="1"><ac:parameter ac:name="maxLevel">1</ac:parameter><ac:parameter ac:name="exclude">On this page</ac:parameter><ac:parameter ac:name="type">flat</ac:parameter><ac:parameter ac:name="separator"> | </ac:parameter></ac:structured-macro></p></ac:rich-text-body></ac:structured-macro><p><ac:structured-macro ac:macro-id="4bad0bee-723c-4395-aba0-67c09afe5b03" ac:name="include" ac:schema-version="1"><ac:parameter ac:name=""><ac:link><ri:page ri:content-title="_RecipeTemplateTopLogo" /></ac:link></ac:parameter></ac:structured-macro></p><h1>Recipe overview</h1><p><span style="color: rgb(51,51,51);">This recipe provides step-by-step instructions on how to create a Confluence space and assign appropriate permissions using the <a href="https://marketplace.atlassian.com/plugins/org.swift.confluence.cli">Confluence Command Line Utility (CLI)</a> add-on. This allows a Confluence Administrator to quickly, consistently and flawlessly create new spaces when requested by your users.<br /></span></p><p>

.....

[Dirk Baumbach]

Hello Michael,

thanks for this answer! This is indeed a good starting point.

Is it possible to also do this using the REST API? I recently heard we won't get any new marketplace addons from Atlassian. I must try to get the content with CURL or other commad line interface tools.

[Michael Kuhl {Appfire}]

Yes.  Everything you can do in the CLI you can do with the REST API.  It may take multiple calls and some scripting to pull it all together though. 

The CLI makes it easier and more productive than hand-coding. :)