Cloning production into test/staging environments

[Xabier Davila]

Hi,

Like many of us, I need to create test environments by clonning our production instance (testing upgrades, plugins, workflows...). This applies to all our Atlassina environments: JIRA, Confluence, etc.

Since this is a repetitive, quite boring task, I've created a script to do this. The script runs from the test enviroment and pulls the data from prod. These are roughly the steps I'm following:

1. Stop JIRA/Confluence...
2. rsync the home and installation folders. I do not need to change paths to home, etc since both environments structure is identical. Same applies to the DB, running in a local PostgreSQL with the same name.
3. SQL Dump the production database and copy it to the test instance
4. Import the database in the test instance
5. Disable email notifications

All working fine up to this point, but there are a couple more things that I have to do manually and I'd like to automate:

1. Change the sandbox Base URL
2. Change the color schema, so it's obvious when you're logged into sandbox vs production
3. Sort out the Application Links. Ideally, I'd like to have the JIRA sandbox linked to Confluence sandbox and so on, but as a minimum, I'd like to remove all the application links from the test instance.
4. Probably worth replacing all the http://jira.domain.com occurences in the SQL dump wit http://jira-test.domain.com. Not 100% sure about this.

Any suggestions on the best way to achieve the last 4 points? My ideal would be to get a clone of all production instances in my test environment, including Application Links, URLs between, say Confluence and JIRA, etc.

Thanks

xabier

[Alex Perez · Accepted solution]

Hi Xabi,

I cant access a sample jira db schema, so I cant tell you the exact tables at this moment ...

1. run the appropiate update in the fresh after importing the db in the new env.

2. I have all my stagging envs with the same color scheme as production, BUT they have a different logo. I have a smiling spongebob (xD), so it-s easy to realize that you're not in production. You have references to the filename in the database, but you can overwrite the file with another logo ..

3. Create the application links in the stagging env, then dump the database tables related to his config (table applinks, I think). Import them after the "content dump" to overwrite the applinks.

4. Most links are created using the url base, so .. this may worth only if you have lots of links inserted in the description or comments Perhaps its easier to do with a sed applied to the sql file ...or a bunch of updates searching for the description/comments ...

HTH

[Xabier Davila]

Hola Alex

Thanks for your reply.

1. This seems to be pretty obvious for JIRA, a bit more complex for Confluence. Haven't checked on how to do this with Crucible or Bamboo. No REST API way??

JIRA: UPDATE propertystring SET propertyvalue='http://jira-test.domain.com' WHERE ID IN (SELECT id FROM propertystring WHERE id IN (SELECT id FROM propertyentry WHERE property_key like '%baseurl%'));

2. Good tip!!

3. Your solution would be ideal, but there seems to be a lot of tables involved in App Links, as per this link. There’s other content in those tables, so I’m not sure your approach will work.

4. Doing a sed replace in the SQL file was my idea. Will give it a go an comment with the results.

Thanks again for your help. The point of my question is to try to automate this task. I know it’ll change from environment to environment, but due to the number of times we (Atlassian users) have to do this, it’d be good to work on automating it as much as possible.

[Ian Smith]

Thanks, the BaseURL was missing from the Atlassian documentation and caught me for a few minutes till I found your page.  Even though it's an old post, still valueable.

This was my original source.

https://confluence.atlassian.com/doc/restoring-a-test-instance-from-production-127369720.html

[Deleted user]

Hello @Xabier Davila

Thank you for sharing the idea.

Could you please share more context about how does your script achieves this.

I'm also keen to know how you are able to accomplish 3. and 4. ?

Thanks,

Bikram

[Justin W. Richeson]

@Xabier Davila- would you be willing to share your script or a scrubbed version of it? I'd like to present something similar to help keep our test/staging environment in line with our production one and I like the idea of a scriptable method to keep this in check.

[William Zanchet [Atlassian]]

Hi Xabier,

Here is the steps for Confluence: https://confluence.atlassian.com/display/DOC/Migrating+Confluence+Between+Servers#MigratingConfluenceBetweenServers-HowtoCreateaTestorDevelopmentSite

This should do the trick.

[Sandeep Nagaraja]

@ Xabier Davila

I want to perform similar task and have a doubt .

What should we do if we have two Separate Crowd Environments for Prod and Test. Your Steps only brings in JIRA data ?

I performed similar steps and i am now not able to login as Adminstrator.Because my Crowd environment is different then production crowd.

Cheers

Sandeep

[Xabier Davila]

Hi Sandeep, apologies for the late reply.

I'm actually using the same Crowd for both, production and test, so all I do is add JIRA test IP to the authorised list in Crowd. Not ideal, I know