Continuous Svn-Git synchronization with Stash

[Alessio Massaro]

I've just imported the whole history of my Svn repository into a Git one using Stash's subgit add-in.

For the test drive, the Git repo is meant to stay read-only.

Will the add-in keep importing any new commits that happen in Svn?

Or do I have to come up with a way to schedule svnsync some other way?

Thanks

[Semyon Vadishev · Accepted solution]

Please refer to the message at our mailing list which Jan has kindly suggested: http://lists.tmatesoft.com/pipermail/subgit-users/2012-December/000042.html

Follow the instructions for remote mirror mode. Find a few notes on that below:

When you've found Git repository directory, run `subgit configure --svn-url $SVN_URL $GIT_REPO` and after that you can adjust SVN polling settings:

By default SubGit polls SVN server every 60 seconds, you can change this time period to 2 hours by setting the following option in $GIT_REPO/subgit/config file:

[svn]

fetchInterval = 7200

Once the configuration is done, run `subgit install $GIT_REPO`, this command starts background process that polls SVN server. Alternatively, you can avoid starting background processes for every Git repository, if there are too many of them. In this case I'd recommend you to specify the following option before running install command:

[daemon]

idleTimeout = 0

Now, in order to poll SVN server please setup a cron job that triggers SubGit translation mechanism every two hours, the following command can start SubGit translation process remotely:

cd path/to/git/working_tree && git push origin +:refs/subgit/command/fetch-sync

Hope that helps.

[Jan Swaelens]

Hello,

For the moment that plugin is a one shot deal - no sync available after the initial import. The folks at subgit are however working on a version that will support this.

We are using the standalone SubGit EAP package which allows you to set up a bi-directional sync between SVN and GIT - we even managed to set it up to sync to the Stash repo (see http://lists.tmatesoft.com/pipermail/subgit-users/2012-December/000042.html).

All in all the installation and setup was very easy and fast, I wouldn't recommend it without thinking.

[Alessio Massaro]

Thanks Jan

I've done as you suggested: recreated an empty git repo and configured/installed subgit into it (without using Stash SVN Importer).

I'm a bit confused because the end result seems very close to what Stash SVN Importer v1.0.4 (subgit 2EAP) had achieved previously, but that is less important.

How do I pull in all the commits that have happened in svn now? Is there a simple one-liner that one can run, e.g. from a scheduled task?

Thanks!

[Semyon Vadishev]

Hi Rudey,

According to the debug log, SubGit does fetch revisions for that repository you mentioned. Can you find any error messages in the log files?

Could you please also send us subgit/logs/daemon.N.log files to support@subgit.comor attach them to an issue in our tracker:

http://issues.tmatesoft.com/issues/SG#newissue=yes

We'll try to find the reason for that problem and fix it as soon as we can.

Regards,

Semyon

[Rudey Yao]

We have subgit-2.0.1 installed in Stash which is a bi-directional mirror setup with SVN. Recently we noticed that the background translation stopped progress, unless someone trigger a push on Stash it then caught up with the latest revision in SVN then - but doing that cost a lot of time upon push and wait so we want to fix this problem.

There are more than 10 Stash repos with subgit module enabled, neither is kept up to date on revision number. By looking into one of the repo, we monitored things seems going well but just no translation is made. Below are settings of one of repo:

* subgit/config file. Having default '60' seconds on 'FetchInteval' and '0' on 'IdleTimeout';

* subgit/logs/subgit-sync.0.log and subgit/daemon.0.log keep growing, it also noticed updated revisions on SVN and did 'fetch' action in log like below lines.

--- Log of daemon.0.log

"[2014-07-14 12:36:43.326][daemon][14] Updating latest fetched revision for svn-remote "svn" to r80983

[2014-07-14 12:36:45.313][daemon][14] Location '/data/sdlc/stash/data/repositories/463' fetch completed, fetch took 5269 ms.

[2014-07-14 12:36:45.314][daemon][14] Repository fetch completed, fetch took 5276 ms.

[2014-07-14 12:36:45.314][daemon][13] Sent '(ok (fetch ))'.

[2014-07-14 12:36:45.315][daemon][13] Command completed, commands running: 0.

----- Log of subgit-sync.0.log

[2014-07-14 12:38:40.988][subgit-sync][49] Read '(message (45: translating SVN revisions to Git commits... ))'.

[2014-07-14 12:38:44.977][subgit-sync][49] Read '(ok (fetch ))'.

[2014-07-14 12:39:39.128][subgit-sync][48] Initialized memory cache logger.

[2014-07-14 12:39:39.128][subgit-sync][48] SubGit version 2.0.1-EAP ('Patrick') build #2682

[2014-07-14 12:39:39.137][subgit-sync][48] Initialized file logger, logs directory is: '/data/sdlc/stash/data/repositories/463/subgit/logs'.

[2014-07-14 12:39:39.138][subgit-sync][48] SubGit version 2.0.1-EAP ('Patrick') build #2682

---- end of log

Question is, subgit background translation process was firing up everying 60 seconds but it seems not working to translate at all. If I disable/enable Stash subgit module once, I'll notice that subgit-upgrade.0.log appeared and Stash repo will catch up with latest SVN repo rev, once only.

Appreciated someone can give your help on this. Thanks.

[Semyon Vadishev]

Hi Rudey,

According to the debug log, SubGit does fetch revisions for that repository you mentioned. Can you find any error messages in the log files?

Could you please also send us subgit/logs/daemon.N.log files to support@subgit.comorattach them to an issue in our tracker:

http://issues.tmatesoft.com/issues/SG#newissue=yes

We'll try to find the reason for that problem and fix it as soon as we can.

Regards,

Semyon

[Alessio Massaro]

I had a more accurate read to the subgit documentation and realized that the the tool hinges on a hook that needs to be installed on the svn repo.

[Semyon Vadishev]

Indeed SubGit 1.0 has to be installed into both Subversion and Git repositories, however, one can install SubGit 2.0 into Git repository without touching Subversion side, i.e. only Git hooks are added into Git repository in this case. Subversion repository may even reside on remote host since SubGit uses one of SVN protocols: http(s)://, svn(+ssh):// or file:///

[Alessio Massaro]

In this case, could you point me in the right direction as to how one sets up a polling mechanism to pull the svn commits into the git repository, please?

I think I know my way around svn decently well, but I'm afraid I'm just too ignorant about git to tell how subgit should be configured.

As I said, I really just care about a local (w.r.t. subgit) read-only git repository that is updated from a remote svn every few hours via http.

Thanks a lot in advance!