How to specify minimalRevision with the SVN Mirror Plugin?

[Joseph Smith]

I am working with a repository that has over 140,000 revisiosn. It would be great to limit the number of revisions during import. Any help is greatly appreciated.

[Semyon Vadishev · Accepted solution]

Hello Joseph,

SVN Mirror Add-on has no UI setting to specify minimal revision at the moment. But since the plugin is built upon SubGit, it's still possible to specify minimal revision as follows:

1. Create new Git repository in Stash;

2. Go to Subversion Mirror page;

3. Fill in all the required fields there and click continue;

4. When the page with authors mapping is displayed (see screenshot below), adjust SubGit config file <STASH_HOME>/data/repositories/<ID>/subgit/config, so it has the following option:

[svn]

minimalRevision = 100000

Hint: you can find repository ID on Settings | Repository Details page.

5. Finally, adjust authors mapping as needed and start initial import by clicking continue. The plugin should start fetching SVN revisions from the one you've specified in SubGit config file.

Please note that in certain cases importing Subversion repository from arbitrary revision may take much more time than importing it from scratch. This happens because Subversion repository may have many branches and tags at minimal revision specified. As result SubGit has to fetch every branch and tag by requesting them separately which is more expensive than processing incremental deltas starting from r0.

Hope this helps,

Semyon

[Joseph Smith]

Very helpful. Thanks much!

[Joseph Smith]

Hey Semyon - I've noticed that without using minrevisions, the import looks like it may take about 2-3 weeks for 145K revisionsm which is a bit extreme. Are there any tips for speeding things up?

[Semyon Vadishev]

Hi Joseph,

Can you describe your repository? Does it have complex branches and tags structure (something beyond standard trunk, branches/* and tags/*)? Was this structure changing over time, so older revisions have some different layout?

Do you store huge files within Subversion repository, e.g. 10 megabytes and more?

You can speed-up intial import process by disabling processing of svn:eol-style, svn:mime-type and svn:ignore properties. Specify the following options in SubGit config file (just like in svn.minimalRevision example above):

[translate]

eols = false

ignores = false

Note that you have to start from empty Git repository in order to use this new configuration.

Regards,

Semyon

[Joseph Smith]

Thanks for the quick response. I will try these settings.

In general we have the standard layout, but there were some big svn mv's early on in the history. I am not importing branches at all, but I am importing a lot of tags.

[Joseph Smith]

So I tried this with the minimalRevision property and after continuing the import, the import seems to be idle and do nothing. This is where the logs leave off:

[2014-01-29 11:53:46.955][subgit-install][19603] There are unfetched revisions in Subversion repository.

[2014-01-29 11:53:46.962][subgit-install][19603] Fetching file:///scratchq/joseph/svn up to revision=HEAD

[2014-01-29 11:53:49.470][subgit-install][19603] SET_PATH '' 142306 empty depth=infinity

[Semyon Vadishev]

These log records mean that SubGit is started fetching some branch, this may take a while.

You can also send us a thread dump, so we'll try to understand whether the import process stuck or not. Here are instructions on getting a thread dump:

https://confluence.atlassian.com/display/STASHKB/Generate+a+Thread+Dump+Externally

Probably, there will be a lot of threads reported, but it should be fairly easy to find SVN import thread.