Skip to content

Commit

Permalink
git-svn: disable _rev_list memoization
Browse files Browse the repository at this point in the history
This memoization appears unneeded as the check_cherry_pick2 cache is
in front of it does enough.

With this change applied, importing from local svn+ssh and http copies
of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn
process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2].
This 60M measurement is only for the git-svn Perl process itself and
does not include memory used by git subprocesses accessing large packs
(subprocess memory usage _is_ measured by my time(1) tool).

Before this change, an import took longer (2:20) on svn+ssh:// but
git-svn used around 240MB during the imports.  Worse yet, git-svn
ballooned to over 400M when writing out the cache to the filesystem.

I also tried removing memoization for `has_no_changes', too, but a
local copy of the R repository(*) was not close to finishing within
10 hours on my system.

[1] http://svn.r-project.org/R
[2] file:// repos causes libsvn to use more memory internally

Signed-off-by: Eric Wong <[email protected]>
Cc: Hin-Tak Leung <[email protected]>
  • Loading branch information
Eric Wong committed Oct 27, 2014
1 parent aee7d04 commit 7676aff
Showing 1 changed file with 0 additions and 10 deletions.
10 changes: 0 additions & 10 deletions perl/Git/SVN.pm
Original file line number Diff line number Diff line change
Expand Up @@ -1600,7 +1600,6 @@ sub tie_for_persistent_memoization {
my %lookup_svn_merge_cache;
my %check_cherry_pick2_cache;
my %has_no_changes_cache;
my %_rev_list_cache;

tie_for_persistent_memoization(\%lookup_svn_merge_cache,
"$cache_path/lookup_svn_merge");
Expand All @@ -1622,14 +1621,6 @@ sub tie_for_persistent_memoization {
SCALAR_CACHE => ['HASH' => \%has_no_changes_cache],
LIST_CACHE => 'FAULT',
;

tie_for_persistent_memoization(\%_rev_list_cache,
"$cache_path/_rev_list");
memoize '_rev_list',
SCALAR_CACHE => 'FAULT',
LIST_CACHE => ['HASH' => \%_rev_list_cache],
;

}

sub unmemoize_svn_mergeinfo_functions {
Expand All @@ -1639,7 +1630,6 @@ sub tie_for_persistent_memoization {
Memoize::unmemoize 'lookup_svn_merge';
Memoize::unmemoize 'check_cherry_pick2';
Memoize::unmemoize 'has_no_changes';
Memoize::unmemoize '_rev_list';
}

sub clear_memoized_mergeinfo_caches {
Expand Down

0 comments on commit 7676aff

Please sign in to comment.