[Monotone-devel] Issue 166 - netsync performance is very poor when new branches are synced with restricted sync patterns (monotone)

[code]

Hello,

A new issue has been created and assigned
to you:

166 - netsync performance is very poor when new branches are synced with 
restricted sync patterns
Project: monotone
Status: New
Reported by: Ethan Blanton
Labels:
 Type:Defect
 Priority:Medium

Description:

When syncing a new branch to/from a server for the first time, netsync 
performance is extremely poor if both the client and server share history for 
an ancestor of the new branch, but that ancestor is not captured by the sync 
pattern.

This is only a serious problem when revision history is very large.

You can reproduce this by cloning the Pidgin revision history and setting up a 
server to serve im.pidgin*.  Create a new branch (say, im.pidgin.slowsync) by 
committing a new revision against the head of im.pigin.pidgin.  Sync (or push) 
*only* im.pidgin.slowsync to the server.

Specific reproduction steps:
1. Fetch the Pidgin mtn bootstrap database from pidgin.im:
   wget http://developer.pidgin.im/static/pidgin.mtn.bz2
   bunzip2 pidgin.mtn
2. Migrate the database if necessary
3. Copy the database:
   cp pidgin.mtn pidgin2.mtn
4. Check out a working copy from pidgin2.mtn:
   mtn -d pidgin2.mtn co -b im.pidgin.pidgin pidgin-test
5. Create a new revision:
   cd pidgin-test
   echo 'make mtn do something stupid' >> README
   mtn ci -b im.pidgin.slowsync -m 'README update'
6. Start a sync:
   mtn sync file://../pidgin.mtn/?im.pidgin.slowsync
7. Go get a cup of coffee
8. Go ahead and have lunch
9. Take a nap
10. Kill the sync because the bug has been demonstrated and there are still 
25,000 revisions to go

Despite the fact that the server and the client share the entire history of the 
branch in question except for the most recent commit, monotone will exchange 
30k+ revisions, because the shared ancestry is on a branch (im.pidgin.pidgin) 
which is not included in the current sync pattern.

This problem can be solved by including a shared ancestor branch in the sync 
pattern.  However, this approach makes sync patterns much more complicated when 
it is not desirable to sync some local branches.

I am aware of the (arguably good) reasons for this performance issue (it has 
been around ~forever and discussed on multiple occasions), but I believe it 
should be addressed nonetheless, if even as a heuristic hack.

mtn version --full

monotone 0.99.1 (base revision: 8973482283db7c36780dce2b54721ccc0f5b7388)
Running on          : Darwin 10.7.0 Darwin Kernel Version 10.7.0: Sat Jan 29 
15:17:16 PST 2011; root:xnu-1504.9.37~1/RELEASE_I386 i386
C++ compiler        : GNU C++ version 4.2.1 (Apple Inc. build 5666) (dot 3)
C++ standard library: GNU libstdc++ version 20070719
Boost version       : 1_45
SQLite version      : 3.7.5 (compiled against 3.7.5)
Lua version         : Lua 5.1
PCRE version        : 8.12 2011-01-15 (compiled against 8.12)
Botan version       : 1.8.10 (compiled against 1.8.10)
Changes since base revision:
format_version "1"

new_manifest [c1270158b7fa91abf8235ad129b0476943bde1ed]

old_revision [8973482283db7c36780dce2b54721ccc0f5b7388]

  Generated from data cached in the distribution;
  further changes may have been made.

--
Issue: https://code.monotone.ca/p/monotone/issues/166/