Issue12

Title "mb scan" may hang, because rsync's "--timeout" option is not effective at all times
Priority bug Status resolved
Superseder Nosy List poeml
Assigned To poeml Keywords scanner

Created on 2009-10-08.11:50:06 by poeml, last changed by poeml.

Messages
msg129 (view) Author: poeml Date: 2010-02-10.04:31:22
I committed a fix to trunk (r7953).

http://svn.mirrorbrain.org/viewvc/mirrorbrain?view=revision&revision=7953
msg105 (view) Author: poeml Date: 2009-12-11.21:16:57
Incidentally, I discovered in the rsync man page that there's an additional timeout setting:

--contimeout=SECONDS     set daemon connection timeout in seconds

...exactly what we need.

It was new in rsync 3.0.0.

Thus, we can use it, except on platforms that still have rsync 2.6. Debian 5.0 is such a case.

So it could be worthwhile to detect the rsync version in the beginning of a scan. The detection can be used 
to also make sure that rsync is installed at all, and give a meaningful error message if it isn't.


 % python -c "import commands; status, output = commands.getstatusoutput('rsync --version'); print status; 
ver = output.splitlines()[0].split()[2]; print ver"
0
3.0.4

Something along these lines needs to be added to mb/testmirror.py.
msg32 (view) Author: poeml Date: 2009-10-08.11:50:55
The impact is that this may cause the whole "mb scan" command to hang in the 
initial phase, where probing for directories is done.
msg31 (view) Author: poeml Date: 2009-10-08.11:50:06
rsync should never hang longer than 20 seconds when we probe for files -- since we run it with --timeout=20.

But this timeout doesn't seem to apply to the connect phase of rsync:



    \_ /usr/bin/python /usr/bin/mb scan -j 8 -a -d distribution/11.2-Milestone7/iso
              |   \_ sh -c { rsync -d --timeout=20 rsync://ftp.novell.co.jp/opensuse/distribution/11.2-Milestone7/iso 
/tmp/mb_probefile_PChalR/ --list-only; } 2>&1
                        |       \_ rsync -d --timeout=20 rsync://ftp.novell.co.jp/opensuse/distribution/11.2-Milestone7/iso 
/tmp/mb_probefile_PChalR/ --list-only



strace shows:

    connect(3, {sa_family=AF_INET6, sin6_port=htons(873), inet_pton(AF_INET6, "2001:278:101f:1::2", &sin6_addr), sin6_flowinfo=0, 
sin6_scope_id=0}, 28
History
Date User Action Args
2010-02-10 04:41:58poemlsetstatus: testing -> resolved
2010-02-10 04:31:22poemlsetstatus: in-progress -> testing
messages: + msg129
2009-12-11 21:16:58poemlsetstatus: chatting -> in-progress
messages: + msg105
2009-12-01 20:55:44poemlsetkeyword: + scanner
2009-10-08 11:50:55poemlsetstatus: unread -> chatting
messages: + msg32
2009-10-08 11:50:06poemlcreate