Re: [mirrorbrain] null-rsync (pseudo file trees)

Hi,

I played with a test setup yesterday, and it worked as expected.

I created a local file tree as a "pseudo mirror" of openSUSE.org's
download.opensuse.org server, with the null-rsync script 
(http://svn.mirrorbrain.org/viewvc/mirrorbrain/trunk/tools/null-rsync?view=markup)
originally 775G in size, and taking 2.2G locally. (548928 files in 37306
directories still take place, even though filled with zeroes...)

Then I set up a MirrorBrain instance http://opensuse.mirrorbrain.org/
and configured it with MirrorBrainMinSize 0, so that it doesn't want to
deliver small files itself (the default is 4096 bytes), and added two
fallback mirrors:
    MirrorBrainFallback eu de http://widehat.opensuse.org/
    MirrorBrainFallback eu de http://ftp5.gwdg.de/pub/linux/suse/opensuse/

This immediately made the server redirect as expected (and create mirror
lists), even before I started scanning real mirrors. After scanning, the
number of redirection targets of course grew, of course. 

Once a good amount of mirrors is available, the above
MirrorBrainFallback mirrors turn into a pure insurance against ever
delivering the pseudo files, which must never happen because they
contain only zeroes.

For testing, I used a Linux box to update itself, by adding the IP
address of the server with the name "download.opensuse.org" in
/etc/hosts. And it worked flawlessly.

Apart from the fact, that even the 775G tree was _not_ complete; it
lacks certain debug packages. Since they are missing, the client
naturally got a 404 File not found on them.

Such incompleteness is particularly prone to happen with the extreme
openSUSE tree, which is very hard to keep in sync in _any_ way. (The
775G is probably only half of the whole tree.) However, in principle,
such inconsistencies can happen in small setups as well, and if only
transitory.

So, the next step could be to implement an intelligent behaviour for
cases where a file is not found - see also 
http://mirrorbrain.org/archive/mirrorbrain/0049.html

Peter


_______________________________________________
mirrorbrain mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain/

Note: To remove yourself from this mailing list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-request_at_mirrorbrain.org