Re: [mirrorbrain] thoughts about multi-instance setups

Am 09.12.2009 um 17:15 schrieb dfarning_at_sugarlabs.org:

> I don't know if it helps.... but have you seen the work being done  
> with cacheboy[1].  They are taking a rather different approach to  
> the problem.

Yes, I shortly looked at it - different approach. Caching is a good  
approach, but requires a lot of effort in deploying IMO, which would  
be a too high barrier in my view. I have seen too many academic  
approaches failing for that reason.

A network of squid caches would all that's needed, but how do you turn  
all existing mirrors into squid caches?

A real CDN would be a very different approach - and it has a huge  
potential of course (as can be seen by the success of commercial  
CDNs). The MirrorBrain approach is "use the material that's there",  
which is of course only one of the possible approaches. But it has  
worked very well for me. (I needed to implement something in limited  
time.)

> The question that comes to mind when thinking about a federated  
> mirror system(Great Idea) is the ability to automatically control  
> _what_ gets mirrored.  Just like other caches there appeares to be  
> significant time and spacial locality in our file download.  On any  
> given day it looks like 90% of Sugar Labs downloads come from 10% of  
> our content.  A way for mirrors to determine what to cache seems  
> very useful.

That's one reason why I am working on a good download counting system;  
because that could be the basis for popularity-based mirroring (or  
whatever one might call it). Ah, and in addition, I also started  
working on a mirror pushing system - where you have complete control  
what goes to mirrors and when (and the redirector immediately knows  
about it). A primitive prototype of such a pushing system runs at  
openSUSE (and feeds 10 or 11 mirrors), and it works nicely - but that  
needs to be implemented properly to be taken to a level where it'd be  
generally useful.

The question whether this would just re-implement things that caches  
already do is a valid one.

I'm curious to see how the Cacheboy thing works out! Thanks for the  
pointer.

Peter

_______________________________________________
mirrorbrain mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain/

Note: To remove yourself from this mailing list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-request_at_mirrorbrain.org