Re: [mirrorbrain] How to make Squid work with mirrorbrain

From: Per Jessen <>
Date: Thu, 14 Jun 2012 11:00:21 +0200
Jack Bates wrote:

>> If looking up the currently cached content is fast/efficient, rewriting
>> the header accordingly sounds okay, but I can't help thinking that it
>> would be easier to do what I do with Squid - rewrite the URLs when they
>> are stored?
>> If<primary>  is the primary location, e.g.
>>, and<mirror1-9>  are mirrors,
>> then files retrieved from<mirror1-9>  are stored as if they were
>> fetched from<primary>.  On subsequent retrievals, you would have a
>> direct cache hit with no need to look at the header.
> Hmm, is there any way to automatically discover the list of mirrors? 
 > I know you automatically retrieve the list of mirrors from
>, and you are looking for 
> something less messy than scraping this HTML. But I think the proxy 
> administrator must manually configure where to find the list of mirrors, 
> for each different content distribution network (openSUSE, OpenOffice, 
> etc.)

Yes, that is correct.  I think it's a reasonable amount of admin work, but
your point is entirely valid.

> A strong motivation for using Metalink is that no manual intervention is 
> required by the proxy administrator. Any content distribution network 
> that supports Metalink should be automatically discovered

Agree, that would be optimal.  I also completely agree that it would be far
more elegant getting the mirror list from the HTTP headers, but in my Squid 
setup, that becomes a chicken-and-egg issue.  I need to know the mirrors first
before I can map them to the storage URL.

Another thing is - I wanted a minimally invasive procedure :-) I wanted to avoid 
fiddling too much with Squid. Trying to get to know the code and finding a place 
where I could hook in to examine the headers would have meant much more effort.

Best regards
Per Jessen

mirrorbrain mailing list

Note: To remove yourself from this mailing list, send a mail with the content
to the address
Received on Thu Jun 14 2012 - 09:00:45 GMT

This archive was generated by hypermail 2.3.0 : Thu Jun 14 2012 - 17:47:03 GMT