Re: [mirrorbrain] mirror list in plain text?

Hi Per,

Am 16.05.2012 um 13:47 schrieb Per Jessen:
> is there a (standardized) way of retrieving the mirror list in plain
> text?  I need this for setting up a URL rewriter for squid which will
> help me cache segmented downloads.  For instance, I can retrieve the
> HTML from http://mirrors.opensuse.org/list/all.html and parse that HTML
> quite easily, but I would prefer just getting a plain text file
> straight from mirrorbrain. 

Do you mean all mirrors? If you have a certain file in mind, then appending .meta4 to the file's URL will give you parseable XML. Not plain text, though.

The list of *all* mirrors can't be requested directly. It would be easy to implement that, but there are some things to keep in mind:

Not all mirrors have all content, especially with openSUSE there is much variation between what the individual mirrors carry.

Some mirrors might want to remain private - which is the case for some mirrors located in countries with poor internationaly connectivity, where requests from outside the country need to be avoided. There is already a hack in the "mb mirrorlist" command (which generates also http://mirrors.opensuse.org/list/all.html) to exclude such mirrors from the listing. That might not be relevant in your case - I don't know if the URL rewriter could be deployed in a country with such a mirror.

The data you want to retrieve is the base URL of the mirrors, or anything else?

With the latest MirrorBrain (newer than what is deployed on openSUSE.org), mirrors are also listed in HTTP headers on requesting a file (Link headers, RFC 6249). Maybe that would be convenient too. A head request would be sufficient to get a list of mirrors. (That list is limited to 5 entries a the moment.)

BTW, I noticed a GSOC project that might share a similar goal with yours, but with another proxy: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/nottheoilrig/1

Peter


_______________________________________________
mirrorbrain mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain/

Note: To remove yourself from this mailing list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-request_at_mirrorbrain.org