Re: [mirrorbrain] How to tell if mirrorbrain is redirecting

On Tue, Oct 6, 2009 at 4:36 PM, Peter Poeml <poeml_at_cmdline.net> wrote:
> Hi David,
>
> On Sun, Oct 04, 2009 at 09:15:43PM -0500, David Farning wrote:
>> Peter,
>>
>> Yet another question...
>>
>> I have manually added, edited, scanned and enabled two mirrors on
>> mirrorbrain-testing.sl.o.
>
> Nice!
>
> Did you also "enable" the mirrors?
>
> For mirrors to be used, three conditions must be met:
>
> 1) mirror needs to be enabled (enabled = True)
> 2) it needs to be reachable (statusBaseurl = True)
> 3) it needs to be known to have the file in question
>
> You see the status when you look at the mirrors with "mb show".
> Alternatively, you could search for files with
> "mb file ls '*abiword-2.6.8-4.fc11.sl.i386.rpm'", which will also display
> the mirrors status.
>
> In addition, you can check the availability of mirrors for a certain
> file by appending "?mirrorlist" to the URL. Example:
> http://mirrorbrain-testing.sugarlabs.org/soas/repositories/2/abiword-2.6.8-4.fc11.sl.i386.rpm?mirrorlist
>
> (As we see here, the mirror list is empty for that file, so something is
> wrong)
>
> (mod_autoindex_mb can be used as a drop-in replacement for
> mod_autoindex, for its capability to include, in the index, a link for
> each file to its mirror list.)
>
>> How can I tell if mirrorbrain is redirecting correctly?  I see no
>> indication in either the access or error logs.
>
> In the (default) access log, Apache logs the HTTP status code directly
> after the URL. So you see
>        "GET /url/... HTTP/1.1" 200 ...
> for a request that was answered with a "200 OK" reply.
>
> For a redirect however, you would see a reply with
>        "GET /url/... HTTP/1.1" 302 ...
> where the number is from "302 Found", a reply that comes with a Location
> header containing the URL to be followed.
>
> So to verify that MirrorBrain is redirecting correctly, you could look
> into the access_log in the first place. The access_log snippet you
> attached contains only "200" responses; thus, no redirects happen. (We'll
> soon find out why not!)

Ahh so we were not getting any redirects that make sense.

> For the record, let me mention an additional way to check for redirects.
> You can look at the protocol level with "curl -sI <URL>", which will
> make the server responses visible (only the header lines due to the -I).
> Example (of a working redirect):
> http://mirrorbrain.org/static/images/screenshots/redirect.jpg
> With a browser like Firefox, it is harder to notice if a redirect
> happens.

Ok

> With some Apache configuration, MirrorBrain allows logging several more
> details out of the box, that complement the normal access log:
>
> - which mirror a client was sent to
> - data about client origin: country, continent, autonomous system number
>  and network prefix.
> - the reason why a particular mirror was selected
> - size of the file in question
> - files for which no mirror was found
> - (with Apache's native logging means) things like the number of bytes
>  actually transferred, byterange headers requested by the client or
>  other headers
>
> All these could be logged conditionally, using Apache's SetEnvIf
> directive combined with "CustomLog env=....".
>
>
> Here's an example for defining a new LogFormat which logs all of the
> above. The format is identical to the "access log" format, but with
> additional data added at the end of the log lines:
>
> # In order to log the mirror we redirected to, we use the
> # %{X-MirrorBrain-Chose-Mirror}o logging variable to log that headers
> # content
> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \
> %{X-MirrorBrain-Mirror}o r:%{MB_REALM}e \
> %{MB_CONTINENT_CODE}e:%{MB_COUNTRY_CODE}e ASN:%{ASN}e P:%{PFX}e \
> size:%{MB_FILESIZE}e %{Range}i" combined_redirect
>
> To use the newly defined log format, you use "combined_redirect" with
> the CustomLog directive of the virtual host in question, instead of the
> usually defined "combined" log format.
>
>
> ... This won't fix the issue you are currently seeing (no mirror being
> used), but it will add transparency to what's going on in general, which
> is why I'm mentioning it.
>
>
done
and restarted apache

>
>
>> Also, I am still getting could not resolve continent, referer:
>> http://mirrorbrain-testing.sugarlabs.org/soas/repositories/2/ error.
>
>
> Hm, did you already update to the newer mod_geoip that I packaged last
> week? With that, it should work fine and this error should disappear
> with the upgrade.

I installed mirror brain via the packages you described after you made
that up date.

> Also, the following should work:
>
>  # geoiplookup_continent 207.118.255.6
> NA
>
>
> Hm^2, do you run with loglevel "Warn" or higher? When no mirror is
> found, you should actually see a message about it in the error_log:
>
>       ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r,
>                "[mod_mirrorbrain] no mirrors found for %s", filename);
>
> It should become visible if you change the loglevel to Info...
> Nevertheless, it is pretty clear that no (active) mirror is found.

I had set MirrorBrainDebug add

just added LogLevel info

> The failure to look up the continent also needs to be fixed, but it
> doesn't prevent mirror redirects as such.
>
> I think that seeing the output of
>        mb file ls '*abiword-2.6.8-4.fc11.sl.i386.rpm'
> would be most helpful at this point. What does it say?

mirrorbrain_at_mirrorbrain:/etc/apache2/sites-available$  mb file ls
'*abiword-2.6.8-4.fc11.sl.i386.rpm'
/usr/lib/python2.6/dist-packages/sqlobject/converters.py:165:
DeprecationWarning: the sets module is deprecated
  from sets import Set, ImmutableSet
na us  100 ok       ok   download2.sugarlabs.org
na us  100 ok       ok   download.sugarlabs.org
na us  100 ok       ok   download2.sugarlabs.org
na us  100 ok       ok   download.sugarlabs.org
na us  100 ok       ok   download2.sugarlabs.org
na us  100 ok       ok   download.sugarlabs.org
na us  100 ok       ok   download2.sugarlabs.org
na us  100 ok       ok   download.sugarlabs.org


I'll add you to sudoer and admin as soon as i figure out how.

david

> Thanks,
> Peter
>

_______________________________________________
mirrorbrain mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain/

Note: To remove yourself from this mailing list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-request_at_mirrorbrain.org