Re: [mirrorbrain] Minimise return footprint

From: Peter Pöml <peter_at_poeml.de>
Date: Wed, 26 Mar 2014 00:12:39 +0100
Hi!

On Sun, Mar 23, 2014 at 11:12:22AM +0100, Kib wrote:
> Hey all,
> 
> I know that mirrorbrain sends multiple mirrors as response to a request.
> However, we currently use mirrorbrain to cater to the needs of our own
> application which currently is not able to follow secondary mirrors in case
> the first one fails.
> 
> In order to minimize the amount of bandwidth our main redirector needs it
> would be nice if there was a mirrorbrain setting which would enable the
> absolute minimum response size possible.
> 
> Ergo, instead of replying
> 
> HTTP/1.1 302 Found
> Date: Sun, 23 Mar 2014 10:05:24 GMT
> Server: Apache
> X-MirrorBrain-Mirror: mirrorservice.org
> X-MirrorBrain-Realm: country
> Link: <http://mirrors.xbmc.org/addons/gotham/addons.xml.meta4>;
> rel=describedby; type="application/metalink4+xml"
> Link: <
> http://www.mirrorservice.org/sites/mirrors.xbmc.org/addons/gotham/addons.xml>;
> rel=duplicate; pri=1; geo=gb
> Link: <http://mirror.ox.ac.uk/sites/xbmc.org/addons/gotham/addons.xml>;
> rel=duplicate; pri=2; geo=gb
> Link: <http://mirror.nl.leaseweb.net/xbmc/addons/gotham/addons.xml>;
> rel=duplicate; pri=3; geo=nl
> Link: <http://ftp1.nluug.nl/mediaplayer/xbmc/addons/gotham/addons.xml>;
> rel=duplicate; pri=4; geo=nl
> Link: <http://ftp2.nluug.nl/mediaplayer/xbmc/addons/gotham/addons.xml>;
> rel=duplicate; pri=5; geo=nl
> Digest: MD5=Mjj/fOv8n9xQ3JkOXzkW1A==
> Digest: SHA=it0SKa/hBEY30NvZf3Y3SBK4i48=
> Digest: SHA-256=y+E2VnsWLyfCkocTPdTVn65r1WKl6A3jfzGmdKq0YRU=
> Location:
> http://www.mirrorservice.org/sites/mirrors.xbmc.org/addons/gotham/addons.xml
> Vary: Accept-Encoding
> Content-Length: 260
> Connection: close
> Content-Type: text/html; charset=iso-8859-1
> 
> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>302 Found</title>
> </head><body>
> <h1>Found</h1>
> <p>The document has moved <a href="
> http://www.mirrorservice.org/sites/mirrors.xbmc.org/addons/gotham/addons.xml
> ">here</a>.</p>
> </body></html>
> 
> I would like to have mirrorbrain reply with the minimum response that would
> still be valid as redirection:
> 
> HTTP/1.1 302 Found
> Location:
> http://www.mirrorservice.org/sites/mirrors.xbmc.org/addons/gotham/addons.xml
> Vary: Accept-Encoding
> Content-Length: 260
> Connection: close
> Content-Type: text/html; charset=iso-8859-1
> 
> Ideally this would be configurable via (one ore more) options.
> What do you think ?

A very reasonable request. Indeed the response got quite large through
all the "bells and whistles". Time to rethink this.

Yes, this should be made configurable.

I just figured that it should be easy to remove the headers (as a
workaround for now) with the Apache module mod_headers. However, I was
surprised that this doesn't work, the reason apparently being that the
response is not a HTTP 200 response, but a 302, and that
mod_mirrorbrain's headers are set in an Apache-internal structure called
err_headers_out, contrary to headers_out which might also be used.

err_headers_out is what I chose a long time ago, but I remember that I
wasn't sure if it's the right choice. But even Apache developers are not
sure about it:
http://mail-archives.apache.org/mod_mbox/perl-modperl/201209.mbox/%3C50472B28.5020905_at_gmx.net%3E

So maybe it's better to use the headers_out table instead of
err_headers_out to set Apache headers with mod_mirrorbrain; I just need
to check if it has any disadvantages (or if it just was a bad choice).
In fact, the most important header of all, "Location", is put into the
headers_out table (and it works).

The headers should be set in a way so that they can easily be tweaked
(or removed) with mod_headers. Of course, not generating them in the
first place is more elegant than removing them later.

By the way, the "document has moved ..." HTML text is a built-in
Apache default text, which is hardly ever seen by anyone; it should be
possible to reconfigure that with the "ErrorDocument" config directive
(but I didn't check that now).

So I'm sorry I can't offer an immediate workaround, but I suppose this
can be fixed relatively quickly.

(Having said that, "quickly" is such a weird term to use when having a
toddler and a newborn in the house, and a dayjob... :-)

Thanks for your input!
Peter



_______________________________________________
mirrorbrain mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain/

Note: To remove yourself from this mailing list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-request_at_mirrorbrain.org
Received on Tue Mar 25 2014 - 23:12:55 GMT

This archive was generated by hypermail 2.3.0 : Fri Mar 28 2014 - 21:17:06 GMT