Re: [mirrorbrain] [PATCH] IPv6 support for mod_asn - Apparently a very low hanging fruit.

Hi,

great patches! 

More inline below.

On Fri, May 08, 2015 at 08:43:13 +0200, Michael Meier wrote:
> > With the adoption of IPv6 for more and more end user connections I believe it's
> > necessary that mod_asn is also able to match IPv6 addresses to their AS and get
> > the users to the closest IPv6-enabled mirror possible.
> > [...]
> > as you can see it works just as it should, giving back the correct order, IPv4
> > and IPv6 prefixes in harmony !
> 
> I played around with your incredibly small patches today, and it really
> does seem to work very well. It really is a very low hanging fruit.
> 
> > The only piece to the IPv6 puzzle that is missing to me at least, is a good
> > source for the IPv6 prefixes <-> AS list in plain just like there is with
> > http://www.routeviews.org/ for IPv4. If someone has BGP full-table running on
> > maybe a looking glass router one can simply get it from there, but that is of
> > course not something for everyone. So this needs some thought and then the
> > appropriate addition to the provided script to gather the data and put it into
> > the database.
> 
> While there are no textual dumps available for IPv6, there is really no
> lack of binary dumps. For example, ripe offers quite a few on
> data.ris.ripe.net. But since mirrorbrain already uses routeviews.org
> data for IPv4, I used their IPv6 data for my tests.
> Unfortunately, there is no "current" symlink as with the IPv4 text data,
> so you need to construct the right filename depending on the current
> date (and maybe time) yourself. For example, the data i used was
> http://archive.routeviews.org/route-views6/bgpdata/2015.05/RIBS/rib.20150508.0000.bz2
> So generally,
> http://archive.routeviews.org/route-views6/bgpdata/YYYY.MM/RIBS/rib.YYYYMMDD.HH00.bz2
> They offer dumps every two hours, but for mirrorbrain use just taking a
> random one from the last week should be perfectly fine.
> The command-sequence I used then was:
> > bzcat rib.20150508.0000.bz2 | ./zebra-dump-parser.pl | ./filteruniqueprefixes.pl > /tmp/v6routedump.dat
> > asn_get_routeviews > /tmp/v4routedump.dat
> > cat /tmp/v6routedump.dat /tmp/v4routedump.dat | asn_import.py
> The helper tools used in this and the slightly modified version of
> asn_import.py that I used can be found in
> https://github.com/poempelfox/mod_asn/tree/ipv6support
> While this works, it could need some more streamlining, e.g.:
> - providing a download-tool like for the IPv4 data
> - providing a mirror of the data used on mirrorbrain.org/routeviews/
> - install the new helper tool(s) in packaging
> - perhaps merging it all together into asn_get_routeviews.py
> It would really be nice if this could be integrated into mirrorbrain.

That's all great. I'm only struggling to package it in "digestible"
form, so the existing users don't run into problems. Unfortunately, the
old mod_asn SQL query doesn't work with the new database schema.
I think I might add the new column alongside the old one first and
duplicate the data, so that mod_asn can continue running until it's updated
as well. 

This database upgrade could be integrated into the mb tool, where there
already is some logic for things like that.. 

Regarding the import machinery, there's quite a zoo of scripts now... it
would be nice to have something more integrated, but it's not trivial to
rewrite everything, and why not use what's there (and what works)... however, at least
some kind of wrapper would be nice to hide the mess below it, and to
ease admministration. Maybe also wrapped into the mb tool. 

So, I have not managed to integrate your patches so easily yet, but I'm
slowly getting there.

> While I'm at it, I also have a small documentation fix at
> https://github.com/poempelfox/mod_asn/tree/mod_asn_docfix1
> This should be trivial to merge, it simply adapts the documentation to
> non-ancient postgresql-versions.
> 

Excellent! Merged. Thanks.

Peter


_______________________________________________
mirrorbrain mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain/

Note: To remove yourself from this mailing list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-request_at_mirrorbrain.org