[mirrorbrain-commits] [mod_stats] r44 - in /trunk/tools: dlcount.py ooo.conf

Author: poeml
Date: Thu Nov 26 15:37:40 2009
New Revision: 44

URL: http://svn.mirrorbrain.org/viewvc/mod_stats?rev=44&view=rev
Log:
use (ip, url, referer, user-agent) as key for memorizing requests,
leaving out redundant data like country and status code.

Modified:
    trunk/tools/dlcount.py
    trunk/tools/ooo.conf

Modified: trunk/tools/dlcount.py
URL: http://svn.mirrorbrain.org/viewvc/mod_stats/trunk/tools/dlcount.py?rev=44&r1=43&r2=44&view=diff
==============================================================================
--- trunk/tools/dlcount.py (original)
+++ trunk/tools/dlcount.py Thu Nov 26 15:37:40 2009
_at_@ -265,12 +265,6 @@
         (ip, url, status, referer, ua, country) = req
         url_raw = url
 
-        # over a window of StatsDupWindow last requests, the same request must
-        # not have occured already
-        m = hashlib.md5()
-        m.update(repr(req))
-        md = m.digest()
-
         skip = False
         for r, mreg in conf['statsignoremask']:
             if r.match(url):
_at_@ -285,6 +279,15 @@
                 skip = True
                 break
         if skip: continue
+
+        # over a window of StatsDupWindow last requests, the same request must
+        # not have occured already
+        m = hashlib.md5()
+        m.update(ip)
+        m.update(url)
+        m.update(referer)
+        m.update(ua)
+        md = m.digest()
 
         # was the requests seen recently? If yes, ignore it.
         # otherwise, put it into the ring buffer.

Modified: trunk/tools/ooo.conf
URL: http://svn.mirrorbrain.org/viewvc/mod_stats/trunk/tools/ooo.conf?rev=44&r1=43&r2=44&view=diff
==============================================================================
--- trunk/tools/ooo.conf (original)
+++ trunk/tools/ooo.conf Thu Nov 26 15:37:40 2009
_at_@ -14,7 +14,7 @@
 
 
 # define the size of a sliding window for remembering the last requests,
-# while parsing the log. Keyed by (ip, url, status, referer, user-agent, country), 
+# while parsing the log. Keyed by (ip, url, referer, user-agent), 
 # every requests is checked whether it has been seen in identical form before.
 StatsDupWindow 200
 




_______________________________________________
mirrorbrain-commits mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain-commits/

Note: To remove yourself from this list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-commits-request_at_mirrorbrain.org