[mirrorbrain-commits] [opensuse-svn] r7624 - trunk/tools/download-redirector-v2/mirrordoctor/mb

Author: poeml
Date: 2009-06-26 06:42:03 -0600 (Fri, 26 Jun 2009)
New Revision: 7624

Modified:
   trunk/tools/download-redirector-v2/mirrordoctor/mb/testmirror.py
Log:
mb probefile:
The size of the process pool isn't configurable yet (to be added later). For now,
don't start more processes than needed for the jobs. (Apparently the
multiprocessing process pool has a habit to always fork the given number of
children processes, more than needed.)


Modified: trunk/tools/download-redirector-v2/mirrordoctor/mb/testmirror.py
===================================================================
--- trunk/tools/download-redirector-v2/mirrordoctor/mb/testmirror.py	2009-06-26 11:42:58 UTC (rev 7623)
+++ trunk/tools/download-redirector-v2/mirrordoctor/mb/testmirror.py	2009-06-26 12:42:03 UTC (rev 7624)
@@ -212,7 +212,13 @@
 
 
     if mp_mod in ['processing', 'multiprocessing']:
-        p = Pool(24)
+        # FIXME make the pool size configurable
+        # we don't need to spawn more processes then we have jobs to do
+        if len(probelist) < 24:
+            pool_size = len(probelist)
+        else:
+            pool_size = 24
+        p = Pool(pool_size)
         result = p.map_async(probe_report, probelist)
         #print result.get(timeout=20)
         return result.get()

_______________________________________________
Opensuse-svn mailing list
Opensuse-svn_at_forge.novell.com
http://forge.novell.com/mailman/listinfo/opensuse-svn


_______________________________________________
mirrorbrain-commits mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain-commits/

Note: To remove yourself from this list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-commits-request_at_mirrorbrain.org