[mirrorbrain-commits] r8001 - /trunk/docs/installation/initial_config.rst

From: <poeml_at_mirrorbrain.org>
Date: Sun, 14 Mar 2010 20:30:28 -0000
Author: poeml
Date: Sun Mar 14 21:30:27 2010
New Revision: 8001

URL: http://svn.mirrorbrain.org/viewvc/mirrorbrain?rev=8001&view=rev
Log:
commit new source file forgotten with last commit.

Added:
    trunk/docs/installation/initial_config.rst

Added: trunk/docs/installation/initial_config.rst
URL: http://svn.mirrorbrain.org/viewvc/mirrorbrain/trunk/docs/installation/initial_config.rst?rev=8001&view=auto
==============================================================================
--- trunk/docs/installation/initial_config.rst (added)
+++ trunk/docs/installation/initial_config.rst Sun Mar 14 21:30:27 2010
_at_@ -1,0 +1,338 @@
+
+.. _initial_configuration:
+
+Initial configuration steps on all platforms
+============================================
+
+Create mirrorbrain.conf
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Create a configuration file named :file:`/etc/mirrorbrain.conf` with the content below::
+
+    [general]
+    instances = main
+    
+    [main]
+    dbuser = mirrorbrain
+    dbpass = 12345
+    dbdriver = postgresql
+    dbhost = 127.0.0.1
+    # optional: dbport = ...
+    dbname = mirrorbrain
+    
+    [mirrorprobe]
+    # logfile = /var/log/mirrorbrain/mirrorprobe.log
+    # loglevel = INFO
+  
+
+.. note::
+   The database password in the above template is only a placeholder and you
+   need to edit it: change it to the actual password, the one that you gave
+   when you ran PostgreSQL's :program:`createuser` command. Likewise, make sure
+   that you picked the same username.
+
+Set the following permissions and privileges on the file::
+
+  sudo chmod 0640 /etc/mirrorbrain.conf 
+  sudo chown root:mirrorbrain /etc/mirrorbrain.conf
+
+
+
+* now you should be able to type 'mb list' without getting an error.
+  It'll produce no output, but exit with 0. If it gives an error, something is
+  wrong.
+
+
+
+Other possible options per MirrorBrain instance are:
+
+.. describe:: scan_top_include
+
+   Directory names separated by spaces. Meaning: Scan only these directories,
+   and ignore all other directories at the top level.
+
+.. describe:: scan_exclude_rsync
+
+   Exclude list for rsync scans (same rules as for rsyncs option ``--exclude``
+   apply). Meaning: Ignore all directories or path names that match, everywhere
+   in the tree.
+
+.. describe:: scan_exclude
+
+   Exclude list for FTP scans. Meaning: Ignore all directories or path names
+   that match, everywhere in the tree.
+
+
+Test the database admin tool
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the following command returns no error, but rather displays its usage info,
+the installation should be quite fine::
+
+  mb help
+
+
+Creating some mirrors
+~~~~~~~~~~~~~~~~~~~~~
+
+* collect a list of mirrors (their HTTP baseurl, and their rsync or FTP baseurl
+  for scanning). For example::
+
+    http://ftp.isr.ist.utl.pt/pub/MIRRORS/ftp.suse.com/projects/
+    rsync://ftp.isr.ist.utl.pt/suse/projects/
+
+    http://ftp.kddilabs.jp/Linux/distributions/ftp.suse.com/projects/
+    rsync://ftp.kddilabs.jp/suse/projects/
+
+
+
+  Now you need to enter the mirrors into the database; it could be done using the
+  "mb" mirrorbrain tool. (See 'mb help new' for full option list.)::
+
+    mb new ftp.isr.ist.utl.pt \
+           --http http://ftp.isr.ist.utl.pt/pub/MIRRORS/ftp.suse.com/projects/ \
+           --rsync rsync://ftp.isr.ist.utl.pt/suse/projects/
+
+    mb new ftp.kddilabs.jp \
+           --http http://ftp.kddilabs.jp/Linux/distributions/ftp.suse.com/projects/ \
+           --rsync rsync://ftp.kddilabs.jp/suse/projects/
+
+
+
+  The tool automatically figures out the GeoIP location of each mirror by itself.
+  But you could also specify them on the commandline.
+
+  If you want to edit a mirror later, use::
+
+    mb edit <identifier>
+
+  To simply display a mirror, you could use 'mb show kddi', for instance.
+
+  Finally, each mirror needs to be scanned and enabled::
+
+    mb scan --enable <identifier>
+
+  See the output of 'mb help' for more commands.
+
+
+
+
+  * mirror monitoring needs to be configured. Put this into /etc/crontab::
+
+      -* * * * *                mirrorbrain   mirrorprobe
+
+    configure mirror scanning::
+
+      45 * * * *                mirrorbrain   mb scan -j 3 -a
+
+    another cron job can remove unreferenced files from the database::
+
+      # Monday: database clean-up day...
+      30 1 * * mon              mirrorbrain   mb db vacuum
+
+
+
+TODO: describe how to test that the install was successful
+    (When testing, consider any excludes that you configured, and which might
+    introduce confusion.)
+
+* Many HTTP clients can be used for testing, but `cURL`_ is a most helpful tool
+  for that. Here are some examples.
+
+  Showy the HTTP response code and the Location header pointing to the new location::
+
+    curl -sI <url>
+
+  Display the metalink::
+
+    curl -s <url>.metalink
+
+  Show a HTML list with the available mirrors::
+
+    curl -s <url>?mirrorlist
+
+.. _`cURL`: http://curl.haxx.se/
+
+
+TODO: describe a decent logging setup
+
+
+* further things that you might want to configure:
+
+  * mod_autoindex_mb, a replacement for the standard module mod_autoindex::
+
+      a2dismod autoindex
+      a2enmod autoindex_mb
+      Add IndexOptions Metalink Mirrorlist
+      # or IndexOptions +Metalink +Mirrorlist, depending on your config
+
+  * add a link to a CSS stylesheet for mirror lists::
+
+      MirrorBrainMirrorlistStylesheet "http://static.opensuse.org/css/mirrorbrain.css"
+
+    and for the autoindex::
+
+      IndexStyleSheet "http://static.opensuse.org/css/mirrorbrain.css"
+
+  * prepare the metalink hashes. 
+
+    * First, add some configuration::
+
+        MirrorBrainMetalinkPublisher "openSUSE" http://download.opensuse.org
+        MirrorBrainMetalinkHashesPathPrefix /srv/hashes/srv/opensuse
+
+    * you need to create a directory where to store the hashes. For instance,
+      :file:`/srv/hashes/srv/opensuse`. Note that the full pathname to
+      the filetree (``/srv/opensuse``) is part of this target path.
+      
+      Make the directory owned by the ``mirrorbrain`` user.
+
+    * now, create the hashes with the following command. It is best run as
+      unprivileged user (``mirrorbrain``)::
+
+        mb makehashes /srv/opensuse -t /srv/hashes/srv/opensuse
+
+
+    * add the hashing command to /etc/crontab to be run every few hours. Alternatively, run
+      it after changes in the file tree happen.
+
+
+
+
+Create a DNS alias for your web host, if needed
+
+
+
+Configure GeoIP
+----------------------------------------------
+
+Edit /etc/apache2/conf.d/mod_geoip.conf::
+
+  <IfModule mod_geoip.c>
+     GeoIPEnable On
+     GeoIPDBFile /var/lib/GeoIP/GeoIP.dat
+     #GeoIPOutput [Notes|Env|All]
+     GeoIPOutput Env
+  </IfModule>
+
+(Change GeoIPOutput All to GeoIPOutput Env)
+
+Note that a caching mode like MMapCache needs to be used, when Apache runs with
+the worker MPM.In this case, use::
+
+  <IfModule mod_geoip.c>
+     GeoIPEnable On
+     GeoIPDBFile /var/lib/GeoIP/GeoIP.dat MMapCache
+     GeoIPOutput Env
+  </IfModule>
+
+
+
+.. configure GeoIP database updates
+
+Seting up automatic updates of the GeoIP database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+New versions of the GeoIP database are released each month. You can set up a
+cron job to automatically fetch new updates as follows. If you do that, make
+sure to set the GeoIPDBFile path (see above) to
+:file:`/var/lib/GeoIP/GeoLiteCity.dat.updated`::
+
+  # update GeoIP database on Mondays
+  31 2 * * mon   root    sleep $(($RANDOM/1024)); /usr/bin/geoip-lite-update
+
+
+
+
+
+  * configure mod_mirrorbrain.
+    You probably want to reate a vhost (e.g.
+    /etc/apache2/vhosts.d/samba.mirrorbrain.org.conf) and add the MirrorBrain
+    configuration like shown here::
+
+      <VirtualHost your.host.name:80>
+          ServerName samba.mirrorbrain.org
+      
+          ServerAdmin webmaster_at_example.org
+      
+          DocumentRoot /srv/samba/pub/projects
+      
+          ErrorLog     /var/log/apache/samba.mirrorbrain.org/logs/error_log
+          CustomLog    /var/log/apache/samba.mirrorbrain.org/logs/access_log combined
+
+          <Directory /srv/samba/pub/projects>
+              MirrorBrainEngine On
+              MirrorBrainDebug Off
+              FormGET On
+              MirrorBrainHandleHEADRequestLocally Off
+              MirrorBrainMinSize 2048
+              MirrorBrainExcludeUserAgent rpm/4.4.2*
+              MirrorBrainExcludeUserAgent *APT-HTTP*
+              MirrorBrainExcludeMimeType application/pgp-keys
+
+              Options FollowSymLinks Indexes
+              AllowOverride None
+              Order allow,deny
+              Allow from all
+          </Directory>
+      
+      </VirtualHost>
+
+  * restart Apache, best while watching the error log::
+
+      tail -F /var/log/apache/*_log &
+      apachectl restart
+
+    
+
+
+Create a virtual host
+---------------------
+
+The following snippet would create a new site as virtual host::
+
+  sudo sh -c "cat > /etc/apache2/sites-available/mirrorbrain << EOF
+  <VirtualHost 127.0.0.1>
+      ServerName mirrors.example.org
+      ServerAdmin webmaster_at_example.org
+      DocumentRoot /var/www/downloads
+      ErrorLog     /var/log/apache2/mirrors.example.org/error.log
+      CustomLog    /var/log/apache2/mirrors.example.org/access.log combined
+      <Directory /var/www/downloads>
+          MirrorBrainEngine On
+          MirrorBrainDebug Off
+          FormGET On
+          MirrorBrainHandleHEADRequestLocally Off
+          MirrorBrainMinSize 2048
+          MirrorBrainExcludeUserAgent rpm/4.4.2*
+          MirrorBrainExcludeUserAgent *APT-HTTP*
+          MirrorBrainExcludeMimeType application/pgp-keys
+          Options FollowSymLinks Indexes
+          AllowOverride None
+          Order allow,deny
+          Allow from all
+      </Directory>
+  </VirtualHost>
+  EOF
+  "
+
+Make the log directory::
+
+  sudo mkdir /var/log/apache2/mirrors.example.org/
+
+Make the download directory::
+
+  sudo mkdir /var/www/downloads
+
+Enable the site::
+
+  sudo a2ensite mirrorbrain
+
+
+Restart Apache, best while watching the error log::
+
+  sudo tail -f /var/log/apache2/error.log &
+  sudo /etc/init.d/apache2 restart
+
+
+




_______________________________________________
mirrorbrain-commits mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain-commits/

Note: To remove yourself from this list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-commits-request_at_mirrorbrain.org
Received on Sun Mar 14 2010 - 20:30:29 GMT

This archive was generated by hypermail 2.3.0 : Mon Feb 20 2012 - 23:47:04 GMT