[mirrorbrain-commits] r8121 - /trunk/docs/intro.rst

Author: poeml
Date: Mon Sep  6 18:50:29 2010
New Revision: 8121

URL: http://svn.mirrorbrain.org/viewvc/mirrorbrain?rev=8121&view=rev
Log:
docs/intro: Create a new chapter "Implementation and design notes", and add a section "Database hash store"

Modified:
    trunk/docs/intro.rst

Modified: trunk/docs/intro.rst
URL: http://svn.mirrorbrain.org/viewvc/mirrorbrain/trunk/docs/intro.rst?rev=8121&r1=8120&r2=8121&view=diff
==============================================================================
--- trunk/docs/intro.rst (original)
+++ trunk/docs/intro.rst Mon Sep  6 18:50:29 2010
_at_@ -74,3 +74,28 @@
 .. _`Sphinx Python Documentation Generator`: http://sphinx.pocoo.org/
 .. _`reStructuredText Primer`: http://sphinx.pocoo.org/rest.html
 
+
+
+Implementation and design notes
+-------------------------------
+
+Database hash store
+^^^^^^^^^^^^^^^^^^^
+
+Since 2.13.0, all hashes are stored in the database. Before, they were kept in
+files in the file system. The old store was suitable only for generation of
+old-style Metalinks.
+
+Inside the database, the hashes are stored as compart binary blobs. For
+transfer, they are converted to hexadecimal. This is due to the following
+design decision: Storage is binary in so-called ``bytea`` columns.  PostgreSQL
+automatically escapes binary (bytea) data on output in its own way. But this
+encoding is not very efficient in space. Hex encoding is more efficient (it
+results in shorter strings, and thus less data to transfer over the wire, and
+it's also faster). The escape format is kind phased out, and it doesn't make
+sense to use it in a new application (which we are).  On the other hand,
+storage in bytea is as compact as it can be, which is good.  So we store the
+data in binary, and provide a database view which converts to hex on the fly.
+The hex encoding function in PostgreSQL seems to be fast.
+
+




_______________________________________________
mirrorbrain-commits mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain-commits/

Note: To remove yourself from this list, send a mail with the content
 	unsubscribe
to the address mirrorbrain-commits-request_at_mirrorbrain.org