[mirrorbrain-commits] r8121 - /trunk/docs/intro.rst

From: <poeml_at_mirrorbrain.org>
Date: Mon, 06 Sep 2010 16:50:29 -0000
Author: poeml
Date: Mon Sep  6 18:50:29 2010
New Revision: 8121

URL: http://svn.mirrorbrain.org/viewvc/mirrorbrain?rev=8121&view=rev
docs/intro: Create a new chapter "Implementation and design notes", and add a section "Database hash store"


Modified: trunk/docs/intro.rst
URL: http://svn.mirrorbrain.org/viewvc/mirrorbrain/trunk/docs/intro.rst?rev=8121&r1=8120&r2=8121&view=diff
--- trunk/docs/intro.rst (original)
+++ trunk/docs/intro.rst Mon Sep  6 18:50:29 2010
_at_@ -74,3 +74,28 @@
 .. _`Sphinx Python Documentation Generator`: http://sphinx.pocoo.org/
 .. _`reStructuredText Primer`: http://sphinx.pocoo.org/rest.html
+Implementation and design notes
+Database hash store
+Since 2.13.0, all hashes are stored in the database. Before, they were kept in
+files in the file system. The old store was suitable only for generation of
+old-style Metalinks.
+Inside the database, the hashes are stored as compart binary blobs. For
+transfer, they are converted to hexadecimal. This is due to the following
+design decision: Storage is binary in so-called ``bytea`` columns.  PostgreSQL
+automatically escapes binary (bytea) data on output in its own way. But this
+encoding is not very efficient in space. Hex encoding is more efficient (it
+results in shorter strings, and thus less data to transfer over the wire, and
+it's also faster). The escape format is kind phased out, and it doesn't make
+sense to use it in a new application (which we are).  On the other hand,
+storage in bytea is as compact as it can be, which is good.  So we store the
+data in binary, and provide a database view which converts to hex on the fly.
+The hex encoding function in PostgreSQL seems to be fast.

mirrorbrain-commits mailing list
Archive: http://mirrorbrain.org/archive/mirrorbrain-commits/

Note: To remove yourself from this list, send a mail with the content
to the address mirrorbrain-commits-request_at_mirrorbrain.org
Received on Mon Sep 06 2010 - 16:50:31 GMT

This archive was generated by hypermail 2.3.0 : Mon Feb 20 2012 - 23:47:04 GMT