Issue37

Title Could the metalink-hasher create Torrent files in the same step?
Priority wish Status resolved
Superseder Nosy List poeml
Assigned To poeml Keywords

Created on 2010-02-03.22:45:30 by poeml, last changed by poeml.

Messages
msg219 (view) Author: poeml Date: 2010-09-05.23:44:55
This issue can be closed as resolved. Way cool!
msg173 (view) Author: poeml Date: 2010-03-31.20:04:21
another interesting link, regarding hashes to be included: 
http://wiki.depthstrike.com/index.php/P2P:Protocol:Specifications:Optional_Hashes
msg169 (view) Author: poeml Date: 2010-03-31.19:29:03
Hash cache redesign largely finished, and in testing now.

Torrent support working already in trunk. Now figuring out things like updating 
the tracker servers, and exact bittorrent format that is suitable for as many 
clients as possible.
msg138 (view) Author: poeml Date: 2010-03-08.20:48:00
This requires a hash cache redesign -- tracked in issue 40.
msg134 (view) Author: poeml Date: 2010-02-25.00:02:38
Links about generation of torrents:

http://en.wikipedia.org/wiki/BitTorrent_(protocol)#Creating_and_publishing_torrent
s
http://www.bittorrent.org/beps/bep_0003.html
http://wiki.theory.org/BitTorrentSpecification
http://en.wikipedia.org/wiki/Bencode
http://pypi.python.org/pypi/BitTorrent-bencode/5.0.8
msg133 (view) Author: poeml Date: 2010-02-24.23:55:12
Looking further, this is all straightforward - metalink-hasher could provide the 
hashes in a form digestible by Apache, and they would contain all that's needed. 
Apache could easily generated the torrent live.

However the question arises whether it might be better to have real .torrent files 
on disk (and maybe even next to the original files), simply for two reasons:

- they can be mirrored then
- to run a seeder (I'm not sure about the exact needs of seeders, but I suppose 
they benefit from already prepared torrent files)

For practical reasons, the latter might be more important than the former.
msg121 (view) Author: poeml Date: 2010-02-04.11:43:28
Darix suggests as first approach to hack make_torrent calls into the python 
thingie (metalink-hasher).

See http://www.rasterbar.com/products/libtorrent/examples.html
msg120 (view) Author: poeml Date: 2010-02-03.22:48:44
Darix suggests to talk to hydri in #libtorrent.
msg119 (view) Author: poeml Date: 2010-02-03.22:45:30
Vittorio suggested that the metalink-hasher might create torrent files along the way.

The metalink-hasher is a Python program which walks the file tree and looks what needs 
to be done; it calls the "metalink" binary which is a standalone binary.

Darix suggests to look at make_torrent example from libtorrent-rasterbar and either 
call that or even link libtorrent-rasterbar into metalink. That way you only need to 
read each block once for hashing.

http://www.rasterbar.com/products/libtorrent/
http://www.rasterbar.com/products/libtorrent/client_test.html

Maybe metalink could be extended easily to do the same. Or, maybe the output of 
"metalink" could be used - after all, it outputs a series of hashes. However, maybe the 
blocks would not fit?
History
Date User Action Args
2010-09-05 23:45:03poemlsetassignedto: poeml
2010-09-05 23:44:56poemlsetstatus: testing -> resolved
messages: + msg219
2010-03-31 20:04:21poemlsetmessages: + msg173
2010-03-31 19:29:05poemlsetstatus: deferred -> testing
messages: + msg169
2010-03-08 21:20:32poemlsetstatus: chatting -> deferred
2010-03-08 20:48:00poemlsetmessages: + msg138
2010-02-25 00:02:38poemlsetmessages: + msg134
2010-02-24 23:55:12poemlsetmessages: + msg133
2010-02-04 11:43:28poemlsetmessages: + msg121
2010-02-03 22:48:44poemlsetmessages: + msg120
2010-02-03 22:45:30poemlcreate