[JDEV] asynchronous, efficient, in-band bytestreams / file transfer

[Tijl Houtbeckers]

Hi Darrell,

Darrell Berry <darrell at ku24.com> wrote on 9-11-2002 16:52:41:
>
>hi
>
>I understand from some googling that this is the kind of topic which 
>may rake over the ashes of ancient flamewars. 
>
>I hope that's not the case: if there's a simple answer to the question,
>so much the better!
>

The only thing Jabber can queue right now is <message>. In theory you 
could use this to transfer files asynchronically, however this is not 
very efficient when you're talking about files several GB big (if not 
impossible). The implementation was never meant for that, and in any 
case you'll have to split the file in more then one pieces when sending 
if (the inband data JEP describes this). Also this does not take care 
of a pub-sub distribution of files, and there are other issues with it. 
This rules out a purely client based implementation. 

However, writing your own component could solve this. Since the 
component will never be offline jabberd itself won't have to do any 
ineffecient queuing. The component can store data more efficiantly, and 
can take care of pub-sub distribution (there are several proposed JEP's 
for jabber based Pub-Sub, and there's a lot of work being done on 
getting a final spec. out), guarantee delivery, etc. I don't know the 
exact requirment of your project but ofcourse both store&forward and a 
combination of store&forward and streaming (Point to Point Proxy or 
multicast) can be implemented (that's all up to you...). 

Since the component most likely will not be firewalled (asumming that 
the jabber server isn't either) it's now possible to consider using OOB 
data too for transfering data to the component. Jabber is perfect for 
signaling and this way you'll have the huge flexability of the jabber 
protocol available for your project, and at the same time the most 
effecient way of datatransfer and storing. You could even consider 
attempting to use P2P OOB streams, and fall back to using the component 
if it doesn't succeed. (using jabber for signaling this would be 
entirely transparant). 

You could build a very efficient, scalable and highly flexibel system 
like this! 

Also, you could still do datatraffic inband (espc. if you turn off 
Karma on the server) or use it as a fall back, jabberd is quite capable 
of handeling that. However it is still no match ofcourse for a simple 
OOB binary stream when it comes to data traffic, speed, CPU power and 
memory requirments. 

-- 
Tijl Houtbeckers
Java/J2ME/GPRS Software Engineer @ Splendo
The Netherlands