[JDEV] File transfers

[Max Metral]

Have you been looking at the fact that no service provider is going to be
able to afford a system that does in-band binary transfer exclusively?  Are
you saying all file transfer and binary data (H.323) belong in band?  Just
trying to make sure I understand your suggestion.

-----Original Message-----
From: Michael F Lin [mailto:MFLIN at us.ibm.com]
Sent: Friday, June 07, 2002 11:38 AM
To: jdev at jabber.org
Subject: Re: [JDEV] File transfers



I would also add that encoding concerns could be alleviated by using
something like Jer's XIB or SOAP attachments, which are extensions to XML
streams that allows inline binary payloads. There are a few things we would
do differently, but that's the general idea.

Again, the key advantage is that all packets are still JID-routed, so we
get through all firewalls and NATs without having to set up complicated
stateful proxy servers in addition to what we already have.

Really, the way to do this is to define a maximum _packet_ size, for all
Jabber packets. This makes everything manageable from the server side. If
you need to transfer a packet larger than that maximum size, either drop
out of band and deal with firewall problems yourself, or chunk the message.
Personally, I think the latter is preferable, because all these problems of
negotiating through firewalls and finding PASS servers just go away.

Unfortunately, it is not easy to define a maximum packet size because we
don't have length-prefixed framing, so you never know the length of a
packet until you've received the whole thing - by which time
you (almost) might as well send it along anyway! This is why we need
something like JEP-0017 or XATP.

Guys, frankly, we've been looking at this problem in this lab for the last
_year_ (some of you may remember me talking about it at JabberCon, over
most of the same objections). We've done the research and the prototypes
for high-performance Jabber-like messaging systems. I know the
disadvantages seem obvious and the advantages seem hard to accept - but we
are positive that high-performance, scalable in-band or parallel-band
transfer of large payloads is both practically advantageous and technically
feasible.

We are preparing a detailed proposal on the matter, which we hope will be a
major concrete step towards JNG. I hope we can soon put this silly argument
to rest for now.

-Mike



|---------+------------------------------>
|         |           "Michael Rothwell" |
|         |           <rothwell at holly-spr|
|         |           ings.nc.us>        |
|         |           Sent by:           |
|         |           jdev-admin at jabber.o|
|         |           rg                 |
|         |                              |
|         |                              |
|         |           06/06/2002 04:23 PM|
|         |           Please respond to  |
|         |           jdev               |
|         |                              |
|---------+------------------------------>
 
>---------------------------------------------------------------------------
---------------------------------------------------|
  |
|
  |       To:       <jdev at jabber.org>
|
  |       cc:
|
  |       Subject:  Re: [JDEV] File transfers
|
  |
|
  |
|
 
>---------------------------------------------------------------------------
---------------------------------------------------|



The jabber server could simply have a config option for "max transfer
size," which ,when set to zero, disables all file transfers. Clients are
notified of the file-transfer capabilities of the server by the server. You
could even take it one step further and allow/disallow file-transfter use
per account ID and/or groups. Other config options which would be useful
are "cache time" and "max cache size" -- if cache time and/or size are
zero, then if the server cannot stream the data directly to the
recipient(s), it informs the client that the recipient isn't accepting
transfers. You could even limit the number of transfers/person/day, etc.

Leave it up to the clients to B64 encode/decode the transfer. The server
doesn't have to care, which will reduce processing load. The "max size"
parameters will be for the encoded versions, which is what server operators
care about, because it's what they pay for.

Server-supported transfers would be a nice config option, esp. for
behind-the-firewall and small-number-of-users servers.

 ----- Original Message -----
 From: Gallo, Felix S.
 To: 'jdev at jabber.org'
 Sent: Thursday, June 06, 2002 3:54 PM
 Subject: RE: [JDEV] File transfers



 Mike writes:
 > Firstly, there is no inherent problem with sending moderately
 > large files through a software server. Sendmail does it all
 > day, every day, on a massive scale, without relying on
 > client-to-client connections.


 However, most mail is accessed through POP, IMAP, or Exchange,
 which are definitely client-to-client connections -- for the
 simple reason that sendmail doesn't scale very well.  For
 every byte send to a sendmail server, two bytes traverse the
 network.  Most unfortunately, those two bytes always involve
 just the one sendmail server and its attached network.


 What positives do you get for having the sendmail server involved
 in a large file transaction between two parties?  You get a
 guarantee of delivery and no need for continued storage on the
 sending party's side (since the file 'moves' to the sendmail server).
 If you're sending the file to multiple users at once, you get less
 traffic on your local network (the server takes the load).
 You also get an opportunity to mediate the file somehow (e.g.,
 virus checking it as a service, converting it from aac into mp3,
 storing it for later delivery to other users..)


 What positives do you get for not having the sendmail server
 involved?  The network on the sendmail server sees 0X load
 rather than 2X load; the latency is lower; the sendmail server
 has no storage requirement; and you have arguably fewer points
 of failure.


 Pragmatically, taking the load off the server is more valuable
 in the normal case than replicating HTTP/FTP/SMTP/FXP yet again.
 The fact that 'SMTP does it' is not a great rationale for forcing
 all the jabber servers to pay 2X bandwidth costs for file transfers
 between their users.


 F.




 **********************************************************************
 E-mail sent through the Internet is not secure. Western Asset therefore
 recommends that you do not send any confidential or sensitive information
 to
 us via electronic mail, including social security numbers, account
 numbers,
 or personal identification numbers. Delivery, and or timely delivery of
 Internet mail is not guaranteed. Western Asset therefore recommends that
 you do not send time sensitive or action-oriented messages to us via
 electronic mail.
 **********************************************************************








_______________________________________________
jdev mailing list
jdev at jabber.org
http://mailman.jabber.org/listinfo/jdev