[JDEV] File transfers
Richard Dobson
richard at dobson-i.net
Thu Jun 6 08:53:09 CDT 2002
Well I myself dont like using HTTP transfers in Jabber, but as thats how
file transfers are already implemented in current clients, if you want to
interoperate with them you will have to support HTTP.
I myself am creating a method of file transfer in my own client which works
differently, by each client negotiating the file transfer connection, in
which they connect directly to each other either way so if one of them is
behind a firewall/nat and the other isnt then they just establish the
connection in the appropriate direction, otherwise they connect via a PASS
server. When a socket has been established between the two clients the file
is transfered using a basic packet framing mechanism to minimise the
overhead.
Also to get around the firewall a fixed port could be mapped at the firewall
to the client, or a PASS server could be setup on the network with a range
of ports mapped to it that could act as a proxy/gateway for external clients
to connect to, or a SOCKS gateway could be setup, there are lots of better
ways to get around the firewall problem.
That is the proper way to send files between clients, putting all the
complexity required into the server to stop potential abuses or in-band file
transfer is unfeasable, it will probably slow down the server overall as it
will have to do extra checking of packets passing thru to see if they are
file transfer packets over the specified limit. If you are wanting all this
limiting and control stuff just use PASS which is designed with that in
mind.
Also I dont see why you are trying to ignore the bandwidth (its not free for
providers, they often pay by the K) aspect of this, for one encoding things
into BASE64 (required to transporting binary in XML) increases the size by
about 33% slowing down the transfer overall at the client ends (especially
when people are on dial-up) and using more bandwidth than really necessary,
it means a lot more bandwidth used than a PASS solution, also instead of
sending all file transfers thru the server dont you think its best to
connect directly if possible.
Also why are you saying that I am saying anything constructive?? The only
reason you have stated for transferring files in-band is that it gets around
firewalls, and I have suggested a better alternative that also gets around
firewalls.
Richard
----- Original Message -----
From: "Andy Beetz" <andy.beetz at clearswift.com>
To: <jdev at jabber.org>
Sent: Thursday, June 06, 2002 12:59 PM
Subject: RE: [JDEV] File transfers
> I hear what you're saying, but there are ways you could easily stop large
> transfers (or any for that matter) from occuring (limit by filesize, limit
> bandwidth p/day or p/month, not allowing it at all full stop, content
> filtering etc etc).
>
> I'm not hearing anything constructive from you. Your use of the oxymoron
> "unofficial standard" (your other post) surprises me. For a technology
that
> wants to be a standard, I would have expected it to try and accommodate
> usage of itself.
>
> The jabber server has it's ports registered with IANA why not register one
> for the clients? Then everyone is operating on the same level and not
having
> to re-invent the wheel every time someone writes a client. All I'm trying
to
> do is get something done which could prevent possible problems later on.
>
>
> -----Original Message-----
> From: Richard Dobson [mailto:richard at dobson-i.net]
> Sent: 06 June 2002 12:11
> To: jdev at jabber.org
> Subject: Re: [JDEV] File transfers
>
>
> Small level of abuse ????
> The problems I outlines are not small they are very significant, there may
> be better ways to get copyrighted files but it does not stop people using
> jabber for that purpose now does it, and any copyrighted files that get
> transfered via the jabber server open the operators of that server to
> serious legal problems because they helped transfer the actual file
between
> the two users, remember napster got shut down because they were helping
> users transfer files between each other, they werent even transfering the
> files in-band via the napster servers, which jabber already supports
opening
> it to possible legal problems already, but transfering files via the
servers
> just takes it up to a whole new level. Also transfering large files
whether
> they are split up or not is just not feasable at all because of the major
> jump in bandwidth use for the server operators. Also there is nothing
> stopping someone from pushing a file on someone wether they wanted it or
> not, the only way files should be sent is that the sender sends a request
to
> the receiver and then the receiver downloads the file from the sender
(HTTP
> or otherwise), not the sender pushing the file to the receiver. Also if
the
> whole reason for this is because of wanting to get around a firewall then
> you need to do it in the correct manor, using something like PASS, so if a
> server provider is prepared to allow the transfering of files this way
they
> just setup a PASS server, you cant just force it on server providers like
> this, that would end up making lots of the free servers either shut down
or
> start charging for using it. Bandwidth is not free.
>
> Richard
>
> ----- Original Message -----
> From: "Andy Beetz" <andy.beetz at clearswift.com>
> To: <jdev at jabber.org>
> Sent: Thursday, June 06, 2002 10:03 AM
> Subject: RE: [JDEV] File transfers
>
>
> > I can see a small level of abuse perhaps, but there are better ways to
> > distribute/get hold of copyrighted files (kazaa to name but one). The
> > fact that the communications are 1 to 1 would not make the sharing of
> > files on
> a
> > massive scale feasible. I can only speak for myself obviously, but
> > I've
> only
> > ever used file transfer in msn messenger for small files. If I wanted
> > to download say a movie, I would use something designed for that
> > purpose,
> even
> > if it was from someone I knew I would find a way around not using an
> > IM
> app.
> >
> > I know in band data transfers present a problem, but I think splitting
> > the data would make it more server friendly. Plus the clients can
> > still send
> and
> > receive messages etc in between parts (given higher priority).
> > Firewalls present a major problem, but if you can get a connection to
> > the server,
> then
> > the problem dissipates.
> >
> > -----Original Message-----
> > From: Richard Dobson [mailto:richard at dobson-i.net]
> > Sent: 06 June 2002 09:35
> > To: jdev at jabber.org
> > Subject: Re: [JDEV] File transfers
> >
> >
> > I think that allowing file transfers of very small files in-band would
> > be cool, but anything over 10k or so should be sent out of band by
> > some other means, allowing it in band at all is also a big problem
> > because of the massive potential for abuse, in ways like DOS attacks
> > against individual clients and the server itself, excessive use of
> > expensive bandwidth, also creates copyright issues if people transfer
> > copyrighted files via the
> server
> > because it then brings the server providors into the line of fire
> > because they facilitated the transfer, etc etc.
> >
> > Because of all of these problems I dont think its a good idea to
> > transfer files in-band at all.
> >
> > Just my 2p
> > Richard
> >
> > ----- Original Message -----
> > From: "Andy Beetz" <andy.beetz at clearswift.com>
> > To: <jdev at jabber.org>
> > Sent: Thursday, June 06, 2002 6:58 AM
> > Subject: RE: [JDEV] File transfers
> >
> >
> > > What about the nntp idea for very large posts? Where the file is
> > > split
> > into
> > > several parts, each part being only small in size could be
> > > transmitted in-band just one at a time. As long as they carry header
> > > information the client at the other end should be able to decode and
> > > re-assemble. It
> > should
> > > be possible to request parts if they're missing.
> > >
> > >
> > > -----Original Message-----
> > > From: Michael F Lin [mailto:MFLIN at us.ibm.com]
> > > Sent: 05 June 2002 19:23
> > > To: jdev at jabber.org
> > > Subject: Re: [JDEV] File transfers
> > >
> > >
> > >
> > > When we generalize the Jabber network to thousands of servers, it
> > > becomes something of a nightmare to transport stuff out of band.
> > > This is of course why HTTP is not too good for this purpose - too
> > > many people are behind firewalls. Any direct client-to-client
> > > connection with whatever protocol will of course have the same
> > > problem. Relying on e-mail routing is one option, but how do you
> > > negotiate what address to send an e-mail to? How do you receive it?
> > > What if you need a file but don't have access to your e-mail?
> > >
> > > There are any number of solutions you can set up with WebDAV and so
> > > forth, but what we would really, really like - particularly when it
> > > comes to
> > Jabber
> > > as a transport for web services - is a way to transport large
> > > payloads if not directly in-band, then in a band that fully adopts
> > > JID routing.
> > Jeremie
> > > has proposed PASS, which is a step forwards but not totally
> > > satisfactory.
> > >
> > > The only "good solutions" I've been able to think of basically
> > > involve running a Jabber server that knows how to route s2s on every
> > > client
> > machine.
> > > Which is, not coincidentally, something I'm working towards.
> > >
> > > -Mike
> > >
> > >
> > >
> > > |---------+---------------------------->
> > > | | Mike Oliver |
> > > | | <ollie at appsaspeer|
> > > | | s.com> |
> > > | | Sent by: |
> > > | | jdev-admin at jabber|
> > > | | .org |
> > > | | |
> > > | | |
> > > | | 06/05/2002 12:21 |
> > > | | PM |
> > > | | Please respond to|
> > > | | jdev |
> > > | | |
> > > |---------+---------------------------->
> > >
> > >
> > >---------------------------------------------------------------------
> > >--
> > >----
> > > ---------------------------------------------------|
> > > |
> > > |
> > > | To: jdev at jabber.org
> > > |
> > > | cc:
> > > |
> > > | Subject: Re: [JDEV] File transfers
> > > |
> > > |
> > > |
> > > |
> > > |
> > >
> > >
> > >---------------------------------------------------------------------
> > >--
> > >----
> > > ---------------------------------------------------|
> > >
> > >
> > >
> > > Why have just one protocol?
> > >
> > > SMTP does pretty well at file transfers that are asynch. The Jabber
> > > protocol can include a header for the attachments and the user at
> > > the
> > other
> > >
> > > end can decide if they want to download the file. The a request can
> > > then
> > be
> > > sent to the originating peer and an SMTP transfer begun and the
> > > remote client can notify the user when the transaction is complete
> > > by asking
> > where
> > >
> > > to put the file. There are SMTP libraries in almost every language
> > > you
> > can
> > >
> > > name, so this doesn't appear to be a big problem.
> > >
> > > FTP is another and offers the ability to transfer files without the
> > > base64 encoding.
> > >
> > > Ollie
> > >
> > > At 11:45 AM 6/5/2002 -0400, you wrote:
> > >
> > > >In-band transport of large payloads is something we and others have
> > > >been looking at pretty intensely. Obviously it would be a nice
> > > >thing to have, but it is also very, very difficult to do properly.
> > > >If you just stick base64 in an X element, you have huge problems
> > > >because if that takes 10 minutes to transmit, you can't send
> > > >anything else for those 10 minutes.
> > > You
> > > >could chunk them, but that hardly makes things simpler for the
> > > >client software. This also makes it massively more difficult to
> > > >distinguish legitimate traffic from a denial of service attack.
> > > >Furthermore, it means the server has to do a whole lot more XML
> > > >processing (which may already be a bottleneck), because all XML
> > > >content has to be at least checked for well-formedness. To speak
> > > >nothing of the bandwidth implications.
> > > >
> > > >Ultimately, I don't believe there is a satisfactory way to
> > > >transport large payloads in-band while keeping things simple for
> > > >the client. The solution to this problem will involve a more
> > > >complex system on the client endpoints
> > > >- though not necessarily in typical client software.
> > > >
> > > >-Mike
> > > >
> > > >
> > > >
> > > >|---------+---------------------------->
> > > >| | Andy Beetz |
> > > >| | <andy.beetz at clear|
> > > >| | swift.com> |
> > > >| | Sent by: |
> > > >| | jdev-admin at jabber|
> > > >| | .org |
> > > >| | |
> > > >| | |
> > > >| | 06/05/2002 10:29 |
> > > >| | AM |
> > > >| | Please respond to|
> > > >| | jdev |
> > > >| | |
> > > >|---------+---------------------------->
> > > >
> > > > >
> > > --------------------------------------------------------------------
> > > --
> > > ----
> > --
> > > --------------------------------------------------|
> > >
> > > > |
> > > > |
> > > > | To: "'jdev at jabber.org'"
> > > > <jdev at jabber.org>
> > > > |
> > > > | cc:
> > > > |
> > > > | Subject: [JDEV] File
> > > > transfers
> > > > |
> > > > |
> > > > |
> > > > |
> > > > |
> > > >
> > > > >
> > > --------------------------------------------------------------------
> > > --
> > > ----
> > --
> > > --------------------------------------------------|
> > >
> > > >
> > > >
> > > >
> > > >I've set up jabberd and got a couple of clients connecting to it
> > > >(winjab). I tried a file transfer which worked no problem. What I
> > > >saw looking at the Winjab source is that the receiver downloads the
> > > >file from the sender on it's own socket based connection.
> > > >
> > > >I'm just thinking that there should be a better way to do this and
> > > >inside the message. I'm not saying my idea is the best or anything,
> > > >but I do
> > > think
> > > >that it would present the client authors with less headaches.
> > > >Anyway, my idea is that a message element can have a child, let's
> > > >say attachment or even an x, which will contain the contents of the
> > > >file. XML can handle
> > > this
> > > >if the file is base64 encoded, as it ends up as plain text.
> > > >
> > > >Just some thoughts
> > > >Andy Beetz
> > > >
> > > >
> > > >
> > > --------------------------------------------------------------------
> > > --
> > > ----
> > --
> > > -----------------------------------
> > >
> > > >
> > > >Clearswift monitors, controls and protects all its messaging
> > > >traffic in compliance with its corporate email policy using
> > > >Clearswift products. Find out more about Clearswift, its solutions
> > > >and services at www.clearswift.com.
> > > >
> > >
> >
>
****************************************************************************
> > > *******
> > >
> > > >
> > > >This communication is confidential and may contain privileged
> > > >information intended solely for the named addressee(s). It may not
> > > >be used or disclosed except for the purpose for which it has been
> > > >sent. If you are not the intended recipient, you must not copy,
> > > >distribute or take any action in reliance on it. Unless expressly
> > > >stated, opinions in this message are those of the individual sender
> > > >and not of Clearswift. If you have received this communication in
> > > >error, please notify Clearswift by emailing support at clearswift.com
> > > >quoting the sender and delete the message and any attached
> > > >documents. Clearswift accepts no liability or responsibility for
> > > >any onward transmission or use of emails and attachments having
> > > >left the Clearswift domain.
> > > >
> > > >This footnote confirms that this email message has been swept by
> > > >MIMEsweeper for Content Security threats, including computer
> > > >viruses.
> > > >
> > > >_______________________________________________
> > > >jdev mailing list
> > > >jdev at jabber.org
> > > >http://mailman.jabber.org/listinfo/jdev
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >_______________________________________________
> > > >jdev mailing list
> > > >jdev at jabber.org
> > > >http://mailman.jabber.org/listinfo/jdev
> > >
> > > Michael Oliver
> > > Chief Technology Officer
> > > AppsAsPeers.com
> > > 7391 S. Bullrider Ave.
> > > Tucson, AZ 85747
> > > 520.574.1150
> > >
> > > _______________________________________________
> > > jdev mailing list
> > > jdev at jabber.org
> > > http://mailman.jabber.org/listinfo/jdev
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > jdev mailing list
> > > jdev at jabber.org
> > > http://mailman.jabber.org/listinfo/jdev
> > >
> > >
> > > --------------------------------------------------------------------
> > > --
> > > ----
> > -------------------------------------
> > > Clearswift monitors, controls and protects all its messaging traffic
> > > in compliance with its corporate email policy using Clearswift
> > > products. Find out more about Clearswift, its solutions and services
> > > at www.clearswift.com.
> > >
> >
>
****************************************************************************
> > *******
> > > This communication is confidential and may contain privileged
> > > information intended solely for the named addressee(s). It may not
> > > be used or disclosed except for the purpose for which it has been
> > > sent. If you are not the intended recipient, you must not copy,
> > > distribute or take any action in reliance on it. Unless expressly
> > > stated, opinions in this message are those of the individual sender
> > > and not of Clearswift. If you have received this communication in
> > > error, please notify Clearswift by emailing support at clearswift.com
> > > quoting the sender and delete the message and any attached
> > > documents. Clearswift accepts no liability or responsibility for any
> > > onward transmission or use
> > of
> > > emails and attachments having left the Clearswift domain.
> > >
> > > This footnote confirms that this email message has been swept by
> > > MIMEsweeper for Content Security threats, including computer
> > > viruses.
> > >
> > > _______________________________________________
> > > jdev mailing list
> > > jdev at jabber.org
> > > http://mailman.jabber.org/listinfo/jdev
> > >
> >
> >
> > _______________________________________________
> > jdev mailing list
> > jdev at jabber.org
> > http://mailman.jabber.org/listinfo/jdev
> > _______________________________________________
> > jdev mailing list
> > jdev at jabber.org
> > http://mailman.jabber.org/listinfo/jdev
> >
>
>
> _______________________________________________
> jdev mailing list
> jdev at jabber.org
> http://mailman.jabber.org/listinfo/jdev
> _______________________________________________
> jdev mailing list
> jdev at jabber.org
> http://mailman.jabber.org/listinfo/jdev
>
More information about the JDev
mailing list