[JDEV] Ramblings - feel free to join in :-)

[David Waite]

----- Original Message -----
From: "Dennis Noordsij" <dennis.noordsij at wiral.com>
To: <jdev at jabber.org>
Sent: Thursday, December 14, 2000 5:28 AM
Subject: [JDEV] Ramblings - feel free to join in :-)


> Hi,
>
> I have had 2 things on my mind for a while and would like to take the
> opportunity to hear from other people what they think would work and
wouldn't
> work, or maybe come up with a better idea or implementation.
>
> The first one concerns bandwidth vs horsepower. I think we can pretty
safely
> assume that :
> - in our own jabber server farm bandwidth is plentily available, and the
only
> thing we are worried about is raw power of our servers. Any optimizations
> would be ones that will get more messages routed in the same time, even if
> that takes up a little more bandwidth inbetween jabber components (think
of
> the main JSM to transports to xdb databases - all on a small LAN).

Sure - you can even put these machines on a separate network to increase
performance - this is one-time hardware costs.

>
> - with regard to the "outside", ie users connecting via TCP/IP over the
> internet we value bandwidth much more. It is alright if the client has to
do
> a little more work if it means it takes less bandwidth to get a message
> across.
>
> How to do this, without affecting jabber server code at all, and clients
> minorly?
>
> Why not bzip2 the xml stream? The client would simply stream through a
bzip2
> function before sending it out over the socket, this would be quite easily
> implemented in clients. On the server side, since any serious setup will
use
> jpolld multiplexing machines only jpolld has to know about bzip2, when the
> XML reaches the jabber server it is plain text XML again. Likewise, why
not
> stream through an SSL component (with compression), once again, on the
client
> side it would make no difference, on the server side the jpolld's could be
> linked again an SSL library making use that hardware SSL acceleration
board I
> see in every issue of LinuxJournal :-)

The first issue that may/probably will come up with this is that the packets
would have to be considered independant documents - otherwise you will have
a hold on a sent packet until enough additional data is sent to complete a
few more bytes of data.

The second issue is the same as with SSL - increased power needed by the
server. In the case of bzip2, a decompressing stream needs at least 1 MB of
memory (usually when you run bzip2 it will use about 9 MB for working
space). This would also be a requirement for working space while processing
a document - it wouldn't just be a 1 MB requirement per working thread. So
either you give a bit over a MB per working user (ouch), or restrict worker
threads from changing tasks until a packet is complete, and recognizing that
as a document (a major opening for DoS attacks).

Compared to both of these, a binary-representation of the protocol would be
a godsend :)

<snipped bit about server farming>

-David Waite