[JDEV] Jabbers disappointment !!

[Dave Smith]

Hi Vijay..

Just to address some of your problems..

1.) Memory usage - the 1.2 server no longer uses a thread per user, opting 
instead for a thread pool approach. What this means is that at any given time, 
for any # of users there is a fixed # of threads running (the default is 
~20 IIRC). Having a thread per user is not an efficient approach for high-end 
scalability -- the O/S would spend more time context switching than it would 
letting a thread run. The implications of this approach means that in 1GB of 
memory, the server can handle ~120k users (or so).

2.) Using multiple processors - Jabber.com is currently in the process of 
refitting the server to support pthreads so that the jabber server can make 
use of multiple processors. This is expected to be complete in late January or early Feburary. Note that while j.com is doing the work, the modified code will
be merged back into j.org's CVS and made available under standard OpenSource
licensing.

3.) TCP connections - You can modify some setting in Linux's /proc filesystem
to make it support more than 1024 connections. Jabber.com has managed to get 
24k simultaneous connections being managed on single Linux box at this time. 
It's possible to do more (prolly up to ~40k) but it becomes very inefficient,
due to the fact that the Linux kernel is not optimized for such large amounts
of I/O and latency is introduced into the entire system. In fact, I would
recommend that a Sun box is used to act as the connection manager as it has
noticeably better performance when it comes to handling huge amounts of I/O.

4.) Distributed approach - In order to attain the numbers I have been talking
about, you definately need a farm of machines. For instance, to handle ~30k 
users you will want to have 2 boxes (or so). One box will run jabberd (the
component hub) which will load JSM (Jabber Session Manager) -- this box tracks
individual user session and the logic associated with knowing when a user is
on/offline. The second box will run a series of jpollds, with multiple virtual
IPs mapped to each. The #2 box will be handling all those thousands of conns
and multiplexing them back to the JSM for processing. Now that only gets you up
to ~30k users or so with a fairly low sustained message throughput 
(~500 msgs/sec). Once a pthread version of the server is out, we expect the 
sustained msg throughput to double or even triple -- note that the # of users
is no longer directly related to message throughput..you _could_ stack 100k
users on a single JSM so long as there where < 500 msgs/sec the server would
be able to keep up with no major problems.

I hope this info helps. 

D.

On Tue, Nov 28, 2000 at 06:45:51PM +0530, Vijay Kumar wrote:
>  Hi!!  all
>         for last couple of days I was figuring out whats the upper limit of
> no of users jabber can support
> concurrently. what I can figure out is that jabber as a server can be used
> for small no of clients only .
> 
> 1) memorey diappointment .
>     With pth, the individual user threads get 32k (correct me pease if I'm
> wrong) so 10,000 users would use up 320MB.
>     if we are having 1GB of RAM then atmost 30,000 user will be supported
> concurrently.
>     beside this pth is not kernel based so if dual propressor is used its of
> no use to jabber .
> 2)TCP connection :
> It should also be noted that by default, Linux will only accept 1024 TCP
> connections. Adding more simo connections requires a kernel patch to
> increase
> this limit. to what limit we can increase this ?? If I am not wrong then one
> connection will be dedicated to one user
> ,I meant each tread will be revoling around one connection . will it means
> maximum atmost 1024 will be online at a time .
> but its also known that :
> sockets absolute limitation is 64k since the port value is 2 bytes wide. So,
> with
> TCP/IP you cannot have more then 64k connections (this is a theoretical
> maximum, in reality more limitations apply) even if you have a
> server-machine that can handle the load
> 
> 3)Distributed approach.
>      well 1st & 2nd point is true for any server which is based on Pth & TCP
>  .
>      but agin Lot is taked about distributed approch in jabber .but how to
> implement it practically is no where written neither
> in jabber docs nor in mailling archives . suppose if I am running 3 jabber
> servers in 3 different machines then how to
> syncronize them or co-relate them .
> or how to install one master jabber server
>   with 3-4 salve servers distributed .
> 
> mine whole aim is to use jabber at very high end of user numbers.
> 
> thanks in advance !!
> 
> vijay
> 
> 
> _______________________________________________
> jdev mailing list
> jdev at jabber.org
> http://mailman.jabber.org/listinfo/jdev