[JDEV] Stress testing and connection limits

[David Waite]

It would be my (barely educated) guess that a single user's session requires
about 50k of state, plus an 8k buffer for the TCP socket (which I wouldn't
think would be swappable). 100k users would take 800M of memory just
kernel-side for the TCP sockets, neglecting any of the user state (thread set
up, separate stack space per user, working data structures). This is based on
past linux-kernel discussions where I've heard 8k stated as the absolute
minimum usuable buffer per tcp socket (I don't know if the size was actually
lowered to this).

-David Waite

Chuck Bohling wrote:

> ulimit on my Linux server that jserver is running on reports 1024 open file
> handles. I've tried changing that number with ulimit but I'm not sure if the
> number of handles really changes. I don't think so. The implication from
> what you've said is that the max number of jabber users connected at one
> point is limited. In my case, 1024. Is that true? Isn't that as fairly
> serious restriction? I was expecting to have a powerful server with 10's
> maybe 100's of thousands of connected users. Can the kernel be rebuilt with
> a larger size? If I change the limit to 100,000 or 200,000, will Linux still
> run?
>
>  -----Original Message-----
> From:   Eric Bowersox [mailto:ebowersox at jabber.com]
> Sent:   Wednesday, June 14, 2000 10:34 AM
> To:     jdev at jabber.org
> Subject:        RE: [JDEV] Stress testing and connection limits
>
> > I've been stress testing Jabber to get a feel for how many
> > concurrent users
> > it can handle. I've hit some limit on the number of
> > connections that Jabber
> > will accept. Hopefully, it's a soft limit in Linux or Jabber,
> > or maybe a
> > mistake in my test.
> [...]
> > This works great up to about 1000 (1024?) clients
> > (threads). Above that number, I start seeing all kinds of
> > failures. I think
> > it may be caused by pth_accept rejecting the connection, but
> > I'm not sure.
> > Anyone have any ideas? I did disable the connection rate
> > check in tlisten.c.
> > Hopefully, it's something like a 1024 limit on the number of
> > connections
> > from the same IP.
>
> You've hit the nail right on the head.  It's not pth_accept() that's the
> problem, though, it's pth_select(), which relies on the select() system call
> in the OS.  Many (most?) implementations of select() won't handle more than
> 1024 file handles.
>
> The obvious solution is to use poll() instead, but the select() is buried
> down inside the pth library, which falls firmly into the realm of Code We
> Don't Dare Tamper With Because It's Impossible For Mere Mortals To
> Understand.
>
> We (the jserver developers, that is) are currently developing a solution
> that uses additional processes as "multiplexers" to aggregate large amounts
> of connections.  With this, it would be  theoretically possible to increase
> the number of server connections to as many network sockets as your system
> will handle (which is more like 64K).  This isn't ready yet, though; if
> someone knows different, please step in and correct me...
>
>                                         Eric
>
> --
> Eric J. Bowersox - Jabber Inc. - ebowersox at jabber.com
> <mailto:ebowersox at jabber.com>   http://www.jabber.com
> OpenProjects IRC #jabber: erbo  - Advogato: Erbo
> "AIM is to Jabber as Notepad is to emacs" - washort, #jabber
>
> _______________________________________________
> jdev mailing list
> jdev at jabber.org
> http://mailman.jabber.org/listinfo/jdev
>
> _______________________________________________
> jdev mailing list
> jdev at jabber.org
> http://mailman.jabber.org/listinfo/jdev