[JDEV] performance numbers.. questions

[Oliver Jones]

Hey, Dustin, congratulations on actually getting real-live 
numbers!  Getting a test harness working is a tremendous 
accomplishment!  Thanks!

At 02:52 PM 5/9/01 -0500, you wrote:
>...here is how the test works:
>active_jab basically connects 1..n users serially, and sends x messages
>when each user connects. It does NOT send messages concurrently for each
>connected user.

If I understand you, the sequence of events is this, for each user:

a. connect
b. log in.
c. send x messages as fast as possible.
d. stay connected, but stop sending messages.

When you run the test, you run these steps for each virtual user in 
turn.  Right?

Some questions about your test sequence:
--Does the log-in step also create an account for the virtual user?
--What kind of authentication are you using?
--To what user do the x messages get sent?
--Is the recipient online or are the messages getting stashed in jspool?
--Does the volume of the x messages sent cross the karma threshold?

You proposed this scenario:

>for user 1..n do
>   connect user to jabber
>   send x messages and track delivery times
>done
>
>for user 1..n do
>   disconnect user from jabber
>done

I'd like to see tests that figure out the load that comes on when many 
users connect and authenticate at once.  The question to ask might be,

  -- when 500 users are already connected to jabber but not doing much,
     how many new user connections can jabber accept each second?

A subsidiary question might be how the choice of authentication method 
affects this, and how the choice of xdb_file or other jspool module affects it.

Another subsidiary question might be how many NEW user accounts can be 
created in a second.

I'd like to see tests that figure out the load that comes on when many 
connected users send lots of messages.  The question to ask might be,

   -- when 1000 users are already connected, how many messages a second
       can each user send to some other user -- what's the message throughput?

>Memory
>=======
>First of all, I assumed that Jabber would consume a lot of memory, which
>turned out to be a wrong assumption.

This agrees with my observation.  Our jabber setup typically runs with 
several hundred users connected -- and neither jabberd nor the half-dozen 
jpollds we operate take much memory at all.

(Check out our jabber-enabled music community at http://dotclick.com.)

>CPU=======
>Big surprise here. Jabber seems to *quickly* begin eating CPU cycles as
>the number of connected users increases. Here is some data:

Our jabberd and jpollds take very little CPU time, but our users stay 
connected for long periods and typically don't send huge numbers of 
messages -- dozens a day.

Your high CPU usage may be due to overhead on new connections, or overhead 
on karma-throttling virtual users who are sending big bursts of data.

Great progress.

Ollie Jones