[jdev] RE: [jadmin] Presence packets bottleneck on huge rosters (was: Forcing roster reload on every client!!! Please help)

[Bresler, Jonathan]

Hello,

Large roster lists on active servers with a number of users are a problem.
All the more so if the connections are encrypted.  The current jabberd14 code
(reading, still have jabberd2 and ejabberd to read) sends out <presence> 
Stanzas immediately.  This can be intense.

One easy to implement option, that might not twist the RFCs to hard, is to schedule
The <presence> stanzas over some short time period using the heartbeats that are
Available in jabberd14 (and presumeably jabberd2 and ejabberd).  

Speaking of the heartbeat code in jabber14.  This code forces a traversal every second of a doubly linked list of all the heartbeats that have been registered. For each heartbeat, the code has to increment a counter.  OUCH!

Two easy speedups:

Use the callout methods of UNIX V7, store the heartbeat in expiration order, use a countdown field to indicate number of seconds till the heartbeat fires (requires examining the first heartbeat only), reinit the countdown field from the freq each time and reinsert into the proper place in the list (requires partial list traversal).

Use the callout timing wheel by Adam M. Costello and George Varghese, published in a technical report entitled Redesigning the BSD Callout and Timer Facilities and modified slightly for inclusion in FreeBSD by Justin T. Gibbs to yield significantly better results when the list grows in size.

Thoughts?

Jonathan


-----Original Message-----
From: jadmin-bounces at jabber.org [mailto:jadmin-bounces at jabber.org] On Behalf Of Mickael Remond
Sent: Wednesday, December 08, 2004 4:45 AM
To: Jabber software development list
Cc: Jabber server administration list
Subject: [jadmin] Presence packets bottleneck on huge rosters (was: Forcing roster reload on every client!!! Please help)


Miguel Ferreira wrote:
> Dear all, 
>  
> I've developed a small script called buddymaker to run on the same server as
> my jabberd 1.4.The idea is to check the jabberd spool/hostname directory for
> changes (added or removed users) and then update the roster entries of every
> user making everyone's presence available to everybody else.

Hello,

This kind of feature implies that every roster will be really huge. 
Suppose you have 1000 users on a server, for each connexion, 1000 
presence packets will be send. And to that the probe answer by online 
contact and you have potentially 1500 presence packets send at a time 
for each new connexion.
When several people connect at the send time this generate a important 
load on the server and can be a bottleneck in huge server in production.

I am thinking on a way to limit the impact of presence packet for huge 
roster. I was wondering if other people were already thinking on this 
problem.

-- 
Mickaël Rémond
  http://www.erlang-projects.org/
_______________________________________________
jadmin mailing list
jadmin at jabber.org
http://mail.jabber.org/mailman/listinfo/jadmin
FAQ: http://www.jabber.org/wiki/index.php/FAQ-JADMIN

---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.804 / Virus Database: 546 - Release Date: 11/30/2004
 

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.804 / Virus Database: 546 - Release Date: 11/30/2004