[jdev] Re: Presence packets bottleneck on huge rosters

[Mickael Remond]

Peter Millard wrote:
> On Thu, 09 Dec 2004 18:09:37 +0100, Mickael Remond
> <mickael.remond at erlang-fr.org> wrote:
> 
>>On a real server (I have the figure of the Jabber.ru website), the
>>biggest roster has more than 900 contacts and 5 percents of users have
>>roster with 200 to 400 users.
>>So I think the load problem on connect with big roster is not only theoric.
> 
> 
> I'm not convinced.. jabber.org has 178000+ registered users (we purge
> inactive accounts, so this is a pretty decent number).
> 
> The largest roster has 1519 items. Only 100 users have 200+ items in
> their roster (this is 0.05%). The average number of roster items is
> currently 5.8. The standard deviation of # of roster items is 18.5. So
> the VAST majority of our users have VERY small rosters. Also note that
> folks with very large rosters are a lot more likely to have a smaller
> percentage of users online. So the total # of presence packets is
> going to be smaller.

Maybe the Jabber.ru site has more power users as Jabber.org is probably 
used by many user for testing.

> Also please checkout http://status.jabber.org. You can see that normal
> operations consume very little bandwidth (for a server) and we're
> serving 10000+ users during our peak operations.

I am not worried by bandwidth but much more by the load on generated on 
the server to route a big number of presence packets simultaneously. You 
are probably right that the behaviour of the server can optimize this to 
an acceptable level. For now the benchmark I did were quite impacted by 
the number of contact in users rosters.
I will investigate more on this and keep you informed of my findings.

> Honestly, I think we're worrying about a non-issue. As someone else
> brought up, a server could also use JEP-131 to allow remote servers to
> do the de-multiplexing of presence packets.

I did not see this aspect of JEP-0131, so I will look into it.

-- 
Mickaël Rémond
  http://www.erlang-projects.org/