[jdev] Re: [jadmin] Re: One million concurrent user

[Richard Dobson]

>> Yup although for a truely scalable solution you wouldnt want to rely on a 
>> single sql server, you would want the servers to replicate the routing 
>> data themselves and store that data in memory for maximum speed and to 
>> help keep the load of your sql servers.
>
> So you are saying have a server that routes individual IM connections to 
> other jabber servers and store the route info in the sql db. Then, how to 
> make the sql servers redundant? I don't have tons of experience in this 
> area, but am willing to work towards it and/or provide code to solve 
> scalability issues if we can lead towards some needed improvements.

Nope thats not what I meant, what I meant is that you wouldnt want data such 
as routes stored into an sql db (otherwise it will load it far too much, and 
makes it a massive single point of failure), the jabber servers themselves 
need to store and replicate this data themselves, all you should need the db 
for is storing stuff like authentication/account details, offline messages, 
roster etc.

> That is one of the options I am considering is to have a main jabber login 
> server that then keeps track of the load of the other jabber servers and 
> then routes connections to the other jabber servers and records which 
> current jabber server a user is on in mysql.
>
> Thus, could have jabber01.blah.com, jabber02.blah.com and just record that 
> in the mysql db as the current server route...but wait, that would mess up 
> the rosters...ugh...still thinking about solutions.

Yea that would be a bit of a mess and rather unprofessional looking, 
ejabberd sounds like it might be a possibility as that is the only 
opensource server ive heard of so far that has really thought about real 
server clustering, jabberd2 really will not work for you as it needs to be 
able to cluster is jsm module to really be able to handle the kind of load 
you want it to, last I heard the tested limit for a jabberd2 server was 
between 10,000 to 20,000 connections (not sure if its been fixed yet but 
there were also run away memory leaks past a certain number of users), which 
is hardly your required figure of a million.

> As I said before, this company I'm doing this work for is in support of me 
> adding code to open source solutions to contribute back to the community, 
> so I hope that that can be the best solution.

If you want to work on it then do, but ive got a feeling it will be a lot of 
work for you, and if you do have tight deadlines on getting this operational 
then I do think a commercial solution from people who have experience with 
the sort of system you want to build will be your only real option, but if 
you have quite a few months then yea you might be ok with developing an 
opensource solution.

Richard