[JDEV] Request for comments on system using jabber

[Thomas Charron]

> IMO, mailservers should break if you try to send to more than ~20 people
> at a time. Not crash, but refuse to send. Jabber should probably work
> the same way in this case (200 users max in a roster or something
> configurable like that)

    Well, I'm not so sure about that, but I suppose if it's configurable.

> Hypothetically, if you had a 10,000 user roster, that would generate
> about 5 MB of XML traffic through the server it was running on everytime
> the bot came online.  This depends a bit on the length of the average
> JID, but the same amount of information is sent whether the user is
> online or not. Even if all of those users are on the same machine, that
> would be 10,000 user rosters it would have to load up via XDB and parsed
> (since the roster is also basically the presence ACL). If the server
> wasn't intelligent, you would also have the converse - any time one of
> those 10,000 users changed their status to 'away' or went on or offline,
> it would have to load and parse a megabyte XML document.

    Now your talking about the improper use of a jabber client.  That bot
should be written as a transport, and NOT as a client.  The c2s system
simply wasn't written to handle that kind of situation..  8-)

> Now imagine this is a portal with a quarter of a million users, and the
> bot is added by default to everyone's roster. Not only would that roster
> be about 25MB, there would be at least a 35MB memory image for the DOM
> tree created. The restarting of that service would cause 150 MB of
> traffic to be routed through the server (25 MB going outside the local
> network), and a 35 MB roster structure would be resident on the server.
> Finally, your 250,000 user roster will be run through for every presence
> change on the server (since every user has the bot in their roster). The
> 0.25 million node XML structure in memory is a linked list, so this
> would equate to a worst-case 250,000 string compares as it walks a
> quarter-million node list. Finally, at least the open source server
> implementation will cache presence for local users while they are
> online. 250,000 - 500,000 cached entries, all XMLNodes, for a
> guesstimated worst-case size of about 250 MB server memory for the bot
> to just be online.

    See above.  Thats why that bot should actually be running off of a
transport that is written specifically to handle that kind fo traffic.  Most
likely storing the user data in some sort of in memory database, while
persisting some of the data it requires for operation to more permenent
storage.

> Moral of the story: if you try to solve every problem with a hammer and
> a crowbar, you just end up breaking a lot of things ;-)

    Yeppers.