[JDEV] Re: Large scale Jabber bots

[Todd Bradley]

> > [I've retitled this thread...]

Thank you!  And boy did I never expect this
thread to be of so much interest to people.

Jens wrote:
> > What, all mailservers?! What about the ones that run 
> > mailing lists or
> > [voluntary] announcements? I've been on music mailing lists 
> > with thousands
> > of subscribers, and I get very useful "what's new" mailings 
> > from Amazon that
> > must have tens of thousands of readers.

Well, I agree with the person that pointed out
this isn't a use you would normally expect of
CLIENT software.  You would want some special
piece of server-side email software to handle 
your mailing list.  Try using Outlook (or what-
ever email program you use) to send out 10,000
messages all at once and let me know how it
reacts!  ;-)

> > > Jabber should probably work
> > > the same way in this case (200 users max in a roster or something
> > > configurable like that)
> > >
> > >
> > Why on earth should that be a requirement? First off, it 
> > penalizes people for something that's not their fault. 

I disagree.  Except in unusual circumstances,
you only have people in your roster that you
know and want to talk to.  So, in most cases,
it IS the user's fault.  

I can think of two exceptions, though.  First,
I have no control over who sends me presence
subscription requests and how often they do so.
So, someone could write a malicious program that
generates 100,000 Jabber accounts and then sends
me a subscription request for each one.  That's
a particularly evil attack you could make on 
a Jabber server, in fact.  You could probably
bring down that server, then, by sending alter-
nating on/off presence packets from just one of 
the users in that big roster.  Each change in
presence forces the server to read and parse that
whole roster.  Or an alternate attack could be
that the program searches through the JUD and
just sends a single subscription request to each
person in the JUD.  Then, it would alternate
presence rapidly, forcing the server to read
and parse ALL rosters on the server.

The second exception is that my Jabber client 
might automatically send a subscription request 
to somebody without my knowledge.

> > Secondly, it ignores very useful aspects of Jabber for 
> > information delivery.
> > Stock price agents, auction agents, news agents, etc. 

> I'm very familiar with bots. I also know that the majority of 
> bots do not
> require presence information at all, all users do is query 
> their services.

I agree.  Most news bots don't have any
need for bidirectional presence.

> I'm saying that you can't take protocol decisions made for an 
> average roster
> size of 10 and scale it to a roster size of a quarter 
> million. 

That definitely makes a lot of sense.  The
roster system in Jabber was designed with
the idea that a user would have just a hand-
ful of friends they're interested in.  I
would bet the major IM systems have some
kind of limits to roster size.  Anyone know
for sure?  I doubt AIM would let me have
10,000 users in my AIM roster.  But, no, I
am not going to try.

Thomas Charron wrote:
>     Yes.  The problem is, the typical c2s componenent takes 
> some things for
> granted, namely, one of them being, that it won't take a 
> tremendouse amount
> of work to do some things.  In the case of a jabber 'bot', 
> that bot should
> actually be written as a transport and *NOT* as a client 
> connecting to the
> c2s component.

As someone pointed out, the downside of
this is portability.  It is definitely
nice to be able to run your bot from any
machine, rather than just from the server,
which only a limited number of people have
access to.  On the other hand, the beauty
of Jabber is that you can always run your
own server!

Nonetheless, I agree that for a bot that
is expecting to handle many users on the
Jabber system as it's currently designed,
a transport (aka agent, aka service) is a
better way to go than a client.  That was
my original point in all of this.

Now a bot that's not expected to have a
lot of users, such as a doppleganger bot
(one that runs in the place of a live user
behind a traditional client), might be
best as a client.

And stpeter wrote:
> I have a roster on jabber.com (my stpeter at jabber.com account, which I 
> used for a few months last year). That roster has about 200 
> people in it 
> and is definitely not the largest roster on the system, so 
> I'd imagine 
> that Jabberbot has at least several thousand users in its roster.

Yes, as of last week there were a little bit 
over 10,000 JIDs in Jabberbot's roster.


Todd.