[jdev] Properly filtering broadcast events

[Peter Saint-Andre]

IMHO this is similar to what pubsub.com used to do. They built their
application on top of XEP-0060 but their filtering semantics were not
built into the protocol.

On 11/26/08 11:52 AM, Bryan Morgan wrote:
> I'm new to XMPP but am very excited about the possibilities.  We are
> designing an application based on XMPP and want to be sure we are
> leveraging the appropriate technologies and standards in the correct
> places, in order to ensure scalability as well as future-proof the app,
> as much as is possible.  My question, at its most basic is what is the
> recommended approach for handling the following scenario:
> 
> (1) Users starts session
> (2) Users submits message (via HTTP or XMTP - doesn't really matter)
> (3) Server picks up message (via asynchronous MQ or via XMPP component -
> doesn't really matter....I think)
> (4) Server needs to flexibly broadcast message out to a large number of
> users but (and this is key) this list of users needs to be dynamically
> filtered
> 
> I originally was hoping PubSub would work but that seems to rely on a
> fixed node with a fixed set of subscribers.  In my scenario, I may have
> fixed nodes but I don't want to push a message out to all subscribers -
> just a dynamically chosen subset of those (based on some filter
> parameters).  Some options I've considered include:
> 
> * Pushing out the messages to all members of the node and letting a
> custom XMPP client filter them.  This makes the server simple but
> results in possibly many messages being sent to many "receiving" clients
> when a subset would have worked.  Considering there may also be a large
> volume of user-submitted messages coming in to trigger this, minimizing
> the number of outbound messages is critical from a scalability standpoint.
> 
> * Not sure if this is possible but in the routing chain could we build a
> component that would intercept the pubsub messages and apply the
> filtering there?  Not sure if it's possible to do this generally for
> "all broadcast recipients" or if the code would have to be called once
> for every message recipient which would then again kill
> performance/scalability as the load grows.
> 
> * The inbound item picked up off the queue could also be used to
> determine the list of valid recipients, and then.....I assume....loop
> through them one at a time and send them individiual messages??  Again,
> seems to work at a small scale but maybe not at a large scale?  Is there
> a way to build a dynamic list of users and have XMPP send messages to
> that list?  Are there practical limitations on handling the size of this
> list or are there "sharding" approaches for big XMPP broadcasts like this?
> 
> Any advice or lessons learned by XMPP veterans would be greatly
> appreciated.  I apologize in advance for the lengthy message.


-- 
Peter Saint-Andre
https://stpeter.im/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6751 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://www.jabber.org/jdev/attachments/20090303/3a391ef2/attachment-0003.bin>