[jdev] The future of Jabber/XMPP?

[Waqas Hussain]

Late response. I have acquired this bad habit of forgetting to hit
send on drafts. This message had been sitting around for 2+ weeks...

On Sat, Aug 28, 2010 at 7:34 AM, Evgeniy Khramtsov <xramtsov at gmail.com> wrote:
> 28.08.2010 02:36, Waqas Hussain wrote:
>>
>> 2. A table of local accounts mapped to list of subscriptions. Each
>> subscription is just two pointers (one to a shared caps hash table,
>> and one to a JID). 1M * 64 bytes = 64MB (just roughly picked 64
>> bytes).
>>
>
> Let me clarify if you don't mind ;) Do you mean 'recipients' table here? If
> yes, then I have some questions:
>
> 1) is it shared/mutable/global table?

It's local to the module. That's to say there's one per-host, and yes
it's mutable.

> 2) I see only one deletion there, in line 128 of mod_pep.lua (I'm looking at
> 0.7.0 release). Does that mean recipients[user] is deleted somewhere else
> when the corresponding user's session terminates?

While subscriptions are deleted, the user's subscription table itself
isn't deleted. There wasn't much point in deleting it as the overhead
was very low (typically less than 128 bytes per subscription table). I
might change that.

> 3) Do you limit the number of recipients you cached for local user?

I'll discuss this below.

>
>>  3. The caps hash table, mapping caps hashes to a list of +notify
>>  namespaces.
>
> I guess you are talking about hash_map table. Do you care about the size of
> this table? I don't see any checks in the code :)
>

No, I don't. This table has an upper bound: It's never greater than
the maximum number of subscriptions.

>
> --
> Regards,
> Evgeniy Khramtsov, ProcessOne.
> xmpp:xram at jabber.ru.

Alright, let's discuss the subscription list and limits.

The main issue you are concerned about is that the list of
subscriptions is unbounded, and is dependent on a remote server. We
are not in disagreement over whether the problem exists. I disagree
with your solution.

Now let's discuss what's wrong with your proposed solution: spam.

In my experience, most users on a given server tend to be offline.
Most contacts in a given roster tend to be offline. A significant
portion of online contacts are not interested in PEP. I went ahead and
gathered some statistics on thiessen.im/org. During the time of the
test, the number of online resources of remote contacts was typically
near 20%, and always below 40% of the total number of contacts of all
online users. This matches what I see in my roster, and what others
have told me about their's.

60-80% of the stanzas your solution would be sending would be useless.
Also, when a new user comes online, they don't get older PEP data, do
they? Or do you resend it each time a contact sends a presence?

A quick and simple fix in my implementation is limiting the number of
subscribed resources per bare JID. This puts an upper bound on the
size of all tables, without harming a typical user's experience.

The 'perfect' solution would be smart (PEP-aware) remote stanza
fan-out. This is something which interests me a lot, and would
significantly cut down on XMPP traffic.

--
Waqas Hussain