[jdev] Routing

[Pedro Melo]

Hi,

On Jun 23, 2008, at 12:42 PM, Jonathan Dickinson wrote:

> Can anyone give me some advice on this problem? From what I can  
> tell, currently routers are a congestion point for all components.  
> E.g.
>
> +--------+
> | ROUTER +-S2S
> +---+----+
>     |
>    C2S
>
> As you can see the performance of the whole server depends on the  
> router, and the router can't be scaled horizontally by clustering  
> it (as far as I can tell). Like as follows:
>
> +--------+   +--------+
> | ROUTER +---+ ROUTER +-C2S
> +--------+   +--------+
> |        |   |        |
> C2S     C2S  C2S      S2S
>
> Can anyone correct me on this?

I might be missing something obvious, but this is exactly what  
ejabberd does, it scales horizontally the router core.

The JSM is another matter altogether, though...

Keeping the "who's online and where" table is the tricky part. On a  
first approach you would have to keep it up-to-date on each node.

A way to bypass that limitation is a clever use of the resource part  
of each jid: just add a contanst-length "tag" to the resource,  
marking the server where that resource is located. This seems to be  
what Google is doing.


> I came up with the following solution to the problem:
>
>               C2S      S2S
>                |        |
>                +--------+
>                | ROUTER |
>                +----+---+
>               /     |    \
> C2S-+--------+ +----+---+ +--------+-C2S
>     | ROUTER +-+ FAILURE+-+ ROUTER |
> C2S-+--------+ +----+---+ +--------+-C2S
>               \     |    /
>                +----+---+
>                | ROUTER |
>                +--------+
>                |        |
>               S2S      S2S
>
> The router network works like a token ring network, only that there  
> is no token. Each router can send the packet along to the next as  
> it wishes (in one direction only). The following flow would apply:
>
> <!-- Router1 -->
> <route from="c2s1.50" to="joe at joe.com">
>   ...
> </route>
>
> <!-- Router2 -->
> <route from="r1" to="joe at joe.com">
>   <visit name="r1"/>
>   ...
> </route>
>
> <!-- Router3 -->
> <route from="r2" to="joe at joe.com">
>   <visit name="r1"/>
>   <visit name="r2"/>
>   ...
> </route>
>
> If at any point one of the routers handles the packet successfully  
> it simply stops passing the route node around. If the route node  
> visits the same router twice it:
>
> a. If the node is from 'this' domain send it to the failure node.
> b. If the node is not from 'this' domain:
>    1. If I have a S2S component attached send it to that.
>    2. If I don't have a S2S component attached send it on.

First, let me state that I don't totally understand what you are  
trying to achieve with this design, so I'm probably missing something.

But with this design, all stanzas need to cross (on average) N/2  
routers where N is the number of routers. More: with the rules above,  
it seems to me that outbound stanzas need a run-around the circle  
once before we decide the stanza is not ours. This of course is the  
only way to do it if each node does not have a notion of local  
domains, but again, if the problem is router congestion, I believe  
your design is even worse than the current ones.


> Components would send failures if no connected client is found:  
> thus making the router continue routing the packet.
>
> The sexy thing about this is that you should be able to remove and  
> connect routers to the network without bringing the whole thing down.

As you can today, with ejabberd for example. In fact one of the funny  
aspects of ejabberd is that you can add or remove nodes in runtime,  
but you cannot add or remove virtual domains in runtime. :)


> Great so far, this would work well with single-sign-in capable  
> systems (i.e. no resources/priorities like MSN). Now, I can't for  
> the life of me figure out a way to handle resources and priorities.  
> One way would be to (but it's ugly):
>
> Component gets route packet:
>
> <route from="r4" to="joe at joe.com">
>   <visit name="r3"/>
>   <visit name="r4"/>
>   <priority name="c2s4.1" resource="home" value="0"/>
>   <priority name="c2s3.50" resource="mobile" value="50"/>
>   ...
> </route>
>
> Thus each stanza would make a complete trip round the network (2 at  
> worst) before it could be sent. Does anyone have any clever ideas  
> how to get around this?

Get an Google Account. Log into Google Talk. Look at the resource you  
get back from the server. It ends with 8 char hex string. Its the  
server that is handling this connection. So each router can find the  
correct server just by looking at the last 8 chars of the destination  
jid.

Best regards,
-- 
HIId: Pedro Melo
SMTP: melo at co.sapo.pt
XMPP: pedro.melo at sapo.pt