[JDEV] The Important Things

[Iain Shigeoka]

On 1/8/02 7:31 PM, "Michael F Lin" <MFLIN at us.ibm.com> wrote:

> 
> Here is Mike Lin's list of Things The Jabber Open Source Community Needs To
> Do, In No Particular Order. Most of these are Really Hard Problems, and
> most of them are problems I don't have a solution to. Not everyone will
> agree with everything on this list. That is OK; we are a heterogeneous
> community.

:)  I like!  Many of these issues seem to require a pretty major update to
the standards... Perhaps a 2.0 version that is completely overhauled?  I'm
just not sure how the standards can be updated and if we should be worrying
about thing like backward compatibility...

> 1. Our transport layer needs a better framing scheme (length prefixing or
> byte sentinel). Requiring the Jabber router (half of the server) to parse
> all payload content is an easily avoidable performance and scalability
> hindrance. Look into BEEP.

I strongly agree. I've been playing around with schemes for separating
delivery (transport) from content.  There are many possibilities but none
seem to mesh well with the existing Jabber setup.  XML is very flexible but
it doesn't help the server at all in being efficient about what it is doing.

Even if we just had a packet byte count for each packet so that the server
could start parsing a packet, see its a <message>, find its delivery point,
then be able to do a simple blind copy of x bytes to the outgoing connection
would save enormous amounts of time...

Right now the transport and content can be mixed freely and in many flexible
ways making efficient handling of packets difficult (x packets being a great
example of "surprises" in the transport layer).  Your only sure bet is to
completely parse everything then analyze what you've got.

> 2. We need to fill in the rest of the sendmail-like infrastructure that
> will allow Jabber to scale across the Internet and into Intranets as one
> interoperating network. This means we need well-defined support for
> multi-hop routing, and we need well-defined, end-to-end store and forward.
> Especially, we need to figure out how presence should behave in a
> failure-prone environment, which is the only new problem we are solving
> here.

Agreed.  The direct routing of messages domain to domain is simple but will
probably cause major scaling problems when Jabber grows.  Major domains like
say a "yahoo.com" jabber server would essentially have to handle the entire
internet...

> 3. We need to figure out how to scalably transport large payloads in-band
> (or at least in a band that adopts JID routing).

I cringe at the thought but agree that it's going to be needed.

> 4. We need to figure out how to have Jabber endpoints that deal with a
> large amount of realtime ("instant") traffic from all over the cloud under
> one JID. Serializing everything across one TCP connection probably will not
> do.

Once again, I agree.  I've often thought that perhaps we should have an
FTP-like break in duties... Perhaps a "presence connection" that persists
and only concerns itself with presence and control (IQ).  Then zero or more
connections to handle messages.  The beauty being that the server can divert
the message connection to other machines...

For example you may connect to central file servers, to p2p routers, to
other clients, etc etc.  In addition you may build up connections to
groupchat servers then drop them as you leave groupchat.  In addition, you
can also then consider "riding" other protocols for the message connection
as needed (i.e. Hit a regular ftp server for some file transfers with all
the benefits of restartable FTP, parallel downloads, etc).

> 5. Everyone thinks that web services (SOAP, UDDI) combined with presence
> (Jabber) is a big and important thing. We have to figure out exactly why,
> and how to build them on top of Jabber. Especially, we need to integrate
> Jabber with existing and upcoming web service tooling (Tomcat, .NET,
> DotGNU).

With all the extra infrastructure this implies.  For starters security and
transactions/quality of service.

> 6. We need to figure out security and authentication. SSL is okay, but
> transport-level encryption is less important than network-wide
> authentication and trust based on interoperable PKI, whether we adopt a CA
> model (like X.509) or a web of trust model (like PGP), or some combination
> of the two (which I consder most likely). We are looking for much more than
> shelling to GPG here. Look into the W3C XML Signature and XKMS work.

Yes.  Security.  :)

-iain


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com