[JDEV] Need help with protocol translation

Sun Mar 2 20:32:45 CST 2003

> I need to translate between XMPP and a private XML based protocol. I think my 
> problem is similar to that of creating a transport between Jabber and AOL. 
> I've never done anything like this before and am looking for general tips and 
> ideas about how to approach this problem. I'm mostly interested in how to 
> create a mapping between the two protocols, but I'm also interested in 
> implementation pointers. Are there general guidelines or patterns that people 
> have discovered? Any books that are recomended?

Basically, you need to look at the functions that are provided on both
sides, and develop a mapping between them. Then, you write a program
that can hook into a Jabber server via its component interface, and also
act as a client (or number of clients) for the foreign service.

I offer here a small tutorial about how I wrote a transport for an
instant messaging system called "Goofey", that is used internally in my
organsation.

Address mapping
---------------

To link two networks, its necessary to make nodes on network A appear on
network B in some way, and vice-versa. Making Goofey nodes (users)
appear on the Jabber network was quite straightforward - there is a
single central server for the service, and each username is just a
simple text string, with no domain or realm. So, the mapping became:

  JID: goofey-user at transport  <->  Goofey: goofey-user

That is, the Goofey user called 'foo' appears on the Jabber network as
'foo at transport'. In this way, every Goofey user has an identifier on the
Jabber network.

Getting Jabber nodes (users) to appear on the Goofey network is not
quite as easy. Since we can't just create an brand new address namespace
on the network like we did above, we have to use existing Goofey
identifiers.

To do this, you get Goofey users to register their username and password
with the transport (done via the 'jabber:iq:register mechanism, see
http://www.jabber.org/protocol/registration.html). When this happens,
the transport connects to the foreign network as a client on your
behalf.

By this point, we have the entire Goofey network mapped onto the Jabber
network, and we have a single user JID mapped into the Goofey network,
which is enough for a single user to send/receive message to/from the
Goofey network. Obviously, additional "clients" can be started, one for
each Jabber user that registers with the transport.

Sending messages
----------------

To send a message to a Goofey user from Jabber, the user simply sends a
message to recip-user at transport.

The transport first searches through its list of registered users to
find the Goofey username for the sending JID. If it doesn't find one, it
bounces the message with a 407 (Registration required) error.

Once it finds the Jabber user's Goofey username, it uses it to identify
the Goofey client connection that the message will be sent out on.

Next, the transport creates a Goofey message from the contents of the
<body/> element of the message (Goofey has no concept of "subject", so
that gets left out).

The "to" address for the Goofey message gets taken from the recipient
JID (recip-user), and attached to the message. The whole message then
gets pushed out (via the Goofey client connection) to the Goofey
network.

Receiving messages
------------------

Receiving messages is basically the same as sending a message. The
transport receives a message from one of its client connections. It
then looks up the corresponding user JID for this client. It should
always find one, because the client connection that the message arrived
on should not exist if the corresponding user did not register with the
transport.

The transport creates a new <message/> packet, fills in the body from
the received message, sets the "to" address to be the user JID found
previously, sets the "from" to be sending-user at transport (where
"sending-user" is the Goofey username of the sender), and injects the
message into the Jabber network.

Mapping presence
----------------

If the foreign network has similar presence semantics as Jabber, then
the mapping is almost exactly the same as message mapping.

Goofey, however, has different (and simpler) presence mechanism, known
as "watches", which has made the transport implementation slightly more
complicated.

Basically, every user has a "watch list". When a user connects or
disconnects from Goofey, every user who has that user on their watch
list is informed of the fact. No authorisation is required to add a user
to their watch list, which makes it exceptionally easy to map Jabber
subscription functions - when a Jabber user requests a subscription to a
Goofey user, the transport simply responds with "subscribed", and issues
an "add user to watch" command to the Goofey server (and vice-versa for
unsubscribes).

Goofey has the concept of "quiet" states, but users are not informed
when a user goes quiet. However, they can find this information about by
polling.

The polling mechanism allows a user to see the current state of all
watched users with a single command, eg:

  hawkeye OUT (Mar  3 11:21)  0u
      box *IN (Mar  3 09:46)  0u I0:09 [Q:eating lunch]
      gub  IN (Mar  3 09:40)  0u I????
      rmi  IN (Mar  3 08:57)  0u I????
    stick OUT (Mar  1 04:39)  2u
    lanks OUT (Mar  1 04:26)  0u
  faramir OUT (Feb 28 06:47)  0u
   portal OUT (Dec 19 09:32)  0u

So, the transport maps presence in the following way:

 - When a watched user connects, the transport is informed, and sends
   <presence/> to the Jabber user.

 - When a watched user disconnects, the transport is informed, and sends
   <presence type='unavailable'/> to the Jabber user.

 - The server polls every five minutes (for each user), and parses the
   returned information. If any states have changed since last time, new
   presence packets are issued, with the <status/> and <show/> set
   appropriately.

(Note: I have plans to make this more intelligent in the future by
keeping a global (rather than per-user) cache of all known Goofey users
and their known states. Information received from every user currently
online will be used to feed this database, which should allow for
updates much closer to realtime).

Other functions
---------------

One function that is used widely within Goofey is the ability to send
messages to multiple users. Some Jabber clients currently provide this
service, but they do it by sending a single messages multiple times,
which means that recipients are unable to see who else received the
message, and thus can't "reply to all". Goofey allows recipients to see
all other recipients.

The "Jabber Packet Headers" extension (JEP-0033) would perform the same
tasks, however, there are no clients that implement this extension yet.
So, when and if I decide to implement this, I expect it will be done in
one of two ways:

 - Jabber users would send/receive messages to/from a JID like
   "foo,bar,baz at transport" which would map to a multiple-recipient
   messages to/from Goofey users "foo", "bar" and "baz". This would
   work, however such a user would appear to be a "different" users to
   foo at transport, bar at transport and baz at transport.

 - The transport could implement a mini-groupchat server that could be
   used for multiple-recipient messaging.

Neither of these are perfect, and I still haven't decided which is
better. This is example is useful to see that not all functions have a
perfect mapping, and some may require some serious compromises if you
wish to implement them.

Implementation
--------------

Implementation depends largely on the complexity of the foreign network.
In implementing the Goofey transport, it was relatively straightforward.
I have an I/O event loop (as provided by a library like MIO) which
informs the application when something happens on a file descriptor.

I have an instance of a Jabber stream library to manage getting packets
in and out of the Jabber network. This instance is tied to the
descriptor that connects to a jabberd router.

When first started, this is all that happens. Then the transport simply
responds to events as they come in from Jabber users. When users
register, connections are made into the Goofey network, and those file
descriptors are entered into the event loop.

Then, as messages and presence/watch information arrives (from either
side), the data gets transformed and spat out the other side.

I also have a simple SIGALRM timer that triggers the watch polling.

Conceptually, there's not a lot more to it than that. In fact, this
method is how you'd bridge to just about any foreign network or service,
not just a foreign IM network. Of course, if the remote service is
particularly unusual, you may find yourself doing some serious
mind-bending to make it "fit" into the Jabber way of thinking, but those
are the breaks :)

Hope this helps. I've been meaning to write something like this for a
long time, so good on you for emailing on a day when I had time and
motiviation to ;)

Rob.

-- 
Robert Norris                                       GPG: 1024D/FC18E6C2
Email+Jabber: rob at cataclysm.cx                Web: http://cataclysm.cx/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <https://www.jabber.org/jdev/attachments/20030303/ed1aca7c/attachment-0002.pgp>