[jdev] Replication & revision protocol

Wed Jan 20 18:39:17 CST 2010

The protocol sequence below is derived from an exchange between instances of
my app. Typos could have crept in during editing!

There are features missing... splitting <revision/> messages to fit the max
payload size for the node, and types for diff formats.

Feedback welcome!

1. Creator makes a pubsub node for incoming messages, and subscribes to it:
(These nodes give us IQ for 1-to-1 messaging, even if recipient is offline.)

<iq from='creator at localhost/
reprev' to='pubsub.localhost'
    type='set' id='9365' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='creator'/>
    <configure><x xmlns='jabber:x:data' type='submit'>
      <field var='FORM_TYPE' type='hidden'><value>
http://jabber.org/protocol/pubsub#node_config</value></field>
      <field var='pubsub#notify_retract'><value>0</value></field>
      <field var='pubsub#persist_items'><value>0</value></field>
      <field var='pubsub#publish_model'><value>open</value></field>
      <field var='pubsub#access_model'><value>whitelist</value></field>
      <field
var='pubsub#send_last_published_item'><value>never</value></field>
</x></configure></pubsub></iq>

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9366' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe node='creator' jid='creator at localhost/reprev'/>
</pubsub></iq>

2. Contributor does the same:

<iq from='contributor at localhost/reprev' to='pubsub.localhost'
    type='set' id='8776' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='contributor'/>
    <configure><x xmlns='jabber:x:data' type='submit'>
      <field var='FORM_TYPE' type='hidden'><value>
http://jabber.org/protocol/pubsub#node_config</value></field>
      <field var='pubsub#notify_retract'><value>0</value></field>
      <field var='pubsub#persist_items'><value>0</value></field>
      <field var='pubsub#publish_model'><value>open</value></field>
      <field var='pubsub#access_model'><value>whitelist</value></field>
      <field
var='pubsub#send_last_published_item'><value>never</value></field>
</x></configure></pubsub></iq>

<iq from='contributor at localhost/reprev' to='pubsub.localhost'
    type='set' id='8777' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe node='contributor' jid='contributor at localhost/reprev'/>
</pubsub></iq>

3. Creator makes a pubsub node for a shared collection, and subscribes:

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9367' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='uuid1.105'/>
    <configure><x xmlns='jabber:x:data' type='submit'>
      <field var='FORM_TYPE' type='hidden'><value>
http://jabber.org/protocol/pubsub#node_config</value></field>
      <field var='pubsub#notify_retract'><value>0</value></field>
      <field var='pubsub#persist_items'><value>0</value></field>
      <field var='pubsub#publish_model'><value>subscribers</value></field>
      <field var='pubsub#access_model'><value>whitelist</value></field>
      <field
var='pubsub#send_last_published_item'><value>never</value></field>
</x></configure></pubsub></iq>

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9368' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe node='uuid1.105' jid='creator at localhost/reprev'/>
</pubsub></iq>

4. Creator revs the collection whitelist, notifies collection members of the
whitelist change, and invites Contributor to join the collection:

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9372' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <affiliations node='uuid1.105'>
      <affiliation jid='contributor at localhost' affiliation='publisher'/>
</affiliations></pubsub></iq>

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9370' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='uuid1.105' jid='creator at localhost/reprev'><item>
      <member xmlns='reprev_ns' uid="contributor at localhost" added="pending">
        contributor at localhost
</member></item></publish></pubsub></iq>

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9369' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='contributor' jid='creator at localhost/reprev'><item>
      <invite xmlns='reprev_ns' collection="uuid1.105">
        <from uid="creator at localhost">Liam</from>
        <date>2010-01-19T20:45:26Z</date>
        <name>Cornucopia</name>
        <blurb>A non-descript invitation</blurb>
</invite></item></publish></pubsub></iq>

5. Contributor subscribes to the collection, and notifies Creator:

<iq from='contributor at localhost/reprev' to='pubsub.localhost'
    type='set' id='8778' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe node='uuid1.105' jid='contributor at localhost/reprev'/>
</pubsub></iq>

<iq from='contributor at localhost/reprev' to='pubsub.localhost'
    type='set' id='8779' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='creator' jid='contributor at localhost/reprev'><item>
      <join xmlns='reprev_ns' collection="uuid1.105"
uid="contributor at localhost"/>
</item></publish></pubsub></iq>

6. Creator replicates collection to Contributor, first meta/data and then
revision history:

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9375' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='contributor' jid='creator at localhost/reprev'><item>
      <replica xmlns='reprev_ns' collection='uuid1.105'>
        <!-- app-defined data -->
</replica></item></publish></pubsub></iq>

<iq from='creator at localhost/reprev' to='pubsub.localhost'
    type='set' id='9376' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='contributor' jid='creator at localhost/reprev'><item>
      <revision xmlns='reprev_ns' collection="uuid1.105">
        <revlist touch="2010-01-19T20:52:34Z" oid="uuid1.118">
          <object oid="uuid1.115" op="+" touch="2010-01-20T00:29:26Z"
diff="uuid1.120">
            <object oid="uuid1.116" op="+" touch="2010-01-20T00:29:26Z"
diff="uuid1.121"/></object></revlist>
        <diff orig="uuid1.115" oid="uuid1.120" type="not-implemented">
          <!-- app-defined content --></diff>
        <diff orig="uuid1.116" oid="uuid1.121" type="not-implemented">
          <!-- app-defined content --></diff>
</revision></item></publish></pubsub></iq>

7. Contributor makes a revision to collection:

<iq from='contributor at localhost/reprev' to='pubsub.localhost'
    type='set' id='8784' xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='uuid1.105' jid='contributor at localhost/reprev'><item
id='uuid2.106'>
      <revision xmlns='reprev_ns' collection="uuid1.105">
        <revlist touch="2010-01-19T21:00:36Z" oid="uuid2.106">
          <object oid="uuid1.116" op="!" touch="2010-01-20T00:29:26Z"
diff="uuid2.107"/></revlist>
        <diff orig="uuid1.116" oid="uuid2.107" type="not-implemented">
          <!-- app-defined content --></diff>
</revision></item></publish></pubsub></iq>

8. Contributor resigns from collection:

<!-- not yet implemented -->

On Wed, Jan 20, 2010 at 3:44 PM, Liam <pubsub at networkimprov.net> wrote:

> I'm building an app which replicates collections of data objects among
> groups of contributors/subscribers.
>
> I use XMPP for transport & store-forward, and I've built a simple
> replication & revision system on top of PubSub.
>
> I'm considering open sourcing this code and documenting the protocol.
> (Source is javascript requiring Strophe.) Creating a separate package for
> this would entail some work, so I'd like to gauge potential interest...
>
> My replication & revision system does the following:
>
> Establishes pubsub nodes for shared collections
> Reliably transmits (reliable requires IQ Notifications):
>   - invitations to join a collection
>   - invitation acceptances
>   - collection replicas to new members
>   - resignations from a collection
>   - revisions to a collection, containing:
>     a) list of revised objects
>     b) diffs for added/modified objects
>        format is app-specific; common diff types may be specified
>
> Obviously, it's very lightweight. It's not focused on real-time apps per
> se, but could be applied thereto.
>
> Also, I'd love pointers to other forums where folks could be interested...
>
> Liam
>
> PS: protocol specifics to follow...
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.jabber.org/jdev/attachments/20100120/6d7f8e4c/attachment-0001.htm>