[JDEV] Jabber DevZone News - Jabber Email?

[travis at thinkvirtual.com]

Interesting concept and I have thought a lot about this before.  The fact is that IM makes e-mail obsolete in a lot of ways.  The only reason e-mail was made the way it is with a mailbox and you pick it up, because it was made in the days of dial ups and not being connected 24/7.  But that is changing as we all know and IM is becoming a way of life.  I would say that after people start using IM's, they probably use it more than e-mail.  

Now the valid points have come up about why we still use e-mail and that is because people still have dial ups, you are pretty positive that the message will get received (this relates to the discussion on guaranteed messaging), and it is more formal.

Now something that I think you and anyone interested in this concept should look at is Palm's developer mailing lists.  From what I can remember, I haven't seen anything that goes as far along the lines of this topic as Palm is.  

Basically, palm records e-mails into a USENET newsgroup and sends e-mails that are posted to that group as well.  So I can read through the messages in my newsgroup reader or I can read them through my e-mail.  This allows people to join at any time, to read through old threads, etc. and at the same time, they could join the mailing list and receive all the new messages through e-mail.  

It's definitely not perfect, but it is a great step forward.  I completely agree that newsgroups are the best way to communicate when you are in a group setting because people can follow up and read the whole conversation at any time and get involved at any time.  And they don't have those reply quotes in each message.  You should receive a message that would have to do with a thread and be able to go back through the thread, but shouldn't be attached to every single message.

The only barrier I can see with IM is that it is informal and messages are generally short and often meaningless.  And recording all of these messages would not benefit anybody.  But maybe the IM could just be the notification?  

Anyways, just thought I'd throw in my thoughts here because I completely agree with the concept of a newer (and better) e-mail system.

Travis Reeder
Chief Software Architect
ThinkVirtual


---- Original Message ----
From: Jabber DevZone <webmaster at jabber.org>
Sent: 2001-05-07 17:08:54.0
To: jdev at jabber.org
Subject: [JDEV] Jabber DevZone News - Jabber Email?

Jabber Email?

The following was posted by stpeter at jabber.org via the Jabber DevZone web site (http://dev.jabber.org/):

A proposal from Michael Hearn (Jabber: tweedledee at jabber.org, email:
mhearn at subdimension.com):



So what is Jabber Email?


Good question. Jabber Email is a reimplementation of the internet
email system in Jabber/XML ... with a few differences. It's not just a
reimplementation of the current system, it's a completely new set of
ideas about how email should work.


Er, what? What's wrong with the current email system then?


Well, lots of things. The current email system has been built over a
number of years to do many different things, and I think there are
several things about it that are less than perfect:



I don't like the nature of email messages at the moment. They are
standalone, so for instance if you send a message to someone, it goes
from you, to them. It sits in their mailbox and can only be accessed
by them, and then when they reply often the only thing that links the
original message to the reply is a series of  marks that quotes the
message. This can lead to a massive buildup of  marks as messages are
replied to, forwarded, etc.


Threading doesn't work often because of this, and it restricts what
you can do with email conversations.


Email can be slow, the time taken to deliver a message can be measured
in seconds or days.


It's hard to secure email messages, both sides need the right
software, etc.


Different media types aren't handled particularly well in email
messages at the moment. For instance, if a message has formatting,
that may not display on some email clients so the message has to be
sent twice, one with formatting, one without. It's not possible to
type in an address and for the mail reader to retrieve information
about that address, for instance by getting client capability
information/public keys etc.






So what would you do to make it better?


Good question. One of the systems I really like on the internet is
USENET. To me, this is a messaging system that makes sense.  Messages
are usually threaded right, people can come into conversations
half-way through, and so on. So a combination of the ideas behind
USENET messaging with the one-to-one privacy of email, combined with
the power of Jabber and XML, would be ideal.


So why would this system be better?


Well, imagine this. I write a message to my pal Eric Murphy, telling
him about some great new feature of Jabber I have been experimenting
with, and I enclose some screenshots etc. The message appears in my
jabber email client in the "message store" or something like that (for
threading to work well, the messages you send should I think appear in
the same tree structure as the messages you receive, or more
accurately, subscribe to). Eric's message store immediately gets the
message.


He happens to be online at the moment, and receives it instantly.  He
reads it, replies to it, and the message then immediately appears
below the one I sent in the tree view in my client. In it he suggests
that we get another mate of ours, Jeremie Miller into the
conversation. So I put Jeremie on the To list of my original message.
Suddenly he has access to the complete conversation, and can insert
replies at any point.


Perhaps this is Jabber.com business or whatever and therefore
confidential, well that's OK, the message is identified by a hash
anyway so we know it's not been tampered with, and because of Jabber's
IQ mechanism (combined perhaps with the Jabber Identity system I
proposed not so long ago) we can easily retrieve information about
Jeremies formatting preferences, public key etc. We carry on like
this, and build up quite a nice little email conversation. Massive
amounts of quoting are not necessary (although that option is still
available if you want it), as the sequencing needed for threading is
all done automatically.


We decide that maybe the whole company should be able to access this
conversation and take part in it, so we add the root message of the
conversation to the company-wide message store. Instantly everyone has
access to those messages and can work on them, reply to them etc. If
someone was to forward a message, well the forwardee could elect to
receive all the followups too. This idea is coming along, so we create
an email that is editable by anyone.  Yes, that's right, from now on
that email becomes more like a shared document than anything else --
as multiple people change it, it updates in the message store (inbox)
before our eyes.


So you see, the whole email/messaging experience is a lot more
seamless and the distinction between private email and more groupware
oriented messaging like newsgroups are blurred.


But we already have instant messaging! Why do we need another
messaging system?


Another good question. Well, IM and email have similar yet different
approaches to the same problem -- allowing people to communicate. IM
is immediate, but has lower capabilities. It's for the type of
conversations we have around the dinner table, immediate, often made
up of short statements. Even if the current email system was instant,
would you want to use it for IM? No, because email has a different
approach. It's for longer messages, and requires much higher
capabilites. For instance, the ability to embed a Flash animation in
an email might be required by some people, but nobody would want that
feature in an IM application. Another of email's features that has
saved many careers I expect is the reliablility. If an instant message
doesn't get through, you get an error. If an email doesn't get through
first time, you're notified and the servers will keep trying until it
does or the message times out. So the email system would need
reliablility. I think a part of email's potential is still unlocked,
and the Jabber platform can help realise it.


OK, you've sold me on it, so how would it work?


Well, the concept of the "email (message)" needs redefining first.  In
Jabber Email, an "email" is no longer an entity that is passed from
one computer (the senders) to another (the receivers).  Instead, all
messages reside server-side, and all messages have a JID. This doesn't
mean they have accounts with a server of course, just that they are
entries in a database somewhere and can be accessed by a JID. For
instance, here are some examples:



AF41c1003FdaV98 at messages.jabber.org -- a simple message, identified by
a hash of its contents


AF41c1003FdaV98 at messages.jabber.org/part/3 -- a reference to the third
part of the email, presumably an attachment



When a user sends a message, a hash of that message is created, and
the message is posted to the "messages" server component, which
records it, along with information about who is allowed to access it
etc. Obviously this loosens the coupling between sender and receiver
possibly making it easier to intercept mail, but strong encryption as
a requirement of the email system would help solve this problem. The
hash acts as a double-check to ensure the message hasn't been tampered
with.


So now the message server has a message stored, which can be accessed
by a JID (assuming that you can prove you are authorized, more on this
later) The next thing is to tell the receiver about its presence, er,
wait a minute, did I just say presence?  Yes, that's right, the idea
of the "message store", which is like an Inbox except that it has your
messages in it too, is based on an extension of the presence
framework. Messages (or rather the messages server) emit presence. If
you want to monitor a message and any associated messages (i.e.,
replies, forwards, or any other arbitrarily associated messages) then
the user subscribes to that messages presence. This means that when
the user logs in to their Jabber Email client (which btw doesn't have
to be related to the Jabber IM client they use, it just shares the
user account and network, it'd have a negative priority so it doesn't
receive instant messages) they receive the presence for each of the
messages in their store. The presence of a message contains
information on any children it has, it's last timestamp so we know if
it's changed internally, and basically any other information that
could change in a messages lifetime. Remember in Jabber Email,
messages aren't static, they can and should change as much as
necessary.


So once the reader has received presence for each of the messages in
its store, it knows what's changed (messages send presence only to
people that are on its "To" list), and can therefore download any
messages that are new, update any that have a changed timestamp, etc.
This is the first part of the system, the part that exchanges
messages.


The next part is the messages themselves. This bit isn't difficult, we
just need a decent XML envelope language to specify the subject line,
etc. Bear in mind that some data, such as the threading information,
timestamp, To list (etc.) would be stored out-of-band, for instance by
using an IQ set on the message as it was being uploaded. I would
imagine something like this:




<message type="email">

<subject>A simple sample message</subject>

<priority>normal</priority>

<body type="text/xml" xmlns="xhtml namespace" for="*">

Hi there, this is a simple <div
style="underline">HTML</div> message.

</body>

 <body type="text/plain" for="*">

Hi there, this is a simple plain text message, with formatting.
Clients should use the first type they understand when selecting a
body to use

</body>

</message>






Of course that's extremely rough, but you get the idea. Bear in mind
that most of the routing data would be stored out-of-band. To post a
message:





<iq to="hash at messages.jabber.org" type="set">

 <query xmlns="jabber:email:message">

 <to jid="tweedledee at jabber.org"/>

 <to jid="ericmurphy at jabber.org"/>

 <message/> etc...

 </query>

</iq>






And an IQ get would retrieve the message.  Simple!


Cool, but what about security?


Security is very important. The hash of a message provides evidence it
hasn't been tampered with. To change a message, it must be obsoleted
(this information is stored OOB), so no message can be changed without
leaving a trace of the original. All messages are encrypted
on-the-wire and stored on the server in an encrypted form. However, if
you want extra security, a public key can be stored in your jabber
identity profile, and used to encrypt the message for individuals.
Because message bodies can be different for different people (the for
attribute) one message can be encrypted for many people.


But how do you stop the system being overloaded if nobody has private
email boxes anymore?


Well, this is an important issue that is both a strength and a
weakness. For instance, now all messages are stored on the server in a
single area, they can be as big as is needed, you don't have to worry
about an attachment bouncing because the recepient doesn't have enough
space. But the servers need to be able to control the amount of space
this takes up, especially as messages can't be deleted by users (see
below). A simple way would be to restrict message uploads to the same
server as the user's account. The messaging server could then monitor
the messages sent by each user, and if active messages goes over a
specified size then message sends could be restricted.


Active Messages?


Yeah, because you don't actually "send" messages, when you delete them
from your message store (remember it's just a database made up of
cached messages -- the actual message store is transient and made up
of presence notifications), all that happens is that you "unsubscribe"
to that message and any replies. This is because if other people are
on the "To" list, and they always will be, then the message can't be
deleted from the server, they may still need it. To solve this
problem, messages can be either "active" or "inactive". An active
message is one that is still broadcasting presence. When the To list
is empty, a message no longer broadcasts and is classed as inactive.
This message will then be compressed and archived, and eventually,
consigned to oblivion by the server. This has some interesting
conseqences:



Messages can be "undeleted", i.e. you just resubscribe to their
presence.
Messages will not stay in existence forever, unless a user
specifically instructs their client to keep a local copy.
The original sender may think a message has been deleted while other
people can still access it, modify it and reply to it.




It sounds great, when do we get it?


I wish I knew, but as I have no Linux/server programming, only client
building skills probably not
until Jabber.com hires me to build it!! :)




http://jabber.org/?oid=1334
_______________________________________________
jdev mailing list
jdev at jabber.org
http://mailman.jabber.org/listinfo/jdev