[JDEV] Module API

[Donn Cave]

Quoth Will Morton <will.morton at cwcom.net>:
| Donn Cave wrote:
|> I was thinking the storage access model here is the very simplest
|> possible.  Each item is retrieved by exactly one user name.  The data
|> is just bytes, it's up to the service to sort out the XML.  Only the
|> service daemons need access to the data.  Is it a bigger job than this?
|> Is LDAP attractive because it's an existing network storage solution,
|> or for features like fast wild-card attribute lookup and hierarchical
|> structure?
| 
| 	LDAP directories (or rather Directories in general) are attractive
| mainly because in a large, multi-services platform (web, mail, ftp,
| chat) they provide an easy-to-administer and highly scalable way to
| store all your user information, which is also very easy to write Apps
| for. They can also be browsed by an admin in a logical fashion, which is
| a big bonus when you have O(1m users) on your system, everything has
| gone tits-up and you're trying to find the problem.
| 	I'm not sure what you mean by 'network storage solution'. Do you mean a
| place to dump all your files, such as a NetApp? Or do you mean a
| repository for user info such as a Database?

I just mean /usr/local/var/jspool on the net.  Jabber-transport 1.0
gets presence and subscription information, offline messages etc.
from a local disk file, like ----------
    file = open('/usr/local/var/jspool/%s.xml' % user)
    data = file.read()
    file.close()
    state = parse_xml(data)

and a network version doesn't need to be much more complex --------
    net.write('get %s\r\n' % user)
    data = net.read()
    state = parse_xml(data)
    verify_user_state(state, user)    # check for ``user not found''

This addresses the specific problem, that the disk file is local to
one host when you might need several hosts to support the service,
and it can solve that problem better than NFS-mounting the file.

The protocol can be bone-head simple, I'd think.  I could write an
application in 15 minutes (in C, 2 minutes in Python), no need for
BER encoding or any of the other fun stuff that requires external
library support for LDAP applications.  The hair is on the service
end, where for a first cut I'd probably go with a jspool-like
filesystem approach (but with an artificial directory hierarchy, to
avoid mammoth directory rewrites.)

My point is not to push one approach, though.  I'm just asking,
is there an application here for LDAP because of its technical
merits, am I missing something about the problem?

I don't really know what all will go in this xdb space.  It looks
to me like /user/spool/mail.  Files would certainly be shorter on
average, but subject to radical ballooning with a few offline
messages with video & audio.  I suppose there would have to
be some kind of quota, and expiration, but you'd have to be able
to store on the order of megabytes per request.

	Donn Cave, donn at u.washington.edu