[JDEV] Jabber DevZone News - on-the-fly XML Parsing

[Jabber DevZone]

on-the-fly XML Parsing

I've been working on a way to increase performance and memory usage in
the jabber server,and I've finnally got it working and commited..


Basically, the issue is that the jabber server is an xml router, as
such, it's only looking
    at the to and from attributes generally speaking, and ignoring any
of the child data.. so, 
    I've created xmlnode objects that are parsed on-the-fly in the
server... basically, the incoming
    stream is parsed once for xml conformity (no bad xml), and a
single xmlnode is created, just    
    the parent, and its attributes.  whereas before, and entire
xmlnode tree was created for the
    xmlnode and all of its children.


the xmlnode is marked incomplete, and the actual text for the node is
stored with it.. when
    any child nodes are accessed, the xmlnode's children are then
created from parsing the stored
    text, one child level at a time, as needed... so only the xmlnode
structures that are needed are
    ever stored in memory.. everything else is simply stored as the
original text stream.


Yes, this means that a single xmlnode could conceivably be needed to
be parsed several times,
    but I assure you, this is not the server's bottleneck!  expat is
incredibly fast at parsing text!
    single xmlnodes are parsed in around 20 *nano* seconds, depending
on size, and I've fed it giant 
    xml files ~12M in size, which it chewed up in around 2.5
seconds...  I'm not sure how long it takes
    to build an xmlnode struct for that same file, because after
several minutes of trying, i ran out  
    of memory, and it locked up my PC...


This method reduces the number of malloc and free calles in the
server, which is where the real
    bottleneck lies, and increases server performance and memory
hogging!


This project is still in developmental stages, and I would like to
have some beta testers, and
    performace experts take a look at it, and see how good/bad it is,
before I commit it to the main
    server codebase.


If you want to try it out, it's located in a branch of the "jabberd"
module that is tagged
    "XMLNODE_TEST".  for example..


Check out the "jabber2" module from CVS:
    
cvs -d :pserver:anonymous at jabber.org:/home/cvs co jabber2
    
cd jabber2


get rid of the "HEAD" jabberd module:

    rm -rf jabberd


check out the experimental jabberd:

    cvs -d :pserver:anonymous at jabber.org:/home/cvs co -r XMLNODE_TEST
jabberd


 Configure and compile:

    ./configure

    make


Please report any problems either to this list, or my email adress at
tsbandit at jabber.org
    thanks for your time, and happy jabbering!