[JDEV] On-The-Fly XML parsing

[Keith Minkler]

JDEV,

    I've been working on a way to increase performance and memory usage in the jabber server,
    and I've finnally got it working and commited.. 

    Basically, the issue is that the jabber server is an xml router, as such, it's only looking
    at the to and from attributes generally speaking, and ignoring any of the child data.. so,
    I've created xmlnode objects that are parsed on-the-fly in the server... basically, the incoming
    stream is parsed once for xml conformity (no bad xml), and a single xmlnode is created, just
    the parent, and its attributes.  whereas before, and entire xmlnode tree was created for the
    xmlnode and all of its children.

    the xmlnode is marked incomplete, and the actual text for the node is stored with it.. when
    any child nodes are accessed, the xmlnode's children are then created from parsing the stored
    text, one child level at a time, as needed... so only the xmlnode structures that are needed are
    ever stored in memory.. everything else is simply stored as the original text stream.

    Yes, this means that a single xmlnode could conceivably be needed to be parsed several times,
    but I assure you, this is not the server's bottleneck!  expat is incredibly fast at parsing text!
    single xmlnodes are parsed in around 20 *nano* seconds, depending on size, and I've fed it giant
    xml files ~12M in size, which it chewed up in around 2.5 seconds...  I'm not sure how long it takes
    to build an xmlnode struct for that same file, because after several minutes of trying, i ran out
    of memory, and it locked up my PC...

    This method reduces the number of malloc and free calles in the server, which is where the real
    bottleneck lies, and increases server performance and memory hogging!  

    This project is still in developmental stages, and I would like to have some beta testers, and
    performace experts take a look at it, and see how good/bad it is, before I commit it to the main
    server codebase.

    If you want to try it out, it's located in a branch of the "jabberd" module that is tagged
    "XMLNODE_TEST".  for example..

    Check out the "jabber2" module from CVS:
    cvs -d :pserver:anonymous at jabber.org:/home/cvs co jabber2
    cd jabber2

    get rid of the "HEAD" jabberd module:
    rm -rf jabberd

    check out the experimental jabberd:
    cvs -d :pserver:anonymous at jabber.org:/home/cvs co -r XMLNODE_TEST jabberd

    Configure and compile:
    ./configure
    make

    Please report any problems either to this list, or my email adress at tsbandit at jabber.org
    thanks for your time, and happy jabbering!

Keith Minkler

-------
Software Developer
Jabber.COM, Inc.
-------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: not available
URL: <https://www.jabber.org/jdev/attachments/20010317/bf600e8b/attachment-0002.pgp>