[jdev] parsing xml (xmpp) with ruby

[Jonathan Dickinson]

I have looked at the code and I think you could get away with no more than 15-20 *small* changes. All in the pull method in baseparser.rb.

However, you really need to decide if using a pull parser is the right tool for the job. You are going to do regex matches, complex branches and buttloads of other stuff every time a user sends you something (which may or may not be complete): so you will get the scenario that the server is under strain when it hasn't even begun to think about what to do with the XML.

If you want, I can send you some C# code to port (I am a bit strapped for time myself). My code supports a event-based push model (not actual events as .Net events are NOT to be used in servers, it uses an interface reference).

e.g.

Parser p = new Parser(this); // this is instance of IXmlNotificationTarget
p.Push(Encoding.UTF8.GetBytes("<stream:stream xmlns:stream='http://ether....'");

// OnDocumentStarted is called.
// OnElementStarted is called, attributes are present, other nodes are not (ditto).

p.Push(Encoding.UTF8.GetBytes("</stream:stream>");

// OnElementEnded is called, all child nodes are present.
// OnDocumentEnded is called.

You want? It's pretty much a hard-coded state machine. I don't use any arcane C# trickery in it, so porting it shouldn't be hard if you don't understand C#. It's also very forgiving, for two reasons: I decided that if the server can assert what the XML should be it might as well, and putting those verification steps really wouldn't be worth the effort. I have been using it for a couple of months now, so I think I have gotten rid of most of the bugs (it was based on some C code that I found, so there were not that many to start with).

> -----Original Message-----
> From: jdev-bounces at jabber.org [mailto:jdev-bounces at jabber.org] On
> Behalf Of Jonathan Dickinson
> Sent: Thursday, October 02, 2008 12:09 PM
> To: Jabber/XMPP software development list
> Subject: Re: [jdev] parsing xml (xmpp) with ruby
>
> Maybe it is something that is amiss in the Ruby world?
>
> Why not try to get REXML to handle streaming XML? Throwing and catching
> exceptions on a server on a regular basis isn't the best idea (the same
> reason why we don't continually give ourselves papercuts and stick a
> plaster on them).
>
> There *must* be a common point where it decides that the XML document
> is incomplete, maybe you can add fragment support there, instead of
> throwing an exception. I think the best idea would be to make a new
> type of XML node (e.g. IncompleteXmlNode) and return that.
>
> Otherwise you will have to roll your own, which is only about a week of
> work: it's not *that* bad.
>
> I will look at the code now and see if I can offer any insight.
>
> > -----Original Message-----
> > From: jdev-bounces at jabber.org [mailto:jdev-bounces at jabber.org] On
> > Behalf Of Eric Will
> > Sent: Wednesday, October 01, 2008 10:04 PM
> > To: Jabber/XMPP software development list
> > Subject: Re: [jdev] parsing xml (xmpp) with ruby
> >
> > On Wed, Oct 1, 2008 at 3:59 PM, Remko Tronçon <remko at el-tramo.be>
> > wrote:
> > > I don't know anything about REXML. Looking at the source code,
> > there's
> > > a bunch of parsers, but all of them call 'pull' somewhere, so that
> > > makes me suspect that something's not right. The only people that
> can
> > > help you are the other Ruby developers that did XMPP work, and I
> > think
> > > all of them already replied in this thread. Maybe you should look
> at
> > > their code to see how they do it, and talk to them if you need
> help?
> >
> > XMPP4R is the only other Ruby project I can find out there. I asked
> > what you use to see if I could find Ruby bindings for it. XMPP4R uses
> > REXML, and as far as I can tell after looking through the code for an
> > hour or so is that it does the same thing I do, and closes the stream
> > when it gets an exception from REXML.
> >
> > > cheers,
> > > Remko
> >
> > -- Eric Will // rakaur --
> > _______________________________________________
> > JDev mailing list
> > FAQ: http://www.jabber.org/discussion-lists/jdev-faq
> > Forum: http://www.jabberforum.org/forumdisplay.php?f=20
> > Info: http://mail.jabber.org/mailman/listinfo/jdev
> > Unsubscribe: JDev-unsubscribe at jabber.org
> > _______________________________________________
> _______________________________________________
> JDev mailing list
> FAQ: http://www.jabber.org/discussion-lists/jdev-faq
> Forum: http://www.jabberforum.org/forumdisplay.php?f=20
> Info: http://mail.jabber.org/mailman/listinfo/jdev
> Unsubscribe: JDev-unsubscribe at jabber.org
> _______________________________________________