[jdev] PubSub & News Feeds

[JabberForum]

Kelly S;562 Wrote: 
> 
> I don't really need the node tree to reflect the url sub directory
> structure if thats what your mentioning in the
> "/subdomain.domain/some/path/feed/" path.
> 
> I've created a root node with the id of "feeds" and inside here is
> really where I just want to populate a whole ton of feeds. However I
> need to relate their web url somehow when trying to fetch these feeds
> because the "service" which is going to download & publish is going to
> need to know where to publish to.
> 
> I have been looking at some APIs of PubSub managers etc and many take
> node id as a string and I wonder if I put a url in there with "/" its
> going to think that is a sub node tree structure when all I really need
> is the following (I think?):
> 
> /feeds/someurl/entries*
> 
> I hope this makes sense. I'm not sure how else to describe it.
> 

Yes this makes sense. I had a fast look at the XEP. I don't see
anything which tell which characters are allowed in an item id. Maybe
there is somewhere, there are so many XEPs linked to each others, and
linking one to another. Someone on this list who knows better the XEPs
than me could probably answer. But maybe an id could simply be anything
which is accepted as an XML parameter. In this case, you can probably
use '/'. But I still need a confirmation from other people, so don't
trust my writing!

And as anyway the node url and the item id are given separatly, I don't
think this would cause major issue of distinguish them. Something like
this to publish an item could maybe do the trick:


Code:
--------------------
    
  <iq type='set'
  from='hamlet at denmark.lit/blogbot'
  to='pubsub.shakespeare.lit'
  id='publish1'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
  <publish node='/feeds/someurl/entries'>
  <item id='http://subdomain.domain/some/path/rssfeed/'>
  <entry xmlns='http://www.w3.org/2005/Atom'>
  ...
  </entry>
  </item>
  </publish>
  </pubsub>
  </iq>
  
--------------------


Yet the problem with this method is that you have only one item for one
rss feed. Is this what you want? I was rather thinking of one leaf node
for one rss feed, and then one item in this node for every rss item.
And in this case, you could transform the url
http://subdomain.domain/some/path/rssfeed into the pubsub node:
/feeds/someurl/entries/subdomain.domain/some/path/rssfeed/
Then inside this leaf node, you can publish all item you want.

> 
> So basically I can request PubSub to send me the latest 1 item, take
> that date, then pull the RSS feed off the web, and only publish items
> newer? This makes sense I think. I was hoping I didn't have to execute
> queries before pushing data as that would put more load on the XMPP
> service but if I am able to request just 1 latest item  atleast thats
> minimizing the hit.
> 

Yes that's it. I think this is the way agregators work. And what you
want to do is finally just some kind of dynamic agregator transforming
rss feeds into Jabber feeds.
Anyway I don't think that you can really avoid queries, because this is
basically the problem of pull systems: you will always have to query
data in order to know simply whether or not you need to do something!
This is basically unefficient!
Your system will bring the advantages of Jabber feed, but I don't think
it can really delete the flaws (efficiency and non-realtime) of RSS as
it still relies on it. To do so, you still need a pure Jabber
notification from the beginning to the end.

Anyway as long as it is not spread enough (but that RSS is), your
system can probably be better than nothing. :-)


-- 
Jehan
------------------------------------------------------------------------
Jehan's Profile: http://www.jabberforum.org/member.php?userid=16911
View this thread: http://www.jabberforum.org/showthread.php?t=149