[jdev] PubSub & News Feeds

[Kelly S]

Thanks for your time Jehan!
 
Sometimes I am unsure how to explain what I have in my head lol so let me try and clarify some more. Responses are found below.



> To: jdev at jabber.org> From: list-jdev at jabberforum.org> Date: Sun, 25 May 2008 21:25:27 +0200> Subject: Re: [jdev] PubSub & News Feeds> > > Kelly S;548 Wrote: > > Hiya.> > From what I can read on PubSub its clear how users subscribe to feeds> > and receive events but how can I handle a user requesting a feed that is> > not indexed on the server yet? So that the server can now start indexing> > that feed and begin Publishing new entries it pulls (I will be writting> > the service which starts pulling feeds off the web).> > > > Here as far as I understand, and remembering your previous post, you> want to be able to transform any RSS publication into a xmpp node> publication. So the point here is that your pubsub tree is "dynamic".> From what I know, this is not in the basic XEP, because it implies a> relation between the web and xmpp, which has not been planned (you> wanted to tie the http url and the pubsub node, for instance a rss in> http://subdomain.domain/some/path/feed/ would be in the node> /subdomain.domain/some/path/feed/, didn't you?).> Yet this is probably very easy to implement such service on a server:> when querying a subscription, it would check whether the address you> entered is indeed linking to a rss feed, then creating the associated> node for beginning to pull.> 
 
I don't really need the node tree to reflect the url sub directory structure if thats what your mentioning in the "/subdomain.domain/some/path/feed/" path.
 
I've created a root node with the id of "feeds" and inside here is really where I just want to populate a whole ton of feeds. However I need to relate their web url somehow when trying to fetch these feeds because the "service" which is going to download & publish is going to need to know where to publish to.
 
I have been looking at some APIs of PubSub managers etc and many take node id as a string and I wonder if I put a url in there with "/" its going to think that is a sub node tree structure when all I really need is the following (I think?):
 
/feeds/someurl/entries*
 
I hope this makes sense. I'm not sure how else to describe it.
> > Also a bunch of concerns pop into my head that I'm still unclear about> > when maintaining all this "feed" data. 1. Is there any way I can just> > publish the latest feeds I pulled down and pubsub discard any duplicates> > that may already exist? Or is the right way to handle this is to query> > every single entry individually to check if they exist before publishing> > them? > > > > Here you would do simply like any normal agregator already does: you> check the time. You know the last time you got any news on a RSS, hence> you stop reading the feed to this date.> 
 
So basically I can request PubSub to send me the latest 1 item, take that date, then pull the RSS feed off the web, and only publish items newer? This makes sense I think. I was hoping I didn't have to execute queries before pushing data as that would put more load on the XMPP service but if I am able to request just 1 latest item  atleast thats minimizing the hit.
> > > > Thanks so much for your time > > > > You're welcome. ;-)> 
 
Again thanks so much. I am quite confused on how to relate the node structure to feed urls so hopefully I can get this figured out.
> > -- > Jehan> ------------------------------------------------------------------------> Jehan's Profile: http://www.jabberforum.org/member.php?userid=16911> View this thread: http://www.jabberforum.org/showthread.php?t=149> > _______________________________________________> JDev mailing list> FAQ: http://www.jabber.org/discussion-lists/jdev-faq> Forum: http://www.jabberforum.org/forumdisplay.php?f=20> Info: http://mail.jabber.org/mailman/listinfo/jdev> Unsubscribe: JDev-unsubscribe at jabber.org> _______________________________________________
_________________________________________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.jabber.org/jdev/attachments/20080525/f70f4030/attachment-0002.htm>