[JDEV] current bots?

[Ryan Eatmon]

The RSS-Agent was shelved due to a similar problem.  It has the ability
to fetch the RSS from the sites itself, which works fine for a fixed
number of sites.  But it also had the ability to allow for custom URLs. 
And that killed it.  There were too many people putitng in custom URLs
and it caused RSS-Agent to bog the machine down and take up the entire
process table. =(

To that end, I was approached by the author of Meerkat. 
http://oreillynetwork.com/meerkat/  It's a database of RSS feeds from
all over the web in a lot of categories (no sports feeds though...).  I
have been working on a Meerkat-Service that fetches the headlines from
Meerkat so only it indexes the sites, and then the multiple
Meerkat-Services can fetch from it.

I'm almost done with it too.  Just working out the new dynamic forms
stuff so that you can register with a custom channel.  Now why Meerkat? 
A couple of quick reasons:

1) It does the fetching work, which makes it easier on the web sites,
and on me the coder.
2) It stores everything in a MySQL database... so you can search it. 
(One of my favorite things to do is subscribe to /[Ll]inux/ and get
every article that contains the word Linux.)

That said, I need to bring RSS-Agent out of the closet and dust it off. 
One of the things that we have been trying to get to is a
perl.jabber.org redesign so that all of the Perl projects for Jabber can
be found in one location.  When that comes about RSS-Agent will be
available for all to
download.  In the meantime, I will make it available from CVS on
jabber.org within a week.

The version that will be in CVS will have the custom URL feature removed
due to the performance reasons listed above.  But it can be used to
serve up a specific RSS feed that you want to design and get working.

Ryan
reatmon at jabber.org

M hirsch wrote:
> 
> I would like to know..
> Is anybody currently running some of the above headline perl scripts. (I would like to nickname them a JabberBot )
> And can they be publiclly be accessable.  ex: I was very happy with the code that goes out and hits a few sites for headlines.
> I am looking for this so that we will not have multiple people getting the same information from a copy of the same script abusing the same website.
> 
> Thank you...  (pointer of this on:  http://home.nyc.ny.us/twiki/bin/view/Jabber/JabberBots )
> 
> Matthew


-- 

Ryan Eatmon                   reatmon at jabber.org 
------------------------------------------------
Jabber.org - Perl Team    jid:reatmon at jabber.org