Dear lazyweb, I’m asking for help. I maintain the ebuilds for a series of software that seems not to use common release notices like Freshmeat. This is the case, for instance, of libarchive, Nimbus theme and sudo, etc.
I usually check the pages every other day, but it starts to get boring, and something I’d rather automate. Does anybody know of a nice software that checks if pages have changed and send me an email, or a log, or an rss, or anything actually, so that I can just tell that to check some given pages and leave it alone?
Optimally, it should just check the header for the Last-Modified date, or the ETag to be different, without doing a full GET of the page. I could probably write something up in bash to do it, it’s just a matter of netcat and grep, or maybe curl, but I’d actually avoid having to write something up myself if it exists already.
The reason why I would like it to use HEAD rather than GET is because there is no point for a script to request the same page every day to check if it is changed, it’s not like a browser requesting it. This way it would save both my and the site’s bandwidth, which if done properly by all services would be quite a save. Even better, if it could use If-Modified-Since, so that after the first request, every other subsequent request would just get a 304 response and no extra data like content type and length, which requires a stat on the server.
If anybody has a suggestion, it’s very welcome. Even if it’s not in portage, I can create ebuilds and maintain them afterward, I just need something to make my job easier! Of course it’s obvious that it has to run on Linux (and possibly FreeBSD 😉 ).
Freshmeat has a subscription service. If you subscribe to an app, you’ll get mails when a new release of the subscribed branch happens. I’ve never used it myself, so I don’t know if the email sent is anything that one would want to parse.cf. http://freshmeat.net/faq/vi…
Freshmeat-based software I already subscribe to and I’m fine with that.The problem is, as I say on the blog, for the software NOT sending freshmeat branch updates. Which seems to be quite a bit of the software I maintain…Rubyforge updates are fine too, as well as SourceForge. But for self hosted projects it is a problem.
Take a look at Yahoo Pipes. You could build valid XML with http://htmltidy.movb.de/ and than create a feed to read with your favorite feed reader.
Try uscan (from Debian) in a cron job.
I’m not entirely sure that this is what you need but emacs and w3m has a feature called antenna that AFAIK does what you need.
How about Mechanize – Python, Perl and Ruby versions seem to be available (also in portage)Here is the Python versionhttp://wwwsearch.sourceforg…
there was one such site, but i cannot remember what was it named.it would simply send you mails that page got modified.i’ll try to find it again, as i haven’t received mails from them for a while.
http://www.changedetection.com will e-mail you whenever a certain page changes.
ck4up is what I use on another distribution. Takes a little setup, but works wonders afterwards 🙂
There is this service here:http://www.changedetection….
rlazo, w3m’s Antenna is almost exactly what I need. Almost because it seems to request user’s intervention. I only tried with emacs-w3m for now, but it doesn’t seem to work much good right now, will probably try w3m proper later.Andy, ck4up would also be good if Ruby allowed to provide extra headers during the request (and supported HTTP/1.1, which it doesn’t iirc), as that would allow the code to handle @If-Modified-Since@.The ChangeDetection site also seems to be interesting, but if I can just let it live in a window on the DWM monitor that would probably be great.
Google Reader lets you subscribe to arbitrary webpages. Also worth looking into is meatoo — talk to pythonhead.
http://rpm5.org/cvs/filevie…Download raw, add perl schebang, run pod2man on the script, read man, create config, run, be happy.
Specto can do this:http://code.google.com/p/sp…
http://www.sudo.ws/mailman/… ?
Okay I missed sudo-announce; doesn’t solve the problem for other projects though ;)Donnie are you sure it’s possible to subscribe to arbitrary pages? I admit I never used it before, and I doubt I’ll be using it too much soon, as it’s not compatible with Konqueror, but I can’t find a way to check a given page for changes…
http://overlays.gentoo.org/… in case you want to test it… 🙂
I did choose net-misc myself, and added it to my overlay already. I was thinking about adding it to portage. My ebuild also fixes some things that the one in sunrise ignore.I’m not sure about the deps though, as I didn’t have dbus-python or gnome-python-extra installed and it is working fine.