Checking for page changes

Dear lazyweb, I’m asking for help. I maintain the ebuilds for a series of software that seems not to use common release notices like Freshmeat. This is the case, for instance, of libarchive, Nimbus theme and sudo, etc.

I usually check the pages every other day, but it starts to get boring, and something I’d rather automate. Does anybody know of a nice software that checks if pages have changed and send me an email, or a log, or an rss, or anything actually, so that I can just tell that to check some given pages and leave it alone?

Optimally, it should just check the header for the Last-Modified date, or the ETag to be different, without doing a full GET of the page. I could probably write something up in bash to do it, it’s just a matter of netcat and grep, or maybe curl, but I’d actually avoid having to write something up myself if it exists already.

The reason why I would like it to use HEAD rather than GET is because there is no point for a script to request the same page every day to check if it is changed, it’s not like a browser requesting it. This way it would save both my and the site’s bandwidth, which if done properly by all services would be quite a save. Even better, if it could use If-Modified-Since, so that after the first request, every other subsequent request would just get a 304 response and no extra data like content type and length, which requires a stat on the server.

If anybody has a suggestion, it’s very welcome. Even if it’s not in portage, I can create ebuilds and maintain them afterward, I just need something to make my job easier! Of course it’s obvious that it has to run on Linux (and possibly FreeBSD 😉 ).

Exit mobile version