How to check if content of webpage has been changed?

Question

Basically I\'m trying to run some code (Python 2.7) if the content on a website changes, otherwise wait for a bit and check it later.

I\'m thinking of comparing

Answer 1

There is no universal solution.

• Use If-modifed-since or HEAD when possible (usually ignored by dynamic pages)
• Use RSS when possible.
• Extract last modification stamp in site-specific way (news sites have publication dates for each article, easily extractable via XPATH)
• Only hash interesting elements of page (build site-specific model) excluding volatile parts
• Hash whole content (useless for dynamic pages)