Crawler that checks links on websites for malware activity (google safebrowsing v2, surbl, spamhaus)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Emiel Kollof f49c85e613 Add todo item 8 years ago
.gitignore Initial commit 8 years ago
LICENSE Initial commit 8 years ago
README.md Add todo item 8 years ago
malcrawler.py Added bs4b dependancy. 8 years ago

README.md

malcrawler

Crawler that checks links on websites for malware activity (google safebrowsing v2, surbl, spamhaus)

Prerequisites

TODO:

  • Maybe split out safebrowse logic in a server part so it can stay running and doesn't need to reinitialize every time.
  • Robuster crawling
  • Unicode niggles.
  • Proper handling of Ctrl-C