Major Description
Add option to updatedb to filter out records with status db_gone (http 404). This is especially useful in cases where a crawl db is targetted at only a specific site. If the site, for some reason, suddenly changes a lot of url's we'll get a crawl db filled with garbage. Since the targetted site is known (or controlled) it is safe to get rid of all these url's: reduce db size, reduce useless http requests.