You Can Look Here

http://mdou8.ru/user/jarlonwogl

A crawler, placed it one more way, is a device to produce a list of URLs that can be fed into your extractor. Provide a website to begin with, and also they will certainly adhere to all these links on that web