A “crawler” is an automatic program that browses the internet to search and collect data from websites, mainly used by search engines. For example, Google’s crawlers visit websites to update the information used in search results.