Allows You to submit a sitemap of your site to Google. A sitemap is sort of a map that can help Google find and comprehend the many webpages on your website. txt file is then parsed and may instruct the robot concerning which web pages usually are not being crawled. https://www.youtube.com/watch?v=DhNbKpYq6Ro