Updated Format Of Robots.txt
Hello,
We all know the importance of Robots.txt in SEO campaign. In fact all the web developers, web designers are well known of it. Robots.txt is mainly work as a convention method to prevent search engine's spider bot from crawling any certain page of a website. But I would like to know about the current, updated and module structure of Robots.txt, search engine are mostly preferring.
Hope will have my answer soon from you. Have a good day.
Thanks...
prithwish