What is the use of robots.txt?

[artijain1900]

Hello Guys,


I would like to know that What is the use of robots.txt? tell me ..

[irfan]

ok

[irfan]

thanks

[chinmay.sahoo]

using a robots.txt file to block a specify page on search engine

[saisaanvi]

Robot.txt is a standard exclusion protocl which will helps to communicate with search engine bots to restrict bot from crawling a particular pages

[Demixl]

thank you for info

[jameskotler]

Robot Exclusion Protocol provides information to search engine spiders on the directories that have to be skipped or disallowed in your website. robot.txt is very important for site content, search engine as well as meta descriptions.