Robots.txt
What is Robots.txt?
A small file on your website that tells search engines and AI crawlers which parts they are and are not allowed to visit.
Also known as: robots
If a restriction is included by mistake, search enginesSearch engine A website used to search the internet, such as Google or Bing. or AIAI Computer programs that perform tasks normally requiring human intelligence, such as writing text or answering questions. will not read your site. This is a common cause of poor visibility.
