What is robots.txt ? A robots.txt file is a file that contains specific instructions that inform the bot what it…