Creating the robots.txt file There is nothing difficult about creating a basic robots.txt file. It can be created using notepad or whatever is your favorite text editor. Each entry has just two lines: User-Agent: [Spider or Bot name] Disallow:...