#The robots.txt file is to give instruction to search engines: # #"User-agent" is the type of search robot/crawler, the value * #is a wildcard, which allows all engines. "Disallow" doesnt #have a value to disallow nothing, so that all and files and #sub-directories can be searched. # #This file is currently standard with no restrictions. User-agent: * Disallow: