# robots.txt
# This file specifies what harvesting robots are allowed to index and not.
#
# For information on the format: 
# http://www.robotstxt.org/
#
# Everything not explicitly disallowed is considered fair game to retrieve.