Use ci/cd to place different robot.txt per environment

Also: add the URL of the sitemap index file to public/robots.txt to help search engines find the sitemap. See here for details on this: https://www.woorank.com/en/blog/how-to-locate-a-sitemap-in-a-robots-txt-file