Google has released a new robots.txt report within Google Search Console. Google also made relevant information around robots.txt available from within the Page indexing report in Search Console.

Google's John Mueller said it may make sense to noindex your LLMs.txt file. This way, the file doesn't get indexed by search engines, and then somehow a user lands on it and is confused. John wrote on ...

Google's John Mueller answers whether llms.txt could be seen as duplicate content and whether it makes sense to use a noindex header with it. Google’s John Mueller answered a question about llms.txt ...