The llms.txt file is a proposed standard designed to help large language models more effectively understand and use content from websites. Rather than allowing AI crawlers to freely scan your entire site, it provides a curated list of the most relevant and important pages. In essence, it tells AI systems which content they should focus on. This makes it especially useful for models that rely on your site’s information to answer questions or generate responses for users.
Where should you create llms.txt file?
The llms.txt file should be created in the web root, which is in the /data/wordpress/htdocs/ directory. Some plugins may create it in an incorrect directory such as /data/wordpress/htdocs/wordpress/. In that case, you can create the following symlink in the /data/wordpress/htdocs/ directory:
ln -s wordpress/llms.txt
How to fix broken letters in the txt file?
Latin letters with diacritics may appear broken in the llms.txt file. This can be fixed by creating a new nginx configuration file in the /data/wordpress/nginx/ directory with the following content:
location = /llms.txt {
charset utf-8;
}
For the changes to take effect, nginx must be restarted using the wp-restart-nginx command.