I would suggest there is probably something wrong with the syntax in your robots.txt file. I haven't used it but Bing has search console similar to Google, there is robots.txt tester available in Google's search console and I would imagine same is true for Bing.since March we've been under attack from Bing/MSN in like fashion.
You can create a "Bad Bots" group in phpBB, add bots that are not listening to robots.txt. Create a new forum role with never permissions and apply that to bad bots group for forum permissions. This is also helpful in reducing server load because the script quickly exits with permission denied message.
This is also useful for other things. If you have forum you don't allow bots to access but want to display Google ads you can create an ads bot group to give it access while denying it to search engine bots.
Statistics: Posted by thecoalman — Tue May 13, 2025 4:19 pm