Bingbot Crawl Control Settings in Bing Webmaster Tools

The crawl control feature in Bing webmaster tools allows webmasters to control the rate at which Bingbot crawls your website. Crawl controlling is required to limit the activities of search engine spiders like Bingbot and Googlebot when there are visitors on your site to manage the bandwidth suitably.

Since visitors on the site are to be given more priority it is obvious that the search engine activities at that time are to be restricted. You can inform Bingbot to crawl faster or slower any point of time during a day according to the visitor’s pattern on your site.

Setting a Bingbot Crawl Pattern

Login to your Bing webmaster tools account and select the site you want to set the crawl control pattern. You must have added and verified you site in your account in order to use this feature. The “Crawl Control” option is available under “Configure My Site” tab.

Setting Preset Pattern

You can see the preset options available under the drop down menu of the question “When do you receive the most traffic to this site for your local time of day?”. “All Day” is selected by default and you can choose any time frame based on the visitors pattern of your site. The pattern automatically changes as per the time frame you selected.

Bingbot Crawl Control - Preset
Bingbot Crawl Control – Preset

Save your settings after you selected the suitable crawl control pattern.

Setting Custom Crawl Pattern

If the preset time frames are not suiting your requirement then select the “Custom” option available in the drop down to set your custom crawl control pattern. Hold the left button of your mouse and move over the blue color bricks to build your own crawl pattern as shown in the below picture:

Bingbot Crawl Control - Custom
Bingbot Crawl Control – Custom

By default 5 blue bricks are selected for anytime and more the blue bricks faster the crawling speed and lesser the blue bricks slower the crawling speed of Bingbot. Save your settings after you selected the suitable crawl control pattern.

You also can set a crawl delay in your robots.txt file. The crawl rate set in your Bing webmaster tools will be superseded with the information available in your robots.txt.

It is important that the webmasters should first ensure the site slowness is due to Bingbot activity or not. Use verify Bingbot tool to check the IP address shown in your server log file belongs to Bingbot and then set the crawl rate accordingly based on the Bingbot crawl pattern and visitors pattern.

Editorial Staff at WebNots are team members who love to build websites and share the learning with webmasters community. Follow us on Facebook and Twitter.

1 thought on “Bingbot Crawl Control Settings in Bing Webmaster Tools”

Leave your response.

  1. Hey,

    I was looking for this solution and I did the changes, waiting for the crawl rate changes.

    I have couple of questions, Does the crawl rate change happen asap? or Do we get a message like we get on the google webmaster?

    Let me know
    Thanks
    Suresh

    Reply

Leave a Comment