High load on Solr server when numerous items are deleted


Description

Removal of numerous items or item versions might result in a high number of requests to the Solr server. This might happen even when the batch mode is enabled. A surge in the number of requests might affect the Solr server performance leading to errors similar to the following:

org.apache.solr.client.solrj.SolrServerException: Max requests queued per destination 3000 exceeded for HttpDestination[http://...]
java.io.IOException: Request processing has stalled for 119004ms with 100 remaining elements in the queue.

Solution

To resolve the issue, consider the following options:

Notes

If your solution is configured to use a separate Solr core/collection for each Sitecore search index (that is the default behavior), then further optimization can be achieved. You can disable the ContentSearch.Solr.RestrictDeleteByIndexName setting by creating a configuration patch file in the \App_Config\Include\zzz folder:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
  <sitecore search:require="solr">
    <settings>
      <setting name="ContentSearch.Solr.RestrictDeleteByIndexName" value="false" /> 
    </settings>
  </sitecore>
</configuration>

For additional recommendations on how to reduce the load on the Solr server, see this article.