Rebuilding of web index results in an error on CD server at reindexing if EXM module installed


Description

When trying to rebuild the web index with the EXM module installed on a Sitecore XP instance, the following exception message might appear in the crawling log of a Content Delivery instance:

FATAL Could not add field {52D572AD-9678-423F-96E2-477528418BA1} : global opt-out list for indexable sitecore://web/{...}...
Exception: System.InvalidOperationException
Message: Could not find configuration node: databases/database[@id='master']
Source: Sitecore.Kernel
   at Sitecore.Configuration.DefaultFactory.GetConfigNode(String xpath, Boolean assert)
   at Sitecore.Configuration.DefaultFactory.CreateObject(String configPath, String[] parameters, Boolean assert)
   at Sitecore.Configuration.DefaultFactory.GetDatabase(String name, Boolean assert)
   at Sitecore.Configuration.DefaultFactory.GetDatabase(String name)
   at Sitecore.Web.UI.HtmlControls.Data.LookupSources.GetDatabase(String source)
   at Sitecore.Data.Fields.CustomField.GetDatabase()
   at Sitecore.Data.Fields.ReferenceField.get_Database()
   at Sitecore.Data.Fields.ReferenceField.get_TargetID()
   at Sitecore.ContentSearch.FieldReaders.LookupFieldReader.GetFieldValue(IIndexableDataField indexableField)
   at Sitecore.ContentSearch.FieldReaders.FieldReaderMap.GetFieldValue(IIndexableDataField field)
   at Sitecore.ContentSearch.SolrProvider.SolrDocumentBuilder.AddField(IIndexableDataField field)
   at Sitecore.ContentSearch.AbstractDocumentBuilder`1.CheckAndAddField(IIndexable indexable, IIndexableDataField field)

Solution #1

As this exception does not bring any impact on the environment, you can safely ignore it. In order not to get it in the crawling log file, you can filter out this kind of messages:

In the /App_Config/Sitecore/ContentSearch/Sitecore.ContentSearch.config file, add the following filter:

<appender name="CrawlingLogFileAppender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
  <file value="$(dataFolder)/logs/Crawling.log.{date}.{time}.txt" />
  <appendToFile value="true" />
  <layout type="log4net.Layout.PatternLayout">
    <conversionPattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
  </layout>
  <filter type="log4net.Filter.StringMatchFilter">
    <stringToMatch value="global opt-out list for indexable"/>
    <acceptOnMatch value="false" />
  </filter>
  <encoding value="utf-8" />
</appender>

Solution #2

Download and apply the following configuration file: