Error due to large terms when rebuilding the xDB search index in Azure Search or Solr


Description

Rebuilding the xDB search index might fail with an error when processing large terms, which are more than 32766 bytes. This happens because Azure Search and Solr providers have a limit on the maximum size of a single term in the search index. Errors similar to the following can be found in the log records:

Failed to repeat processing: key: c315df92-d019-0000-0000-054e908dff7c
msg: Field 'facets_keybehaviorcache_pageevents_data_ss' contains a term that is too large to process. The max length for UTF-8 encoded terms is 32766 bytes. The most likely cause of this error is that filtering, sorting, and/or faceting are enabled on this field, which causes the entire field value to be indexed as a single term. Please avoid the use of these options for large fields.

Solution

To resolve the issue, consider one of the following options: