Search Engine traffic type is not resolved if keywords are not provided in the Referer HTTP header


Description

Sitecore CMS/XP determines traffic types using the Referer HTTP header. There is a predefined set of pairs (a hostname and a query string parameter name) that are used to determine the traffic type. This configuration is located in the Sitecore.Analytics.config file:

<parseReferrer>
   <processor type="Sitecore.Analytics.Pipelines.ParseReferrer.ParseGenericSearchEngine,Sitecore.Analytics">
        <engines hint="raw:AddHostParameterName">
            <engine hostname="www.google" parametername="q"/>
            <engine hostname="search.yahoo" parametername="p"/>
            <engine hostname="www.bing" parametername="q"/>
            <engine hostname="search.lycos" parametername="query"/>
            <engine hostname="www.baidu" parametername="wd"/>
        </engines>
   </processor>
</parseReferrer>

For example, if the Referer header starts with www.google and contains the q query string parameter, the traffic type is determined as "Search Engine - Organic", and the keywords set to the query string parameter value.

However, Google and other search engines do not always provide keywords in the query string of the Referer header. In this case, Google Analytics determines such a request as a Search Engine one and sets the keywords to the (not provided) value. Sitecore Analytics determines such requests as referral ones because the required query string parameter is missing.

Solution

The following patch makes Sitecore Analytics determine the search engine traffic type using only the hostname value if the corresponding query string parameter is missing. In this case, the keywords are set to the (not provided) value.

Download and install the patch compatible with the affected product version found on this page:

https://github.com/SitecoreSupport/Sitecore.Support.96029/releases/tag/8.2.3.2

Contact Sitecore Support if there is no patch for your version.

Additional Search Engine Configuration

Additional search engine configuration can be added to the <engines> section of the Sitecore.Analytics.config file to make the traffic type reports data more accurate. The full list of search engines and their query strings used by Google Analytics can be found here.

For example, the MSN search engine can be added to Sitecore CMS/XP configuration as follows:

<parseReferrer>
   <processor type="Sitecore.Analytics.Pipelines.ParseReferrer.ParseGenericSearchEngine,Sitecore.Analytics">
      <engines hint="raw:AddHostParameterName">
         <engine hostname="www.google" parametername="q"/>
         <engine hostname="search.yahoo" parametername="p"/>
         <engine hostname="www.bing" parametername="q"/>
         <engine hostname="search.lycos" parametername="query"/>
         <engine hostname="www.baidu" parametername="wd"/>
         <engine hostname="www.msn" parametername="q"/>
       </engines>
   </processor>
</parseReferrer>