How to populate Analytics database with test data


Description

The following article describes how to quickly populate the Sitecore analytics database with the test data.

The approach could be useful for demoing Sitecore xDB/DMS features, verifying the analytics functionality and configuration, etc.

Prerequisites

  1. Java Runtime Environment (JRE) 6 or later
  2. Apache JMeter: http://jmeter.apache.org.
  3. XDB.RandomWalk.JMeter.jmx script. Latest version is available here: https://bitbucket.org/scSPL/jmeter-randomwalk.
  4. Well-defined link structure on your site.
  5. (optional) Properly configured xDB/DMS tools (goals, automations, campaigns, etc.) on your Sitecore solution for the enhanced analytics reports in result.

Note: To generate data on the clean Sitecore solution, the package from http://launchsitecore.net/ could be used. It contains required well-defined link structure and appropriate marketing features enabled.

Solution

Perform the following actions to run the script:

  1. Run Apache JMeter via the \bin\ApacheJMeter.jar file.
  2. Press File > Open and select the XDB.RandomWalk.JMeter.jmx script.
  3. Open the "UDV Global" node and configure the script:

    Make sure that you set up the global.host, global.visitors and sitecore.version variables correctly, before starting the script.
  4. Press the Start button to start the process. Then, JMeter starts browsing your web solution via multiple visitors/threads, each user has his unique random path on the site.
  5. Wait until the script stops working. The indicator will show 0 / "total number of threads" and the green light changes to gray.
  6. After some time all analytics reports will be filled with the data emulating multiple visitors visiting the site and viewing different set of content pages on it.

Script Settings

The following variables are available for configuration in the "UDV Global" node:

  1. global.host – the target solution host, should be specified without "http://"
  2. global.visitors – the number of unique visitors/contacts to create
  3. sitecore.version – must be 7.2 if Sitecore CMS 7.2 or earlier used. You can leave it blank if you use Sitecore XP 7.5 and higher
  4. global.visits.min – the minimum number of visits/interactions to generate per visitor/contact
  5. global.visits.max – the maximum number of visits/interactions to generate per visitor/contact
  6. global.rampuptime – time in seconds needed to start all visitor threads. For example: if there are 10 visitors and the global.rampuptime time value is set to 100 seconds, then each visitor will be initiated 10 seconds after the previous one
  7. global.landingpage – the start page for all visitors
  8. global.pagepervisit.min – the minimum possible number of pages for one visit
  9. global.pagepervisit.max – the maximum possible number of pages for one visit
  10. global.referrers – the list of referrers separated by #. The default value contains google.com, sitecore.net and bing.net referrers
  11. global.useragents – the list of user agents separated by #
  12. file.path – path to the .csv file, where the test result data will be written. The file contains the needed information about every request (page url, response code, response message, etc.)

Note:

In Sitecore XP 7.5+, the visit data is saved into the collection database on the session end event. By default, session state timeout is set to 20 minutes in the web.config file. You can change the setting in the following way to speed up the process:

<sessionState mode="InProc" cookieless="false" timeout="1" ...>

Collecting GeoIp Information While Testing

If you want to collect the GeoIp information while testing, you should perform the following actions:

  1. In your \App_Config\Include\Sitecore.Analytics.Tracking.config (Sitecore.Analytics.config in Sitecore CMS 7.2 and earlier versions) file set the Analytics.ForwardedRequestHttpHeader setting as follows:
    <setting name="Analytics.ForwardedRequestHttpHeader" value="X-Forwarded-For" />
  2. In the JMeter script open the "IP Adress Configuration" node and specify the range for the IP address.

Notes:

Tuning JMeter Performance

While running the tests with many visitors, an out of memory error may appear:

ERROR - jmeter.util.BeanShellInterpreter: Error invoking bsh method: eval Java heap space

The issue can be resolved by increasing the JMeter heap memory:

  1. In the JMeter bin folder, open the jmeter.bat file using Notepad.
  2. Find the "set HEAP=-Xms512m -Xmx512mset" line in the text and change it to "set HEAP=-Xms1024m -Xmx1024mset".
  3. Save the file and run it.