Description

When MongoDB replica sets are used, occasional network issues can lead to rollbacks caused by a replica set failover. You can find a detailed description of this behavior in the MongoDB documentation and mLab blog post. If you have experienced a rollback in one of the MongoDB databases used by Sitecore XP, you can restore the data lost because of the rollback.

Solution

There is no automated way to apply data from rollback files to the database. You must examine the individual documents and compare them to what is in the database to see if any of them need to be merged. The following sections describe how to handle documents from rollback data.

Identifiers collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, compare the values of the updated field of the document in the collection and the document from the rollback data. Overwrite the existing document in the collection with a version of the document from the rollback data if the document from the rollback data has a later date/time in the updated field.

Devices collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, update the existing document to match the version of the document from the rollback data.

Contacts collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, do the following:

• Ignore the changes in the BehaviorProfiles and System fields. These fields will be updated automatically once the interactions of this contact from the rollback data are correctly restored and processed.
• If the contact in the rollback data has an identifier and the contact in the collection does not have an identifier, assign the identifier from the rollback data to the contact. If the contact in the rollback data and the contact in the collection have identifiers, choose the most recent identifier (check the updated field of the document in the Identifiers collection) and assign it to the contact.
  Note: We do not recommend that you set the identifier manually. Use the contact repository (contactRepository node in Sitecore XP configuration) to load the contact, set the identifier, and save the contact.
• Merge tags of the contact (Tags field). Tags are usually updated by custom code, so we cannot provide recommendations on how to merge them.
• Merge custom values and custom facets of the contact. Custom values and custom facets are usually updated by custom code, so we cannot provide recommendations on how to merge them.

OperationStatuses collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, update the existing document to match the version from the rollback data.

Locations collection in the analytics database

If a document with such an _id does not exist in the collection, add a document from the rollback data to the collection.
If a document with such an _id exists in the collection, update the existing document to match the version from the rollback data. If the value of the Classification field of the location was changed, you must call the UpdateVisitorClassification() method of the Sitecore.Analytics.Data.DataAccess.Classification class to update the classification of contacts affected by the change of location classification. Call this method only after you have processed all rollback files and applied the necessary changes to the collections in the databases.

UserAgents collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection and the date in the Fetched field of the document in the collection is less than the date in the Fetched field of the document from the rollback data, update the existing document to match the version from the rollback data. If the document was updated and the value of the Classification field was changed, you must call the UpdateVisitorClassification() method of the Sitecore.Analytics.Data.DataAccess.Classification class to update the classification of contacts affected by the change of the user agent classification. Call this method only after you have processed all rollback files and applied the necessary changes to the collections in databases.

GeoIps collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection and the date in the Fetched field of the document in the collection is less than the date in the Fetched field of the document from the rollback data, update the existing document to match the version from the rollback data. If the document was updated and the value of the Classification field was changed, you must call the UpdateVisitorClassification() method of the Sitecore.Analytics.Data.DataAccess.Classification class to update the classification of the contacts affected by the change of GeoIP classification. Call this method only after you have processed all rollback files and applied the necessary changes to the collections in the databases.

ReferrringSites collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, update the existing document to match the version from the rollback data.

ClassificationMap collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, do not update the existing document. The existing document will be updated after the execution of the UpdateVisitorClassification() method of the Sitecore.Analytics.Data.DataAccess.Classification class if the classification of the related contact is affected by the change of the classification of location, GeoIP, or user agent.

Interactions collection in the analytics database

If a document with such an _id does not exist in the collection, we do not recommend adding this document manually. When a new interaction is added to the system, the system needs to update the data in several other collections too. So, instead of inserting documents manually, we recommend using the Interaction Registry API (Sitecore.Analytics.Tracking.External.InteractionRegistry class) to add new interactions programmatically. If this API is used, the interaction is added, and the data in related collections is also updated correctly.
If a document with such an _id exists in the collection, then most likely the interaction got into the rollback data as a result of a contact merge, because contact merge is the only operation that can edit the existing interaction documents. In this case, it is necessary to change the value of the ContactId field in the collection to match the value of the ContactId field of the document from the rollback data.

KeyBehaviorCache collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, we recommend rebuilding the key behavior cache of the contact instead of manually merging the documents. The key behavior cache of the contact can be rebuilt by calling the Process() method of the Sitecore.Analytics.Processing.Contact.KeyBehaviorCache.RebuildKeyBehaviorCacheWorker class.

AutomationStates collection in the analytics database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection.
If a document with such an _id exists in the collection, then you must perform one of the following actions:

• If the document in the collection does not have a _t field with a value equal to "TransitionDocument", compare the values of the Generation field of the document in the collection and the document from the rollback data. Overwrite the document in the collection with a version of the document from the rollback data if the value of the Generation field of the document from the rollback data is higher than the value of the Generation field of the document in the collection.
• If the document in the collection has a _t field with a value equal to "TransitionDocument", then it is necessary to compare the values of the StateTransitions fields. The StateTransitions field is an array of state transitions. If the document from the rollback data contains more state transitions, missing state transitions must be added to the document in the collection.

AutomationRanges collection in the analytics database

This is a system collection, which is used by the code responsible for processing engagement automation-related data. We do not recommend that you merge the data in this collection manually.

HistoryTasks collection in the analytics database

Documents in this collection are created during the rebuild of the reporting database. Consider rebuilding the reporting database instead of trying to manually restore documents in this collection.

HistoryRanges collection in the analytics database

Documents in this collection are created during the rebuild of the reporting database. Consider rebuilding the reporting database instead of trying to manually restore documents in this collection.

ProcessingPool collection in the tracking_live database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection. Add a new document to the collection only after you have processed all rollback files and applied the necessary changes to the collections in databases.
If a document with such an _id exists in the collection, no actions are required.

TaskQueue collection in the tracking_live database

Instead of trying to manually restore the data in this collection, analyze the documents in the rollback data and rerun the operations that caused the tasks to appear in the TaskQueue collection of the rollback data (changing the classification of locations, moving contacts between states of an engagement plan, and so on).

ProcessingPool collection in the tracking_contact database

If a document with such an _id does not exist in the collection, add the document from the rollback data to the collection. Add a new document to the collection only after you have processed all rollback files and applied the necessary changes to the collections in databases.
If a document with such an _id exists in the collection, no actions are required.

ProcessingPool collection in the tracking_history database

Documents in this collection are created during the rebuild of the reporting database. Consider rebuilding the reporting database instead of trying to manually restore documents in this collection.