Merging Changes

One of the challenges when syncing objects is merging changes. The ZeroDark framework comes with a set of tools to make this surprisingly easy.

 


When & Why

Before we talk about merging changes, it's important to understand when & why merges are needed.

Imagine that we're making a "Business Contacts" application. Our collaborative feature allows teammates to share the same set of contacts. So updates to a contact object get synced across the entire team. Now imagine that Alice & Bob are teammates, and they both decide to modify Carol's information simultaneously. Here's what happens:

  • Alice: Edits Carol's information by updating her business phone number
  • Bob: Adds a note about Carol's business, and a tip ("she loves baseball")

Both of these modifications happen at around the same time. This means that Alice & Bob both started with the same version of Carol's information. Let's call this version 'a'. But Alice's changes brought her to version 'b1', and Bob's changes brought him to version 'b2':

   (a)     <= original version of object
   / \
(b1) (b2)  <= but now: Alice is at 'b1', Bob is at 'b2'

Both Alice & Bob will attempt to upload their changes to the server. Let's assume that Alice wins this race. Meaning that Bob's request will get rejected with a conflict error. There are 2 options to resolve the conflict:

  • Bob's device can discard his changes, OR
  • Bob's device can merge the changes made by Alice

If Bob's device can merge the changes, then the app will work more smoothly, and will contribute to a better user experience. Here's what that merge operation looks like:

   (a)       <= original version of object
   / \
(b1) (b2)    <= but now: Alice is at 'b1', Bob is at 'b2'
 | \   |
 |   (b1&b2) <= Bob downloads 'b1', and merges it with 'b2'
 |  /
(c)          <= Bob pushes the merged version 'c' to the cloud

 

The Challenge

The ZeroDark framework will inform you of these conflicts. You can choose how to react (discard your changes, or perform a merge). If you'd like to perform a merge, the framework makes it easy to download the latest version from the cloud. But after that, you're faced with the merge challenge.

The difficulty stems from the information you have:

  • the latest version of the contact object (Carol), as it exists on Bob's device (b2)
  • the latest version of the contact object (Carol), as it exists in the cloud (b1)

Comparing these 2 objects, you can see there are differences. For example, the 'business phone number' is different. However, it's impossible to merge the changes effectively with only this information. Because we don't know which of the following is true:

  • The local device (Bob) modified the property
  • Some other device (Alice) modified the property
  • Both (Alice & Bob) modified the property

We are missing the required information needed to perform an algorithmically deterministic merge:

  • the list of changes made by the local device (Bob), which haven't been pushed to the cloud yet

 

Storing changesets

The ZeroDark framework helps you out by providing a place to store these changesets. This is integrated with the API you use to push changes to the cloud:

cloudTransaction.queueDataUpload(forNodeID: node.uuid, withChangeset: changeset)
//                                                     ^^^^^^^^^^^^^^^^^^^^^^^^
//                                   We can store our changeset here: ^

And then when you're ready to perform a merge, there's a corresponding API to fetch the list of changesets that are queued, but haven't been pushed to the cloud yet:

let pendingChangesets = cloudTransaction.pendingChangesets(forNodeID: nodeID)

 

Performing the merge

We recommend you use the ZDCSyncable open-source framework. It's a Swift project that automatically tracks changes made to your objects, and supports performing merges too. (There's also an objective-c version here.)

Important: You do NOT have to use ZDCSyncable. You can also build your own system, or use an alternative open-source project. The choice is completely up to you.

If you choose to use ZDCSyncable, here's how easy it is to use:

/**
 * STEP 1:
 * You simply extend the ZDCRecord object
 */

class FooBar: ZDCRecord { // < Just extend ZDCRecord

  @objc dynamic var someString: String? // add your properties
  @objc dynamic var someInt: Int = 0    // and make sure they're '@objc dynamic'

  // you can even use "smart containers" !
  let dict = ZDCDictionary<String, String>()
}

/**
 * STEP 2:
 * Changes you make to your object are now tracked automatically !
 */

let foobar = self.fetchFooBar()
foobar.someInt = 1
foobar.dict["foo"] = "bar"

let changeset = foobuzz.changeset()
^^^^^^^^^^^^^
// The changeset contains a dictionary that includes:
// - which properties were changed
// - what the original values were (prior to change)


/**
 * STEP 3:
 * Store your changeset when you queue uploads
 */

let changeset = foobar.changeset() ?? Dictionary()
cloudTransaction.queueDataUpload(forNodeID: node.uuid, withChangeset: changeset)

/**
 * STEP 4:
 * Simply perform a merge whenever you download the latest version
 * of an object from the cloud.
 */

func processDownload(_ downloadedFoobar: FooBar, forNodeID nodeID: String, withETag eTag: String) {

  let rwConnection = self.rwDatabaseConnection()
  rwConnection.asyncReadWrite() {(transaction) in

    guard
      let node = transaction.object(forKey: nodeID, inCollection: kZDCCollection_Nodes) as? ZDCNode,
      let cloudTransaction = zdc.cloudTransaction(transaction, forLocalUserID: node.localUserID)
    else {
      return
    }

    if var existingFooBar = cloudTransaction.linkedObject(forNodeID: nodeID) as? FooBar {

      // Merging downloaded cloud version with local version
      existingFooBar = existingFooBar.copy() as! FooBar

      // Fetch queued changesets (may or may not be empty)
      let pendingChangesets = cloudTransaction.pendingChangesets(forNodeID: nodeID)

      do {
        // Perform merge
        let _ = try existingFooBar.merge(cloudVersion: downloadedFoobar, pendingChangesets: pendingChangesets)

        // Notify framework that conflicts have been resolved.
        cloudTransaction.didMergeData(withETag: eTag, forNodeID: nodeID)

      } catch () {
        // If merge fails, fallback to using cloud version here
      }

    } else {

      // This is a NEW FooBar object.
      // (Nothing to merge. Just save new FooBar object.)
    }
  }
}

Yup, that's pretty much all there is to it. The ZDCSyncable system does all the heavy lifting for us. And our users get a smooth sync experience that "just works".

To see this code in action, check out the sample project included with the repository: ZeroDarkTodo