Treesystem

ZeroDark.cloud provides a separate treesystem in the cloud for each user.

Treesystem != Filesystem

A traditional filesystem has directories & files. This design forces all content to reside in the leaves. That is, if you think about a traditional filesystem as a tree, you can see that all files are leaves, and all non-leaves are directories.

In contrast, the ZeroDark.cloud treesystem acts as a generic tree, where each item in the tree is simply called a "node". A node can be whatever you want it to be - an object, a file, a container, etc. Additionally, all nodes are allowed to have children.

For example, consider the following treesystem:

       (home)
       /    \
     (A)    (B)
    / | \    |
  (D)(E)(F) (G)

If this were a filesystem, then (A) would have to be a directory. However, in a treesystem (A) can be anything you want it to be. Perhaps (A) is a Recipe object. And (D), (E) & (F) are images of the recipe. Or perhaps (A) is a Conversation object, and (D), (E), & (F) are messages within the conversation. Or maybe (A) is an Album, and (D), (E) & (F) are songs in the album. You get the idea.

Using a treesystem allows you model the tree hierarchy according to the natural relationships between items within your application. Plus it simplifies many operations. For example, if you delete (A) from the cloud, then (D), (E), & (F) are automatically deleted as well.

 

User Treesystem

Every user gets their own treesystem, which comes with a "home" container, and some other special containers. Here's what it looks like:

       (Alice's Treesystem)
       /     /    \      \ 
      /     /      \      \
 (Home) (Prefs) (Outbox) (Inbox)

 

  • Home — This is where your application will store the majority of its data.
  • Prefs — A simple container designed for storing (synced) user preferences.
  • Inbox & Outbox — Containers used by the messaging system of the framework.

 

Treesystem = Cloud Organization

The treesystem is how your application will organize data in the cloud. But it has nothing to do with how your application organizes data locally, on the device.

That is, your application will be continue to be architected as you prefer. Maybe by storing objects in a local database, with various indexes, etc. And these objects are still your own objects. (No subclassing required.) The treesystem comes into play when you want to upload items to the cloud. In other words, it's how you organize data in the cloud, but not how you organize data on the local device.

So your job (as developer) will be to determine the best tree structure for your app. This is a process of optimizing data for the cloud. Things to consider:

  • How to optimize downloads, such that you can minimize the amount of data downloaded (and thus speed up the app's performance)
  • This may include separating things such as images into child nodes that can be downloaded independently.
  • How to exploit natural parent-child relationships in the data, so deleting items from the cloud is easier. (e.g. atomically delete a Recipe and all associated images by deleting a single node)

When you design the treesystem for your app, think about long-time users of your app. Imagine them using your app heavily for several years, and then deciding to upgrade their phone. Then they login to your app on their new phone. How can you exploit the treesystem to minimize the amount of information that must be downloaded? How can you make your app quickly restore its previous state?

 

We'll walk through a few concrete examples so you can get the hang of it.

 

Example #1 - Todo List

A simple example is a todo list. In particular, the ZeroDarkTodo sample app. Here's the app architecture:

At the top level are List's

  • Each list has a name. For example: "Groceries", or "Weekend Chores"
  • The list is simply a container for a bunch of todo items

Within each list are an array of Task's

  • A task has a title. For example: "milk" or "mow the lawn"
  • And some other attributes like: priority, isComplete, lastModified

A task can also have an optional picture.

  • For example, a picture of the particular type of flour we're supposed to purchase at the grocery store.

Now, on the local device, this is architected as you would imagine. Both List's & Task's get stored in the local database. And we have and index to assist with sorting all the List's. And we have other indexes to sort the Task's within each List. And then we want to cache the task images on the local filesystem.

In terms of the cloud, the sample app structures the treesystem like this:

               (home)
              /      \
        (listA)      (listB)
        /   \         /    \
  (task1)  (task2) (task3) (task4)
                      |
                   (photoA)

This simple example helps to highlight the difference between a Treesystem and a Filesystem. A filesystem would force all content into the leaves of the tree. A Treesystem, in contrast, allows content to reside in any node. And all nodes are allowed to have children.

So, when you're thinking about the objects within your application, you can often ask yourself, "Does this object have a parent that owns it?" If the answer is YES, then this generally translates into a simple tree structure with a parent & child node.

There are several advantages to this design. The most obvious is that deleting a parent node will automatically delete all its children (and grandchildren, etc). In this example, deleting a task node will automatically delete its attached photos (if it has any). And deleting a list node will automatically delete all associated task items (and any attached photos).

  • To learn more about pushing data up to the cloud, see the push article.

  • To learn more about pulling data down from the cloud, see the pull article.

 

Example #2 - Collaborative Todo List

Now let's add a feature to our todo list: the ability to share a list (i.e. collaboration). The classic example is the grocery list. Spouses or roommates sharing a grocery list for the house.

This can be solved through the use of grafting:

           (Alice's home)               (Bob's home)
              /      \                  /          \
        (listA)      (listB)<-----(listC*)        (listD)
        /   \         /    \                       /   \
  (task1)  (task2) (task3) (task4)           (task5)    (task6)
                      |
                   (photoA)

This is similar to a link within a filesystem. A graft can be visualized as a pointer to another node, within a different treesystem.

In other words, Alice is sharing a list with Bob. Technically, she could give Bob read-only permission to view the list (without the ability to modify it). But in the context of this app, she's giving Bob read-write permission so that he can:

  • add new tasks to the list
  • delete tasks from the list
  • or modify tasks (e.g. mark them as completed)

Permissions work similar to standard unix permissions:

  • Since Bob has write permission on listB, he can delete task3
  • Bob can also delete listC (which deletes his pointer, but not listB in Alice's treesystem)
  • Bob cannot delete listB (because he doesn't have write permission on it's parent node - Alice's home node)

In other words, Bob can remove the shared list from his app, but he doesn't have permission to delete the list from Alice's app.

So pointers are pretty sweet. But you might be wondering: How does Alice's app tell Bob's app about the shared list ? And that's where messaging comes in.

Alice simply has to send a message to Bob. And a message can be whatever you want it to be. So for this app, we might come up with a custom JSON structure that looks something like this:

{
  "listID": "abc123",
  "listName": "Groceries",
  "message": "Hi Bob, let's share a grocery list. We've got a lot to buy for the upcoming holidays!!!" 
}

The message will be delivered to Bob's msgs folder:

               (Bob)
               /   \
          (msgs)    (home)
          /          /   \
(msg from Alice) (listC*) (listD)

And when the message arrives in Bob's msgs folder in the cloud, the server will send out push notifications to all of Bob's devices. And eventually your app (Bob) receives the message. At that point your app can do whatever you want. For the ZeroDarkTodo sample app, we prompt Bob:

'Alice' would like to share a list with you: "Hi Bob, let's share a grocery list. We've got a lot to buy for the upcoming holidays!!!" (Accept) (Reject)

If Bob accepts the invitation, then all your app has to do is create the pointer node (listC). Once the pointer node is created, the ZeroDark framework will automatically follow the link, and start updating the filesystem so that Bob can see all of the existing todo items in the list.

 

Treesystem Names

  • All node names are encoded in UTF-8
  • There are no restricted characters
  • There is no limit on the number of characters in a name
  • There is no limit on the depth of a tree, or length of a pathname

The ZeroDark framework has a custom class for treepath's. Internally, it stores the path as an array, where each item in the array is a path component. This means there is no such thing as a separator character. In fact, there are no illegal characters at all. If it's valid UTF-8, it's fine.

With that being said, it's still really convenient to display paths the same way programmers have been seeing them their entire life. So throughout the documentation you'll still see path names written in a traditional style. Just remember, in the back of your mind, that it doesn't come with the usual set of baggage.

The one thing to be aware of is that the treesystem is case insensitive. All of the following are considered to be equal:

  • /foo/bar is the same as:
  • /FOO/BAR is the same as:
  • /fOo/BaR

This choice of case-insensitivity is curious to some developers. So its important to point out two things. First is that this is just the name of the cloud representation. And oftentimes this has little to do with your app's content. For example, a messaging application may represent a message using the following treepath:

  • /{conversation_uuid}/{message_uuid}

The content of the message is obviously completely different: "Don't forget to pickup eggs on your way home."

Second, a case-insensitive system turns out to be the more pragmatic of the two because:

  • It's easy to map a case-sensitive system atop a case-insensitive system
  • However it's more difficult to map the inverse in practice

Mapping a case-sensitive system atop a case-insenstive system is as easy as using an escape sequence for all upper-case letters:

  • "Bob Johnson" => "\bob \johnson"

Such a technique makes it easy to map without any loss of information. Further, since there are no length restrictions for node names, this becomes a simple search & replace function.

The inverse is more involved: attempting to map a case-insensitve name onto a case-sensitive system. The solution generally involves converting everything to lowercase. However this results in a loss of case-sensitive information. Which necessitates storing the properly cased name elsewhere, and thus requires additional overhead within your codebase.

We realize its impossible to make everyone happy. However, if your app requires a case-sensitive system, the Treepath class contains utility functions that will perform the above mapping for you.

 

Permissions

Permissions in the ZeroDark treesystem are more fine-grained than an ordinary filesystem. So instead of the unix-like {owner / group / everybody} setup, you instead explicitly set permissions for every user that you wish to grant access.

Also, there are some unique permissions that are tailor-made for the cloud, designed to facilitate collaboration.

Read

If a user has this flag, the server assumes the user knows how to decrypt the content, and will send push notifications to the user when the file is created, modified or deleted.

If the node has children, the read permission also means the user is allowed to list the children. For example, consider the following treesystem:

         (alice: home)
          /        \
      (foo)       (bar)
      /   \        /   \
  (dog)  (cat)  (cow) (duck)
                         |
                       (quack)

If Alice gives Bob read permission for the (bar) node, then Bob will be able to fetch a list of the (bar)'s children: (cow) & (duck). However, without read permission for (foo), Bob won't be able to see (dog) or (cat). Similarly, Bob will only be able to see (quack) if he has read permission for (duck).

So only the owner of a bucket is allowed to list all items within that bucket. All other users must be given read permission for each node in which they're allowed to list the direct children.

Write

If the user has this flag, they are are allowed to modify the content of the node. Keep in mind that this permission (by itself) does not allow the user to modify the permissions set of a node - only the content.

If the node has children, the write permission also means the user is allowed to modify existing children, delete existing children, or add new children.

Share

The share permission means the user is allowed to modify the node's set of permissions. This means they can add someone to the list, remove someone, or even modify existing permissions. However, the bucket owner's permissions can never be modified. That is, if the node exists in Alice's bucket, and Bob has share permission on the node, Bob cannot remove Alice from the list of permissions, nor can he modify her permissions.

Leafs Only

All children of the node are restricted to leafs. That is, the children cannot have their own children. Using this permission is a way of preventing abuse. A user's inbox & messages containers use this permission.

Users Only

Users with this permission are allowed to create/modify a single child node, whose name matches their userID. For example, if Alice's userID is z55tqmfr9kix1p1gntotqpwkacpuoyno, then she will be allowed to create/modify a child node named z55tqmfr9kix1p1gntotqpwkacpuoyno.

Write Once

Users with this permission are allowed to create child nodes. However, the nodes are considered "write once", in that the user can create them, but doesn't have permission to modify them afterwards.

A user's messages folder utilizes this flag. For example, Alice is allowed to write a message into Bob's messages folder, but Alice doesn't have permission to modify that message afterwards.

Burn

If a user has this permission, they're allowed to delete the node. In other words, they don't require write permission on the parent node, just the burn permission on the node to be deleted.