Encryption

This section is an overview of how ZeroDark.cloud utilizes encryption. For deeper technical details of how our cryptography is implemented, please refer to our cryptography white paper

All data stored in the cloud is encrypted using keys that only your app knows. In other words, the data stored in the cloud can NEVER be read by the ZeroDark team (nor any other party for that matter).

Depending on how you code your app, this could mean that the user is the only person who can decrypt the content. Or perhaps only the user, and other members of their team. Or perhaps the organization has access in order to comply with industry regulations. You control access, and can write code as needed for your situation.

Old-fashioned & outdated systems attempt to control access to data via a gatekeeper. You tell the gatekeeper who has access, and then you trust the gatekeeper to enforce those permissions. ZeroDark is the next evolution of security. Permissions are controlled via modern cryptography. You determine who has access, and the ZeroDark.cloud framework enforces those permissions using state-of-the-art encryption (not fragile gatekeepers).

Public/Private Key Pairs

It starts with public/private key pairs. All users in ZeroDark.cloud generate a public/private key pair. This pair is generated on the client's device, and the client is the only one who knows the private key. The ZeroDark server does NOT know the private keys of users.

Public Keys are verified via Ethereum Smart Contract

Public keys are hosted in the cloud, and can easily be downloaded by anyone.

ZeroDark.cloud uses the Ethereum Blockchain to secure public keys, and protect against the man-in-the-middle attack.

The man-in-the-middle attack (in brief):

If Alice downloads Bob's public key from a server, how can she be sure it's really Bob's public key ? That is, how can she be sure the server hasn't been compromised, and that Bob's public key has actually been replaced by somebody looking to eavesdrop on Bob's conversations ?

ZeroDark.cloud uses an Ethereum Smart Contract to solve this problem. Using the ethereum blockchain, one can now deploy small computer programs that cannot be modified. That is, the source code cannot be modified or upgraded in any way.

(You read that correctly - zero changes can be made to the code once it's deployed to Ethereum. That's why they call it a "contract".)

Once deployed, the smart contract can be interacted with via any ethereum node in the world. So we wrote and deployed a smart contract that:

  • allows the public key information for a user to be stored in the contract
  • allows this information to be written once and ONLY once

This effectively acts as a trusted 3rd party because:

  • The source code doesn't allow the public key information to be tampered with
  • The Ethereum Blockchain doesn't allow the source code to be tampered with

For detailed information about how the smart contract works, see the Ethereum article.


All data in the cloud is encrypted with different random keys

Every file uploaded to the cloud gets encrypted with a different randomly generated key. This is called the file encryption key.

Clients control read-permissions with cryptography

Since all files are encrypted with a different file encryption key, the ability to read a file is controlled by the abililty to access the file encryption key. Clients control this by selectively "wrapping" the file encryption key with the public keys of those who are given read-permission.

The term "wrapping" means to take keyA, and encrypt it using keyB. In this case, the client is taking the file encryption key, and wrapping it using a public key. Therefore, to unwrap the file encryption key will require the corresponding private key.

Here's a real-world example:

{
    "fileID": "E21E04505CD4430EBBBB989E92CE90BF",
    "keys": {
        "UID:z55tqmfr9kix1p1gntotqpwkacpuoyno": {
            "perms": "rws",
            "key": "eyJ2ZXJzaW9uIjoxLCJlbmNvZGluZyI6IkN1cnZlNDE0MTciLCJrZXlJRCI6InlLcVl3U2dzL2drUTRSTVlsME9wYVE9PSIsImtleVN1aXRlIjoiVGhyZWVGaXNoLTUxMiIsIm1hYyI6IjV4ZEZCMEprNjVvPSIsImVuY3J5cHRlZCI6Ik1JSEVCZ2xnaGtnQlpRTUVBZ01FZFRCekF3SUhBQUlCTkFJMEdxSDN3Q3hNdDRWaGp6MS9aaFZrbmk4WGtQWTFUMFp4SlZrZytJZEgra1pBNHl1bFZadER2cC9yUlIvUlRtRUZ0K2JjUEFJMElFenFMeVA5RmpoNXRmZnNxa1AraDNSNlVuZmxpZnVDQVgyU291L0R1eGRncFU0L2RyelNkN0lrZUp3WGV1Z0dXNWhBM1FSQUQrYkhSTFhtbzk3eTB3Y1MrOUtmUnZ5dm55M1MxYUNlZmRzNWk4M1lEWE8rbW00WmdpSU1TbDhncnM2andPMVVOajQwZjZ4MW9ZVC9pTStzMUdRc3hRPT0ifQ=="
        }
    },
    ...
}

In this example, only a single user is given permission to read the file. (userID: z55tqm...) So this user will need to use their private key to decrypt the key blob ("eyJ2ZXJ..."), which will reveal the file encryption key. And the file encryption key can then be used to decrypt the corresponding file.

To add additional permissions, the client simply adds another item to the "keys" section. This scales easily for small to medium group sizes. For larger numbers, a groups feature is used, which is discussed elsewhere.


Node names are encrypted too

ZeroDark.cloud provides a treesystem in the cloud. In order to do this, the framework has 2 challenges to solve:

  1. It must map /any/valid/treesystem/path into AWS S3
  2. It must encrypt all node names to maintain zero-knowledge in the cloud

AWS S3 is not a filesystem. It's actually a simple key/value store. So every treepath needs to get mapped to a valid S3 key. But S3 has a limitation on the length of the key. Thus we cannot simply use /the/node/path because this could easily become too long.

So to accomplish both goals, ZeroDark uses a unique but elegant solution. To see how this works, let's walk through an example. Imagine 2 nodes:

  • /foo
  • /foo/bar

Starting with /foo:

  • the object resides in the root directory
  • the cleartext name of the node is "foo", and this information must be encrypted in the cloud (server must not be able to read the name "foo")

The cleartext path (/foo) gets mapped into a cloudpath which becomes the node's location within S3. So the cloudpath of /foo ends up being something like this:

00000000000000000000000000000000/58fidhxeyyfzgp73hgefpr956jaxa6xs.rcrd

The general form is: <diPrefix>/<hashedNodeName>

dirPrefix
  • Every node has a randomly generated dirPrefix
  • These are UUID's (32 characters, hexadecimal, 128 bits of entropy)
  • The root node is the only exception, which is hard-coded to be all zeros.
  • Thus all direct children of node X share the same /dirPrefix/
hashedNodeName
  • Every node has a randomly generated dirSalt (160 bits of entropy)
  • The root node's dirSalt is derived from the private key (using a key derivation function)
  • The cleartext node-name is hashed with the dirSalt of its parent node
  • The hashed result is 160 bits, encoded using zBase32

This means that cleartext names are never leaked. Further, since the dirSalt is different for each parent, two nodes with the exact same name, but with different parents (i.e. in different "directories"), will have completely different hashedNodeName's.

Now let's see what this file actually looks like in the cloud. It's JSON, so it's easy to read:

{
    "fileID": "E21E04505CD4430EBBBB989E92CE90BF",
    "keys": {
        "UID:z55tqmfr9kix1p1gntotqpwkacpuoyno": {
            "perms": "rws",
            "key": "eyJ2ZXJzaW9uIjoxLCJlbmNvZGluZyI6IkN1cnZlNDE0MTciLCJrZXlJRCI6InlLcVl3U2dzL2drUTRSTVlsME9wYVE9PSIsImtleVN1aXRlIjoiVGhyZWVGaXNoLTUxMiIsIm1hYyI6IjV4ZEZCMEprNjVvPSIsImVuY3J5cHRlZCI6Ik1JSEVCZ2xnaGtnQlpRTUVBZ01FZFRCekF3SUhBQUlCTkFJMEdxSDN3Q3hNdDRWaGp6MS9aaFZrbmk4WGtQWTFUMFp4SlZrZytJZEgra1pBNHl1bFZadER2cC9yUlIvUlRtRUZ0K2JjUEFJMElFenFMeVA5RmpoNXRmZnNxa1AraDNSNlVuZmxpZnVDQVgyU291L0R1eGRncFU0L2RyelNkN0lrZUp3WGV1Z0dXNWhBM1FSQUQrYkhSTFhtbzk3eTB3Y1MrOUtmUnZ5dm55M1MxYUNlZmRzNWk4M1lEWE8rbW00WmdpSU1TbDhncnM2andPMVVOajQwZjZ4MW9ZVC9pTStzMUdRc3hRPT0ifQ=="
        }
    },
    "dirPrefix": "F8622C33B26C43C7B7DB3A6B26C60057",
    "version": 3,
    "metadata": "zL9FiZFl5Pbo4NXQkrWanA6yWI0B4QTdYSKISnqAsKfX38Kwd2/KNE0bOTysRk9wXq0XKswWPaxEtv0OVuf4MLaZubQTQPTvCt4k4J1/K9Et1pWuYZjOS7uKvZrwQ9rsVLNh9Ne6lJjeQ5Rny+CrI4HM6pI="
}
fileID
  • Every node has a server-assigned fileID
  • This value is immutable, and is used by the sync system
  • In particular, it helps to facilitate move & rename operations
keys
  • Who has permission to read this node ?
  • Permissions are explained in the more detail here
  • The big key blob (eyJ2ZX....) is: Base64(Wrap(fileKey, pubKey))
  • In other words, the content cannot be decrypted without the file encryption key
  • And the file encryption key cannot be decrypted without the matching private key
dirPrefix
  • As explained above
  • This value is immutable - server enforced
  • Server also protects against dirPrefix collisions
version
  • Just a version number for the format of the RCRD file itself
metadata
  • Contains the filename ("foo")
  • Also contains the dirSalt for /foo, which will be used by all its children
  • This information is encrypted with the file encryption key
  • So the server cannot read it, because the server cannot decrypt it

Moving on to the next node in our example: /foo/bar. This cleartext path gets mapped to the cloudpath:

F8622C33B26C43C7B7DB3A6B26C60057/91b1pjdm9g9j9bi5ezh8jofap7ddeztn.rcrd

Here we see that the dirPrefix comes from the parent.

And here's what the file looks like in the cloud:

{
    "fileID": "5D7B0110D320444C8FAFC0B3FCBA40B4",
    "keys": {
        "UID:z55tqmfr9kix1p1gntotqpwkacpuoyno": {
            "perms": "rws",
            "key": "eyJ2ZXJzaW9uIjoxLCJlbmNvZGluZyI6IkN1cnZlNDE0MTciLCJrZXlJRCI6InlLcVl3U2dzL2drUTRSTVlsME9wYVE9PSIsImtleVN1aXRlIjoiVGhyZWVGaXNoLTUxMiIsIm1hYyI6ImJQT3k5TWRIZi9RPSIsImVuY3J5cHRlZCI6Ik1JSEVCZ2xnaGtnQlpRTUVBZ01FZFRCekF3SUhBQUlCTkFJMFBSOHY4a2tZMHdOTmwyR29BVGc5YUp3d2NIUFd1ZWhFSlZ4emVibDdFenJBRzlkc2FhRU85YjZpbE10QlZNOEgyMDVjYUFJME0zMWl2YWhRNE5CazRnS256Ykcwdjc0Q21WZ2dBOTdkVW00UEJXdk1ISzNiaXNZWDlrbmw3SFpLNE9HZEEyMkhXaGE5NlFSQXloVUt3U0taaExSVTlzMlJxbEZ3WTJtMXpCQ05WSzFKWnFIOTlyYjdFYzQ1cjE2Y3IvQk1JdGd0WU5MckFEQW9paEVGcFZBeVEreU5jN1RzcnliWld3PT0ifQ=="
        }
    },
    "version": 3,
    "dirPrefix": "027CB39F3DFB4AE0A173468B8932377C",
    "metadata": "5bDiqt/wcepnSTYtkdXYOd7R77T8FFi2359tuS9PvS2NXjG1IZm03uC/7mDsYK+B6DoYmf+V6qyHNIkiPykCfWZjcwBGNCua6Fc3QK9YX8+2+K+Pu8nOGKAB2c0ZVIyDoUE+s8Fw5U1/QcoRn8IpIunL80Y=",
}

As you can see, this design protects the cleartext names of all nodes. But it also has performance benefits. It becomes easy to move/rename any node in the heirarchy, because doing so doesn't affect the cloudpath of children.

 


Treesystem Access

All data is stored in S3. And every user gets their own bucket.

Only a bucket's owner is allowed to list the contents of their bucket. For example, Alice can list every item in her bucket. However, Bob is NEVER allowed to perform this action on Alice's bucket.

Alice can give Bob access to particular nodes in her treesystem. For example, if Alice chooses to give Bob access to /foo, then Bob would be able to ask the system to list children of that node, which would reveal /foo/bar.

Note that, due to hashing & dirSalt, it's effectively impossible to guess the location of files. For example, imagine I told you the following information:

  • My Bitcoin private key is stored in my bucket
  • The bucket name is com.4th-a.user.z55tqmfr9kix1p1gntotqpwkacpuoyno-e0975e8e
  • It has "lots of money" (i.e. you're super motivated to hack me now)
  • It's in the root folder
  • The cleartext name is bitcoinPrivatekey.txt

Here's what you'd know:

  • the dirPrefix is 00000000000000000000000000000000

But you don't know:

  • the dirSalt
  • the hashedNodeName

Which means there are 160 bits of entropy separating you from the actual S3 keypath. Good luck. Of course, that's not even the end of the story. Even if I handed you the S3 keypath, you would only recieve an encrypted file. The content is encrypted using Threefish-512, with a randomly generated 512 bit key. You'd have to break that too.

ZeroDark.cloud gives you to tools to provide this same level of security & privacy for your customers.


Summary

ZeroDark.cloud was architected from the ground up to provide a usable zero-knowledge sync & messaging system. Designed by veterans in the security world, we've seen too many "solutions" where security was either added in as an after-thought (and thus leaked too much information), or it was architected to be so "secure", it ended up being unusable. In contrast, ZeroDark.cloud servers know the absolute minimum possible to implement a usable system:

  • who owns the file (i.e. who's paying for it)
  • who was write permission (so server can enforce it)
  • who needs to be notified of changes (push notifications are not optional today)

Frequently Asked Questions

How do user's transfer private keys to another device ?

If a users wants to login to the same account on another device, they will need to transfer the private key. ZeroDark.cloud client SDK's make this process super easy, by supplying drop-in user interfaces for cloning accounts via:

  • a scannable QR-code
  • Simply point a camera, and transfer is complete
  • or easy-to-copy / easy-to-type mnemonics
  • The private key is encoded using BIP-39
How do user's backup their private keys ?

ZeroDark.cloud client SDK's make this process easy by supplying drop-in user interfaces for backing up private keys. Three different options are provided:

Social key backup
  • The private key is split into parts via a user selected algorithm (e.g. 2-of-3), such that any combination of X parts out of the original Y can be used to restore the key.
  • The UI facilitates sending these parts to other people (why it's called "social key backup"), either by utilizing ZeroDark.cloud messaging or by exporting the parts.
QR-code
  • An easy-to-save image file for backup
Text-based mnemonics
  • An easy-to-save text-based representation of the file
What happens if a user loses their private key ?

User's can login to their account from multiple devices. If they lose one device, and they're logged in on another, they can easily clone their private key from a logged in device. Further, users are encouraged to backup their private key - and the client SDK provides numerous ways to accomplish this. (See previous question.)

That being said, if the user loses their private key, and all devices that contained their private key, then they've lost access to their data. If this wasn't true, it wouldn't be called zero-knowledge.

As app developers, we realize there's a trade-off here. Stereotypical users want everything. They want a phone that's as thin as possible... and they also want the battery to last 7 days. They want your app to available yesterday... and they also want it to be free of bugs... and infinitely fast... and free... and without ads. You know the drill. And security is no different. Users want infinite privacy and security... and they also want zero responsibility. But the reality is that encryption involves keys, privacy relies on these keys being private, and life requires these keys to be backed up.

ZeroDark.cloud provides the infrastructure to solve these problems in various ways, but YOU determine the tradeoffs that work best for your app. Let's look at some examples:

The enterprise app

Enterprise apps have their own regulations to worry about. Using a zero-knowledge sync system built atop AWS is just good business - it means lower prices, better scalability, and less worries about hackers. It means protecting the business. But zero-knowledge in the cloud doesn't mean zero-knowledge for the enterprise. The client has full control over read permissions. Thus most enterprises will provide read-access for the employee's team, or a corporate key for auditability requirements. And in these environments, the enterprise generally holds the private key for the user.

The privacy app

Let's face it - backing up a private key is a pain in the arse. But the industry has already solved similar pains elsewhere! Logging in used to be a pain too - until the industry started adopting "social login", where you can just click a button that says "Login with FooBar". Click once and done - you're in.

Regardless of your feelings about this trend, you should be able to see the abstraction. The user spends time setting up an account once. And from that point forward, the process of logging in becomes much easier. The same is true in ZeroDark.cloud.

If you take advantage of the ZeroDark identity system, users can create an account in app X, and backup their key ONCE. Then they can later login to your app (app Y) using the same account - no backup needed. Further, the client SDK's automatically facilitate transferring access. The UI allows the user to launch app X and transfer access to your app:

Application Y would like permission to access ....

You'll notice this is similar to social login, and is thus familiar to the user. Plus it simplifies the login for the user, while maintaining privacy. And reduces signup/signin friction when onboarding users for your app.

The custom app

ZeroDark.cloud is flexible. Tools are provided to solve problems related to sync & messaging, but it's understood that things like identity, authentication & key management are not a one-size-fits all approach. Apps are welcome to provide their own identity & authentication system, and also their own key backup approach. Perhaps you have a custom environment that provides an alternative solution? Great. Bring your own solution, and blaze a trail !