Billing
- Part 1 - Overview & explanation
- Part 2 - Details & calculations
Overview
The prices you pay are set by AWS. There are 4 categories of cloud costs:
- Storage (S3)
- Bandwidth
- Push Notifications (SNS)
- Compute (Lambda + API Gateway)
For most applications, the bulk of your costs will come from storage & bandwidth. The other costs tend to be negligible.
We'll walk through each category below and:
- link to the official AWS pricing page
- explain the terms used by AWS
- explain how ZeroDark.cloud uses the service
- provide a blueprint for back-of-the-napkin estimates
Storage
All data is stored in S3, and the official prices are here.
Basics
You get unlimited storage, and you only pay for what you use. At the time of writing, the costs were (for us-west-2):
| Storage Type | Price |
|---|---|
| Standard | $0.023 per GiB |
| Infrequent Access | $0.0125 per GiB |
| Glacier | $0.004 per GiB |
Note that AWS uses the common "GB", but they actually mean "GiB". In other words, they're counting in binary, not decimal. When they say "gigabyte" they actually mean 2^30 bytes. (source) We use the "GiB" designation instead to prevent confusion.
AWS breaks down costs into 2 categories
- the costs for the storage itself (data sitting on hard drives)
- the costs to interact with the storage system (get, put, list, etc)
Storage costs
The numbers you see on their official pricing page are in "gigabyte months", which means 1 GiB stored for an entire month. As a simple example, to help explain the concept:
Imagine the price for a particular region is $0.03 per gigabyte month. And you store exactly 3 gigabytes for 1/3 of a month (before deleting the data). The total cost would be 3 cents (math: 3 gigs * 1/3 month * $0.03 = $0.03). Similarly if you store 1/3 gigabytes for an entire month, the cost is 1 cent (math: 1/3 gigs * 1 month * $0.03 = $0.01).
ZeroDark.cloud makes use of 3 different tiers
- Standard
- Infrequent Access
- Glacier
Most app data starts life in the standard tier. During that time, it's being actively synced among devices, and updated with changes. The server tracks when each file was last accessed, and can automatically transition it to lower priced tiers. By default, if a file's last access time exceeds 30 days, the file is automatically transitioned to Infrequent Access. If the file isn't accessed for another 60 days, it is then automatically transitioned to Glacier.
In addition to automatic transitioning, the server supports automatic burning. So a file can be flagged with a burn date, after which time the server will automatically delete the file.
Requests
The second category is operational costs, represented by the interactions you make with the storage system. The primary operations are:
- GET - whenever a user downloads a file from the cloud
- PUT - whenever a user uploads a file to the cloud
At the time of writing, the costs were (for standard tier on us-west-2)
| Operation | Price |
|---|---|
| GET | $0.0004 per 1,000 requests |
| PUT | $0.005 per 1,000 requests |
All downloads go directly through S3 - no intermediate server is required as read-permissions in ZeroDark.cloud are managed using cryptography. So each download translates into exactly 1 GET request.
For uploads, the ZeroDark server needs to verify write permissions, check eTags, send push notifications, and a bunch of other stuff. So each upload translates into approximately 2 PUT requests.
Estimating costs
You'll need to come up with a storage estimate. This is easier for some apps than others. However, a good rule of thumb is to assume the minimum filesize for every file is 1 KiB. This accounts for various bookkeeping stuff such as permissions & encryption keys.
For estimating operational costs, it's often easier to start by estimating the number of uploads performed. You have a little more leeway with downloads because the client SDK gives you full control over what to download & when. This means you can skip downloading items from the cloud until you need them.
The bulk of your storage costs usually come from the storage itself. The operational costs tend to be negligible. And remember, the best way to estimate your costs is by testing early versions of your app.
Bandwidth
You can find the official pricing within the S3 pricing page here.
(Look for the "data transfer" section. Specifically: "Data Transfer OUT From Amazon S3 To Internet".)
Basics
AWS only charges for download bandwidth. (So you are NOT charged for uploads, such as when uploading node data.)
At the time of writing, the costs were (for us-west-2):
| Type | Price |
|---|---|
| Data transfer OUT to Internet | $0.09 per GB |
Estimating costs
The bulk of your bandwidth costs will come from downloading nodes from S3. Which is heavily dependent on the type of application you're creating. However, the ZeroDark framework gives you a number of tools to decrease this amount. The framework will tell you when something has changed, but you (the developer) have full control over IF and WHEN to perform the download.
Also, the framework supports the ability to include additional metadata & thumbnails alongside the node's data. This allows client applications to download only the thumbnails from the cloud, reducing your bandwidth costs, and speeding up your application too.
Push Notifications
Push notifications utilize AWS SNS, and the official prices are here.
Basics
You get unlimited push notifications, and you only pay for what you use. At the time of writing, the costs were (for us-west-2):
| Operation | Price |
|---|---|
| Publish | $0.50 per 1 million |
| Mobile Push Notifications | $0.50 per 1 million |
Explanation
AWS charges for something called “publishes”, and then they charge per “mobile push notification”. What this means in practice is that every user will have 1 or more devices associated with their account. For example, say Alice has logged in using her phone & an iPad (so 2 devices). When we want to notify Alice that something changed in her account, we use SNS to “publish” a notification to her channel. And since she has 2 devices, this will in turn send 2 different push notifications (1 per device). Therefore, we performed 1 publish, and 2 mobile push notifications.
For each operation that changes a node in the cloud, all users with read permissions for the node are notified. For many apps, this simply means the owner of the data. But for messaging systems, this often means the sender & recipient. Remember that if Alice sends a message to Bob (by uploading a file into her bucket with read permissions for Bob, and read-write permissions for herself), then we need to notify all of Bob's devices, and also all of Alice's devices.
Estimating costs
It may be useful to use the following formula
- Assume N active users
- Assume X operations per user, per month (by operations, we mean changes to the cloud which generate push notifications)
- Assume an average of Z devices per user. A good estimate may exceed 1, such as 1.1.
Plugging in an example cost structure from AWS
- $0.50 per million publishes
- $0.50 per million mobile push notifications
Gives us the following cost estimate:
cost = N * X * (1 + Z) / 1 million * 0.50
Compute
All requests are handled by AWS Lambda, and the official prices are here.
Basics
This is the hardest part to estimate, and it's easy to understand why:
- Storage is familiar. It's common to know how big your hard drive is. Or approximately how big certain files are.
- Bandwidth is related to storage. The amount of bandwidth used is a product of the size of files downloaded.
- And push notifications feel familiar because we've seen them on our phone. And we understand that they are sent in response to actions taken.
But how are we to understand computation? How long does it take to compute something? This concept seems nebulous. But fear not!
The good news is that compute costs are often negligible — the bulk of your costs usually come from storage & bandwidth. But it's also not difficult to understand compute costs. Like push notifications, compute is triggered from actions taken.
Explanation
The ZeroDark server is 100% serverless. Here's what that means:
- We write all the server code, and then hand the code to AWS
- AWS waits until it sees a request on our system that needs to be processed
- When that happens, it runs our code so it can process the request
- Afterwards, AWS makes a note of how long it took our code to run
There are many benefits of serverless design:
- You don't pay fixed costs for servers that aren't being used
- You get massive scalability, handled automatically by AWS
For billing, this process is split into 2 parts
- The request to run the code
- The amount of time the code took to run
At the time of writing, the costs were (for us-west-2):
| AWS Lambda | Price |
|---|---|
| Requests | $0.20 per 1 million |
| Price per 100 milliseconds | $0.000000208 |
Our slim design utilizes the smallest memory size (128 MB). And in terms of compute time, nearly all requests are fulfilled within 100 milliseconds (which is the smallest billable unit of time).
To get an idea of compute costs, we first need to understand when compute services are required.
- Uploading files to the cloud will trigger a lambda operation to process the request
- Additionally, the clients will utilize the REST API for various purposes such as
- requesting a list of changes since their last sync
- searching for other users
- various admin tasks, such as registering for push notifications
When the REST API is used, the system utilizes API Gateway to connect an incoming HTTPS request to the lambda code. And the official prices for API Gateway are here.
Estimating costs
To estimate your compute costs, it's easiest to differentiate between treesystem operations (whenever the user is trying add/modify/delete a node), and REST operations (everything else).
Treesystem Operations work like this:
- the user uploads a request into the staging area of an S3 bucket
-
this triggers the ZeroDark code to process the request (via AWS Lambda)
-
the client uses a REST API to fetch the ZeroDark response
REST Operations work like you'd expect:
- API Gateway receives the incoming HTTPS request
- this triggers the ZeroDark code to process the request (via AWS Lambda)
For estimating your costs, it may be useful to use the following formula
- Assume N active users
- Assume X operations per user per month
- Assume Y REST requests per user per month
Plugging in an example cost structure from AWS:
- $0.0000002 per lambda request
- $0.000000208 per 100 milliseconds of lambda compute
- $3.50 per 1 million API Gateway requests
Would yield an estimate that looks like this:
Lambda Cost = (N * X * 0.0000002) + (N * X * 0.000000208)
API Gateway Cost = (N * X) + (N * Y) / 1 million * $3.50
This is always the hardest to estimate, especially because usage patterns differ so much between apps. However, things become much easier to estimate once you start development because ZeroDark.cloud tracks all this information on a per-user basis. This means you can query the system, and it will report (on a per-user basis):
- number of lambda requests
- lambda CPU time accumulated
- number of API Gateway requests
So as you begin to develop your app, you'll gain a much better understanding concerning your estimated compute costs.
ZeroDark.cloud Fees
An additional fee will be charged according to the following table:
| Application Type | Fee |
|---|---|
| Free & Open Source Software | No Fee |
| Cryptocurrency applications | 1% |
| All other applications | 5% |