Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serverless Follow-up #11989

Merged
merged 8 commits into from
Oct 25, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions _includes/cockroachcloud/planning-your-serverless-cluster.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
### Request Units

{{ site.data.products.serverless }} cluster resource usage is measured by two metrics: storage and Request Units, or RUs. RUs represent the compute and I/O resources used by a query. All database operations cost a certain amount of RUs depending on the resources used. For example, a "small read" might cost 2 RUs, and a "large read" such as a full table scan with indexes could cost a large number of RUs. You can see how many request units your cluster has used on the [Cluster Overview](#view-cluster-overview) page.
{{ site.data.products.serverless }} cluster resource usage is measured by two metrics: storage and Request Units, or RUs. RUs represent the compute and I/O resources used by a query. All database operations cost a certain amount of RUs depending on the resources used. For example, a "small read" might cost 2 RUs, and a "large read" such as a full table scan with indexes could cost a large number of RUs. You can see how many Request Units your cluster has used on the [Cluster Overview](#view-cluster-overview) page.

### Cluster scaling

{{ site.data.products.serverless }} clusters scale based on your workload. Baseline performance for a Serverless cluster is 100 RUs per second, and any usage above that is called [burst performance](architecture.html#concepts). Clusters start with 10M RUs of free burst capacity each month and earn 100 RUs per second up to a maximum of 250M free RUs per month. Earned RUs can be used immediately or accumulated as burst capacity. If you use all of your burst capacity, your cluster will revert to baseline performance.

You can set your spend limit higher to maintain a high level of performance with larger workloads. If you have a spend limit, your cluster will not be throttled to baseline performance once you use all of your free earned RUs. Instead, it will continue to use burst performance as needed until you reach your spend limit. You will only be charged for the resources you use up to your spend limit. If you reach your spend limit, your cluster will revert to the baseline performance of 100 RUs per second.

For example, you have an application that processes sensor data at the end of the week. Most of the week it handles only occasional read requests, and spends under the 100 RUs per second baseline. At the end of the week the sensors send in their data to the application, requiring a burst over the 100 RUs per second baseline. When the cluster requires more than 100 RUs per second to cover the burst, it first spends the earned RUs from the previous week. If the burst requires more RUs than were earned the previous week, the cluster will spend RUs to cover the burst, up to your maximum spend limit. If you reach your spend limit at any point during the month, the cluster will continue to earn RUs during lower load periods and spend earned RUs during bursts, but after using the earned RUs during bursts it will not scale up beyond the 100 RUs per second baseline.
All [Console Admins](console-access-management.html#console-admin) will receive email alerts when your cluster reaches 75% and 100% of its burst capacity or storage limit. If you set a spend limit, you will also receive alerts at 50%, 75%, and 100% of your spend limit. To see a graph of your monthly resource usage and recommended spend limit, navigate to the [Edit cluster](#edit-your-spend-limit) page.

All [Console Admins](console-access-management.html#console-admin) will receive email alerts when your cluster reaches 50%, 75%, and 100% of its spend limit, burst capacity, or storage limit. To see a graph of your monthly resource usage and recommended spend limit, navigate to the [Edit cluster](#edit-your-spend-limit) page.
### Example

Let's say you have an application that processes sensor data at the end of the week. Most of the week it handles only occasional read requests and uses under the 100 RUs per second baseline. At the end of the week the sensors send in their data to the application, requiring a performance burst over the 100 RUs per second baseline. When the cluster requires more than 100 RUs per second to cover the burst, it first spends the earned RUs that accrued over the previous week and the 10M free burst RUs given to the cluster each month.

If you have a free cluster, it will be throttled to baseline performance once all of the free and earned burst RUs are used. The sensor data will still be processed while the cluster is throttled, but it may take longer to complete the job. If you have a spend limit set, the cluster will be able to scale up and spend RUs to cover the burst, up to your maximum spend limit. If you reach your spend limit at any point during the month, your cluster will be throttled to baseline performance.

If your cluster gets throttled after using all of its burst capacity during the high load period, it will still earn RUs during lower load periods and be able to burst again. At the end of the month, your usage will reset and you will receive another 10M free burst RUs.
6 changes: 6 additions & 0 deletions _includes/v21.1/sidebar-data-reference.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@
"urls": [
"/${VERSION}/architecture/reads-and-writes-overview.html"
]
},
{
"title": "CockroachDB Cloud Architecture",
"urls": [
"/cockroachcloud/architecture.html"
]
}
]
},
Expand Down
6 changes: 6 additions & 0 deletions _includes/v21.2/sidebar-data-reference.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@
"urls": [
"/${VERSION}/architecture/reads-and-writes-overview.html"
]
},
{
"title": "CockroachDB Cloud Architecture",
"urls": [
"/cockroachcloud/architecture.html"
]
}
]
},
Expand Down
2 changes: 1 addition & 1 deletion cockroachcloud/alerts-page.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ toc: true
The **Alerts** page allows you to toggle {{ site.data.products.db }} alerts, send test alerts, and view the email recipients and alert history for your {{ site.data.products.db }} Organization. To view the Alerts page, [log in](https://cockroachlabs.cloud/) and click **Alerts**.

{{site.data.alerts.callout_info}}
The Alerts page is accessible on {{ site.data.products.dedicated }} clusters. For {{ site.data.products.serverless }} cluters, all [Console Admins](console-access-management.html#console-admin) will automatically receive email alerts when your cluster reaches 50%, 75%, and 100% of its spend limit, burst capacity, or storage limit.
The Alerts page is accessible on {{ site.data.products.dedicated }} clusters. For {{ site.data.products.serverless }} cluters, all [Console Admins](console-access-management.html#console-admin) will automatically receive email alerts when your cluster reaches 75% and 100% of its burst capacity or storage limit. If you set a spend limit, you will also receive alerts at 50%, 75%, and 100% of your spend limit.
{{site.data.alerts.end}}

If alerts are enabled, {{ site.data.products.db }} will send alerts to [specified email receipients](#configure-alerts) when the following usage metrics are detected:
Expand Down
20 changes: 14 additions & 6 deletions cockroachcloud/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,20 @@ toc: true
Term | Definition
-----|-----------
**Serverless cluster** | A cluster that’s automatically billed and scaled in response to the resources it consumes (as opposed to a Dedicated cluster, which is billed for resources provisioned).
**Request Unit (RU)** | Request Units represent the compute and I/O resources used by a query. All database operations in {{ site.data.products.serverless-plan }} cost a certain amount of RUs depending on the resources used. For example, a "small read" might cost 2 RUs, and a "large read" such as a full table scan with indexes could cost a large number RUs. You can see how many Request Units your cluster has used on the [**Cluster Overview**](serverless-cluster-management.html#view-cluster-overview) page.
**Request Unit (RU)** | Request Units represent the compute and I/O resources used by a query. All database operations in {{ site.data.products.serverless-plan }} cost a certain amount of RUs depending on the resources used. For example, a "small read" might cost 2 RUs, and a "large read" such as a full table scan with indexes could cost a large number of RUs. You can see how many Request Units your cluster has used on the [Cluster Overview](serverless-cluster-management.html#view-cluster-overview) page.
**Spend limit** | The maximum amount of money you indicate you want to be billed in a particular billing period for a cluster. The actual amount you are billed is based on the resources used during that billing period. A cluster's budget is allocated across storage and burst performance.
**Projected usage** | The amount of usage that Cockroach Labs projects a cluster will consume during a billing period. This is important for setting a cluster’s spend limit, because we always reserve enough budget to pay for storage for the rest of the billing period. You can see your projected usage while [editing your spend limit](serverless-cluster-management.html#edit-your-spend-limit).
**Baseline performance** | The minimum compute and I/O performance that you can expect from your cluster at all times. This is 100 RUs per second for all Serverless clusters (free and paid). The actual usage of a cluster may be lower than the baseline performance depending on application traffic, because not every application will need 100 RU/s at all times. 
**Burst performance** | Burst performance is the ability of the Serverless cluster to perform above the baseline. Supporting application traffic that “bursts,” i.e., can fluctuate above baseline traffic, is a key feature of Serverless clusters. Every Serverless cluster starts with a certain amount of burst performance capacity. If the actual usage of a cluster is lower than the baseline performance, the cluster can earn Request Units that can be used for burst performance for the rest of the month. 
**Storage** | Disk space for permanently storing data over time. All data in {{ site.data.products.serverless }} is automatically replicated three times and distributed across Availability Zones to survive outages. Storage is measured in units of GiB-months, which is the amount of data stored multiplied by how long it was stored. Storing 10 GiB for a month and storing 1 GiB for 10 months are both 10 GiB-months. The storage you see in the [Cluster Overview](serverless-cluster-management.html#view-cluster-overview) page is the amount of data before considering the replication multiplier.
{% comment %}
**Projected usage** | The amount of usage that Cockroach Labs projects a cluster will consume during a billing period. This is important for setting a cluster’s spend limit, because we always reserve enough budget to pay for storage for the rest of the billing period. You can see your projected usage while [editing your spend limit](serverless-cluster-management.html#edit-your-spend-limit).
{% endcomment %}

### Architecture

Traffic comes in from the public internet and is routed by the cloud provider’s load balancer to a Kubernetes (K8s) cluster that hosts CockroachDB. K8s pods allow {{ site.data.products.serverless }} to limit SQL resource consumption for each user. It also minimizes interference between pods that are scheduled on the same machine, giving each user a high-quality experience even when other users are running heavy workloads.
{{ site.data.products.serverless }} has separate compute and storage layers. The storage pods (KV pods) can be shared across users, and the compute pods (SQL pods) are unique to each user. These shared resources make {{ site.data.products.serverless }} architecture "multi-tenant," in contrast to the single-tenant architecture of [{{ site.data.products.dedicated }}](?filters=dedicated#security-and-connection).

Traffic comes in from the public internet and is routed by the cloud provider’s load balancer to a Kubernetes (K8s) cluster that hosts CockroachDB. K8s pods allow {{ site.data.products.serverless }} to limit SQL resource consumption for each user. They also minimize interference between pods that are scheduled on the same machine, giving each user a high-quality experience even when other users are running heavy workloads.

The following diagram is a high-level representation of what a typical Serverless cluster looks like:

Expand Down Expand Up @@ -61,9 +65,11 @@ Storage always gets first priority in the budget since you need to be able to st

Serverless clusters also have the ability to scale to zero and consume no compute resources when there are no active queries. When there are no active queries, you will pay for the storage your app is using, but not for Request Units. To avoid wasted resources, {{ site.data.products.db }} automatically pauses Serverless clusters that are inactive, which is defined by having no connection to the cluster for five consecutive minutes. Once the user attempts to reconnect to the cluster, the cluster will automatically resume. Pausing, resuming, and scaling clusters is a fully-managed process and will not disrupt or affect the user experience. However, it is important for your application to have connection retry logic in the event of node restarts or network disruptions. For more information, see the [Production Checklist](production-checklist.html).

The diagram below shows how {{ site.data.products.serverless }} autoscales in four different scenarios:
The diagrams below shows how {{ site.data.products.serverless }} autoscales with your application's traffic:

<img src="{{ 'images/cockroachcloud/serverless-low-traffic.png' | relative_url }}" alt="Serverless low traffic state" style="max-width:100%" />

<img src="{{ 'images/cockroachcloud/serverless-scaling.png' | relative_url }}" alt="Serverless scaling" style="max-width:100%" />
<img src="{{ 'images/cockroachcloud/serverless-high-traffic.png' | relative_url }}" alt="Serverless scaling" style="max-width:100%" />

</section>

Expand All @@ -79,14 +85,16 @@ We use the Kubernetes offerings in AWS and GCP (EKS and GKE respectively) to run

### Security and Connection

{{ site.data.products.dedicated }} clusters are single-tenant. This means each new cluster gets its own project in GCP or its own account in AWS. No two Dedicated clusters share any resources with each other. Since these clusters are within their own accounts and projects, they are also in a default virtual private cloud (VPC). Users connect to a Dedicated cluster by using a load balancer in front of each region which leads to one connection string per region. Unless you set up [VPC peering](network-authorization.html#vpc-peering) or [AWS PrivateLink](network-authorization.html#aws-privatelink), your cluster will use TLS 1.2 protocol for encrypting inter-node and client-node communication.
{{ site.data.products.dedicated }} clusters are single-tenant. This means each new cluster gets its own project in GCP or its own account in AWS. No two Dedicated clusters share any resources with each other. Since these clusters are within their own accounts and projects, they are also in a default virtual private cloud (VPC). Users connect to a Dedicated cluster by using a load balancer in front of each region which leads to one connection string per region. Unless you set up [VPC peering](network-authorization.html#vpc-peering) or [AWS PrivateLink](network-authorization.html#aws-privatelink), your cluster will use TLS 1.3 protocol for encrypting inter-node and client-node communication.

{{ site.data.products.db }} clusters also use digital certificates for inter-node authentication, [SSL modes](authentication.html#ssl-mode-settings) for node identity verification, and password authentication for client identity verification. See [Authentication](authentication.html) for more details.

[Backups](backups-page.html) are encrypted in S3 and GCS buckets using the cloud provider keys.

### Multi-region architecture

The diagram below shows a high-level representation of a {{ site.data.products.dedicated }} multi-region cluster:

<img src="{{ 'images/cockroachcloud/multiregion-diagram.png' | relative_url }}" alt="Multi-region architecture" style="max-width:100%" />

</section>
Expand Down
Binary file modified images/cockroachcloud/multiregion-diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/cockroachcloud/serverless-low-traffic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified images/cockroachcloud/serverless-scaling.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.