Databricks details GA access controls and beta serverless rate limits

Databricks has outlined its cost-governance options for serverless compute: access controls generally available and rate limits in beta. Workspace administrators can control who may run serverless workloads and limit how far some automated workloads can autoscale, while shared budgets for interactive work remain forthcoming.

The rate-limit rollout is narrower than the announcement’s beta label suggests. Databricks calls the feature a Private Preview, directs customers to contact their account team to join, and then requires a workspace administrator to enable the Serverless Compute Rate Limit preview. The limits apply to jobs and Spark Declarative Pipelines on Lakeflow, not to interactive notebooks or Databricks Connect.

A rate limit is assigned to a custom automated compute object and caps Spark executor autoscaling for each job or pipeline using that object. It is not a fixed allocation or a guaranteed ceiling on the final bill: drivers, REPL virtual machines, GPUs, and some refresh workloads sit outside the cap, and Databricks says actual usage can exceed its directional DBU-per-hour estimates. The AWS documentation lists five sizes, from Small at about 60 DBUs per hour to 2X-Large at about 960 DBUs per hour.

The access-control system uses two default serverless compute objects in each workspace: one for interactive workloads and one for automated workloads. All workspace users receive Can Use permission by default; administrators can remove users or groups and grant Can Use or Can Manage to selected principals. Databricks dates the feature’s general availability to August 3, making the October 29 post a broader cost-controls update rather than the initial GA launch.

Databricks says a future control for interactive workloads will combine a group-shared budget with a per-session maximum. The company has not provided a release date for that capability.