dstack introduced Fleets, a new way to manage cloud and on-prem clusters

The AI-focused open-source container orchestration engine dstack.ai recently introduced Fleets, a feature available in version 0.18.7 and onwards, that enables developers to equip and manage cloud or on-prem clusters and instances. As dstack founder and CEO Andrey Cheptsov describes Fleets, "It allows AI engineers to define clusters as code and provision them with a single command. A provisioned fleet can be easily used for running dev environments, tasks, and services. No Ops involvement is required."

Configuring or creating a fleet is remarkably easy: first, a YAML file with a name ending in .dstack.yml or fleet.dstack.yml should be added to the project folder. The basic configuration differs depending on whether the fleet is intended for cloud or on-prem use. Cloud-provisioned fleets using the configured backends require specifying the required resources, number of nodes, and other optional parameters. Likewise, fleets equipped from on-prem servers need a configuration file that includes their hosts, user, port, and SSH key for connection via SSH. On-prem server provision requires CUDA 12.1 and NVIDIA Docker pre-installed, in addition to sudo access for the user.

Additionally, setting the "placement" setting to "cluster" entails the user wanting the nodes to be interconnected (for multi-node tasks). This configuration will automatically equip all nodes in the same backend and region in cloud provisionings. In on-prem contexts, dstack will detect the private network automatically, although it can also be set manually. Fleets are created or updated by calling the dstack apply command once the YAML file is in place. Fleets are ready to use once their status is displayed as 'idle'.

Creation and termination policies can be established to modify dstack's behavior. The dstack run command attempts to create a new fleet when it cannot find a suitable idle fleet. This behavior can be stopped by passing --reuse to the dstack run command. Similarly, setting the termination_idle_time property automatically deletes fleets that have been idle for the specified time. More details are available in the official documentation, and the .dstack.yml reference file, which lists all supported configurations.