Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -578,6 +578,7 @@
"root": "machines/index",
"pages": [
"machines/overview",
"machines/cache-disks",
{
"group": "Machines API",
"root": "machines/api/index",
Expand Down
129 changes: 129 additions & 0 deletions machines/cache-disks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
---
title: Cache disks
description: "Give a Fly Machine fast local scratch storage with a cache disk: what it keeps, when it's wiped, performance, limits, and setup with fly.toml, flyctl, or the Machines API."
---

A cache disk is local disk space attached to a single Fly Machine, for data your app can rebuild if it's lost. Think build caches, downloaded model weights, package caches, or temporary files that are too large for the Machine's root file system.

Fly.io makes a best effort to keep a cache disk's contents across Machine stops, restarts, and updates, but there's no guarantee. Your app should expect to start with an empty cache disk at any time. If you need the data back, store it on a [Fly Volume](/volumes/overview) or in [object storage](/tigris) instead.

## Cache disks, volumes, and the root file system

| | Root file system | Cache disk | Fly Volume |
| --- | --- | --- | --- |
| Survives stop and restart | No | Usually, not guaranteed | Yes |
| Survives [Machine migration](/reference/machine-migration) | No | No | Yes, the volume is forked to the new host |
| Survives Machine destroy | No | No | Yes, until you delete the volume |
| Snapshots and forks | No | No | Yes |
| Created and managed | With the Machine | In the Machine config | Separately, with `fly volumes` |
| IOPS and bandwidth | Image layer: 2,000 IOPS, 8 MiB/s. Writes: same as volumes | [Same as volumes](/volumes/overview#volume-limits) | [By Machine size](/volumes/overview#volume-limits) |
| Max size | Depends on image | 50 GiB | 500GB |

{/* TODO: add a billing row once pricing is set. */}

## What happens to the data

A cache disk belongs to its Machine. It's created when you create or update a Machine with a cache disk config, and it lives on the same host as the Machine.

| Event | Cache disk contents |
| --- | --- |
| Machine stops, starts, or restarts on the same host | Usually kept |
| Machine update that doesn't change the cache disk's size | Usually kept |
| Cache disk `path` changed | Kept |
| Cache disk size changed | Wiped. The disk is destroyed and a new empty one is created |
| [Machine migration](/reference/machine-migration) or recovery on another host | Wiped. A new empty disk is created on the new host |
| Machine cloned | The clone starts with an empty disk |
| Cache disk removed from the config | Disk destroyed |
| Machine destroyed | Disk destroyed |

## Performance

A cache disk has the same IOPS and bandwidth limits as a Fly Volume on the same Machine size. See [Volume limits](/volumes/overview#volume-limits) for the numbers.

The root file system works differently. It has two layers:

- The image layer, from your container image, is read-only and limited to 2,000 IOPS and 8 MiB/s on every Machine size.
- The overlay, where your Machine's writes to the root file system land, has the same limits as a volume.

So reading from a cache disk is faster than reading from the image layer on every Machine size: twice the IOPS and bandwidth on `shared-cpu-1x`, and 16 times on `performance-8x` and larger. If your app reads large files that ship in the image, such as model weights, consider copying them to a cache disk.

## Limits

- One cache disk per Machine.
- Available in all regions.
- Maximum size is 51200 MiB (50 GiB).
- Minimum size is 16 MiB.
- The mount path must be an absolute path other than `/`, and it can't overlap `/dev`, `/proc`, `/sys`, `/.fly`, or a volume mount path on the same Machine.

## Add a cache disk in fly.toml

Add a `[[cache_disk]]` section to your app's `fly.toml`, then run `fly deploy`:

```toml
[[cache_disk]]
size = "1gb"
path = "/cache"
```

- `size`: the disk size, with a unit, such as `512mb` or `1gb`.
- `path`: where to mount the disk in the Machine.
- `processes`: optional. The [process groups](/launch/processes) that get the cache disk. Without it, only Machines in the default process group get one.

Each Machine can have only one cache disk, so a process group can match only one `[[cache_disk]]` entry. To remove the cache disk, delete the section and deploy again.

{/* TODO: add the minimum flyctl version once a release includes flyctl#5469. */}

## Add a cache disk with flyctl

Use `--cache-disk-size` and `--cache-disk-path` with [`fly machine run`](/flyctl/cmd/fly_machine_run), [`fly machine create`](/flyctl/cmd/fly_machine_create), or [`fly machine update`](/flyctl/cmd/fly_machine_update):

```bash
fly machine update <machine-id> --cache-disk-size 1gb --cache-disk-path /cache
```

To resize an existing cache disk, pass only `--cache-disk-size`. To remove it, set `--cache-disk-size 0`.

## Add a cache disk with the Machines API

Set `cache_disk` in the Machine config when you [create](/machines/api/machines-resource#create-a-machine) or [update](/machines/api/machines-resource#update-a-machine) a Machine:

- `size_mb`: the disk size in MiB.
- `path`: where to mount the disk in the Machine. The disk is formatted as ext4 and mounted at this path.

```bash
curl -i -X POST \
-H "Authorization: Bearer ${FLY_API_TOKEN}" -H "Content-Type: application/json" \
"${FLY_API_HOSTNAME}/v1/apps/my-app-name/machines" \
-d '{
"config": {
"image": "registry-1.docker.io/library/ubuntu:latest",
"guest": {
"cpu_kind": "shared",
"cpus": 1,
"memory_mb": 256
},
"cache_disk": {
"size_mb": 1024,
"path": "/cache"
}
}
}'
```

This creates a Machine with a 1 GiB cache disk mounted at `/cache`.

To resize the disk, update the Machine with a new `size_mb`. The old disk and its contents are discarded. To remove the disk, update the Machine config without `cache_disk`.

See the [`fly.MachineCacheDrive`](/api/machines/models#fly-machinecachedrive) reference for the full schema.

{/* TODO: the API reference still says cache_drive; check the schema name and anchor after the next spec refresh. */}

## Pricing

{/* TODO: pricing not decided yet. Add it here and in /about/pricing. */}

## Related

- [Fly Volumes overview](/volumes/overview)
- [Machine migration](/reference/machine-migration)
- [Databases & Storage](/database-storage-guides)
2 changes: 1 addition & 1 deletion volumes/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ source_url: https://fly.io/docs/volumes/overview/

<img src="/images/volumes.png" alt="Illustration by Annie Ruygt of a filing cabinet" />

The root file systems of a Fly Machine are ephemeral and should only be used for temporary data, application code, and runtime files that can be rebuilt from scratch during deployments or restarts. The performance of these ephemeral disks is limited based on machine type and size, the same as [Volumes](/volumes/overview#volume-limits).
The root file systems of a Fly Machine are ephemeral and should only be used for temporary data, application code, and runtime files that can be rebuilt from scratch during deployments or restarts. Reads from the container image are limited to 2,000 IOPS and 8 MiB/s on every Machine size. Writes to the root file system land in an overlay with the same limits as [Volumes](/volumes/overview#volume-limits). For fast scratch space that usually survives restarts, use a [cache disk](/machines/cache-disks).

Fly Volumes are local persistent storage for [Fly Machines](/machines). You can access and write to a volume on a Machine just like a regular directory. Use volumes to store your database files, to save your app's state, such as configuration and session or user data, or for any information that needs to persist after deploy or restart. Maximum volume performance varies by Machine type and noted in [Volume limits](/volumes/overview#volume-limits)

Expand Down
Loading