Restic backups to S3: init, snapshots, prune and restore

restic is the best general-purpose tool for backing a server up to S3 storage. Each run uploads only the chunks that changed since the last one, everything is encrypted before it leaves the machine, every run is a snapshot you can restore on its own, and retention is one command - restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune. Setting it up is four environment variables and restic init. The parts that need thought are the password, which nobody can recover for you, the way you feed it database dumps, and the habit of restoring something occasionally to prove the whole chain works. This post covers all of it against any S3-compatible endpoint.

How restic stores a backup#

Understanding the repository format explains restic's behaviour - why the second backup is fast, why forget frees no space, and why the password matters so much.

• Chunks. restic splits files into variable-size chunks using content-defined chunking, so inserting bytes in the middle of a file only changes the chunks around the insertion. Each chunk is identified by its SHA-256 hash.
• Deduplication. A chunk already in the repository is never uploaded again - from the same file, another file, another snapshot or another machine backing up to the same repository.
• Packs. Chunks are bundled into pack files, which are the objects you see in the bucket under data/. Indexes in index/ record which chunk lives in which pack.
• Snapshots. A snapshot in snapshots/ is a small record pointing to a tree of directories and files, which point to chunks. Every snapshot is a complete backup you can restore, even though it shares almost all its data with its neighbours.
• Encryption. Everything - data, file names, indexes, snapshots - is encrypted with AES-256 and authenticated with Poly1305 before upload. The storage provider sees opaque objects.
• Compression. Repository format version 2, the default since restic 0.14, compresses data with zstd before encrypting it.

The repository's master key is encrypted with your password. There is no recovery. Lose the password and the repository is unreadable, by you or by anybody else. That is the point of the design and the reason the first section below is about where the password lives.

Setting up the repository#

Install restic from your distribution if it is recent, or download the binary from the project's GitHub releases; restic version should show 0.17 or newer for everything in this post, and restic self-update upgrades an official binary in place.

Keep the settings in a file only root can read:

export AWS_ACCESS_KEY_ID=RNAKEXAMPLE123export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEYexport AWS_DEFAULT_REGION=us-east-1export RESTIC_REPOSITORY=s3:https://s3.example.com/my-bucket/restic/web-01export RESTIC_PASSWORD_FILE=/root/.restic-password

$ chmod 600 /root/.restic-env /root/.restic-password$ . /root/.restic-env$ restic -o s3.bucket-lookup=path initcreated restic repository 2f1c9e7a8b at s3:https://s3.example.com/my-bucket/restic/web-01

The pieces:

• `RESTIC_REPOSITORY` is s3: followed by the endpoint URL, the bucket and an optional prefix. A prefix per machine (restic/web-01) keeps repositories apart in one bucket. For a plain-HTTP endpoint, write s3:http://host:port/bucket/prefix.
• `-o s3.bucket-lookup=path` forces path-style addressing. restic's default, auto, already uses path-style for endpoints that are not Amazon's or Google's, but stating it costs nothing and survives a future default change.
• The region comes from AWS_DEFAULT_REGION, or -o s3.region=us-east-1. Any name works against an endpoint that accepts any region, but it must be consistent.
• `RESTIC_PASSWORD_FILE` points to a file holding the repository password. Generate a long random one (openssl rand -base64 32), and store a copy in your password manager now, before the first backup, not after.

On RE:NODE, the access key and secret key are generated for the storage server and shown in the panel, and a first bucket already exists to put the repository in. The endpoint is the plan's address and port over plain HTTP, or a hostname of yours over HTTPS through the plan's proxy slot, which issues and renews the certificate. restic encrypts everything before upload either way, but HTTPS also hides your access key ID and request metadata, so prefer it.

Backing up files#

$ restic backup /srv/minecraft /etc \    --exclude-file /root/restic-excludes.txt \    --exclude-caches --one-file-system \    --tag nightly

/srv/minecraft/logs/srv/minecraft/cache*.tmp**/node_modules

The options in that command:

• --exclude-file reads patterns one per line; --exclude takes one on the command line. --iexclude variants are case-insensitive.
• --exclude-caches skips directories containing a standard CACHEDIR.TAG file. --exclude-if-present .nobackup skips directories containing a marker file you create.
• --one-file-system stays on the file systems of the paths given, so a mounted network share or /proc is not dragged in.
• --tag labels the snapshot; tags can be used to select snapshots in forget and restore.

The first run uploads everything. Later runs read every file's metadata, re-read files whose size or modification time changed, and upload only new chunks - a 30 GB server with a few hundred megabytes of daily change typically finishes in minutes. The summary line at the end shows how much was added; that number is worth logging, because a nightly backup that suddenly adds nothing, or ten times the usual, is your earliest warning that something changed.

Back up consistent data. A game server writing its world during the backup produces a snapshot of a half-written world. Stop the server, or flush and pause saving first, as backups that actually restore explains; for game servers specifically, game server backups to S3 covers the save commands per game.

Backing up databases#

Never back up a running database's data directory with a file tool. Back up a dump. restic can store a dump without a temporary file by reading it from a command:

$ restic backup --stdin-from-command --stdin-filename app.sql --tag mysql \    -- mysqldump --single-transaction --routines --triggers app

--stdin-from-command (restic 0.17 and later) runs the command and stores its output as a file called app.sql in the snapshot. Crucially, if mysqldump exits with an error, restic does not create the snapshot. The older form, mysqldump app | restic backup --stdin --stdin-filename app.sql, cannot see the dump's exit status: a dump that failed halfway is saved as a successful, truncated backup. If you are stuck on an older restic, use set -o pipefail in bash and check the exit status yourself.

Do not compress the dump before giving it to restic. Uncompressed dumps deduplicate well - most of yesterday's rows are still there today - and restic compresses on its own. A gzipped dump changes almost entirely from one day to the next and defeats deduplication.

The same pattern works for pg_dump, mongodump --archive and friends. Credentials for the dump belong in a client option file, such as ~/.my.cnf for MySQL, not on the command line where ps shows them. Database dumps to S3 on a schedule covers each engine, and mysqldump backup and restore the dump flags.

Snapshots, retention, forget and prune#

$ restic snapshotsID        Time                 Host    Tags     Paths----------------------------------------------------------------4f2a1c9e  2026-10-06 04:30:12  web-01  nightly  /etc, /srv/minecrafta81d0b37  2026-10-07 04:30:09  web-01  nightly  /etc, /srv/minecraftc5e9f210  2026-10-08 04:30:11  web-01  nightly  /etc, /srv/minecraft

Retention is two steps. forget removes snapshot records according to a policy; prune removes the data no remaining snapshot references. Until you prune, no space is freed.

Policies combine - a snapshot is kept if any rule keeps it - and they apply separately to each group of snapshots with the same host and paths (--group-by host,paths is the default), so the database snapshots and the file snapshots each keep their own history.

$ restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --dry-run$ restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

Always read the --dry-run output the first time. --keep-tag locked is a useful addition: tag a known-good snapshot locked and no policy will remove it.

Prune is the expensive step. It rewrites packs that are partly unused, which means downloading and re-uploading them, and it needs an exclusive lock, so no backup can run while it does. Its --max-unused option, 5% by default, leaves partly used packs alone until the waste exceeds that share, trading a little space for much less traffic. Run forget nightly if you like, but prune weekly is plenty.

Checking the repository#

restic check verifies the repository's structure: that every snapshot's tree is complete and every referenced chunk exists in a pack listed in the index. It reads metadata only and is quick.

$ restic check$ restic check --read-data-subset=5%$ restic check --read-data-subset=1/12   # a different twelfth each month

Structure checks do not prove the data inside the packs is intact. --read-data downloads and verifies everything, which on a large repository is a lot of traffic; --read-data-subset reads a fraction. A rotating n/12 subset each month reads the whole repository once a year at a twelfth of the cost per run.

Restoring#

The command you are actually doing all this for:

# Everything from the latest snapshot into a scratch directory$ restic restore latest --target /tmp/restore# One folder from a specific snapshot$ restic restore a81d0b37 --target /tmp/restore --include /srv/minecraft/world# A stdin backup, written back out as a file$ restic dump latest /app.sql > /tmp/app.sql# Browse every snapshot as a filesystem (Linux, macOS with FUSE)$ restic mount /mnt/restic

Restore to a scratch directory first, check it, then move it into place - never straight over a live server, which destroys the evidence if you picked the wrong snapshot. latest respects --host, --path and --tag, so on a repository shared by several machines use restic restore latest --host web-01.

To restore onto a new machine, you need exactly three things: the endpoint, the access keys and the repository password. Keep all three somewhere that does not die with the server.

Running it on a schedule#

A wrapper script loaded by cron or a systemd timer keeps the job consistent:

#!/bin/bashset -euo pipefail. /root/.restic-envrestic backup /srv /etc --exclude-file /root/restic-excludes.txt \    --one-file-system --tag nightly --retry-lock 10mrestic backup --stdin-from-command --stdin-filename app.sql --tag mysql \    --retry-lock 10m -- mysqldump --single-transaction apprestic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 \    --keep-tag locked --retry-lock 10mif [ "$(date +%u)" = 7 ]; then    restic prune --retry-lock 30m    restic check --read-data-subset=5%fi

--retry-lock (restic 0.16 and later) waits for another process's lock instead of failing at once. If a run is killed, it can leave a stale lock; restic unlock removes locks whose process no longer exists and is safe to run, while restic unlock --remove-all removes every lock and is only safe when you are certain nothing else is running.

restic keeps a local cache of repository metadata in ~/.cache/restic, which speeds up every operation. It can grow to several gigabytes on large repositories; restic cache --cleanup removes caches of repositories you no longer use. Send the script's output to a log and alert on a non-zero exit - a backup that fails silently every night for a month is the most common way to discover you have no backup.

Where restic on S3 fits in a 3-2-1 plan#

restic makes the copy trustworthy - encrypted, deduplicated, verifiable - but it cannot make the storage behind it more durable than it is. RE:NODE's S3 storage keeps one copy of your data on NVMe in one location, not replicated. That makes a restic repository on it a good off-machine copy: separate from the server it protects, in a format you can restore anywhere restic runs. It does not make it your only copy.

For data you cannot lose, keep a second repository somewhere else and copy snapshots to it:

$ restic -r s3:https://other-storage.example.net/backups/web-01 \    copy --from-repo s3:https://s3.example.com/my-bucket/restic/web-01 \    --from-password-file /root/.restic-password

restic copy transfers snapshots between repositories, re-encrypting with the destination's key, and only sends chunks the destination lacks. Initialise the second repository with init --from-repo ... --copy-chunker-params so that both use the same chunking and deduplication carries across. S3 storage sizing and the 3-2-1 rule works through how much space each copy needs.

FAQ#

What happens if I lose the restic password?

The repository cannot be decrypted, by anyone. You can add several keys with restic key add, each with its own password, so that more than one person or vault can open it - but if every password is lost, so is the data. Store the password in a password manager outside the server.

Why did forget not free any space?

forget only deletes snapshot records. The data they referenced stays until prune removes chunks no snapshot uses. Run restic forget ... --prune, or restic prune afterwards.

Can several servers back up to one repository?

Yes, and they deduplicate against each other, which saves space when servers share files. The cost is shared risk and shared locks: prune locks the repository for every machine. One repository per server, each in its own prefix, is simpler and is what this post assumes.

restic or rclone?

They do different jobs. rclone copies and mirrors files; restic makes versioned, encrypted, deduplicated snapshots with retention. For backups with history use restic; for moving or mirroring files, rclone - see rclone with S3 storage.

How much space will restic use?

Roughly the size of the data once, after compression and deduplication, plus the changed chunks for each kept snapshot. A server with 20 GB of data and 1% daily change, keeping 7 daily, 4 weekly and 6 monthly snapshots, typically needs somewhere between 25 and 40 GB. restic stats --mode raw-data reports the real figure for your repository.