How to take consistent backups with LVM snapshots

Why a snapshot makes the backup consistent

tar, rsync and restic read files one at a time. A backup that runs from 02:00 to 02:40 can come back as a mix of old and new: uploads that don't match the rows pointing at them, or a file caught halfway through a write.

A copy-on-write (COW) snapshot removes the timing problem. It starts empty. The first time a chunk of the original volume (the origin) is about to be overwritten, LVM copies the old chunk into the snapshot's space, so reading the snapshot returns the origin exactly as it was, however long the backup takes. The result is crash-consistent: what a power cut at that instant would leave, which journaling filesystems such as ext4 and XFS are built to recover from. Databases need one more step, below.

Check for free space in the volume group

sudo vgs

A COW snapshot takes its space from the origin's volume group, shown as VFree. sudo lvs lists the logical volumes; this guide uses www in the group vg0. If VFree is 0, add space first, for example an empty disk with sudo vgextend vg0 /dev/sdb (vgextend initializes it as a physical volume if needed).

Create the snapshot

sudo lvcreate --snapshot --size 10G --name www-backup vg0/www

The size only has to hold the chunks that change while the snapshot exists; rewriting a chunk again costs nothing more. lvcreate's manual says 20% of the origin is often enough; Red Hat suggests 10–15% for quiet volumes and 30% or more for busy ones. A worked example: backing up a 200 GiB volume takes 45 minutes and the application changes about 4 GiB an hour, so about 3 GiB of chunks change during the run. A 10G snapshot (5% of the origin) leaves three times the room needed.

Mount it read-only

sudo mkdir -p /mnt/www-backup

sudo mount -o ro /dev/vg0/www-backup /mnt/www-backup

The snapshot is a block-for-block image, so it has the origin's filesystem UUID. XFS refuses to mount a second filesystem with a UUID that is already mounted; on XFS use -o ro,nouuid. xfs(5) adds that nouuid is often paired with norecovery, which skips log replay and can leave files inaccessible, so add that only if the mount fails without it.

Back up from the snapshot

sudo tar -czf /backup-disk/www-$(date +%F).tar.gz -C /mnt/www-backup .

-C stores paths relative to the snapshot. Write the archive anywhere but the origin: every write to the origin, even into free space, first copies the old chunk into the snapshot, so a 30 GB archive on vg0/www uses up to 30 GB of snapshot space. With restic, run as root with the settings from how to back up with restic loaded:

restic backup --tag lvm-www /mnt/www-backup

Use the same mount point every run. restic takes the latest snapshot with the same host and paths as its parent and skips files whose path, size, timestamps and inode match; a block-level snapshot keeps the origin's inode numbers, so later runs read only what changed.

Keep the snapshot from filling up

At 100% full, LVM marks the snapshot invalid and reads from it return errors: the backup fails, the origin carries on. Watch Data%:

sudo lvs -o lv_name,origin,lv_size,data_percent vg0

To add room while it is in use:

sudo lvextend --size +5G vg0/www-backup

Or let LVM do it: in the activation section of /etc/lvm/lvm.conf, snapshot_autoextend_threshold = 70 and snapshot_autoextend_percent = 20 grow a snapshot by 20% whenever it passes 70%, while the volume group has free extents. The default, 100, turns it off; values under 50 count as 50. dmeventd does the work, via the lvm2-monitor service, which Red Hat says to restart after the edit. An invalid snapshot shows I as the fifth Attr character in lvs.

Remove the snapshot

sudo umount /mnt/www-backup

sudo lvremove vg0/www-backup

lvremove refuses a volume that is open and asks before removing an active one; -f or -y skips the question in scripts. Remove the snapshot as soon as the backup ends: while it exists, the first write to each chunk of the origin costs an extra copy. Reads from the origin are not slowed.

Databases: lock or checkpoint first

MySQL. The MySQL manual's method: FLUSH TABLES WITH READ LOCK in a client, the snapshot from another shell, UNLOCK TABLES in the first client, then copy files from the snapshot. The mysql client's system command runs a shell command inside the session, so one heredoc does it. Run it as root, connecting as a MySQL user with the RELOAD or FLUSH_TABLES privilege:

mysql <<'SQL'
FLUSH TABLES WITH READ LOCK;
system lvcreate --snapshot --size 10G --name mysql-backup vg0/mysql
UNLOCK TABLES;
SQL

Writes wait only while lvcreate runs. If lvcreate fails, the client still unlocks, so check the snapshot exists (lvs vg0/mysql-backup). The volume must hold the whole data directory, logs included. Since MySQL 8.4.3, --skip-system-command can disable system.

PostgreSQL. Its manual accepts a snapshot of the volume holding the whole data directory, WAL included, taken while the server runs: started from it, PostgreSQL replays the WAL as after a crash, and a CHECKPOINT just before shortens that. If WAL or a tablespace is on another volume, two snapshots are not simultaneous; the manual then says to stop the server while taking them, or to take a base backup with pg_backup_start and pg_backup_stop (point-in-time recovery). Often a pg_dump or mysqldump is simpler.

Thin-provisioned volumes

If the origin is a thin volume (V as the first Attr character), leave the size out: lvcreate makes a thin snapshot that shares blocks in the pool and reserves nothing. It carries an "activation skip" flag, so activate it with -K before mounting:

sudo lvcreate --snapshot --name www-backup vg0/www

sudo lvchange -ay -K vg0/www-backup

Roll a volume back with lvconvert --merge

Merging a snapshot returns the origin to the moment it was taken, which makes a snapshot taken before an upgrade an undo button. Stop what uses the volume, unmount it (and the snapshot), then:

sudo lvconvert --merge vg0/www-backup

With both volumes closed, the merge starts at once and you can mount the origin again: while it runs, reads and writes already see the snapshot's contents, and lvs shows O as the origin's first Attr character. If either was open, the merge waits for the next activation, which for the root filesystem means a reboot. Thin snapshots merge with the same command.

• Everything written to the origin after the snapshot is lost.
• The merged snapshot is removed. Take a new one for another restore point.
• An invalid (overflowed) snapshot cannot bring anything back, and none of this helps if the disk fails.

A backup script with cleanup on failure

This script snapshots vg0/www, backs it up with restic and always removes the snapshot, also when the mount or restic fails:

#!/usr/bin/env bash
# Back up the vg0/www logical volume with restic, from an LVM snapshot.
set -euo pipefail

VG=vg0
LV=www
SNAP="${LV}-backup"
SIZE=10G
MNT="/mnt/${SNAP}"
MOUNT_OPTS=ro            # for XFS use: ro,nouuid

cleanup() {
  set +e
  if mountpoint -q "$MNT"; then umount "$MNT"; fi
  if lvs "$VG/$SNAP" >/dev/null 2>&1; then lvremove -f "$VG/$SNAP"; fi
}

# A snapshot left by a crashed run keeps growing: stop and look at it.
if lvs "$VG/$SNAP" >/dev/null 2>&1; then
  echo "$VG/$SNAP already exists; check it, then remove it with lvremove" >&2
  exit 1
fi

trap cleanup EXIT
lvcreate --snapshot --size "$SIZE" --name "$SNAP" "$VG/$LV"
mkdir -p "$MNT"
mount -o "$MOUNT_OPTS" "/dev/$VG/$SNAP" "$MNT"

set -a; . /etc/restic/env; set +a
restic backup --tag "lvm-$LV" "$MNT"

# A snapshot that filled up is invalid: 5th lv_attr character "I".
ATTR=$(lvs --noheadings -o lv_attr "$VG/$SNAP" | tr -d ' ')
if [ "${ATTR:4:1}" = "I" ]; then
  echo "$VG/$SNAP overflowed during the backup; raise SIZE" >&2
  exit 1
fi
echo "Snapshot used $(lvs --noheadings -o data_percent "$VG/$SNAP" | tr -d ' ')% of $SIZE"

• trap cleanup EXIT runs cleanup however the script ends and keeps the exit status, so a failed backup still exits non-zero. set +e inside makes it try the removal even if the unmount fails.
• The trap is set after the leftover check, so the script never deletes a snapshot it did not create.
• The final check fails the run if the snapshot overflowed while restic read it.

Make it executable (sudo chmod 700 /usr/local/sbin/lvm-snapshot-backup) and run it as root from cron, with PATH set, because cron's default leaves out /usr/sbin and /usr/local/bin:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# m  h  dom mon dow  user  command
30 2  *   *   *    root  flock -n /run/lock/lvm-snapshot-backup.lock /usr/local/sbin/lvm-snapshot-backup >> /var/log/lvm-snapshot-backup.log 2>&1

Scheduling backups with cron explains each part; backup failure alerts covers runs that fail or never start.

Restore and test

Restores come from the backup; the snapshot is gone by then. With restic, --path picks the newest snapshot of this mount point:

restic restore latest --path /mnt/www-backup --target /srv/www-restore

restic keeps full paths, so files land in /srv/www-restore/mnt/www-backup/. A tar archive extracts with tar -xzf www-2026-10-04.tar.gz -C /srv/www-restore. Restore everything onto a spare server now and then: how to test a backup restore.

Common errors

Frequently asked questions

How big should an LVM snapshot be?

Big enough for the data that changes while it exists. lvcreate's manual says 20% of the origin is often enough; for a short backup much less works. Watch Data% and enable autoextend.

What happens when an LVM snapshot is full?

LVM marks it invalid and reads from it fail, so the backup fails. The origin is unaffected. Take a bigger one.

Does an LVM snapshot slow down the server?

Writes, yes: the first write to each chunk of the origin copies the old chunk first. Reads go straight to the disk. Keep snapshots short-lived.

Is an LVM snapshot a backup?

No. It lives on the same disks as the origin. Use it to take a consistent copy and keep that copy elsewhere.