A Linux backup is useful only if someone can restore the required workload. An archive of /home may protect user files while omitting service configuration, databases, boot layout, package state, certificates and the instructions needed to put them together.
Define recovery outcomes
List the failures to survive: deleted file, broken configuration, failed disk/host, ransomware, site loss and application/database corruption. Set an acceptable recovery point and recovery time for each important service.
Decide the restore target: same host, replacement hardware, VM/cloud instance or a different distribution release. This determines whether you need file backup, filesystem/block snapshot, application-native backup, system image—or a combination.
Inventory the workload
Record:
- distribution/release, architecture and boot/storage layout;
- mounted filesystems, LVM/ZFS/Btrfs/RAID and encryption;
- users/groups, UID/GID mapping, ACLs and extended attributes;
- configuration under
/etcplus service-specific locations; - application data, databases, queues and logs required for recovery;
- packages/repos, containers/VMs and systemd services/timers;
- certificates, keys and secret-store recovery;
- firewall/network/DNS dependencies; and
- monitoring and backup-agent configuration.
Do not back up volatile pseudo-filesystems such as /proc, /sys and /run as ordinary files. Treat mounted external or network filesystems deliberately so a broad traversal does not copy enormous remote data or miss an intended mount.
Choose consistency for each data type
tar and rsync are capable file tools, but copying a live transactional database may capture mutually inconsistent files. Use the database/application’s supported dump, backup mode, snapshot integration or quiescing process.
Filesystem/LVM/ZFS snapshots can create a consistent point for file copying, but they remain on related storage until exported. A snapshot is not an independent retained backup.
For virtual machines or containers, reconcile host-level backup with guest application consistency and external volumes/bind mounts.
Design destination and retention
Keep more than one recovery point and at least one copy separated from source credentials/failure. Define encryption in transit and at rest, immutable/offline protection where risk warrants it, capacity monitoring and retention deletion.
A mounted NFS/object credential available to root can be destroyed by a compromised root process. Separate write and delete authority where the platform permits it.
Store backup configuration and decryption/repository recovery in an independent controlled location. An encrypted archive without recoverable keys is data loss with better mathematics.
Automate with explicit failure evidence
A backup job should record scope, start/end time, bytes/items, application dump result, repository integrity and retention outcome. Alert on missed schedules as well as failed commands.
Shell scripts need safe quoting, predictable paths, exclusive execution and checked exit statuses. A final echo backup complete does not override an earlier failed pipeline. Avoid embedding secrets in scripts, process arguments or logs.
Test capacity and rotation. Canonical’s examples demonstrate archive rotation, but a sample schedule is not automatically appropriate retention or off-site protection for a business workload.
Restore into isolation
Rehearse on an empty or isolated destination using the documented recovery credentials and media. Restore in dependency order:
- storage/filesystems and encryption access;
- base supported OS and required packages;
- identities/UIDs/GIDs and security context;
- configuration and secrets;
- application-native data;
- services, network and scheduled jobs; and
- monitoring and future backup.
Do not overwrite a live server to “test” restore. Keep recovered services off production networking until hostname, IP, certificates, cluster membership and scheduled actions are safe.
Verify ownership, modes, ACLs, extended attributes, links, sparse files and timestamps where they matter. Open files and run application-level integrity checks; an archive listing alone is weak evidence.
Prove the workload
After restore:
- the system boots or the rebuilt host starts cleanly;
- intended services start in the correct dependency order;
- representative reads and writes succeed;
- database/application consistency checks pass;
- identities and permissions allow intended and deny unintended access;
- network/DNS/TLS behaviour is correct;
- data age meets the recovery point objective;
- measured recovery time meets the objective; and
- the restored system produces a new backup that can be read.
Record gaps as recovery debt. If the restore required undocumented improvisation, update the runbook and repeat the affected test.
The backup is complete only when the organisation can recover the service from its retained evidence without depending on the still-working original machine.
