pgmoneta has had its incremental backup support for PostgreSQL 17+ since v0.16. We have been improving the workflow ever since and it has become quite stable and efficient (shoot us bug reports if you find anything!). But we found that not many people know about our cool features, I hope this blog may intrigue you to try it out, leave a star and perhaps even incorporate it into your DBA workflow.
Intro
pgmoneta receives incremental backups following postgres’ streaming replication protocal. And it combines and restores incremental backups following how postgres does that in pg_combinebackup, we have our own native implementation so you don’t need a postgres binary to combine backups. And it is WAY MORE THAN THAT. Here are some more cool features that PostgreSQL doesn’t support.
Hassle-free incremental backup restoration with pgmoneta
If you have ever used pg_combinebackup, you’ll know that it’s a hassle. You have to input all the backup locations in the chain in the exact order. pgmoneta, on the other hand, keeps track of each backup’s parent for you. As a result, to restore an incremental backup is just the same as restoring a full backup. All you need is to take the incremental backup id and do:
pgmoneta-cli restore primary <incr_or_full_backup_id | newest | oldest | latest>
Delete an incremental backup with pgmoneta
And it is more than that, pgmoneta is also capable of squashing incremental backups, i.e. combining incremental backups into a new incremental backup. That opens the door for it to do even cooler things. Have you thought about what happens if you delete an incremental backup in the middle of the chain? All child backups will be impacted and cannot be restored. But as time goes by, you almost inevitably accumulate a long long chain of backups. How do you manage all that? pgmoneta will save your day – we allow you to delete incremental backup in the middle of the chain, you can even delete the root full backup without damaging any of the backups in the chain.
How does it do that? It rolls changes upwards, to the child backup, essentially combining the child backup and the backup to delete into a new incremental or full backup. So the backup is gone, but all the changes preserved in the next backup! As a result, deleting an incremental backup is as easy as deleting a full backup, no strings attached:
pgmoneta-cli delete primary <incr_or_full_backup_id | newest | oldest | latest>
Just do it whenever you want on whatever you want.
File-by-file backup combination
Another cool improvement I’m proud of is the file-by-file combination. Backup files in pgmoneta are encrypted and compressed, so we cannot combine in place. Previously the chain is at backup level, we decrypt and decompress the entire backup chain to a workspace, combine it and write back the combined full backup, it was bulky and slow, and takes a lot of disk space. Now we identify the chain at file level, and only decrypt and decompress the chain of files to the workspace, combine the files and write back, delete the chain and move on to the next file. As a result, we have a significant lower disk space overhead, and we are now able to parallelize the combination process using our worker pool, with multiple workers combining different files at the same time, the restore process is drastically accelerated.
Native incremental backup support for PostgreSQL 14 - 16
Since 0.19, we have also added our own WAL summarizer support (yes, we natively support WAL parsing and even filtering, check out our demos and stay tuned, as more are on the way), which enables us to support incremental backup for PostgreSQL 14 - 16 as well, which isn’t natively supported by these older versions. It is not perfect, we are yet to make it stabler and faster, and it does not support tablespace for now. But we are working towards it.
Try and star!
If you are intrigued by what pgmoneta is capable of, please check out our official repo, and consider giving it a star! We are a small community, but we strive to become one of the go-to choices for PostgreSQL backup/restore. And your stars matter to us more than you can imagine. Feel free to leave us feature request and bug reports. We welcome your input!