一、系统
[root@barman ~]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@barman ~]# uname -a
Linux barman 2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/hosts
192.168.100.146 barman
192.168.100.147 pg
二、安装
2.1 简介
Barman (backup and recovery manager) is an administration tool for disaster recovery of PostgreSQL servers written in Python. Barman can perform remote backups of multiple servers in business critical environments, and helps DBAs during the recovery phase.
Barman’s most wanted features include: backup catalogues, retention policies, remote recovery, archiving and compression of WAL files and of backups. Barman is written and maintained by PostgreSQL professionals 2ndQuadrant.
Barman的版本更新情况:
2.2 安装依赖包
已安装软件版本检查:
[root@barman ~]# python --version
Python 2.6.6
[root@barman ~]# rpm -qa | grep rsync
rsync-3.0.6-9.el6_4.1.x86_64
下载地址:http://dl.fedoraproject.org/pub/epel/6/x86_64/
[root@barman ~]# rpm -ivh python-argparse-1.2.1-2.el6.noarch.rpm
warning: python-argparse-1.2.1-2.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]
1:python-argparse ########################################### [100%]
[root@barman ~]# rpm -ivh python-argcomplete-0.6.7-2.el6.noarch.rpm
warning: python-argcomplete-0.6.7-2.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]
1:python-argcomplete ########################################### [100%]
[root@barman ~]# rpm -ivh python-argh-0.23.2-1.el6.noarch.rpm
warning: python-argh-0.23.2-1.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]
1:python-argh ########################################### [100%]
[root@barman ~]# yum install python-dateutil python-psycopg2
2.3 安装barman
下载地址:http://sourceforge.net/projects/pgbarman/files/1.3.0/
[root@barman ~]# rpm -ivh barman-1.3.0-1.rhel6.noarch.rpm
Preparing... ########################################### [100%]
1:barman ########################################### [100%]
三、安装配置数据库
3.1 安装
在barman、pg上分别安装pg数据库,安装过程略
在barman上安装PostgreSQL仅为用于客户端,确保barman能够连上需要备份的pg
3.2 创建用户
在barman上创建barman用户并设置密码:
[root@barman ~]# useradd barman
[root@barman ~]# passwd barman
在pg上创建postgres用户并设置密码:
[root@pg ~]# useradd postgres
[root@pg ~]# passwd postgres
配置barman/postgres用户环境变量:
export PATH=/opt/pgsql/bin:$PATH:$HOME/bin
export PGDATA=/opt/pgsql/data
export PGUSER=postgres
export PGPORT=5432
export LD_LIBRARY_PATH=/opt/pgsql/lib:$LD_LIBRARY_PATH
3.3 配置
[postgres@pg data]$ vim postgresql.conf
listen_addresses = '*'
port = 5432
wal_level = archive 若为HOT STANDBY环境则此处设置为hot_standby
archive_mode = on
archive_command = 'rsync -a %p barman@barman:/backup/pg1/incoming/%f'
[postgres@pg data]$ vim pg_hba.conf
添加如下一条
host all all 192.168.100.146/32 trust
四、配置barman
4.1 创建归档目录
[root@barman ~]# mkdir -p /backup/pg1/incoming
[root@barman ~]# chown barman:barman /backup -R
4.2 配置ssh互信
在barman上设置:
[barman@barman ~]$ ssh-keygen -t rsa
[barman@barman ~]$ ssh-copy-id -i .ssh/id_rsa.pub postgres@pg
[barman@barman ~]$ ssh postgres@pg date
Thu Apr 10 02:11:43 CST 2014
在pg上设置:
[postgres@pg ~]$ ssh-keygen -t rsa
[postgres@pg ~]$ ssh-copy-id -i .ssh/id_rsa.pub barman@barman
[postgres@pg ~]$ ssh barman@barman date
Thu Apr 10 02:14:01 CST 2014
配置完成后启动数据库:
[postgres@pg ~]$ pg_ctl start
server starting
测试:
[barman@barman ~]$ psql -c 'SELECT version()' -h pg -U postgres
version
--------------------------------------------------------------------------------------------------------------
PostgreSQL 9.3.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4), 64-bit
(1 row)
4.3 barman.conf设置
(设置/etc/barman.conf或~/.barman.conf)
[root@barman ~]# vi /etc/barman.conf
[barman]
barman_home = /var/lib/barman 备份文件路径
barman_user = barman
log_file = /var/log/barman/barman.log
compression = gzip
[main]
description = "Main PostgreSQL Database"
incoming_wals_directory = /backup/pg1/incoming
ssh_command = ssh postgres@pg
conninfo = host=pg user=postgres
4.4 主要参数说明
immediate_checkpoint
1.3.0加入了该参数,当设置为true时,barman在执行备份时不需要再像之前一样等待PostgreSQL服务器发生一个checkpoint,而是主动触发一次checkpoint。默认设置为false。 该参数也可即设即用,可在执行barman backup时加上--immediate-checkpoint / --no-immediate-checkpoint。
bandwidth_limit / tablespace_bandwidth_limit
用于限制带宽使用,避免备份过多占用带宽资源。默认设置为0,即不限制。bandwidth_limit用作全局带宽限制,也可通过tablespace_bandwidth_limit对每个表空间所用带宽做特定限制。
network_compression = true|false
设定是否启用网络数据压缩。默认为false。
minimum_redundancy
最小冗余设置。当备份集个数减少到该值时,剩下的备份集将禁止删除,避免误删了所有的备份。
例如:当设置为1时,
[barman@barman ~]$ barman list-backup main
main 20140411T015540 - Fri Apr 11 01:55:45 2014 - Size: 19.5 MiB - WAL Size: 82.8 KiB
[barman@barman ~]$ barman delete main 20140411T015540
Skipping delete of backup 20140411T015540 for server main due to minimum redundancy requirements (1)
[barman@barman ~]$ barman list-backup main
main 20140411T015540 - Fri Apr 11 01:55:45 2014 - Size: 19.5 MiB - WAL Size: 82.8 KiB
保留策略
retention_policy:指定备份集的保留策略
wal_retention_policy:指定归档的保留策略
retention_policy_mode:策略的执行模式,目前只有auto
保留策略的执行为自动或手动,目前只实现了自动,手动将在之后版本中提供。自动根据保留策略对无用的备份或归档进行清理是通过barman cron。
五、barman管理
[barman@barman ~]$ barman --help
usage: barman [-h] [-v] [-c CONFIG] [-q] [-f {console}]
{status,list-server,list-files,list-backup,show-backup,cron,rebuild-xlogdb,recover,backup,check,show-server,delete}
...
positional arguments:
{status,list-server,list-files,list-backup,show-backup,cron,rebuild-xlogdb,recover,backup,check,show-server,delete}
cron Run maintenance tasks
list-server List available servers, with useful information
show-server Show all configuration parameters for the specified
servers
status Shows live information and status of the PostgreSQL
server
check Check if the server configuration is working. This
command returns success if every checks pass, or
failure if any of these fails
backup Perform a full backup for the given server
list-backup List available backups for the given server (supports
'all')
show-backup This method shows a single backup information
list-files List all the files for a single backup
recover Recover a server at a given time or xid
delete Delete a backup
rebuild-xlogdb Rebuild the WAL file database guessing it from the
disk content.
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-c CONFIG, --config CONFIG
uses a configuration file (defaults: ~/.barman.conf,
/etc/barman.conf, /etc/barman/barman.conf)
-q, --quiet be quiet (default: False)
-f {console}, --format {console}
output format (default: console)
5.1 检查备份信息
5.1.1 配置检查
[barman@barman ~]$ barman show-server main
Server main:
conninfo: host=pg user=postgres
ssh_command: ssh postgres@pg
retention_policy: None
archive_mode: None
minimum_redundancy: 0
custom_decompression_filter: None
compression: gzip
retention_policy_mode: auto
archive_command: None
basebackups_directory: /var/lib/barman/main/base
data_directory: None
tablespace_bandwidth_limit: None
lock_file: /var/lib/barman/main/main.lock
current_xlog: None
description: Main PostgreSQL Database
bandwidth_limit: None
custom_compression_filter: None
incoming_wals_directory: /backup/pg1/incoming
immediate_checkpoint: False
backup_directory: /var/lib/barman/main
active: True
network_compression: False
post_backup_script: None
wals_directory: /var/lib/barman/main/wals
last_shipped_wal: None
server_txt_version: None
wal_retention_policy: main
pre_backup_script: None
5.1.2 状态检查
[barman@barman ~]$ barman check main
Server main:
ssh: OK
PostgreSQL: OK
archive_mode: OK
archive_command: OK
directories: OK
retention policy settings: OK
compression settings: OK
minimum redundancy requirements: OK (have 0 backups, expected at least 0)
[barman@barman ~]$ barman status main
Server main:
Description: Main PostgreSQL Database
PostgreSQL version: 9.3.4
PostgreSQL Data directory: /opt/pgsql/data
PostgreSQL 'archive_command' setting: rsync -a %p barman@barman:/backup/pg1/incoming/%f
Archive status: last shipped WAL segment 000000010000000000000008
Current WAL segment: 000000010000000000000009
Retention policies: not enforced
No. of available backups: 3
First available backup: 20140410T031606
Last available backup: 20140410T032409
Minimum redundancy requirements: satisfied (3/0)
5.2 查看可监听的数据库服务器
[barman@barman ~]$ barman list-server
main - Main PostgreSQL Database
5.3 查看指定备份集的详细信息
[barman@barman ~]$ barman show-backup main 20140410T031606
Backup 20140410T031606:
Server Name : main
Status : DONE
PostgreSQL Version: 90304
PGDATA directory : /opt/pgsql/data
Base backup information:
Disk usage : 19.5 MiB
Timeline : 1
Begin WAL : 000000010000000000000005
End WAL : 000000010000000000000005
WAL number : 1
Begin time : 2014-04-10 03:16:06.343719
End time : 2014-04-10 03:16:11.085018
Begin Offset : 40
End Offset : 184
Begin XLOG : 0/5000028
End XLOG : 0/50000B8
WAL information:
No of files : 1
Disk usage : 16.0 KiB
Last available : 000000010000000000000006
Catalog information:
Retention Policy: not enforced
Previous Backup : - (this is the oldest base backup)
Next Backup : 20140410T031738
5.4 执行全备
[barman@barman ~]$ barman backup main --若想对多个库同时进行备份,可执行barman backup all
Starting backup for server main in /var/lib/barman/main/base/20140410T031606
Backup start at xlog location: 0/5000028 (000000010000000000000005, 00000028)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup end at xlog location: 0/50000B8 (000000010000000000000005, 000000B8)
Backup completed
基础备份:
[barman@barman ~]$ ls /var/lib/barman/main/base/
20140410T031606
wal备份:
[barman@barman ~]$ ls /var/lib/barman/main/wals/
0000000100000000 xlog.db xlog.db.lock
5.5 查看备份集
[barman@barman ~]$ barman list-backup main
main 20140410T032409 - Thu Apr 10 03:24:13 2014 - Size: 19.5 MiB - WAL Size: 0 B
main 20140410T031738 - Thu Apr 10 03:17:41 2014 - Size: 19.5 MiB - WAL Size: 0 B
main 20140410T031606 - Thu Apr 10 03:16:11 2014 - Size: 19.5 MiB - WAL Size: 16.0 KiB
5.6 恢复
确保用户在指定的恢复路径中有权限创建PGDATA目录
barman recover --help
usage: barman recover [-h] [--target-tli TARGET_TLI]
[--target-time TARGET_TIME] [--target-xid TARGET_XID]
[--target-name TARGET_NAME] [--exclusive]
[--tablespace NAME:LOCATION]
[--remote-ssh-command SSH_COMMAND]
server_name backup_id destination_directory
Recover a server at a given time or xid
positional arguments:
server_name specifies the server name for the command
backup_id specifies the backup ID to recover
destination_directory
the directory where the new server is created
optional arguments:
-h, --help show this help message and exit
--target-tli TARGET_TLI
target timeline (default: None)
--target-time TARGET_TIME
target time. You can use any valid unambiguous
representation. e.g: "YYYY-MM-DD HH:MM:SS.mmm"
(default: None)
--target-xid TARGET_XID
target transaction ID (default: None)
--target-name TARGET_NAME
target name created previously with
pg_create_restore_point() function call (default:
None)
--exclusive set target xid to be non inclusive (default: False)
--tablespace NAME:LOCATION
tablespace relocation rule (default: None)
--remote-ssh-command SSH_COMMAND
This options activates remote recovery, by specifying
the secure shell command to be launched on a remote
host. It is the equivalent of the "ssh_command"
serveroption in the configuration file for remote
recovery. Example: "ssh postgres@db2" (default: None)
5.6.1 本地恢复
[barman@barman ~]$ barman recover main 20140410T032409 /tmp/data
Starting local restore for server main using backup 20140410T032409
Destination directory: /tmp/data
Copying the base backup.
Copying required wal segments.
The archive_command was set to 'false' to prevent data losses.
Your PostgreSQL server has been successfully prepared for recovery!
Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.
5.6.2 远程恢复
[barman@barman ~]$ barman recover --remote-ssh-command "ssh postgres@pg" main 20140410T032409 /tmp/data
Starting remote restore for server main using backup 20140410T032409
Destination directory: /tmp/data
Copying the base backup.
Copying required wal segments.
The archive_command was set to 'false' to prevent data losses.
Your PostgreSQL server has been successfully prepared for recovery!
Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.
5.6.3 迁移表空间
测试未成功
5.6.4 PITR
[barman@barman ~]$ barman backup main
[barman@barman ~]$ barman list-backup main
main 20140410T071548 - Thu Apr 10 07:15:53 2014 - Size: 19.5 MiB - WAL Size: 0 B
postgres=# create table t1(id int);
CREATE TABLE
[barman@barman ~]$ barman backup main
[barman@barman ~]$ barman list-backup main
main 20140410T072226 - Thu Apr 10 07:22:32 2014 - Size: 19.5 MiB - WAL Size: 0 B
main 20140410T071548 - Thu Apr 10 07:15:53 2014 - Size: 19.5 MiB - WAL Size: 0 B
5.6.4.1 target-time
[barman@barman ~]$ barman recover --target-time "2014-04-10 07:16:00" main 20140410T071548 /opt/pgsql/data
Starting local restore for server main using backup 20140410T071548
Destination directory: /opt/pgsql/data
Doing PITR. Recovery target time: '2014-04-10 07:16:00'
Copying the base backup.
Copying required wal segments.
Generating recovery.conf
The archive_command was set to 'false' to prevent data losses.
Your PostgreSQL server has been successfully prepared for recovery!
Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.
[barman@barman ~]$ pg_ctl start
server starting
postgres=# \d
No relations found.
5.6.4.2 target-xid
[barman@barman ~]$ barman backup main
[barman@barman ~]$ barman list-backup main
main 20140411T015540 - Fri Apr 11 01:55:45 2014 - Size: 19.5 MiB - WAL Size: 0 B
postgres=# create table t1(id int);
CREATE TABLE
postgres=# select txid_current();
txid_current
--------------
1824
(1 row)
postgres=# create table t2(id int);
CREATE TABLE
postgres=# select txid_current();
txid_current
--------------
1826
(1 row)
postgres=# create table t3(id int);
CREATE TABLE
postgres=# select txid_current();
txid_current
--------------
1828
(1 row)
postgres=# checkpoint ;
CHECKPOINT
postgres=# select pg_switch_xlog();
pg_switch_xlog
----------------
0/18016390
(1 row)
包含所指事务
恢复:
[barman@barman ~]$ barman recover --target-xid "1825" main 20140411T015540 /opt/pgsql/data
验证:
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------
public | t1 | table | postgres
public | t2 | table | postgres
(2 rows)
{说明包括了1825事务}
不包含所指事务
恢复:
[barman@barman ~]$ barman recover --target-xid "1825" --exclusive main 20140411T015540 /opt/pgsql/data
验证:
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------
public | t1 | table | postgres
(1 row)
5.6.4.3 target-name
接着上一部分执行:
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------
public | t1 | table | postgres
public | t2 | table | postgres
public | t3 | table | postgres
(3 rows)
postgres=# create table t4(id int);
CREATE TABLE
postgres=# select pg_create_restore_point('target1');
pg_create_restore_point
-------------------------
0/19015098
(1 row)
postgres=# create table t5(id int);
CREATE TABLE
postgres=# select pg_create_restore_point('target2');
pg_create_restore_point
-------------------------
0/19016170
(1 row)
postgres=# create table t6(id int);
CREATE TABLE
postgres=# select pg_create_restore_point('target3');
pg_create_restore_point
-------------------------
0/19019160
(1 row)
postgres=# select pg_switch_xlog();
pg_switch_xlog
----------------
0/19019180
(1 row)
恢复:
[barman@barman ~]$ barman recover --target-name "target1" main 20140411T015540 /opt/pgsql/data
[barman@barman pgsql]$ cat data/recovery.conf
restore_command = 'cp barman_xlog/%f %p'
recovery_end_command = 'rm -fr barman_xlog'
recovery_target_name = 'target1'
[barman@barman pgsql]$ pg_ctl start
server starting
验证:
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------
public | t1 | table | postgres
public | t2 | table | postgres
public | t3 | table | postgres
public | t4 | table | postgres
(4 rows)
5.6.4.4 target-tli
用于恢复时间线的设置
5.7 删除备份集
[barman@barman ~]$ barman delete main 20140410T032409
Deleting backup 20140410T032409 for server main
Done
5.8 快捷的备份集ID代替(latest/oldest)
适用:
show-backup | delete | list-files | recover
例如:
barman show-backup main latest
5.9 重新生成归档
[barman@barman ~]$ barman rebuild-xlogdb main
Rebuilding xlogdb for server main
Done rebuilding xlogdb for server main (history: 0, backup_labels: 1, wal_file: 4)
5.10 列举备份文件
[barman@barman ~]$ barman list-files main 20140411T015540
/var/lib/barman/main/base/20140411T015540/backup.info
/var/lib/barman/main/base/20140411T015540/pgdata/pg_hba.conf
/var/lib/barman/main/base/20140411T015540/pgdata/postmaster.opts
/var/lib/barman/main/base/20140411T015540/pgdata/pg_ident.conf
/var/lib/barman/main/base/20140411T015540/pgdata/postgresql.conf
/var/lib/barman/main/base/20140411T015540/pgdata/postgresql.conf.origin
/var/lib/barman/main/base/20140411T015540/pgdata/backup_label.old
/var/lib/barman/main/base/20140411T015540/pgdata/backup_label
/var/lib/barman/main/base/20140411T015540/pgdata/PG_VERSION
/var/lib/barman/main/base/20140411T015540/pgdata/pg_clog/0000
/var/lib/barman/main/base/20140411T015540/pgdata/pg_notify/0000
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12733
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12707_fsm
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12718_fsm
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12702
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12848
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12707
……
同时可指定需要列出的文件类型,如:
[barman@barman ~]$ barman list-files --target data main 20140411T015540
--target 值为data/standalone/wal/full,含义如下:
data: lists just the data files;
standalone: lists the base backup files, including required WAL files;
wal: lists all WAL files from the beginning of the base backup to the start of the following one (or until the end of the log);
full: same as data + wal.
5.11 附加执行脚本
barman支持在备份之前或之后/归档之前或之后附带执行定制脚本,关于该脚本在barman.conf有相关设置:
; Pre/post backup hook scripts
;pre_backup_script = env | grep ^BARMAN
;post_backup_script = env | grep ^BARMAN
; Pre/post archive hook scripts
;pre_archive_script = env | grep ^BARMAN
;post_archive_script = env | grep ^BARMAN
六、总结
barman功能特点总结如下:
使用简单,通过简单的命令就可实现所有功能;
支持本地/远程备份恢复;
能够同时管理多个备份端,并能同时对多个备份端进行操控;
支持在线全备、wal增量备份;
支持多种压缩功能,如:gzip、bzip2及用户自定义压缩工具;
提供灵活多变的备份、归档保留策略;
支持流量压缩,可定义到表空间;
支持迁移表空间(未测);
支持自动维护;
支持PITR。
七、参考文献
官方网站:
http://www.pgbarman.org/
http://2ndquadrant.com/
八、license
Barman is the exclusive property of 2ndQuadrant Italia and its code is distributed under GNU General Public License 3.
Contributions to Barman are welcome, and will be listed in the file AUTHORS.
2ndQuadrant Italia requires that any contributions provide a copyright assignment and a disclaimer of any work-for-hire ownership claims from the employer of the developer.
This lets us make sure that all of the Barman distribution remains free code.
来源:oschina
链接:https://my.oschina.net/u/1011289/blog/223701