send and receive ZFS snapshot between machines
背景
zfs的另一个强大的功能, 将snapshot导出到文件, 再将文件恢复成dataset.
导出时支持管道传输, 所以可以很方便的实现跨主机传输.
例子 :
查看A主机的snapshot.
[root@db-172-16-3-150 mnt]# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
zptest/disk1@2014-05-1815:43:51 0 - 48.6M -
zptest/disk1@2014-05-1815:43:54 0 - 48.6M -
zptest/pg93@2014-05-1721:54:55 143M - 185M -
zptest/pg93@2014-05-1723:17:23 99.0M - 3.65G -
zptest/pg93@2014-05-1723:18:11 5.10M - 3.65G -
zptest/pg93@2014-05-1723:35:32 2.49G - 3.65G -
zptest/pg93_clone1@2014-05-1800:31:03 21K - 3.62G -
将snapshot导出到一个文件.
[root@db-172-16-3-150 mnt]# zfs send zptest/pg93@2014-05-1723:18:11 > /ssd4/pg93.img
[root@db-172-16-3-150 mnt]# ll /ssd4/pg93.img
-rw-r--r-- 1 root root 3926704168 May 18 16:42 /ssd4/pg93.img
将snapshot导出到管道, 传送给一个压缩软件, 直接压缩成压缩文件.
[root@db-172-16-3-150 mnt]# zfs send zptest/pg93@2014-05-1723:18:11 | xz > /ssd4/pg93.img.xz
将snapshot导出到管道, 传送给压缩软件, 再通过管道传送给加密软件, 输出压缩后的加密文件.
[root@db-172-16-3-150 mnt]# zfs send zptest/pg93@2014-05-1723:18:11 | xz | openssl enc -aes-256-cbc -a -salt > /ssd4/pg93.img.xz.asc
反过来, 使用压缩过的加密文件, 先解密, 然后解压, 然后管道传送给zfs receive, 使用导出的snapshot创建一个新的dataset.
[root@db-172-16-3-150 ~]# openssl enc -d -aes-256-cbc -a -in /ssd4/pg93.img.xz.asc | unxz | zfs receive zptest/test3
或者直接使用导出的snapshot文件, 导入到zpool并新建一个dataset.
[root@db-172-16-3-150 ~]# zfs receive zptest/test2 < /ssd4/pg93.img
[root@db-172-16-3-150 ~]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zptest 15.3G 23.8G 34K /zptest
zptest/disk1 1.08G 24.8G 48.6M -
zptest/pg93 9.89G 23.8G 3.63G /zptest/pg93
zptest/pg93_clone1 662M 23.8G 3.62G /zptest/pg93_clone1
zptest/test2 3.65G 23.8G 3.65G /zptest/test2
send和receive可以直接通过管道对接, 不用产生临时文件.
[root@db-172-16-3-150 ~]# zfs send zptest/pg93@2014-05-1723:18:11 | zfs receive zptest/test3
[root@db-172-16-3-150 ~]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zptest 18.9G 20.2G 35K /zptest
zptest/disk1 1.08G 21.2G 48.6M -
zptest/pg93 9.89G 20.2G 3.63G /zptest/pg93
zptest/pg93_clone1 662M 20.2G 3.62G /zptest/pg93_clone1
zptest/test2 3.65G 20.2G 3.65G /zptest/test2
zptest/test3 3.65G 20.2G 3.65G /zptest/test3
接下来的这个例子是通过管道传输给另一台主机.
在B主机创建zpool.
[root@spark01 digoal]# dd if=/dev/zero of=./zfs.log1 bs=1k count=1024000
1024000+0 records in
1024000+0 records out
1048576000 bytes (1.0 GB) copied, 2.00667 s, 523 MB/s
[root@spark01 digoal]# dd if=/dev/zero of=./zfs.log bs=1k count=1024000
1024000+0 records in
1024000+0 records out
1048576000 bytes (1.0 GB) copied, 2.04975 s, 512 MB/s
[root@spark01 digoal]# mv zfs.log zfs.log2
[root@spark01 digoal]# dd if=/dev/zero of=./zfs.disk1 bs=1024k count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 1.27549 s, 1.7 GB/s
[root@spark01 digoal]# dd if=/dev/zero of=./zfs.disk2 bs=1024k count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 28.575 s, 75.2 MB/s
[root@spark01 digoal]# cp zfs.disk2 zfs.disk3
[root@spark01 digoal]# cp zfs.disk2 zfs.disk4
[root@spark01 digoal]# zpool create zp /home/digoal/zfs.disk1 /home/digoal/zfs.disk2 /home/digoal/zfs.disk3 /home/digoal/zfs.disk4 log mirror /home/digoal/zfs.log1 /home/digoal/zfs.log2
使用ssh连接B主机接受A主机的管道信息, 在B主机直接使用zfs receive接收snapshot, 并创建一个新的dataset.
[root@db-172-16-3-150 ~]# zfs send zptest/pg93@2014-05-1723:18:11 | ssh root@172.16.3.220 "/opt/zfs0.6.2/sbin/zfs receive zp/test1"
root@172.16.3.220's password:
如果要防止密码传输的话, 可以使用ssh key认证.
[root@db-172-16-3-150 /]# ssh-keygen -t rsa
[root@db-172-16-3-150 ~]# cat ~/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEApn/LgD98MRFTHTW9Unt3fOHmY2k7g/2vVF7KeuRlAu7IpCNpTg+FrdCcICUmsQBIyfC6YaSagWvmuPD+ZRO3poazwtt+3xi+mV8KDEWUPnSRMsqJ9atKzNOmZQhZo0P5yOMwC6gVtObM7bi9JKEgumHkiwvdTxgQVprwvkYTRtPvT84VvXSdADuiBxd/yZlnL4eoPeXODNBuCb5wNRmWcAnkH+mIyspFDWiT0f+ygoSOqZ+Zdy8MFmXIYqSPw9YpHZjUJgpvIH04jsHWASYAJNS4iL8vYVRlzmKZE8GFmXym/OZ9k7xJfFrhzOAVrEiXxYy5mbnTiBVAm+drKmqZDQ== root@db-172-16-3-150.sky-mobi.com
[root@spark01 ~]# cd ~
[root@spark01 ~]# mkdir .ssh
[root@spark01 ~]# vi .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEApn/LgD98MRFTHTW9Unt3fOHmY2k7g/2vVF7KeuRlAu7IpCNpTg+FrdCcICUmsQBIyfC6YaSagWvmuPD+ZRO3poazwtt+3xi+mV8KDEWUPnSRMsqJ9atKzNOmZQhZo0P5yOMwC6gVtObM7bi9JKEgumHkiwvdTxgQVprwvkYTRtPvT84VvXSdADuiBxd/yZlnL4eoPeXODNBuCb5wNRmWcAnkH+mIyspFDWiT0f+ygoSOqZ+Zdy8MFmXIYqSPw9YpHZjUJgpvIH04jsHWASYAJNS4iL8vYVRlzmKZE8GFmXym/OZ9k7xJfFrhzOAVrEiXxYy5mbnTiBVAm+drKmqZDQ== root@db-172-16-3-150.sky-mobi.com
[root@spark01 ~]# setenforce 0
[root@spark01 ~]# vi /etc/selinux/config
SELINUX=disabled
SELINUXTYPE=targeted
[root@db-172-16-3-150 ~]# ssh root@172.16.3.220 date
Mon May 19 02:01:40 CST 2014
[root@db-172-16-3-150 ~]# zfs send zptest/pg93@2014-05-1723:18:11 | ssh root@172.16.3.220 "/opt/zfs0.6.2/sbin/zfs receive zp/test"
[root@spark01 ~]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zp 3.65G 4.16G 31K /zp
zp/test 3.65G 4.16G 3.65G /zp/test
[root@spark01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 31G 1.2G 29G 5% /
tmpfs 12G 0 12G 0% /dev/shm
/dev/sda3 89G 11G 74G 13% /home
zp 4.2G 0 4.2G 0% /zp
zp/test 7.9G 3.7G 4.2G 47% /zp/test
[root@spark01 ~]# cd /zp/test
[root@spark01 test]# ll
total 2
drwx------. 16 digoal digoal 28 May 17 23:15 pg_root
因为zfs的snapshot是一致性镜像, 通过这种方法导出镜像到文件, 可以起到额外备份snapshot到其他主机或文件系统的作用.
参考
1. https://pthree.org/2012/12/20/zfs-administration-part-xiii-sending-and-receiving-filesystems/
2. http://blog.163.com/digoal@126/blog/static/163877040201441694022110/
3. http://blog.163.com/digoal@126/blog/static/163877040201441723450443/
4. http://blog.163.com/digoal@126/blog/static/16387704020144183320807/