use export and import move ZPOOL’s underdev from one machine to another OR upgrade a zfs version OR recover destroyed pools

4 minute read

背景

前面我们介绍了zfs的pool, 类似LVM. 由多个块设备组成.  
如果这些块设备要从一个机器转移到另一台机器的话, 怎么实现呢?  
zfs通过export和import来实现底层块设备的转移.  
在已有POOL的主机上, 先将会读写POOL或dataset的正在运行的程序停止掉, 然后执行export.  
执行export会把cache flush到底层的块设备, 同时卸载dataset和pool.  
import时, 可能需要指定块设备的目录, 但是并不需要指定顺序.  
例如 :   
[root@spark01 ~]# zpool status  
  pool: zp  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME                       STATE     READ WRITE CKSUM  
        zp                         ONLINE       0     0     0  
          /home/digoal/zfs.disk1   ONLINE       0     0     0  
          /home/digoal/zfs.disk2   ONLINE       0     0     0  
          /home/digoal/zfs.disk3   ONLINE       0     0     0  
          /home/digoal/zfs.disk4   ONLINE       0     0     0  
        logs  
          mirror-4                 ONLINE       0     0     0  
            /home/digoal/zfs.log1  ONLINE       0     0     0  
            /home/digoal/zfs.log2  ONLINE       0     0     0  
  
errors: No known data errors  
导出 :   
[root@spark01 ~]# zpool export zp  
[root@spark01 ~]# zpool status  
no pools available  
因为我这里用的是文件, 所以把文件拷贝到对应的其他主机, 然后指定文件所在目录,  
zpool  
        import [-d dir] [-D]  
        import [-d dir | -c cachefile] [-F [-n]] <pool | id>  
        import [-o mntopts] [-o property=value] ...   
            [-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]] -a  
        import [-o mntopts] [-o property=value] ...   
            [-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]]  
            <pool | id> [newpool]  
  
[root@spark01 ~]# zpool import -d /home/digoal/  
   pool: zp  
     id: 2426254839125485600  
  state: ONLINE  
 action: The pool can be imported using its name or numeric identifier.  #此时并没有真正的导入, 提示你可以使用name或id导入.  
 config:  
  
        zp                         ONLINE  
          /home/digoal/zfs.disk1   ONLINE  
          /home/digoal/zfs.disk2   ONLINE  
          /home/digoal/zfs.disk3   ONLINE  
          /home/digoal/zfs.disk4   ONLINE  
        logs  
          mirror-4                 ONLINE  
            /home/digoal/zfs.log1  ONLINE  
            /home/digoal/zfs.log2  ONLINE  
如果是块设备的话, 不需要指定目录.  
[root@spark01 ~]# zpool status  
no pools available  
使用name 导入.  
[root@spark01 ~]# zpool import -d /home/digoal/  zp  
[root@spark01 ~]# zpool status  
  pool: zp  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME                       STATE     READ WRITE CKSUM  
        zp                         ONLINE       0     0     0  
          /home/digoal/zfs.disk1   ONLINE       0     0     0  
          /home/digoal/zfs.disk2   ONLINE       0     0     0  
          /home/digoal/zfs.disk3   ONLINE       0     0     0  
          /home/digoal/zfs.disk4   ONLINE       0     0     0  
        logs  
          mirror-4                 ONLINE       0     0     0  
            /home/digoal/zfs.log1  ONLINE       0     0     0  
            /home/digoal/zfs.log2  ONLINE       0     0     0  
  
errors: No known data errors  
[root@spark01 ~]# df -h  
Filesystem      Size  Used Avail Use% Mounted on  
/dev/sda1        31G  1.2G   29G   5% /  
tmpfs            12G     0   12G   0% /dev/shm  
/dev/sda3        89G   11G   74G  13% /home  
zp              7.9G     0  7.9G   0% /zp  
接下来我export后修改一下目录和文件名, 并且拷贝到另一台主机.  
SERVER B :   
[root@db-172-16-3-150 ~]# mkdir /ssd4/import  
SERVER A :   
[root@spark01 ~]# zpool export zp  
[root@spark01 digoal]# scp zfs.disk* zfs.log* 172.16.3.150:/ssd4/import  
SERVER B :   
[root@db-172-16-3-150 ~]# mkdir /ssd4/import/zfslog  
[root@db-172-16-3-150 ~]# cd /ssd4/import  
[root@db-172-16-3-150 import]# mv zfs.log* zfslog/  
[root@db-172-16-3-150 import]# mv zfs.disk1 zfs.disk01  
[root@db-172-16-3-150 import]# zpool import -d /ssd4/import -d /ssd4/import/zfslog/  
   pool: zp  
     id: 2426254839125485600  
  state: ONLINE  
 action: The pool can be imported using its name or numeric identifier.  
 config:  
  
        zp                                ONLINE  
          /ssd4/import/zfs.disk01         ONLINE  
          /ssd4/import/zfs.disk2          ONLINE  
          /ssd4/import/zfs.disk3          ONLINE  
          /ssd4/import/zfs.disk4          ONLINE  
        logs  
          mirror-4                        ONLINE  
            /ssd4/import/zfslog/zfs.log1  ONLINE  
            /ssd4/import/zfslog/zfs.log2  ONLINE  
甚至可以使用新的pool名字.  
[root@db-172-16-3-150 import]# zpool import -d /ssd4/import -d /ssd4/import/zfslog/ zp newzp  
[root@db-172-16-3-150 import]# zpool status  
  pool: newzp  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME                              STATE     READ WRITE CKSUM  
        newzp                             ONLINE       0     0     0  
          /ssd4/import/zfs.disk01         ONLINE       0     0     0  
          /ssd4/import/zfs.disk2          ONLINE       0     0     0  
          /ssd4/import/zfs.disk3          ONLINE       0     0     0  
          /ssd4/import/zfs.disk4          ONLINE       0     0     0  
        logs  
          mirror-4                        ONLINE       0     0     0  
            /ssd4/import/zfslog/zfs.log1  ONLINE       0     0     0  
            /ssd4/import/zfslog/zfs.log2  ONLINE       0     0     0  
  
errors: No known data errors  
  
恢复destroy的zpool, 当pool被destroy后, 还可以恢复, 因为数据都还在, 只是pool删掉了.  
[root@spark01 digoal]# zpool status  
  pool: zp  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME                       STATE     READ WRITE CKSUM  
        zp                         ONLINE       0     0     0  
          /home/digoal/zfs.disk1   ONLINE       0     0     0  
          /home/digoal/zfs.disk2   ONLINE       0     0     0  
          /home/digoal/zfs.disk3   ONLINE       0     0     0  
          /home/digoal/zfs.disk4   ONLINE       0     0     0  
        logs  
          mirror-4                 ONLINE       0     0     0  
            /home/digoal/zfs.log1  ONLINE       0     0     0  
            /home/digoal/zfs.log2  ONLINE       0     0     0  
  
errors: No known data errors  
[root@spark01 digoal]# zpool destroy zp  
[root@spark01 digoal]# zpool status  
no pools available  
  
必须加-D参数才可以恢复destroy的pool  
[root@spark01 digoal]# zpool import -d /home/digoal/  
no pools available to import  
[root@spark01 digoal]# zpool import -d /home/digoal/ -D  
   pool: zp  
     id: 2426254839125485600  
  state: ONLINE (DESTROYED)  
 action: The pool can be imported using its name or numeric identifier.  
 config:  
  
        zp                         ONLINE  
          /home/digoal/zfs.disk1   ONLINE  
          /home/digoal/zfs.disk2   ONLINE  
          /home/digoal/zfs.disk3   ONLINE  
          /home/digoal/zfs.disk4   ONLINE  
        logs  
          mirror-4                 ONLINE  
            /home/digoal/zfs.log1  ONLINE  
            /home/digoal/zfs.log2  ONLINE  
[root@spark01 digoal]# zpool import -d /home/digoal/ -D zp  
[root@spark01 digoal]# zpool status  
  pool: zp  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME                       STATE     READ WRITE CKSUM  
        zp                         ONLINE       0     0     0  
          /home/digoal/zfs.disk1   ONLINE       0     0     0  
          /home/digoal/zfs.disk2   ONLINE       0     0     0  
          /home/digoal/zfs.disk3   ONLINE       0     0     0  
          /home/digoal/zfs.disk4   ONLINE       0     0     0  
        logs  
          mirror-4                 ONLINE       0     0     0  
            /home/digoal/zfs.log1  ONLINE       0     0     0  
            /home/digoal/zfs.log2  ONLINE       0     0     0  
  
errors: No known data errors  
  
zpool版本升级, 比如一个pool从版本20的机器迁移到版本28的机器, 那么可以升级到28, 但是这样的话, 这个pool就不能再回到版本20的机器上使用了.  
zpool   
        upgrade  
        upgrade -v  
        upgrade [-V version] <-a | pool ...>  
[root@spark01 digoal]# zpool upgrade zp  
This system supports ZFS pool feature flags.  
  
Pool 'zp' already has all supported features enabled.  
查看当前zfs版本信息  
[root@spark01 digoal]# zpool upgrade -v  
This system supports ZFS pool feature flags.  
  
The following features are supported:  
  
FEAT DESCRIPTION  
-------------------------------------------------------------  
async_destroy                         (read-only compatible)  
     Destroy filesystems asynchronously.  
empty_bpobj                           (read-only compatible)  
     Snapshots use less space.  
lz4_compress                           
     LZ4 compression algorithm support.  
  
The following legacy versions are also supported:  
  
VER  DESCRIPTION  
---  --------------------------------------------------------  
 1   Initial ZFS version  
 2   Ditto blocks (replicated metadata)  
 3   Hot spares and double parity RAID-Z  
 4   zpool history  
 5   Compression using the gzip algorithm  
 6   bootfs pool property  
 7   Separate intent log devices  
 8   Delegated administration  
 9   refquota and refreservation properties  
 10  Cache devices  
 11  Improved scrub performance  
 12  Snapshot properties  
 13  snapused property  
 14  passthrough-x aclinherit  
 15  user/group space accounting  
 16  stmf property support  
 17  Triple-parity RAID-Z  
 18  Snapshot user holds  
 19  Log device removal  
 20  Compression using zle (zero-length encoding)  
 21  Deduplication  
 22  Received properties  
 23  Slim ZIL  
 24  System attributes  
 25  Improved scrub stats  
 26  Improved snapshot deletion performance  
 27  Improved snapshot creation performance  
 28  Multiple vdev replacements  
  
For more information on a particular version, including supported releases,  
see the ZFS Administration Guide.  
不能降级  
[root@spark01 digoal]# zpool upgrade -V 20 zp  
This system supports ZFS pool feature flags.  
  
Pool 'zp' is already formatted using more current version '5000'.  

参考

1. https://pthree.org/2012/12/10/zfs-administration-part-v-exporting-and-importing-zpools/

Flag Counter

digoal’s 大量PostgreSQL文章入口