ZFS on Linux performance tuning when zfs_iput_taskq take near 100% CPU
背景
一台机子的zfs_iput_taskq进程CPU占用100%.
处理方法 :
# cat /etc/modprobe.d/zfs.conf
options zfs zfs_arc_max=10240000000
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zp1 548M 267G 136K /zp1
zp1/data01 302M 267G 302M /data01
zp1/pg_log 980K 267G 980K /pg_log
zp1/site 234M 267G 234M /site
# zfs set xattr=sa zp1
# zfs set xattr=sa zp1/data01
# zfs set xattr=sa zp1/pg_log
# zfs set xattr=sa zp1/site
# vi /etc/sysctl.conf
vm.swappiness = 10
vm.min_free_kbytes = 512000
reboot
参考
ZFS on Linux needs a bit of basic tuning in order to operate well under load. There's a bit of a struggle between the ZFS ARC and the Linux virtual memory subsystem.
For your CentOS systems, try the following:
Create an /etc/modprobe.d/zfs.conf configuration file. This is read during the module load/boot.
Add something like:
options zfs zfs_arc_max=40000000000
options zfs zfs_vdev_max_pending=24
Where zfs_arc_max is roughly 40% of your RAM in bytes (Edit: try zfs_arc_max=1200000000). The compiled-in default for zfs_vdev_max_pending is 8 or 10, depending on version. The value should be high (48) for SSD or low-latency drives. Maybe 12-24 for SAS. Otherwise, leave at default.
You'll want to also have some floor values in /etc/sysctl.conf
vm.swappiness = 10
vm.min_free_kbytes = 512000
Finally, with CentOS, you may want to install tuned and tuned-utils and set your profile to virtual-guest with tuned-adm profile virtual-guest.
Try these and see if the problem persists.
Edit:
Run zfs set xattr=sa storage. Here's why. You may have to wipe the volumes and start again (I'd recommend).
1. https://github.com/zfsonlinux/zfs/issues/457
2. http://serverfault.com/questions/583630/gluster-zfs-deadlock-during-benchmarking-zfs-iput-taskq-100-cpu?rq=1