ZFS module parameters in Linux (OR kernel parameter in FreeBSD or zfs-in-kernel OS)

5 minute read

背景

ZFS 在linux中以模块的形式加载, ZFS的内核参数没有整合到linux内核参数中, 所以无法使用sysctl来修改.

只能通过修改模块的参数来变更这些参数.

ZFS内核参数的意义可参见modinfo $modname, 或者直接查看zfs源代码. 有些参数是根据系统读数设定的(如内存)

与Linux不同, 在FreeBSD里zfs是在内核中的, 所以这些参数可以通过sysctl.conf保存, 或使用sysctl -w直接修改.

例如我的一台安装了0.6.3 zfs的系统, CentOS 6.5 x64.

内存24GB

# free  
             total       used       free     shared    buffers     cached  
Mem:      24530656   20768340    3762316          0     115636    2609916  
-/+ buffers/cache:   18042788    6487868  
Swap:      8388600          0    8388600  

zfs相关模块.

# lsmod|grep zfs  
zfs                  1195533  1   
zcommon                46278  1 zfs  
znvpair                80974  2 zfs,zcommon  
zavl                    6925  1 zfs  
zunicode              323159  1 zfs  
spl                   266655  5 zfs,zcommon,znvpair,zavl,zunicode  

zfs模块信息, parm是参数部分, 在FreeBSD里面, 这些参数可以通过sysctl.conf保存, 或使用sysctl -w直接修改.

# modinfo zfs  
filename:       /lib/modules/2.6.32-431.el6.x86_64/extra/zfs.ko  
version:        0.6.3-1  
license:        CDDL  
author:         Sun Microsystems/Oracle, Lawrence Livermore National Laboratory  
description:    ZFS  
srcversion:     C29A443E3D2B93F605A540B  
depends:        spl,znvpair,zcommon,zunicode,zavl  
vermagic:       2.6.32-431.el6.x86_64 SMP mod_unload modversions   
parm:           zvol_inhibit_dev:Do not create zvol device nodes (uint)  
parm:           zvol_major:Major number for zvol device (uint)  
parm:           zvol_threads:Number of threads for zvol device (uint)  
parm:           zvol_max_discard_blocks:Max number of blocks to discard (ulong)  
parm:           zio_injection_enabled:Enable fault injection (int)  
parm:           zio_bulk_flags:Additional flags to pass to bulk buffers (int)  
parm:           zio_delay_max:Max zio millisec delay before posting event (int)  
parm:           zio_requeue_io_start_cut_in_line:Prioritize requeued I/O (int)  
parm:           zfs_sync_pass_deferred_free:Defer frees starting in this pass (int)  
parm:           zfs_sync_pass_dont_compress:Don't compress starting in this pass (int)  
parm:           zfs_sync_pass_rewrite:Rewrite new bps starting in this pass (int)  
parm:           zil_replay_disable:Disable intent logging replay (int)  
parm:           zfs_nocacheflush:Disable cache flushes (int)  
parm:           zil_slog_limit:Max commit bytes to separate log device (ulong)  
parm:           zfs_read_chunk_size:Bytes to read per chunk (long)  
parm:           zfs_immediate_write_sz:Largest data block to write to zil (long)  
parm:           zfs_flags:Set additional debugging flags (int)  
parm:           zfs_recover:Set to attempt to recover from fatal errors (int)  
parm:           zfs_expire_snapshot:Seconds to expire .zfs/snapshot (int)  
parm:           zfs_vdev_aggregation_limit:Max vdev I/O aggregation size (int)  
parm:           zfs_vdev_read_gap_limit:Aggregate read I/O over gap (int)  
parm:           zfs_vdev_write_gap_limit:Aggregate write I/O over gap (int)  
parm:           zfs_vdev_max_active:Maximum number of active I/Os per vdev (int)  
parm:           zfs_vdev_async_write_active_max_dirty_percent:Async write concurrency max threshold (int)  
parm:           zfs_vdev_async_write_active_min_dirty_percent:Async write concurrency min threshold (int)  
parm:           zfs_vdev_async_read_max_active:Max active async read I/Os per vdev (int)  
parm:           zfs_vdev_async_read_min_active:Min active async read I/Os per vdev (int)  
parm:           zfs_vdev_async_write_max_active:Max active async write I/Os per vdev (int)  
parm:           zfs_vdev_async_write_min_active:Min active async write I/Os per vdev (int)  
parm:           zfs_vdev_scrub_max_active:Max active scrub I/Os per vdev (int)  
parm:           zfs_vdev_scrub_min_active:Min active scrub I/Os per vdev (int)  
parm:           zfs_vdev_sync_read_max_active:Max active sync read I/Os per vdev (int)  
parm:           zfs_vdev_sync_read_min_active:Min active sync read I/Os per vdev (int)  
parm:           zfs_vdev_sync_write_max_active:Max active sync write I/Os per vdev (int)  
parm:           zfs_vdev_sync_write_min_active:Min active sync write I/Osper vdev (int)  
parm:           zfs_vdev_mirror_switch_us:Switch mirrors every N usecs (int)  
parm:           zfs_vdev_scheduler:I/O scheduler (charp)  
parm:           zfs_vdev_cache_max:Inflate reads small than max (int)  
parm:           zfs_vdev_cache_size:Total size of the per-disk cache (int)  
parm:           zfs_vdev_cache_bshift:Shift size to inflate reads too (int)  
parm:           zfs_txg_timeout:Max seconds worth of delta per txg (int)  
parm:           zfs_read_history:Historic statistics for the last N reads (int)  
parm:           zfs_read_history_hits:Include cache hits in read history (int)  
parm:           zfs_txg_history:Historic statistics for the last N txgs (int)  
parm:           zfs_deadman_synctime_ms:Expiration time in milliseconds (ulong)  
parm:           zfs_deadman_enabled:Enable deadman timer (int)  
parm:           spa_asize_inflation:SPA size estimate multiplication factor (int)  
parm:           spa_config_path:SPA config file (/etc/zfs/zpool.cache) (charp)  
parm:           zfs_autoimport_disable:Disable pool import at module load (int)  
parm:           metaslab_debug_load:load all metaslabs during pool import (int)  
parm:           metaslab_debug_unload:prevent metaslabs from being unloaded (int)  
parm:           zfs_zevent_len_max:Max event queue length (int)  
parm:           zfs_zevent_cols:Max event column width (int)  
parm:           zfs_zevent_console:Log events to the console (int)  
parm:           zfs_top_maxinflight:Max I/Os per top-level (int)  
parm:           zfs_resilver_delay:Number of ticks to delay resilver (int)  
parm:           zfs_scrub_delay:Number of ticks to delay scrub (int)  
parm:           zfs_scan_idle:Idle window in clock ticks (int)  
parm:           zfs_scan_min_time_ms:Min millisecs to scrub per txg (int)  
parm:           zfs_free_min_time_ms:Min millisecs to free per txg (int)  
parm:           zfs_resilver_min_time_ms:Min millisecs to resilver per txg (int)  
parm:           zfs_no_scrub_io:Set to disable scrub I/O (int)  
parm:           zfs_no_scrub_prefetch:Set to disable scrub prefetching (int)  
parm:           zfs_dirty_data_max_percent:percent of ram can be dirty (int)  
parm:           zfs_dirty_data_max_max_percent:zfs_dirty_data_max upper bound as % of RAM (int)  
parm:           zfs_delay_min_dirty_percent:transaction delay threshold (int)  
parm:           zfs_dirty_data_max:determines the dirty space limit (ulong)  
parm:           zfs_dirty_data_max_max:zfs_dirty_data_max upper bound in bytes (ulong)  
parm:           zfs_dirty_data_sync:sync txg when this much dirty data (ulong)  
parm:           zfs_delay_scale:how quickly delay approaches infinity (ulong)  
parm:           zfs_prefetch_disable:Disable all ZFS prefetching (int)  
parm:           zfetch_max_streams:Max number of streams per zfetch (uint)  
parm:           zfetch_min_sec_reap:Min time before stream reclaim (uint)  
parm:           zfetch_block_cap:Max number of blocks to fetch at a time (uint)  
parm:           zfetch_array_rd_sz:Number of bytes in a array_read (ulong)  
parm:           zfs_pd_blks_max:Max number of blocks to prefetch (int)  
parm:           zfs_send_corrupt_data:Allow sending corrupt data (int)  
parm:           zfs_mdcomp_disable:Disable meta data compression (int)  
parm:           zfs_nopwrite_enabled:Enable NOP writes (int)  
parm:           zfs_dedup_prefetch:Enable prefetching dedup-ed blks (int)  
parm:           zfs_dbuf_state_index:Calculate arc header index (int)  
parm:           zfs_arc_min:Min arc size (ulong)  
parm:           zfs_arc_max:Max arc size (ulong)  
parm:           zfs_arc_meta_limit:Meta limit for arc size (ulong)  
parm:           zfs_arc_meta_prune:Bytes of meta data to prune (int)  
parm:           zfs_arc_grow_retry:Seconds before growing arc size (int)  
parm:           zfs_arc_p_aggressive_disable:disable aggressive arc_p grow (int)  
parm:           zfs_arc_p_dampener_disable:disable arc_p adapt dampener (int)  
parm:           zfs_arc_shrink_shift:log2(fraction of arc to reclaim) (int)  
parm:           zfs_disable_dup_eviction:disable duplicate buffer eviction (int)  
parm:           zfs_arc_memory_throttle_disable:disable memory throttle (int)  
parm:           zfs_arc_min_prefetch_lifespan:Min life of prefetch block (int)  
parm:           l2arc_write_max:Max write bytes per interval (ulong)  
parm:           l2arc_write_boost:Extra write bytes during device warmup (ulong)  
parm:           l2arc_headroom:Number of max device writes to precache (ulong)  
parm:           l2arc_headroom_boost:Compressed l2arc_headroom multiplier (ulong)  
parm:           l2arc_feed_secs:Seconds between L2ARC writing (ulong)  
parm:           l2arc_feed_min_ms:Min feed interval in milliseconds (ulong)  
parm:           l2arc_noprefetch:Skip caching prefetched buffers (int)  
parm:           l2arc_nocompress:Skip compressing L2ARC buffers (int)  
parm:           l2arc_feed_again:Turbo L2ARC warmup (int)  
parm:           l2arc_norw:No reads during writes (int)  

如果要看模块对应的这些参数的值, 可以到/sys/module/$modname/parameters 中查看.

# cd /sys/module/zfs/parameters/  
# grep ".*" *  
l2arc_feed_again:1  
l2arc_feed_min_ms:200  
l2arc_feed_secs:1  
l2arc_headroom:2  
l2arc_headroom_boost:200  
l2arc_nocompress:0  
l2arc_noprefetch:1  
l2arc_norw:0  
l2arc_write_boost:8388608  
l2arc_write_max:8388608  
metaslab_debug_load:0  
metaslab_debug_unload:0  
spa_asize_inflation:24  
spa_config_path:/etc/zfs/zpool.cache  
zfetch_array_rd_sz:1048576  
zfetch_block_cap:256  
zfetch_max_streams:8  
zfetch_min_sec_reap:2  
zfs_arc_grow_retry:5  
zfs_arc_max:0  
zfs_arc_memory_throttle_disable:1  
zfs_arc_meta_limit:0  
zfs_arc_meta_prune:1048576  
zfs_arc_min:0  
zfs_arc_min_prefetch_lifespan:1000  
zfs_arc_p_aggressive_disable:1  
zfs_arc_p_dampener_disable:1  
zfs_arc_shrink_shift:5  
zfs_autoimport_disable:0  
zfs_dbuf_state_index:0  
zfs_deadman_enabled:1  
zfs_deadman_synctime_ms:1000000  
zfs_dedup_prefetch:1  
zfs_delay_min_dirty_percent:60  
zfs_delay_scale:500000  
zfs_dirty_data_max:2511939174  
zfs_dirty_data_max_max:6279847936  
zfs_dirty_data_max_max_percent:25  
zfs_dirty_data_max_percent:10  
zfs_dirty_data_sync:67108864  
zfs_disable_dup_eviction:0  
zfs_expire_snapshot:300  
zfs_flags:1  
zfs_free_min_time_ms:1000  
zfs_immediate_write_sz:32768  
zfs_mdcomp_disable:0  
zfs_nocacheflush:0  
zfs_nopwrite_enabled:1  
zfs_no_scrub_io:0  
zfs_no_scrub_prefetch:0  
zfs_pd_blks_max:100  
zfs_prefetch_disable:0  
zfs_read_chunk_size:1048576  
zfs_read_history:0  
zfs_read_history_hits:0  
zfs_recover:0  
zfs_resilver_delay:2  
zfs_resilver_min_time_ms:3000  
zfs_scan_idle:50  
zfs_scan_min_time_ms:1000  
zfs_scrub_delay:4  
zfs_send_corrupt_data:0  
zfs_sync_pass_deferred_free:2  
zfs_sync_pass_dont_compress:5  
zfs_sync_pass_rewrite:2  
zfs_top_maxinflight:32  
zfs_txg_history:0  
zfs_txg_timeout:5  
zfs_vdev_aggregation_limit:131072  
zfs_vdev_async_read_max_active:3  
zfs_vdev_async_read_min_active:1  
zfs_vdev_async_write_active_max_dirty_percent:60  
zfs_vdev_async_write_active_min_dirty_percent:30  
zfs_vdev_async_write_max_active:10  
zfs_vdev_async_write_min_active:1  
zfs_vdev_cache_bshift:16  
zfs_vdev_cache_max:16384  
zfs_vdev_cache_size:0  
zfs_vdev_max_active:1000  
zfs_vdev_mirror_switch_us:10000  
zfs_vdev_read_gap_limit:32768  
zfs_vdev_scheduler:noop  
zfs_vdev_scrub_max_active:2  
zfs_vdev_scrub_min_active:1  
zfs_vdev_sync_read_max_active:10  
zfs_vdev_sync_read_min_active:10  
zfs_vdev_sync_write_max_active:10  
zfs_vdev_sync_write_min_active:10  
zfs_vdev_write_gap_limit:4096  
zfs_zevent_cols:80  
zfs_zevent_console:0  
zfs_zevent_len_max:128  
zil_replay_disable:0  
zil_slog_limit:1048576  
zio_bulk_flags:0  
zio_delay_max:30000  
zio_injection_enabled:0  
zio_requeue_io_start_cut_in_line:1  
zvol_inhibit_dev:0  
zvol_major:230  
zvol_max_discard_blocks:16384  
zvol_threads:32  

这些文件可以直接修改. 例如 echo 1 >/sys/module/zfs/parameters/zfs_prefetch_disable

如果要重启生效的话, 可以修改/dev/modprobe.d/$modname.conf 如 /dev/modprobe.d/zfs.conf.

# cd /sys/module/zfs/parameters/  
# cat zvol_threads  
# modinfo zfs|grep zvol_threads  
parm:           zvol_threads:Number of threads for zvol device (uint)  
# vi /etc/modprobe.d/zfs.conf  
options zfs zvol_threads=64  
  
# modinfo spl|grep spl_kmem_cache_slab_limit  
parm:           spl_kmem_cache_slab_limit:Objects less than N bytes use the Linux slab (uint)  
# cat /sys/module/spl/parameters/spl_kmem_cache_slab_limit  
0  
# vi /etc/modprobe.d/spl.conf  
options spl spl_kmem_cache_slab_limit=16384  

修改后重启操作系统. 查看是否生效.

# cat /sys/module/spl/parameters/spl_kmem_cache_slab_limit  
16384  
# cat /sys/module/zfs/parameters/zvol_threads  
64  

参考

1. http://zfsonlinux.org/

2. http://open-zfs.org/wiki/Main_Page

3. zfs-0.6.3/module/zfs

4. Brian的回复

You can set these like any other module parameter on Linux.  If you want   
them to be persistent the easiest way is to add them to a zfs.conf file   
in /etc/modprobe.d/.  For example something like this.  
  
$ cat /etc/modprobe.d/zfs.conf  
options spl spl_kmem_cache_slab_limit=16384  
options spl spl_kmem_cache_reclaim=0  
  
Alternately, the ones which are runtime tunable can be set directly   
though /sys/module/zfs/parameters/.  Something like  
  
echo 1 >/sys/module/zfs/parameters/zfs_prefetch_disable  
  
Hope that helps.  
  
Thanks,  
Brian  

Flag Counter

digoal’s 大量PostgreSQL文章入口