Systemtap Syscall probes

背景

The syscall.* aliases define several hundred probes. They use the following syntax:  
syscall.NAME  
syscall.NAME.return  
Generally, two probes are defined for each normal system call as listed in the syscalls(2) manual page: one for entry and one for return. System calls that never return do not have a corresponding .return probe.  
系统调用探针, 与mark或trace探针不同, 系统调用探针需要安装debuginfo包.  
系统调用探针一般定义了入口和返回两个别名. 分别代表探针在函数入口和函数返回的代码位置.   
  
Each probe alias defines a variety of variables. Look at the tapset source code to find the most reliable source of variable definitions. Generally, each variable listed in the standard manual page is available as a script-level variable.   
For example, syscall.open exposes file name, flags, and mode.   
syscall探针的定义在tapset中可以找到, 如  
/usr/share/systemtap/tapset/syscalls.stp  
/usr/share/systemtap/tapset/syscalls2.stp  
/usr/share/systemtap/tapset/syscalls_cfg_trunc.stp  
本文例子中用到的chmod探针源码  
# chmod ______________________________________________________  
# long sys_chmod(const char __user * filename, mode_t mode)  
probe syscall.chmod = kernel.function("sys_chmod").call  
{  
        name = "chmod"  
        path = user_string($filename)  
        mode = $mode  
        argstr = sprintf("%s, %#o", user_string_quoted($filename), mode)  
}  
probe syscall.chmod.return = kernel.function("sys_chmod").return  
{  
        name = "chmod"  
        retstr = return_str(1, $return)  
}  
从这个syscall的定义我们也就知道为什么syscall探针需要debuginfo包了, 因为它实际上是DWARF-based探针的别名.  
  
In addition, a standard suite of variables is available at most aliases, as follows:  
argstr: A pretty-printed form of the entire argument list, without parentheses.  
name: The name of the system call.  
retstr: For return probes, a pretty-printed form of the system call result.  
syscall定义中一般有这些变量, argstr, name, retstr. 分别代表已经转换好的可读变量, 系统调用的名字, 返回值.  
例如chmod中定义的本地变量如下 :   
        name = "chmod"  
        path = user_string($filename)  
        mode = $mode  
        argstr = sprintf("%s, %#o", user_string_quoted($filename), mode)  
chmod.return定义的本地变量如下 :   
        name = "chmod"  
        retstr = return_str(1, $return)  
  
Not all probe aliases obey all of these general guidelines. Please report exceptions that you encounter as a bug.  

举例 :

列出当前系统支持的系统调用探针 :

[root@db-172-16-3-39 ~]# stap -l 'syscall.**'|less  
syscall.accept  
syscall.accept.return  
syscall.access  
syscall.access.return  
syscall.acct  
syscall.acct.return  
syscall.add_key  
syscall.add_key.return  
syscall.adjtimex  
syscall.adjtimex.return  
syscall.alarm  
syscall.alarm.return  
syscall.arch_prctl  
syscall.arch_prctl.return  
syscall.bdflush  
syscall.bdflush.return  
syscall.bind  
syscall.bind.return  
syscall.brk  
... 略  

syscall.chmod探针使用举例 :

例如要给以下文件修改权限 :

[root@db-172-16-3-39 ~]# ll  
total 84900  
-rw-------  1 root root     1849 Oct 21  2011 anaconda-ks.cfg  

开启探针 :

[root@db-172-16-3-39 ~]# stap -e 'probe syscall.chmod { printf ("%s\n%s\n%s\n", name, $$vars, argstr) }'  

修改权限 :

[root@db-172-16-3-39 ~]# chmod 777 anaconda-ks.cfg   

探针handler输出, $$vars输出了在syscall.chmod中定义的本地变量, 而argstr则输出了已经转换好的可读信息(sprintf(“%s, %#o”, user_string_quoted($filename), mode)) :

chmod  
filename=0x140110b0 mode=0x1ff  
"anaconda-ks.cfg", 0777  

如果我们单独输出$filename$$, 显然mode没有正确转换, 因为它使用了user_string_quoted来转换

[root@db-172-16-3-39 ~]# stap -e 'probe syscall.chmod { printf ("%s\n%s\n", $filename$$, $mode$$) }'  
"anaconda-ks.cfg"  
511  

user_string_quoted这个函数的解释参考

https://sourceware.org/systemtap/tapsets/API-user-string-quoted.html

从源码中我们也能看到chmod.return中没有定义argstr本地变量, 所以会有如下告警 :

[root@db-172-16-3-39 ~]# stap -e 'probe syscall.chmod.return { printf ("%s\n%s\n%s\n%s\n", name, $$vars, argstr, retstr) }'  
WARNING: never-assigned local variable 'argstr' (alternatives: _dwarf_tvar_tid _dwarf_tvar_$vars_3_tmp name retstr _dwarf_tvar_$vars_3 _dwarf_tvar_$vars_3_ctr): identifier 'argstr' at <input>:1:72  
 source: probe syscall.chmod.return { printf ("%s\n%s\n%s\n%s\n", name, $$vars, argstr, retstr) }  
                                                                                ^  
chmod  
filename=0x58d80b0 mode=0x1ff  
  
0  

在未安装debuginfo包的系统中使用syscall探针报错如下 :

[root@db-172-16-3-40 ~]# stap -e 'probe syscall.chmod.return { printf ("%s\n%s\n%s\n%s\n", name, $$vars, argstr, retstr) }'  
semantic error: while resolving probe point: identifier 'kernel' at /usr/share/systemtap/tapset/syscalls.stp:300:30  
        source: probe syscall.chmod.return = kernel.function("sys_chmod").return  
                                             ^  
  
semantic error: missing x86_64 kernel/module debuginfo under '/lib/modules/2.6.18-274.el5/build'  
semantic error: while resolving probe point: identifier 'syscall' at <input>:1:7  
        source: probe syscall.chmod.return { printf ("%s\n%s\n%s\n%s\n", name, $$vars, argstr, retstr) }  
                      ^  
  
semantic error: no match  
Pass 2: analysis failed.  Try again with another '--vp 01' option.  
Missing separate debuginfos, use: debuginfo-install kernel-2.6.18-274.el5.x86_64  

参考

1. https://sourceware.org/systemtap/langref/Probe_points.html

2. man syscalls

3. stap -l ‘syscall.**’

4.

/usr/share/systemtap/tapset/syscalls.stp

/usr/share/systemtap/tapset/syscalls2.stp

/usr/share/systemtap/tapset/syscalls_cfg_trunc.stp

digoal’s 大量PostgreSQL文章入口