SystemTap User-Space Stack Backtraces for x86 processors arch only

背景

systemtap 支持user-space的堆信息回溯输出.

使用方法参考 :

https://sourceware.org/systemtap/SystemTap_Beginners_Guide/ustack.html

用到stap的 -d MODULE, --ldd选项.  
同时需要安装被跟踪的进程的debuginfo包, 例如本例跟踪的/bin/ls, 在没有安装debuginfo包的时候会报错 :   
[root@db-172-16-3-150 pg93]# stap -d /bin/ls --ldd -e 'probe process("ls").function("xmalloc") {print_usyms(ubacktrace())}' -c "ls /"  
WARNING: cannot find module /bin/ls debuginfo: No DWARF information found [man warning::debuginfo]  
semantic error: while resolving probe point: identifier 'process' at <input>:1:7  
        source: probe process("ls").function("xmalloc") {print_usyms(ubacktrace())}  
                      ^  
semantic error: no match  
Pass 2: analysis failed.  [man error::pass2]  
查找/bin/ls所在的包名  
[root@db-172-16-3-150 pg93]# rpm -qf /bin/ls  
coreutils-8.4-19.el6.x86_64  
开启debuginfo源.  
[root@db-172-16-3-150 pg93]# vi /etc/yum.repos.d/CentOS-Debuginfo.repo  
[debug]  
name=CentOS-6 - Debuginfo  
baseurl=http://debuginfo.centos.org/6/$basearch/  
gpgcheck=1  
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Debug-6  
# 加一条:  
enabled=1  
安装coreutils对于的debuginfo包.  
[root@db-172-16-3-150 pg93]# yum install -y coreutils-debuginfo-8.4-19.el6.x86_64  
注意debuginfo的版本号必须与coreutils-8.4-19.el6.x86_64包一致.  
安装完debuginfo后, stap可以正常的运行 :   
[root@db-172-16-3-150 pg93]# stap -d /bin/ls --ldd -e 'probe process("ls").function("xmalloc") {ut=ubacktrace(); println(ut); print_usyms(ut)}' -c "ls /"  
bin   cgroup      data01  etc   lib    lost+found  misc  net  proc  sbin     srv   ssd2  ssd4  tmp  var  
boot  core.19981  dev     home  lib64  media       mnt   opt  root  selinux  ssd1  ssd3  sys   usr  
0x4116b0 0x4116ec 0x40e6bb 0x408834 0x39bf41ecdd 0x402809  
 0x4116b0 : xmalloc+0x0/0x20 [/bin/ls]  
 0x4116ec : xmemdup+0x1c/0x40 [/bin/ls]  
 0x40e6bb : clone_quoting_options+0x3b/0x50 [/bin/ls]  
 0x408834 : main+0x3b4/0x1900 [/bin/ls]  
 0x39bf41ecdd : __libc_start_main+0xfd/0x1d0 [/lib64/libc-2.12.so]  
 0x402809 : _start+0x29/0x2c [/bin/ls]  
... 略  

参考

1. https://sourceware.org/systemtap/SystemTap_Beginners_Guide/ustack.html

2. man stap

       -d MODULE  
              Add symbol/unwind information for the given module into the kernel object module.  This may enable  sym-  
              bolic  tracebacks  from  those  modules/programs, even if they do not have an explicit probe placed into  
              them.  
  
       --ldd  Add symbol/unwind information for all shared libraries suspected by ldd to be necessary  for  user-space  
              binaries  being  probe or listed with the -d option.  Caution: this can make the probe modules consider-  
              ably larger.  
  
       --all-modules  
              Equivalent to specifying "-dkernel" and a "-d" for each kernel module that is  currently  loaded.   Cau-  
              tion: this can make the probe modules considerably larger.  

3. https://sourceware.org/systemtap/tapsets/API-uaddr.html

4. https://sourceware.org/systemtap/tapsets/API-ubacktrace.html

5. https://sourceware.org/systemtap/tapsets/API-ucallers.html

6. https://sourceware.org/systemtap/tapsets/API-print-ubacktrace.html

7. https://sourceware.org/systemtap/tapsets/API-print-ubacktrace-brief.html

8. https://sourceware.org/systemtap/tapsets/API-print-ustack.html

9. https://sourceware.org/systemtap/tapsets/API-print-usyms.html

10. /usr/share/systemtap/tapset/ucontext*.stp

11.

Name  
    function::print_usyms — Print out user stack from string  
  
Synopsis  
    print_usyms(callers:string)  
  
Arguments  
    callers  
    String with list of hexadecimal (user) addresses  
  
Description  
    This function performs a symbolic lookup of the addresses in the given string, which are assumed to be the result of prior calls to ustack, ucallers, and similar functions.  
    Prints one line per address, including the address, the name of the function containing the address, and an estimate of its position within that function, as obtained by usymdata. Returns nothing.  
12.   
Name  
    function::ubacktrace — Hex backtrace of current user-space task stack.  
  
Synopsis  
    ubacktrace:string()  
  
Arguments  
    None  
  
Description  
    Return a string of hex addresses that are a backtrace of the stack of the current task. Output may be truncated as per maximum string length. Returns empty string when current probe point cannot determine user backtrace. See backtrace for kernel traceback.  
  
Note  
    To get (full) backtraces for user space applications and shared shared libraries not mentioned in the current script run stap with -d /path/to/exe-or-so and/or add --ldd to load all needed unwind data.  

digoal’s 大量PostgreSQL文章入口