ubuntu+systemtap進行Linux內核和用戶空間開發測試
Sailor_forever sailing_9806#163.com
(本原創文章發表於Sailor_forever 的個人blog,未經本人許可,不得用於商業用途。任何個人、媒體、其他網站不得私自抄襲;網絡媒體轉載請註明出處,增加原文鏈接,否則屬於侵權行爲。如有任何問題,請留言或者發郵件給sailing_9806#163.com)
【摘要】本文主要介紹在ubuntu平臺 + 自定義內核上如何安裝systemtap工具包及解決編譯加載運行過程中的一些問題,如何利用systemtap工具監測分析內核函數,內核模塊及用戶態程序。
【關鍵字】ubuntu, systemtap, utrace, backtrace, dtrace,callstack
1.2.1 卸載通過apt-get自動安裝的systemtap. 2
6 Systemtap加載KO異常,Unknown symbol in module. 9
7 error:implicit declaration of function ‘unregister_uprobe’13
Systemtap 是學習內核一個必不可少的工具,它不僅可以偵測內核空間的相關信息,還可以偵測用戶空間的信息,是研究內核源代碼、優化系統性能及調試診斷系統問題的一個必備工具。相關原理信息可參見附錄https://wiki.ubuntu.com/Kernel/Systemtap及“Linux 下的一個全新的性能測量和調式診斷工具 Systemtap, 第 3 部分: Systemtap”
http://www.ibm.com/developerworks/cn/linux/l-cn-systemtap3/#main
1 Systemtap安裝
1.1 Apt-get自動安裝systemtap
用系統工具自動安裝systemtap時,用戶不用考慮各個軟件包的依賴關係,系統會自動根據當前內核版本下載相應的軟件包。
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ sudoapt-get install systemtap
。。
The following extra packages will beinstalled:
libdw1 systemtap-commonsystemtap-runtime
The following NEW packages will beinstalled:
libdw1 systemtap systemtap-common systemtap-runtime
。
Get:1 http://us.archive.ubuntu.com/ubuntu/quantal-updates/main libdw1 i386 0.153-1ubuntu1.1 [216 kB]
。。
如上可知,系統自動安裝了相關的4個軟件包libdw1 systemtap-common systemtap-runtime
1.2 源代碼編譯安裝systemtap
有時候系統自動安裝的不一定能用,這個時候需要升級,此時只能通過下載源代碼自己編譯
1.2.1 卸載通過apt-get自動安裝的systemtap
首先要卸載系統之前安裝的相關package,避免後續帶來其他副作用。
dd@ubuntu:/mnt/hgfs/systemtap$ sudo apt-getremove systemtap systemtap-common systemtap-runtime
。。
Removing systemtap ...
Removing systemtap-common ...
Removing systemtap-runtime ...
Processing triggers for man-db ...
1.2.2 下載systemtap源碼包
下載相應版本的源碼包。源碼包並不是越新越好,因爲他們之間有一定的依賴關係,最簡單的是按照內核源碼包發佈的時間來選擇systemtap。
https://fedorahosted.org/releases/e/l/elfutils/
ftp://sources.redhat.com/pub/systemtap/releases/
1.2.3 編譯安裝
dd@ubuntu:/mnt/hgfs/systemtap/systemtap-2.0$./configure --with-elfutils=../elfutils-0.156 --prefix=/usr/local
。。。。。
configure: ./configure '--with-elfutils=../elfutils-0.156''--prefix=/usr/local' --prefix=/home/dd/systemtap-2.0-4155
configure: Running systemtap uninstalled, entirely out of the buildtree,
configure: is not supported.
重啓系統之後再執行,仍然如此,不知是否有其他副作用
dd@ubuntu:/mnt/hgfs/systemtap/systemtap-2.0$sudo make
。。
ln -fs libdw.so libdw.so.1
ln: failed to create symbolic link `libdw.so.1': Operation notsupported
make[4]: *** [libdw.so] Error 1
windows不支持符號鏈接,因此system包不能放在windows目錄編譯
dd@ubuntu:~/systemtap/systemtap-2.0$ sudomake
。。。
m4 -Di386 -DDISASSEMBLER../../../elfutils-0.156/libcpu/defs/i386 > i386_defsT
/bin/bash: m4: command not found
1.3 安裝內核debug info
System調試內核或者application,必須有debuginfo,因此必須安裝內核debug symbol。具體參見附錄。
2 Hello World基本測試
Systemtap安裝完畢後,進行簡單的測試。如果能打印出helloworld則認爲基本功能OK。
dd@ubuntu:/usr/src$ sudo stap -ve 'probebegin { log("hello world") exit() }'
Pass 1: parsed user script and 81 libraryscript(s) using 22640virt/13560res/2192shr kb, in 80usr/10sys/93real ms.
Pass 2: analyzed script: 1 probe(s), 2function(s), 0 embed(s), 0 global(s) using 22904virt/14088res/2264shr kb, in0usr/0sys/3real ms.
Pass 3: translated to C into"/tmp/stapNgfERv/stap_202537bef6262d2233e5759ff826c9d3_744_src.c"using 22904virt/14364res/2508shr kb, in 0usr/0sys/0real ms.
Pass 4: compiled C into"stap_202537bef6262d2233e5759ff826c9d3_744.ko" in4760usr/900sys/6360real ms.
Pass 5: starting run.
hello world
Pass 5: run completed in 0usr/20sys/359realms.
3 內核函數測試
3.1 基本功能
dd@ubuntu:/usr/src$ sudo stap -e 'probekernel.function("sys_open") {log("hello world") exit()}'
hello world
3.2 函數調用棧
dd@ubuntu:/mnt/hgfs/systemtap/example$ cat./timer.stp
#!/usr/bin/stap
probekernel.function("do_timer").return {
print_backtrace();
printf("\n");
exit();
}
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap ./timer.stp
Returning from: 0xc1091520 : do_timer+0x0/0x40 [kernel]
Returning to : 0xc1097e4a : tick_do_update_jiffies64+0xca/0x130 [kernel]
0xc1098074 : tick_sched_timer+0xc4/0xd0[kernel]
0xc1068f70 : __run_hrtimer+0x70/0x190 [kernel]
0xc1069be7 : hrtimer_interrupt+0xe7/0x2a0 [kernel]
0xc15d0769 :smp_apic_timer_interrupt+0x59/0x8d [kernel]
0xc15c9815 : apic_timer_interrupt+0x31/0x38[kernel]
0xc1036f35 : native_safe_halt+0x5/0x10[kernel]
0xc1017b66 : default_idle+0x46/0x190 [kernel]
0xc1018716 : cpu_idle+0xb6/0xe0 [kernel]
0xc159f715 : rest_init+0x5d/0x68 [kernel]
0xc18ae9be : start_kernel+0x35d/0x363 [kernel]
0xc18ae303 : i386_start_kernel+0xa6/0xad[kernel]
4 內核模塊測試
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ sudostap ./vmx_inject_irq.stp
[sudo] password for dd:
semantic error: while resolving probepoint: identifier 'kernel' at ./vmx_inject_irq.stp:1:7
source: probe kernel.function("vmx_inject_irq") {
^
semantic error: no match
Pass 2: analysis failed. Try again with another '--vp 01' option.
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ cat/proc/kallsyms | grep vmx_inject_irq
00000000 t vmx_inject_irq [kvm_intel]
kvm_intel以模塊形式加載,其symbol不能以kernel.function形式訪問。
Ubuntu平臺上需要修正動態module的debug info信息
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ catfix-module.sh
#!/bin/sh
for file in `find /usr/lib/debug -name'*.ko' -print`
do
buildid=`eu-readelf-n $file| grep Build.ID: | awk '{print $3}'`
dir=`echo$buildid | cut -c1-2`
fn=`echo$buildid | cut -c3-`
mkdir-p /usr/lib/debug/.build-id/$dir
ln-s $file /usr/lib/debug/.build-id/$dir/$fn
ln-s $file /usr/lib/debug/.build-id/$dir/${fn}.debug
done
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ sudostap -l 'module("*kvm*").function("*")' >kvm-module-sysm.log
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ catkvm-module-sysm.log | grep vmx_inject_irq
module("kvm_intel").function("vmx_inject_irq@/build/buildd/linux-3.5.0/arch/x86/kvm/vmx.c:4036")
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ catvmx_inject_irq.stp
probemodule("kvm_intel").function("vmx_inject_irq") {
printf("-------------------------------------\n")
print_backtrace()
printf("-------------------------------------\n")
}
很多符號都無法解析啊??????? 根本原因待分析。
-------------------------------------
0xf86e4230 : vmx_inject_irq+0x0/0xe0[kvm_intel]
0xf998e694 [kvm]
0xc10654cd (inexact)
0xf997b25b [kvm] (inexact)
0xf998f11f [kvm] (inexact)
0xf997922b [kvm] (inexact)
0xc1075dee (inexact)
0xc1075ebf (inexact)
0xc1099a12 (inexact)
0xc109be08 (inexact)
0xf9978dd0 [kvm] (inexact)
0xc115e8fa (inexact)
0xc1069c65 (inexact)
0xc109c120 (inexact)
0xc115ee88 (inexact)
0xc15cff5f (inexact)
0xc15c0000 (inexact)
-------------------------------------
5 用戶態監控Utrace
5.1 Utrace patch
用戶態監控需要utrace支持,但utrace默認只有redhat系列的kernel支持,如fedora等。其他發行版Linux需要patch
Systemtap的README中有關於utrace支持的描述:
- Consider applying the utrace kernelpatches, if you wish to probe
user-space applications. http://sourceware.org/systemtap/wiki/utrace
Orif your kernel is near 3.5, apply the uprobes and related patches
(see NEWS). Or if your kernel is>= 3.5, enjoy the built-in uprobes.
- Build the kernel using your normalprocedures. Enable
CONFIG_DEBUG_INFO, CONFIG_KPROBES, CONFIG_RELAY, CONFIG_DEBUG_FS,
CONFIG_MODULES, CONFIG_MODULE_UNLOAD, CONFIG_UTRACE if able
- % make modules_install install headers_install
- Boot into the kernel.
Ubuntu 要支持utrace有兩種方式:
1. 自己給 Ubuntu 自帶的老 kernel 應用 utrace 補丁,並重新編譯。但好像redhat提供的patch不能訪問了
http://people.redhat.com/roland/utrace/
Ubuntu 11.10 (Oneiric)上編譯帶utrace補丁的內核
http://chaoslawful.iteye.com/blog/1463564
2. 將 kernel 升級到官方最新的 3.5 或以上的版本。最新的kernel 默認包含了 uprobes 機制,不再需要 utrace 補丁了。
Kernel hacking --->
[*] Tracers --->
[*] Enable uprobes-based dynamic events
新的內核裏已經沒有選項CONFIG_UTRACE,選擇utrace相關的選項即可,包括CONFIG_ARCH_SUPPORTS_UPROBESCONFIG_UPROBES CONFIG_UPROBE_EVENT CONFIG_KPROBES CONFIG_DEBUG_FS 等
dd@ubuntu:/mnt/hgfs/systemtap/example$ cat/boot/config-3.5.0-17-generic | grep UPROBE
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENT=y
dd@ubuntu:/mnt/hgfs/systemtap/example$ cat/boot/config-3.5.0-17-generic | grep KPROBE
CONFIG_KPROBES=y
CONFIG_HAVE_KPROBES=y
# CONFIG_KPROBES_SANITY_TEST is not set
CONFIG_KPROBE_EVENT=y
如果不是必須用某一個版本,建議直接升級到3.5以後的內核版本,如Ubuntu 12.10
5.2 函數跟蹤測試
首先寫了一個用戶態的測試函數,可參考如下
http://blog.csdn.net/astrotycoon/article/details/8142588
Utrace調試的程序需要在編譯時加 -g 選項
dd@ubuntu:/mnt/hgfs/systemtap/example$./utrace-test
Obtained 8 stack frames.nm
./utrace-test() [0x80485c1]
[0xb77a3400]
./utrace-test() [0x804869f]
./utrace-test() [0x80486ac]
./utrace-test() [0x80486b6]
./utrace-test() [0x80486eb]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75ff4d3]
./utrace-test() [0x80484b1]
有函數調用棧,但是沒有函數名稱,莫非是編譯優化導致無符號表?經查需要傳遞(-rdynamic),-rdynamic可用來通知鏈接器將所有符號添加到動態符號表中
dd@ubuntu:/mnt/hgfs/systemtap/example$ gcc-g -rdynamic utrace-test.c -o utrace-test
dd@ubuntu:/mnt/hgfs/systemtap/example$./utrace-test
Obtained 7 stack frames.
./utrace-test(print_trace+0x1f) [0x80487ab]
./utrace-test(dummy_function_3+0xb)[0x8048840]
./utrace-test(test2+0xb) [0x804884d]
./utrace-test(test1+0xb) [0x804885a]
./utrace-test(main+0xb) [0x8048867]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75f44d3]
./utrace-test() [0x80486c1]
OK,能夠正確打印出函數調用順序,參考基準已經建立起來了。
如下測試顯示stap可以監測到待調試程序utrace-test的函數列表。
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap -l 'process("./utrace-test").function("*")'
process("/mnt/hgfs/systemtap/example/utrace-test").function("dummy_function_3@/mnt/hgfs/systemtap/example/utrace-test.c:26")
process("/mnt/hgfs/systemtap/example/utrace-test").function("main@/mnt/hgfs/systemtap/example/utrace-test.c:39")
process("/mnt/hgfs/systemtap/example/utrace-test").function("print_trace@/mnt/hgfs/systemtap/example/utrace-test.c:7")
process("/mnt/hgfs/systemtap/example/utrace-test").function("test1@/mnt/hgfs/systemtap/example/utrace-test.c:34")
process("/mnt/hgfs/systemtap/example/utrace-test").function("test2@/mnt/hgfs/systemtap/example/utrace-test.c:30")
當任意一個函數被調用時,打印其函數名及參數
dd@ubuntu:/mnt/hgfs/systemtap/example$ catutrace.stp
#!/usr/bin/stap
probeprocess("./utrace-test").function("*") {
printf("%s(%s)\n", probefunc(), $$parms);
}
實際檢測到的函數調用順序與實際匹配
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap ./utrace.stp
_start()
__libc_csu_init()
__i686.get_pc_thunk.bx()
_init()
frame_dummy()
register_tm_clones()
main()
test1()
test2()
dummy_function_3()
print_trace()
__do_global_dtors_aux()
deregister_tm_clones()
_fini()
5.3 打印用戶態調用棧
利用print_backtrace打印監測點的函數調用棧。
dd@ubuntu:/mnt/hgfs/systemtap/example$ catutrace-stack.stp
#!/usr/bin/stap
probeprocess("./utrace-test").function("dummy_function_3"){
printf("%s(%s)\n", probefunc(),$$parms);
print_backtrace();
printf("\n");
}
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap ./utrace-stack.stp
semantic error: while resolving probepoint: identifier 'process' at ./utrace.stp:3:7
source: probeprocess("./utrace-test").function("dummy_function_3").return{
semantic error: process return probes not available withinode-based uprobes
Pass 2: analysis failed. Try again with another '--vp 01' option.
去掉return,再執行
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap ./utrace-stack.stp
WARNING: no or bad debug frame hdr
WARNING: No binary search table for eh frame, doing slow linear searchfor stap_3fb2bcd721ab82a6de577773eff766b_10596
dummy_function_3()
0xf8887e10 [stap_3fb2bcd721ab82a6de577773eff766b_10596+0xbe10/0x0]
0xf8888209[stap_3fb2bcd721ab82a6de577773eff766b_10596+0xc209/0x0]
0xf88868b0[stap_3fb2bcd721ab82a6de577773eff766b_10596+0xa8b0/0x0]
0xc113eb3f
0xc104e51b (inexact)
0xc161ac9d (inexact)
沒有打印出用戶態的調用堆棧,具體原因待分析。
6 Systemtap加載KO異常,Unknown symbol inmodule
6.1 異常現象
加載動態編譯出來的module時失敗Unknown symbol inmodule
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ sudostap -ve 'probe begin { log("hello world") exit() }'
。。
Pass 4: compiled C into"stap_aeecf973d1b00387ff715fbb9a761ae9_688.ko" in6010usr/1080sys/10345real ms.
Pass 5: starting run.
Error inserting module'/tmp/stapoo63sX/stap_aeecf973d1b00387ff715fbb9a761ae9_688.ko': Unknown symbolin module
WARNING: /usr/bin/staprun exited withstatus: 1
Pass 5: run completed in 0usr/0sys/11realms.
Pass 5: run failed. Try again with another '--vp 00001' option.
dd@ubuntu:/mnt/hgfs/systemtap/kvm$ sudostap -e 'probe kernel.function("sys_open") {log("helloworld") exit()}'
Error inserting module'/tmp/stapPq4Mh5/stap_b3efd67e86a8df14579c7f419ddcd07f_826.ko': Unknown symbolin module
WARNING: /usr/bin/staprun exited withstatus: 1
Pass 5: run failed. Try again with another '--vp 00001' option
6.2 分析異常原因
查看內核log,分析Unknown symbol inmodule的意義,這種問題通常都是某個symbol無法找到。
dd@ubuntu:~/systemtap/systemtap-2.5$ dmesg| tail
…
[ 76.855961] stap_880c6f2fc0076129ea6feb7db23da703_899: Unknown parameter `_stp_bufsize'
“-k”參數保留運行過程中生產的文件,查找線索
dd@ubuntu:~$ sudo stap -k -ve 'probe begin{ log("hello world") exit() }'
。。
Pass 5: starting run.
Error inserting module'/tmp/staplCMn6Z/stap_4270.ko': Unknown symbol in module
WARNING: /usr/bin/staprun exited withstatus: 1
Pass 5: run completed in 0usr/0sys/3realms.
Pass 5: run failed. Try again with another '--vp 00001' option.
Keeping temporary directory"/tmp/staplCMn6Z"
dd@ubuntu:~$ sudo nm/tmp/staplCMn6Z/stap_4270.ko | grep _stp_bufsize
00000032 r __mod__stp_bufsize76
00000050 r __mod__stp_bufsizetype75
00000000 r __param__stp_bufsize
00000515 r __param_str__stp_bufsize
00000130 b _stp_bufsize
“--vp00005”啓動systemtap的調試選項獲得更多的log。 具體用法可參考http://blog.163.com/digoal@126/blog/static/163877040201391434530674/
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap --vp 00005 -ve'probe begin{printf("hello world\n") exit()}'
。。。
Pass 5: starting run.
Running /usr/local/bin/staprun -v -v -v -v -v-R /tmp/stapwlfJOn/stap_333448f0b355beeb59e41a501a5421fd_817.ko
staprun:parse_modpath:386
。。。
staprun:insert_module:71 inserting module/tmp/stapwlfJOn/stap_333448f0b355beeb59e41a501a5421fd_817.ko
staprun:insert_module:97module options: _stp_bufsize=0
staprun:insert_module:105 module pathcanonicalized to '/tmp/stapwlfJOn/stap_333448f0b355beeb59e41a501a5421fd_817.ko'
。。
staprun:insert_module:183 Modulestap_333448f0b355beeb59e41a501a5421f_6340 inserted from file/tmp/stapwlfJOn/stap_333448f0b355beeb59e41a501a5421fd_817.ko
ERROR:Couldn't insert module '/tmp/stapwlfJOn/stap_333448f0b355beeb59e41a501a5421fd_817.ko':Unknown symbol in module
Spawn waitpid result (0x100): 1
WARNING: /usr/local/bin/staprun exited withstatus: 1
Pass 5: run completed in 0usr/0sys/2real ms.
Pass 5: run failed. [man error::pass5]
Running rm -rf /tmp/stapwlfJOn
Spawn waitpid result (0x0): 0
Removed temporary directory"/tmp/stapwlfJOn"
dd@ubuntu:~/systemtap/systemtap-2.5$ grep -nr"_stp_bufsize" ./
Binary file ./staprun/staprun matches
Binary file ./staprun/staprun-staprun.omatches
./staprun/staprun.c:190: /* Add the _stp_bufsize option. */
./staprun/staprun.c:192: "_stp_bufsize=%d",buffer_size))
./runtime/transport/transport.c:75:static int_stp_bufsize;
./runtime/transport/transport.c:76:module_param(_stp_bufsize,int, 0);
./runtime/transport/transport.c:77:MODULE_PARM_DESC(_stp_bufsize,"buffer size");
./runtime/transport/transport.c:408: if (_stp_bufsize) {
./runtime/transport/transport.c:409: unsigned size = _stp_bufsize * 1024 *1024;
./runtime/transport/ring_buffer.c:83: unsigned long buffer_size = _stp_bufsize* 1024 * 1024;
./runtime/transport/transport.h:48:static int_stp_bufsize;
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudomodinfo stapoMK4aC/stap_12809.ko [sudo] password for dd:
filename: stapoMK4aC/stap_12809.ko
。
parm: _stp_bufsize:buffer size (int)
從上面的代碼看,模塊參數_stp_bufsize確實存在,且加載的過程沒有問題啊,很奇怪爲什麼會fail。難道是版本不匹配?嘗試卸載2.0版本,安裝2.5版本,依然有問題
6.3 Workaround
於是嘗試分析其內在原理,看是否能真正fix或者workaround這個問題。
./runtime/transport/transport.c接收模塊參數_stp_bufsize,但是從代碼看並沒有特別的用處;另外通過“staprun:insert_module:97module options: _stp_bufsize=0”知道./staprun/staprun.c:192實際傳入的參數值是0,因此可去掉special_options參數試試
static int insert_stap_module(privilege_t*user_credentials)
{
charspecial_options[128] = {0};
#if 0
/* Add the _stp_bufsize option. */
if (snprintf_chk(special_options,sizeof (special_options),
"_stp_bufsize=%d", buffer_size))
return -1;
#endif
stap_module_inserted= insert_module(modpath, special_options,
modoptions,
assert_stap_module_permissions,
user_credentials);
if (stap_module_inserted != 0)
err("Error insertingmodule '%s': %s\n", modpath, moderror(errno));
returnstap_module_inserted;
}
重新編譯加載後,一切正常。
7 error: implicit declaration offunction ‘unregister_uprobe’
dd@ubuntu:/mnt/hgfs/systemtap/example$ sudostap -ve 'probe process("./utrace-test").function("*"){printf("-------------------------------------\n") print_backtrace()printf("-------------------------------------\n")}'
。。。
In file included from/tmp/stapRI0noH/stap_db3a2cd3f06ff0a78a55d936a32da8b0_1325_src.c:184:0:
/usr/share/systemtap/runtime/uprobes-inode.c:In function ‘stp_inode_uprobes_unreg’:
/usr/share/systemtap/runtime/uprobes-inode.c:94:3:error: implicit declaration of function ‘unregister_uprobe’[-Werror=implicit-function-declaration]
/usr/share/systemtap/runtime/uprobes-inode.c:In function ‘stp_inode_uprobes_reg’:
/usr/share/systemtap/runtime/uprobes-inode.c:107:3:error: implicit declaration of function ‘register_uprobe’ [-Werror=implicit-function-declaration]
cc1: all warnings being treated as errors
make[1]: ***[/tmp/stapRI0noH/stap_db3a2cd3f06ff0a78a55d936a32da8b0_1325_src.o] Error 1
make: *** [_module_/tmp/stapRI0noH] Error 2
WARNING: make exited with status: 2
Pass 4: compiled C into"stap_db3a2cd3f06ff0a78a55d936a32da8b0_1325.ko" in280usr/170sys/681real ms.
Pass 4: compilation failed. Try again with another '--vp 0001' option.
https://bugs.launchpad.net/ubuntu/+source/systemtap/+bug/1075772
stap fails with: "error: implicitdeclaration of function ‘unregister_uprobe’"
dd@ubuntu:/mnt/hgfs/systemtap$ stap -V
Systemtap translator/driver (version 1.7/0.153 Debian version1.7-1ubuntu1 (quantal))
Copyright (C) 2005-2012 Red Hat, Inc. andothers
This is free software; see the source forcopying conditions.
enabled features: AVAHI LIBSQLITE3 NSSBOOST_SHARED_PTR TR1_UNORDERED_MAP NLS
內核中沒有systemtap/runtime引用的API register_uprobe。在新版本的systemtap中已經fix了,因此需要下載源代碼手動升級
dd@ubuntu:/mnt/hgfs/systemtap$ sudo grep"\suprobe.*register" /proc/kallsyms
c10fe1d0 T uprobe_register
c10fe430 T uprobe_unregister
升級到2.0版本之後,正常
dd@ubuntu:~/systemtap/systemtap-2.0$ stap-V
Systemtap translator/driver (version2.0/0.156, non-git sources)
Copyright (C) 2005-2012 Red Hat, Inc. andothers
This is free software; see the source forcopying conditions.
enabled features: TR1_UNORDERED_MAP NLS
8 參考資料
Systemtap WiKi
https://wiki.ubuntu.com/Kernel/Systemtap
A guide on how to install Systemtap on anUbuntu system
https://sourceware.org/systemtap/wiki/SystemtapOnUbuntu
Linux 下的一個全新的性能測量和調式診斷工具Systemtap, 第 3 部分: Systemtap
http://www.ibm.com/developerworks/cn/linux/l-cn-systemtap3/#main
使用systemtap調試Linux內核示例
http://lenky.info/archives/2013/02/2209
User-Space Probing
https://sourceware.org/systemtap/SystemTap_Beginners_Guide/userspace-probing.html
Dynamic Tracing with DTrace and SystemTap
https://gist.github.com/dhedlund/9635035
http://guiquanz.me/2013/04/21/systemtap-intro/
Systemtap : stap PROCESSING 5 stepsintroduce
http://blog.163.com/digoal@126/blog/static/163877040201391434530674/