Ftrace in Linux kernel

• A look at ftrace http://lwn.net/Articles/322666/


• trace-cmd - command line reader for ftrace http://lwn.net/Articles/341902/


• Debugging the kernel using Ftrace - part 1 http://lwn.net/Articles/365835/


• Debugging the kernel using Ftrace - part 2 http://lwn.net/Articles/366796/


• Secrets of the Ftrace function tracer http://lwn.net/Articles/370423/

NT debugging

http://blogs.msdn.com/b/ntdebugging/

How to debug Linux kernel using QEMU?

How to debug Linux kernel using QEMU?
1. Install QEMU and install a QEMU VM, the Linux on VM is the one being debugged.
2. Enable some debug configurations, then recompile the kernel. The debugging configurations is described on LDD3 (Linux device driver), chapter 4.
3. Start QEMU VM, use –S argument. It means the VM will be suspended when start.
4. Click on the window of the QEMU VM, press ctrl+alt+2 to switch to QEMU monitor, then type: gdbserver 1234. This will start gdbserver built in QEMU and it listens on the port 1234. Then press “c” to “continue”. Then press alt+ctrl+1 to switch back to OS console.
5. Start a GDB on the host. To get the symbols of the linux kernel, put the source files of the compiled kernel on the same path on the host. Use “gdb vmlinux” to start gdb. Note the vmlinux is an uncompressed file and its size is about 30MB. If it is too small, it may not have necessary debug symbols.
6. In gdb command line, type: “target remote localhost:1234”. This 1234 is the port number used in the gdbserver in step 4.
7. Press “enter”, then the VM should be stopped, and gdb will display some source files, telling where the current function is.
8. To setup a breakpoint, use “b xxx” in gdb, then press “c” to continue.

Get vmlinux imge

xen:
http://www.redhat.com/archives/fedora-xen/2007-May/msg00141.html

make vmlinux

OutputDebugString on Linux?

OutputDebugString is very easy to use on Windows platform. But it is not availabe on Linux. However, we can use syslog() for similar purpose. Then use tail -f /var/log/messages to watch the log.

Ref:
http://www.ifm-services.com/people/jamesk/papers/kylix/adv_in_kylix-2.php
http://forum.kde.org/outputdebugstring-equivalent-t-46658.html

Also , check man 3 syslog.

Debug Python

http://www.ferg.org/papers/debugging_in_python.html
http://infohost.nmt.edu/tcc/help/pubs/python22/debugger.html

winpdb:
http://winpdb.org/

debug linux

kernel oops:
http://www.91linux.com/html/article/kernel/20070623/2921.html
Mastering Linux debugging techniques
http://idcnews.net/html/edu/20070101/294479.html
kgdb
http://zhanglinbao.bokee.com/2942457.html
Inside the Linux kernel debugger
http://www.stuttgarter.org/forum/redirect.php?tid=22540&goto=lastpost
EDB
http://www.codef00.com/projects.php#Debugger
Use QEMU to debug kernel
http://issaris.blogspot.com/2007/12/download-linux-kernel-sourcecode-from.html

Debugging Linux kernel with VMware

From vmware
http://stackframe.blogspot.com/2007/04/debugging-linux-kernels-with.html
From IDA pro
http://www.hex-rays.com/idapro/debugger/gdb_vmware_linux.pdf
GDB for windows:
http://www.redantigua.com/gdb-win.html

debug Xen

1. debug dom0 and dom u
http://lists.xensource.com/archives/html/xen-devel/2008-10/msg00195.html

2. build option :

Xen provides a number of build-time options which should be set as environment variables or passed on make's command-line.

verbose=y

Enable debugging messages when Xen detects an unexpected condition. Also enables console output from all domains.

debug=y

Enable debug assertions. Implies verbose=y. (Primarily useful for tracing bugs in Xen).

debugger=y

Enable the in-Xen debugger. This can be used to debug Xen, guest OSes, and applications.

perfc=y

Enable performance counters for significant events within Xen. The counts can be reset or displayed on Xen's console via console control keys.

3. use serial cable
http://en.opensuse.org/How_to_Capture_Xen_Hypervisor_and_Kernel_Messages_using_a_Serial_Cable

4. KDB
http://lists.xensource.com/archives/html/xen-devel/2008-11/msg00165.html