How can I get python stack trace information using GDB?

Question

I am using GDB to debug a segmentation fault in my python application on Kubuntu 12.04. Supposedly GDB version 7 has built-in macros for extracting information about the python stack (http://docs.python.org/devguide/gdb.html), but I am having trouble getting it to work. I have installed python-dbg.

When I ask for a python stack trace in GDB, the result looks like this:

(gdb) py-bt
#5 (unable to read python frame information)
#16 (unable to read python frame information)
#26 (unable to read python frame information)
...

My GDB version is 7.4-2012.04-0ubuntu2, Python is 2.7.3-0ubuntu3.

Answer 1

Here's the problem: to have access to debugging symbols in GDB, you must invoke a different binary: "python-dbg" instead of "python" (found this in /usr/share/doc/python2.7-dbg/README.debug).

Answer 2

On Ubuntu 16.04 I managed to get Python stack trace in Python 3.5 by:

1. Installing python3-dbg and python3-dev:

   $ sudo apt install python3-dbg python3-dev

   python3-dbg package comes with short documentation how to use it in /usr/share/doc/python3-dbg/README.debug which I will use in the next step.

2. Appending unpacked GDB helper script /usr/share/doc/python3.5/gdbinit.gz to ~/.gdbinit:

   zcat /usr/share/doc/python3.5/gdbinit.gz >> ~/.gdbinit

Now gdb will be able to find symbols for Python binary and py-bt works for displaying Python stack trace in gdb:

$ gdb -p 4762
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 4762
[New LWP 4852]
[New LWP 4853]
[New LWP 4854]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f38e43deb5d in poll () at ../sysdeps/unix/syscall-template.S:84
84      ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/bin/indicator-cpufreq", line 80, in <module>
    Gtk.main()
(gdb)

Answer 3

Sometimes, it is not possible to get debuginfo; for example if you are using conda Python, these packages do not come with debuginfo. It turns out that py-spy (https://github.com/benfred/py-spy) has a more robust implementation of Python backtrace dumping that doesn't rely on having accurate debug symbols available. However, as it also uses ptrace, you will have to detach gdb first. Here's how to make it work:

• Do your thing in gdb, getting to the breakpoint you want to get the Python backtrace for.
• Make gdb pass through SIGSTOP with handle SIGSTOP noprint nostop pass
• Pause the process with signal SIGSTOP
• Detach gdb with detach. Note the PID
• Run py-spy dump --pid $PID in another terminal. This gives you your backtrace.
• You can keep running gdb with attach $PID. Note that you can continue the process even if it is SIGSTOP'ed and it will run as normal, and will continue to be paused if you detach again

Answer 4

Maybe this helps someone: The binary is named python2.7-dbg on my Debian system, coming from the python2.7-dbg package. I also installed the python2.7-dev package and apt-get source python2.7-dbg, so that gdb could find the source files to the Python interpreter.

With all this in place, I managed to debug the SIGSEGV I was running into: https://bugs.python.org/issue34870