WAD (Wrapped Application Debugger) Author(s): David M. Beazley (beazley@cs.uchicago.edu) Copyright (C) 2001 University of Chicago This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA See the file COPYING for a complete copy of the LGPL. $Id: README 10001 2007-10-17 21:33:57Z wsfulton $ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!! DISCLAIMER !!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! THIS IS EXPERIMENTAL UNMAINTAINED RESEARCH SOFTWARE THAT REPRESENTS WORK IN PROGRESS. IT IS NOT PORTABLE, IT HAS NOT BEEN EXHAUSTIVELY TESTED, AND IT MIGHT NOT WORK AT ALL. PLEASE KEEP AWAY FROM SMALL CHILDREN, PETS, NUCLEAR REACTORS, AIR-TRAFFIC CONTROL, AND VOTING MACHINES. 0. Supported Platforms This software is currently only known to work with 32-bit applications on Sun Sparc Solaris 2.8 and recent i386-Linux systems. In addition, there are numerous issues concerning the interaction of this software with signal handling, thread libraries, and compilers. Please read this entire document before proceeding. 1. Introduction WAD is an embedded error-recovery mechanism that attempts to convert fatal errors such as SIGSEGV, SIGBUS, and SIGFPE into sensible error messages and exceptions. It is primarily designed to support scripting language extension programming although it can also be used with stand-alone C programs. The primary goal of this system is to explore an alternative approach to mixed scripting-compiled debugging. It requires no modifications or recompilation of existing software. Therefore, it should be relatively easy to try out. Feedback is welcome. Contributions and modifications are even more welcome. 2. Compilation and Installation WAD is not particularly portable and at this time, only two platforms are supported: Sun Sparc Solaris and i386-Linux. Installation is as follows: ./configure make make install The build process creates the following shared libraries: libwad.so - Standalone WAD. Can be linked with C/C++ programs. libwadpy.so - Python WAD. Can be linked to Python extension modules or imported on its own as 'import libwadpy' libwadtcl.so - Tcl WAD. Can be linked to Tcl extension modules or loaded as 'load libwadtcl.so'. libwadpl.so - Perl WAD. Can be linked to Perl extension modules or loaded as 'libwadpl'. To install the libraries, simply type 'make install'. This copies the libraries to /usr/local/lib (unless you modify the makefile). Notes: - The Sun version of WAD has only been tested when compiled with the Sun Workshop C/C++ compilers. However WAD works with other programs that have been compiled with gcc. If gcc is installed on your machine, you may want to set the following environment variables before running configure: setenv CC cc setenv CXX CC ./configure - You may need to modify the Makefile to point to the installed locations of various scripting language libraries if you have installed them in non-traditional locations. - The Linux version has only been tested with 2.2-12 and 2.2-14 kernels and the RedHat 6.x distribution. Your mileage may vary. There may be some compatibility issues related to glibc and other parts of the system as well. 3. Using WAD WAD has no functional API nor does it have any command line options so it's pretty easy to describe---simply link the appropriate WAD library with your C code. For example: % cc blah.c -lwad Once linked, fatal errors will now produce stack traces. For example: % ./a.out seg starting. Segmentation fault. #2 0x400571eb in __libc_start_main() in 'libc-start.c', line 90 #1 0x08048b39 in main(argc=0x2,argv=0xbffffce4) in 'debug.c', line 62 #0 0x080489b3 in seg_crash(n=0x0) in 'debug.c', line 9 /r0/beazley/Projects/WAD/Wad/debug.c, line 9 int *a = 0; if (n > 0) seg_crash(n-1); => *a = 3; return 1; } For scripting languages, WAD works in a similar manner--simply link your scripting language extension module with the appropriate WAD library (wadpy, wadtcl, wadpl). For example: % ld -G $(OBJS) -lwadpy -o pymodule.so When the scripting module is loaded into the interpreter, WAD should automatically initialize. 4. Debugging Modes Due to WAD's experimental nature, a number of debugging modes can be set through the use of environment variables. These variables control WAD's runtime behavior and cause the system to dump debugging information for various stages of error recovery. A lot of this data is pretty ugly and probably only of interest to you if you are trying to debug WAD itself. WAD_DEBUG_SEGMENT - Displays information about the virtual memory map and mapping of addresses to segments. WAD_DEBUG_SYMBOL - Symbol table mapping. WAD_DEBUG_OBJECT - Loading/Unloading of object files. WAD_DEBUG_FILE - Loading/Unloading of raw files. WAD_DEBUG_HOLD - Freezes WAD before it returns from the signal handler. Useful if you need to attach a debugger to WAD itself. WAD_DEBUG_STABS - Display stabs data. WAD_DEBUG_RETURN - Display information about WAD return points. WAD_DEBUG_SYMBOL_SEARCH - Display all symbols in the symbol table that are searched. WAD_DEBUG_UNWIND - Display information about stack unwinding. WAD_DEBUG_SIGNAL - Display information about signal handling. WAD_DEBUG_INIT - Print initialization information. WAD_NOSTACK - Do NOT use an alternative signal handling stack. This may be necessary on certain Linux systems when threads are being used. WAD_ONESHOT - Disable WAD signal handler after first signal has been received. WAD_DEBUG_MEMORY - Print information about WAD memory use. WAD_DEBUG_STRINGS - Print information about WAD string manager. 5. Platform Specific Issues General: - WAD does not gracefully recover from errors that corrupt the call stack (i.e., buffer overlow). - Errors that destroy the process heap may or may not be recoverable depending on what has been destroyed. - WAD does not currently support 64 bit applications on any platform. - If executables have been stripped, their symbol tables might not have enough information to recover from errors. Therefore, if you are using Python, Tcl, or Perl from a binary distribution, you may want to rebuild non-stripped versions of these packages yourself. - WAD only works with programs that utilize the ELF32 linking format and stabs debugging data. Newer formats such as DWARF2 are not supported at this time. - WAD does not correctly report argument values for structures or floating point numbers yet. - Overly aggressive compiler optimization may lead to very strange WAD output. Solaris: - WAD is extremely slow at collecting debugging information from large applications. Linux: - The interaction of threads and signals are particularly problematic on this platform and may cause WAD not to work at all. Here are some specific thread-based issues that may arise: 1. WAD causes the program to crash immediately upon startup. This appears to be caused by a bug in in the implementation of sigaction() and the initialization of signals. This only occurs if WAD is directly linked to an executable using threads. It does not occur when WAD is dynamically loaded into a threaded application. 2. Programs may lock up when an error occurs. This is sometimes caused by an apparently broken implementation of sigaltstack(). One solution to this is to set the following environment variable: setenv WAD_NOSTACK in which case the WAD signal handler will use the same stack as the thread/process that generates the error. 3. WAD just crashes altogether and doesn't seem to do anything. It appears that some versions of Linux threads do *not* pass CPU context information correctly to signal handlers defined in threaded programs. There is no known fix to this at this time. Upgrade your system. - WAD does not work if it is compiled as PIC code. The WAD libraries should be compiled *without* the -fpic option. - WAD has to rely upon a heuristic register recovery scheme when it returns to scripting language interpreters. It seems to work, but it relies upon a very specific compiler code generation convention for saving registers in function prologues. It also relies upon the register save conventions described in the Linux Assembly HOWTO. - If you are using WAD with pre-installed binaries for Python, Tcl, and other scripting languages, it may not work correctly due to stripped symbol tables. Most Linux installs such as Redhat strip symbol tables from executables. This makes it difficult for WAD to determine context correctly (although it may still work since the dynamic loading symbol table is still available in most cases). 6. Language specific issues If WAD is linked with a normal C/C++ program, errors simply produce a stack trace that is printed on standard error. Python: WAD tries to raise a Python exception and return. At this time, the exception merely contains a traceback string. However, in future versions, it may be possible to access a complete exception object. Tcl: WAD returns a Tcl and places the stack trace into the Tcl variable $errorInfo. The wish shell uses this to dump error information. Perl: Perl doesn't seem to have a very well-defined exception handling mechanism. Standard functions tend to just exit. The WAD handler produces a C stack trace and produces a Perl stack trace using some code derived from the sigtrap module. Note: 3/23/01 - Perl support is currently broken. 7. Testing and Examples The Test directory contains some very simple code for testing WAD. In the most simple form, compile the stand-along test program 'debug' as follows: % cd Test % make Now, running it: % debug WAD debug program. Usage: debug type seg - Fail with an uninitialized pointer. bus - Fail with a bus error. abort - Fail with an assertion error. math - Fail with a math error. heap - Blow the process heap. overflow - Buffer overflow on the stack. % debug seg WAD debug program. Segmentation fault. #2 0x400581eb in __libc_start_main() in 'libc-start.c', line 90 #1 0x08048b61 in main(argc=0x2,argv=0xbffffc54) in 'debug.c', line 85 #0 0x080489d0 in seg_crash() in 'debug.c', line 15 /r0/beazley/Projects/WAD/Test/debug.c, line 15 int seg_crash() { int *a = 0; => *a = 3; return 1; } Additional targets 'make python', 'make tcl', and 'make perl' are also available. The scripts debug.py, debug.tcl, debug.pl can be used to test these extensions. 8. Documentation No official documentation exists at this time. However, details of WAD's design and implementation can be found in papers presented at the Ninth International Python Conference and the 2000 USENIX Technical Conference. Both papers can be obtained at: http://systems.cs.uchicago.edu/wad 9. To-Do If you find WAD to be interesting or useful, there are a variety of ways to contribute. Here is the short to-do list: - Better register management. Try to implement in a more portable way. Add some support code for recovering local variables that happen to be stored in registers. - Add heuristic for recovering functions called through an -fomit-frame-pointer compiler optimization scheme. This can probably be determined by looking at the function preamble machine code. Then one can back-trace to the calling function and look at it's preamble. - Continued clean up and modularization of the core. Many of the internal APIs could be greatly improved. - Support for ELF64 linking format. - Support for DWARF2 debugging data. - Improved support for stack-overflow and heap-corruption. Although WAD probably won't be able to recover, it still might be able to produce some informative diagnostics. - Removal of printf() and other high-level library calls which may not operate with a corrupted heap. - Better integration with scripting languages. - Support for new platforms. - Support for new scripting languages. Please contact me if you are interested in working on any of these projects. Dave Beazley (beazley@cs.uchicago.edu) June 24, 2001