Debug Symbols

Debug Symbols covers creating two part executables on Linux platforms. Two part executable is the term used for the fully linked executable stripped of debug symbols (the program or shared object, but not the archive) and its debug information file (the file with the debug symbols). Microsoft Windows already creates them by default using PDB files, so there's nothing to discuss for the platform.

The process below uses a modified makefile to create two new files for use under the debugger and crash reporting. First is cryptest.exe.debug, and second is libcryptopp.so.debug. The files are created with make symbols recipe, and the files are installed as part of the make install rule. Once installed, the GDB debugger will be able to find and use them.

Debug information has minimal impact on application's load times because the link-loader does not map the debug information at runtime. For a discussion, see How does the gcc -g option affect performance? on the GCC mailing list. Symbol visibility will likely have more of an influence on an application's size and load times. Its also discussed below.

Note: if you are stripping symbols to reduce your program's size, then also see the make lean target. It adds function and data sections so your program can be dead code stripped by the linker.

Note: stripping does not work on OS X because the operating system provides a down level version of Binutils.

Note: this is not part of the Crypto++ library. You must download and install the patch below.

Binary Size

Creating two part executables has a profound effect on the size of the resulting executables. As an example, here is the size of cryptest.exe and libcryptopp.so before stripping:

cryptopp-symbols$ ls -l *.exe *.so *.debug
ls: cannot access *.debug: No such file or directory
-rwxrwxr-x 1 jwalton jwalton 43088263 Mar  1 19:05 cryptest.exe
-rwxrwxr-x 1 jwalton jwalton 29870086 Mar  1 19:04 libcryptopp.so

After running make symbols, the executable sizes are reduced by nearly 85%:

cryptopp-symbols$ ls -l *.exe *.so *.debug
-rwxrwxr-x 1 jwalton jwalton  3586144 Mar  1 19:08 cryptest.exe
-rw-rw-rw- 1 jwalton jwalton 39505207 Mar  1 19:08 cryptest.exe.debug
-rwxrwxr-x 1 jwalton jwalton  3401392 Mar  1 19:08 libcryptopp.so
-rw-rw-rw- 1 jwalton jwalton 26471470 Mar  1 19:08 libcryptopp.so.debug

Makefile Patch

To create a two part executable, you need to download and apply the patch below. The patch below adds the following to the GNUmakefile:

# Used for 'make symbol' recipe. Only executables that have been fully linked can be stripped
DEBUG_LIB_SYM = libcryptopp.so.debug
DEBUG_EXE_SYM = cryptest.exe.debug
...

symbol symbols:
	-objcopy --only-keep-debug cryptest.exe $(DEBUG_EXE_SYM)
	-objcopy --only-keep-debug libcryptopp.so $(DEBUG_LIB_SYM)
	-chmod a+r *.debug
	-chmod a-x *.debug
	-strip --strip-debug --strip-unneeded cryptest.exe
	-strip --strip-debug --strip-unneeded libcryptopp.so
	-objcopy --add-gnu-debuglink=$(DEBUG_EXE_SYM) cryptest.exe
	-objcopy --add-gnu-debuglink=$(DEBUG_LIB_SYM) libcryptopp.so

Additionally, it adds some cleanup for the files it creates. For example, the make clean and make remove recipes each removes cryptest.exe.debug and libcryptopp.so.debug, if present.

Creating Two Part Executables

The makefile produces a fully linked executable with symbols. You need to run objcopy and strip on it to create a two part executable. Running objcopy and strip ensures the BuildID is maintained across both parts of the executable.

The two part executable consists of a stripped executable and its debug information file. The debug information file holds the symbols for a fully linked executable. It is the equivalent to a Microsoft Visual Studio Program Database (PDB).

To create the debug information files and strip the executables, perform the following. Note the addition of the make symbols in a typical make/install workflow:

cd cryptopp

# Build the static library, dynamic library and the cryptest program
make static dynamic cryptest.exe

# Create the two part executable on the dynamic library and the cryptest program
make symbols

# Verify the library with the cryptest program
./cryptest.exe v

# Install as normal
sudo make install PREFIX=/usr/local

GDB and Symbols

To ensure GDB locates the executable's symbols, the debug information file should be installed side-by-side with the executable.

$ ls /usr/local/lib/libcryptopp.*
/usr/local/lib/libcryptopp.a   /usr/local/lib/libcryptopp.so.debug
/usr/local/lib/libcryptopp.so

$ file /usr/local/lib/libcryptopp.so
/usr/local/lib/libcryptopp.so: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV),
dynamically linked, BuildID[sha1]=1eda381ddb5526ca6f0740efd167865f6b9d62e6, stripped

$ file /usr/local/lib/libcryptopp.so.debug 
/usr/local/lib/libcryptopp.so.debug: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV),
dynamically linked, BuildID[sha1]=1eda381ddb5526ca6f0740efd167865f6b9d62e6, not stripped

$ gdb
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
...

(gdb) file /usr/local/lib/libcryptopp.so
Reading symbols from /usr/local/lib/libcryptopp.so... done.
Reading symbols from /usr/local/lib/libcryptopp.so.debug... done.

Considerable time was wasted trying to get GDB to locate and load the symbols if the symbol file was placed in /usr/lib/debug (for example, allegedly used by Debian, Ubuntu, Red Hat and Fedora). If you can provide steps to actually get the feature to work, then please provide the information on the Crypto++ User Group.

Symbol visibility

Earlier it was said, Symbol visibility will likely have more of an influence on an application's size and load times. This topic is discussed on the GCC's wiki under Symbol Visibility. The patch below includes the following to help reduce size and increase performance as it relates to symbol visibility.

Be careful of adding symbol visibility to shared objects. Keep in mind it is an unsupported configuration, so be sure the things you expect are exported. A lot of stuff is missing, and you will need to add CRYPTOPP_DLL to classes that are missing. A lot of stuff is missing because CRYPTOPP_DLL is used for the FIPS DLL, and the FIPS DLL only includes core crypto algorithms like AES and SHA and nothing more.

With the warnings aside, and if you are certain you want to use visibility, then add the following after the ifeq ($(CXX),gcc) test around line 25 of the GNUmakefile:.

GCC40_OR_LATER = $(shell $(CXX) -v 2>&1 | $(EGREP) -c "^gcc version ([4-9])")
ifeq ($(GCC40_OR_LATER),1)
  CXXFLAGS += -fvisibility=hidden
endif

CLANG32_OR_LATER = $(shell $(CXX) -v 2>&1 | $(EGREP) -c "clang version (3.[2-9]|[4-9])")
ifeq ($(CLANG32_OR_LATER),1)
  CXXFLAGS += -fvisibility=hidden
endif

GNU_LD216_OR_LATER = $(shell $(LD) -v 2>&1 | $(EGREP) -i -c '^gnu ld .* (2\.1[6-9]|2\.[2-9])')
ifeq ($(GNU_LD216_OR_LATER),1)
  LDFLAGS += -Wl,--exclude-libs,all
endif

The LD 2.16 test is used to keep the Crypto++ library from re-exporting symbols it encounters (like symbols from the standard C++ library). Then add CXXFLAGS and LDFLAGS to the libcryptopp.so recipe:

libcryptopp.so: $(LIBOBJS)
	$(CXX) -o $@ -shared $(CXXFLAGS) $(LDFLAGS) $(LIBOBJS)

Finally, add the following to the end of config.h. A good place is after the __MWERKS__ tests to set CRYPTOPP_DLL:

#if __GNUC__ >= 4
  #undef CRYPTOPP_DLL
  #define CRYPTOPP_DLL __attribute__ ((visibility ("default")))
#endif

Downloads

Gnumakefile-symbols.zip - Patch to update GNUMakefile to create a two part executable. The first is the stripped executable, and the second is the debug information file.

cryptopp-combined-datatdir-symbols.zip - Rollup patch that combines DataDir patch and Debug Symbols patch to avoid resolving conflicts by hand.