Autotools

From Crypto++ Wiki
Jump to navigation Jump to search

The easiest way to build Crypto++ for distros is the unofficial Autotools files. There are six steps to the process and they are the same as other Autotools projects once you fetch the files. The six steps are detailed below and begin at Obtain Source Files and finish at Install the Library.

Though Autotools is unofficial it is well supported by the Crypto++ and Debian projects. One of the gaps for Autotools is we don't support a lot of Autoconf options, like --disable-static or --disable-PIC. If you don't want the static library then manually delete it after building the library. If you don't want position independent code then modify configure.ac.

The Autotools project files were first available in 2016 for Crypto++ 5.6.3. Autotools users should thank László Böszörményi who is our Debian package maintainer. We ripped most of his work when we started supporting the build system.

The Autotools project files are maintained by the community, and not the library. If you want to use Autotools then download it from GitHub | cryptopp-autotools. If you have an improvement, then make the change and submit a pull request. It is a separate project to ensure folks don't accidentally use it. (Our unofficial Android.mk and CMake projects are separate projects, too).

A releated topic is Requirements. Crypto++ attempts to minimize external dependencies like Autotools and CMake, so Autotools will likely never be part of the official sources.

Note: Building a dist tarball is broken. The configure script is hard-wired for the machine where configure.ac was run. It will be Ok if you run configure.ac and configure on the same architecture, but it will be wrong for a different architecture. For example running configure.ac on x86_64 will produce incorrect results if configure is run on aarch64. Another Autotools mystery...

Bug Reports

Autotools questions should be asked on the mailing list. Autotools bug reports should be reported at cryptopp-autotools. Please do not ask questions or report bugs at the Crypto++ GitHub.

Using Autotools

The easiest way to build Crypto++ for distros is the unofficial Autotools files. Though Autotools is unofficial it is well supported by the Crypto++ and Debian projects.

The Autotools project keeps in stride with Crypto++ Master. You can work from the tip of master on both Crypto++ and Autotools and things will just work for you. Crypto++, Autotools and CMake tag releases so you can also download release zips. The OpenCSW Crypto++ 7.0 example uses master and checks out the CRYPTOPP_7_0_0 tag.

Autotools does not support a lot of Autoconf options, like --disable-static or --disable-PIC, because distros don't vary much in their build specs. If you don't want the static library then manually delete it after building the library. If you don't want position independent code then modify configure.ac.

Obtain Source Files

There are two sets of source files to obtain. The first is the Crypto++ sources and the second is the Autotools project files.

To obtain the latest Crypto++ source files clone them from Wei Dia's GitHub.

$ git clone https://github.com/weidai11/cryptopp
Cloning into 'cryptopp'...
remote: Counting objects: 18015, done.
remote: Compressing objects: 100% (33/33), done.
...

$ cd cryptopp

The Autotools files are a separate project so they must be cloned separately as shown below.

$ git clone https://github.com/noloader/cryptopp-autotools
$ cd cryptopp-autotools
$ cp configure.ac Makefile.am libcryptopp.pc.in ../
$ cd ..

At this point you should open Makefile.am and verify the library version number around line 140. Make sure it looks reasonable because we often forget to increment it.

## Very important... Version number of the library
libcryptopp_la_LDFLAGS = $(AM_LDFLAGS) -version-info 6:0:0

Bootstrap Autotools

The second step bootstraps the Autotools project and produces a configure file.

$ mkdir m4/
$ autoreconf -f -i
...

libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
...

After autoreconf finishes you will have fodder sprayed throughout the root directory like a feral dog sprays its urine. Autotools is not exactly known for being tidy.

config.sub and config.guess

As an optional step you can update config.sub and config.guess. GNU recommends you update the files whenever you build a tarball. See 13.4.6 config.guess, config.sub at top level in the GNU manual.

The commands to download config.sub and config.guess directly from GNU Savannah are shown below.

echo "Updating config.sub"
wget 'https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub' -O config.sub

echo "Updating config.guess"
wget 'https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess' -O config.guess

Configure the Library

The third step configures the library. The project follows GNU coding standards and honors a user's CXX, CXXFLAGS and LDFLAGS. The project does not use C related flags like CC or CFLAGS. Below the project uses SunCC compiler from Developer Studio 12.6.

After this steps completes the library will be ready to build. This step also deletes the GNUmakefiles so make just works without surprises. Otherwise you would need extra options like make -f Makefile to build the library with the makefiles you just produced.

$ CXX=clang++ ./configure
checking for a BSD-compatible install... /bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
...

After configure completes you will see a summary as shown below.

Auto-configuration complete. A summary of options are below. If
something looks wrong then please modify config.h and please report
it at http://github.com/noloader/cryptopp-autotools.

   Build triplet: x86_64-pc-linux-gnu
 Compiler target: x86_64-unknown-linux-gnu
Compiler version: clang version 6.0.1 (tags/RELEASE_601/final)

Static library: yes
Shared library: yes

CRYPTOPP_SSE_FLAG: -msse2
CRYPTOPP_ARIA_FLAG: -mssse3
CRYPTOPP_BLAKE2_FLAG: -msse4.1
CRYPTOPP_CHAM_FLAG: -mssse3
CRYPTOPP_CRC_FLAG: -msse4.2
CRYPTOPP_LEA_FLAG: -mssse3
CRYPTOPP_GCM_FLAG: -mssse3 -mpclmul
CRYPTOPP_AES_FLAG: -msse4.1 -maes
CRYPTOPP_SHA_FLAG: -msse4.2 -msha
CRYPTOPP_SIMECK_FLAG: -mssse3
CRYPTOPP_SIMON_FLAG: -msse4.1
CRYPTOPP_SPECK_FLAG: -msse4.1
CRYPTOPP_SM4_FLAG: -mssse3 -maes

Automake flags (can be overridden by user flags):
AM_CXXFLAGS:  -pthread -fdata-sections -ffunction-sections -DCRYPTOPP_DISABLE_SSE4 -DCRYPTOPP_DISABLE_AESNI -DCRYPTOPP_DISABLE_SHANI -pipe
AM_LDFLAGS:  -pthread -Wl,--gc-sections -Wl,--exclude-libs,ALL -Wl,--as-needed

User flags (overrides Automake flags on conflict):
CXXFLAGS: -g -O2
LDFLAGS:

CPPFLAGS does not include -DNDEBUG. You should consider building
with NDEBUG defined so an assert does not inadvertently crash your
program and egress sensitive data to an error reporting service
like AppPort, Windows Error Reporting, Crash Reporter, etc.

CXXFLAGS does not include -O3. You should consider building at -O3
to engage compiler vectorizations and enhance performance.

The machine is 64-bit capable but -m64 is not present in CXXFLAGS

Compile Source Files

The fourth step compiles the source files.

$ make -j 5
make  all-am
make[1]: Entering directory '/home/build/cryptopp'
/bin/sh ./libtool  --tag=CXX   --mode=compile clang++ -DHAVE_CONFIG_H -I.    -pthread -fdata-sections -ffunction-sections -DCRYPTOPP_DISABLE_SSE4 -DCRYPTOPP_DISABLE_AESNI -DCRYPTOPP_DISABLE_SHANI -pipe -g -O2 -MT adhoc.lo -MD -MP -MF .deps/adhoc.Tpo -c -o adhoc.lo adhoc.cpp
/bin/sh ./libtool  --tag=CXX   --mode=compile clang++ -DHAVE_CONFIG_H -I.    -pthread -fdata-sections -ffunction-sections -DCRYPTOPP_DISABLE_SSE4 -DCRYPTOPP_DISABLE_AESNI -DCRYPTOPP_DISABLE_SHANI -pipe  -g -O2 -MT libcryptlib_la-cryptlib.lo -MD -MP -MF .deps/libcryptlib_la-cryptlib.Tpo -c -o libcryptlib_la-cryptlib.lo `test -f 'cryptlib.cpp' || echo './'`cryptlib.cpp
/bin/sh ./libtool  --tag=CXX   --mode=compile clang++ -DHAVE_CONFIG_H -I.    -pthread -fdata-sections -ffunction-sections -DCRYPTOPP_DISABLE_SSE4 -DCRYPTOPP_DISABLE_AESNI -DCRYPTOPP_DISABLE_SHANI -pipe  -g -O2 -MT libcpu_la-cpu.lo -MD -MP -MF .deps/libcpu_la-cpu.Tpo -c -o libcpu_la-cpu.lo `test -f 'cpu.cpp' || echo './'`cpu.cpp
...

Test the Library

There are several artifacts created by the make all recipe. They are the shared object, the static library, the cryptest program called cryptest and a special cryptest program called cryptestcwd. The two test programs exist because of a small design issue we have not worked around.

The difference between cryptest and cryptestcwd is, cryptest looks for datafiles in ${prefix}/share while cryptestcwd looks for datafiles in $PWD. You run cryptestcwd to test the library in-place, and you use cryptest once the library has been installed. The make install recipe does not install cryptestcwd.

Since you want to test the library before staging or installation you run the cryptestcwd program as shown below.

$ ./cryptestcwd v
Using seed: 1532473756

Testing Settings...

passed:  Your machine is little endian.
passed:  Aligned data access.
passed:  sizeof(byte) == 1
passed:  sizeof(word16) == 2
passed:  sizeof(word32) == 4
passed:  sizeof(word64) == 8
passed:  sizeof(hword) == 2, sizeof(word) == 4, sizeof(dword) == 8
passed:  cacheLineSize == 64
hasSSE2 == 1, hasSSSE3 == 1, hasSSE4.1 == 1, hasSSE4.2 == 1, hasAVX == 0, hasAVX2 == 0,
  hasAESNI == 1, hasCLMUL == 1, hasRDRAND == 0, hasRDSEED == 0, hasSHA == 0, isP4 == 0

Testing operating system provided blocking random number generator...

passed:  it took 0 seconds to generate 16 bytes
passed:  16 generated bytes compressed to 19 bytes by DEFLATE
passed:  GenerateWord32 and Crop
...

A second, more comprehensive test is available that runs test vectors as shown below.

$ ./cryptestcwd tv all
Using seed: 1532473795

Testing FileList algorithm all.txt collection.

Testing SymmetricCipher algorithm TEA/ECB.
................................................................
Testing SymmetricCipher algorithm XTEA/ECB.
.................................................................
Testing SymmetricCipher algorithm WAKE-OFB-LE.
.
Testing SymmetricCipher algorithm WAKE-OFB-BE.
...

Install the Library

The final step in the process is installing the library. Installation is accomplished with a make install.

$ sudo make install
make[1]: Entering directory '/home/build/cryptopp'
 /bin/mkdir -p '/usr/local/lib'
 /bin/sh ./libtool   --mode=install /bin/install -c   libcryptopp.la '/usr/local/lib'
libtool: install: /bin/install -c .libs/libcryptopp.so.7.0.1 /usr/local/lib/libcryptopp.so.7.0.1
libtool: install: (cd /usr/local/lib && { ln -s -f libcryptopp.so.7.0.1 libcryptopp.so.7 || { rm -f libcryptopp.so.7 && ln -s libcryptopp.so.7.0.1 libcryptopp.so.7; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libcryptopp.so.7.0.1 libcryptopp.so || { rm -f libcryptopp.so && ln -s libcryptopp.so.7.0.1 libcryptopp.so; }; })
...

The install recipe places artifacts as follows. Also see Variables for Installation Directories in the GNU manual.

Artifact Location Comments
Header files ${includedir}/cryptopp Library header files. Test header files are not included.
Libraries ${libdir} Named libcryptopp.{a|so}. Soft links to versioned shared object.
Test program ${exec_prefix}/bin cryptest utility and test program.
Data files ${datadir}/cryptopp Library data files. *.data and *.txt files.

Documentation

The library uses Doxygen for its online manual at https://www.cryptopp.com/docs/ref/. The project does not use man pages. If you want to build the documents then Doxygen must be installed.

As of Commit 789a4a2cb48d from July 2018 you can build the docs using Autotools. However, the install recipe does not install them because we don't know how to craft the install rule that uses ${pkghtmldir}. You will have to manually copy the local html-doc folder to ${pkghtmldir}. Also see How to conditionally install documentation to pkghtmldir using Automake on Stack Overflow.

You can also use the GNUmakefile to build the docs as shown below. You may need to checkout the GNUmakefile if it was deleted by configure.

$ make -f GNUmakefile docs
$ make docs
doxygen Doxyfile -d CRYPTOPP_DOXYGEN_PROCESSING
Notice: Output directory `html-docs' does not exist. I have created it for you.
Searching for include files...
Searching for files in directory /home/build/cryptopp-doc
Searching for example files...
Searching for files in directory /home/build/cryptopp-doc
Searching for images...
Searching for dot files...
Searching for msc files...
Searching for dia files...
...

The output of the process is a directory named ref/ with an index.html. The ref/ is also zipped up and named CryptoPPRef.zip. The names ref/ and CryptoPPRef.zip are convenience items for the project and match the layout of directories on the web server.

Library Version

We don't always remember to bump the library version number when working with the Autotools project. If you use the project files then please double check configure.ac around line 130:

AC_INIT([Crypto++], [8.7], ...)

And Makefile.am around line 220:

## Very important... Version number of the library
libcryptopp_la_LDFLAGS = $(AM_LDFLAGS) -release 8.7.0 -version-info 8:7

List Source Files

The library ships with a GNUmakefile to build the library. The GNUmakefile has recipes not present in the Autotools project. One recipe is make sources. The recipe lists the headers and source files used for the library and test program as shown below.

You could use this recipe to ensure the cryptopp-dev package included all the headers for programming with the library. In fact we use it for the Autotools project when building the file lists for targets.

$ make -f GNUmakefile sources
***** Library sources *****
cryptlib.cpp cpu.cpp integer.cpp 3way.cpp adler32.cpp algebra.cpp algparam.cpp a
rc4.cpp aria-simd.cpp aria.cpp ariatab.cpp asn.cpp authenc.cpp base32.cpp base64
.cpp basecode.cpp bfinit.cpp blake2-simd.cpp blake2.cpp blowfish.cpp blumshub.cp
p camellia.cpp cast.cpp casts.cpp cbcmac.cpp ccm.cpp chacha.cpp cham-simd.cpp ch
am.cpp channels.cpp cmac.cpp crc-simd.cpp crc.cpp default.cpp des.cpp dessp.cpp
dh.cpp dh2.cpp dll.cpp dsa.cpp eax.cpp ec2n.cpp eccrypto.cpp ecp.cpp elgamal.cpp
 emsa2.cpp eprecomp.cpp esign.cpp files.cpp filters.cpp fips140.cpp fipstest.cpp
 gcm-simd.cpp gcm.cpp gf256.cpp gf2_32.cpp gf2n.cpp gfpcrypt.cpp gost.cpp gzip.c
pp hc128.cpp hc256.cpp hex.cpp hight.cpp hmac.cpp hrtimer.cpp ida.cpp idea.cpp i
terhash.cpp kalyna.cpp kalynatab.cpp keccak.cpp lea-simd.cpp lea.cpp luc.cpp mar
s.cpp marss.cpp md2.cpp md4.cpp md5.cpp misc.cpp modes.cpp mqueue.cpp mqv.cpp nb
theory.cpp neon-simd.cpp network.cpp oaep.cpp ospstore.cpp osrng.cpp padlkrng.cp
p panama.cpp pkcspad.cpp poly1305.cpp polynomi.cpp ppc-simd.cpp pssr.cpp pubkey.
cpp queue.cpp rabbit.cpp rabin.cpp randpool.cpp rc2.cpp rc5.cpp rc6.cpp rdrand.c
pp rdtables.cpp rijndael-simd.cpp rijndael.cpp ripemd.cpp rng.cpp rsa.cpp rw.cpp
 safer.cpp salsa.cpp scrypt.cpp seal.cpp seed.cpp serpent.cpp sha-simd.cpp sha.c
pp sha3.cpp shacal2-simd.cpp shacal2.cpp shark.cpp sharkbox.cpp simeck-simd.cpp
simeck.cpp simon-simd.cpp simon.cpp skipjack.cpp sm3.cpp sm4-simd.cpp sm4.cpp so
cketft.cpp sosemanuk.cpp sparx.cpp speck-simd.cpp speck.cpp square.cpp squaretb.
cpp sse-simd.cpp strciphr.cpp tea.cpp tftables.cpp threefish.cpp tiger.cpp tiger
tab.cpp trdlocal.cpp ttmac.cpp tweetnacl.cpp twofish.cpp vmac.cpp wait.cpp wake.
cpp whrlpool.cpp xtr.cpp xtrcrypt.cpp zdeflate.cpp zinflate.cpp zlib.cpp

***** Library headers *****
3way.h adler32.h adv-simd.h aes-armv4.h aes.h algebra.h algparam.h androidpay.h
arc4.h argnames.h aria.h asn.h authenc.h base32.h base64.h basecode.h blake2.h b
lowfish.h blumshub.h camellia.h cast.h cbcmac.h ccm.h chacha.h cham.h channels.h
 cmac.h config.h cpu.h crc.h cryptlib.h default.h des.h dh.h dh2.h dll.h dmac.h
drbg.h dsa.h eax.h ec2n.h eccrypto.h ecp.h ecpoint.h elgamal.h emsa2.h eprecomp.
h equihash.h esign.h fhmqv.h files.h filters.h fips140.h fltrimpl.h gcm.h gf256.
h gf2_32.h gf2n.h gfpcrypt.h gost.h gzip.h hashfwd.h hc128.h hc256.h hex.h hight
.h hkdf.h hmac.h hmqv.h hrtimer.h ida.h idea.h integer.h iterhash.h kalyna.h kec
cak.h lea.h lubyrack.h luc.h mars.h md2.h md4.h md5.h mdc.h mersenne.h misc.h mo
darith.h modes.h modexppc.h mqueue.h mqv.h naclite.h nbtheory.h network.h nr.h o
aep.h oids.h ospstore.h osrng.h ossig.h padlkrng.h panama.h pch.h pkcspad.h poly
1305.h polynomi.h ppc-simd.h pssr.h pubkey.h pwdbased.h queue.h rabbit.h rabin.h
 randpool.h rc2.h rc5.h rc6.h rdrand.h rijndael.h ripemd.h rng.h rsa.h rw.h safe
r.h salsa.h scrypt.h seal.h secblock.h seckey.h seed.h serpent.h serpentp.h sha.
h sha3.h shacal2.h shark.h simeck.h simon.h simple.h siphash.h skipjack.h sm3.h
sm4.h smartptr.h socketft.h sosemanuk.h sparx.h speck.h square.h stdcpp.h strcip
hr.h tea.h threefish.h tiger.h trap.h trdlocal.h trunhash.h ttmac.h tweetnacl.h
twofish.h vmac.h wait.h wake.h whrlpool.h winpipes.h words.h xtr.h xtrcrypt.h zd
eflate.h zinflate.h zlib.h

***** Test sources *****
adhoc.cpp test.cpp bench1.cpp bench2.cpp validat0.cpp validat1.cpp validat2.cpp
validat3.cpp validat4.cpp datatest.cpp regtest1.cpp regtest2.cpp regtest3.cpp dl
ltest.cpp fipsalgt.cpp

***** Test headers *****
bench.h factory.h validate.h

make: Nothing to be done for 'sources'.

Testing Autotools

The Autotools project is tested on Noloader's GitHub. The test results are available at NoLoader | Crypto++. The test is performed through a script located in TestScripts. The Autotools test script can be run manually.

The test script downloads the Autotools project files to the cryptopp directory. The script then autoreconf's, configure's, builds the library and finally executes the self tests.

cd cryptopp
cp TestScripts/cryptest-autotools .

./cryptest-autotools
...

Downloads

cryptopp-autotools - GitHub with the latest Autotools sources. Releases are tagged so you can download a version of the Autotools project files to fit a version of the library.