Building the AP Bridge Software

Owned by Jonathan Simon
Last updated: Jun 23, 2020 by Dave Bacher [analog]Version comment
10 min read

Introduction

This section describes the requirements and process for building the AP Bridge software on a Linux system. It assumes familiarity with software development tools and how to build software from source packages. 

The build process consists of the following high-level steps:

  • Organize your development environment
  • Build the required libraries
  • Build the AP Bridge

The AP Bridge software can be built on a host platform for deployment on a possibly different target platform.  When the host and target are the same, i.e. you are building for a Raspberry Pi on a Raspberry Pi, then you build the libraries for the Raspberry Pi, and install the libraries on the Raspberry Pi.

Cross-compilation, i.e. when the host and target platforms are different, can require certain workarounds when building from source.  When the host and target are different, i.e. you are building on an Ubuntu machine, but will run the AP Bridge software on a Raspberry Pi, you build (cross-compile) the libraries on the Ubuntu host - these don't "run" on the host, but they need to be available to SCons so that you can build the AP Bridge for the target.  Because you don't intend to use the libraries on the host, we recommend installing them in a different location than other host libraries, which are installed in /usr/local. Also, you will need to copy the libraries you built to the target in the location that the AP Bridge expects to find them. 

Additionally, you have the option to build the libraries and the AP Bridge in your user directory (~) or make them available to all users on the host. In this document, the build examples will use your user directory. 

While this document talks about cross-compilation at a high level, it does not cover the specifics of how to set up compilation for your target with your host development environment. 

Requirements

To build and deploy the AP Bridge software, you will need:

  • A GCC compiler for your target platform. The AP Bridge software uses several features of C++11, so GCC 4.8 or later is required. 
  • A git client is required to download the AP Bridge sources from the public repository on GitHub. A command line git client is included in most Linux distributions. 
  • A local compiler is needed to build certain programs. These examples assume GCC is installed. 
  • The AP Bridge software uses the SCons build system. Python and SCons must be installed on the host platform. Python is included in most Linux distributions and SCons is typically available as a package. 
  • The AP Bridge software is packaged using the fpm packaging tool, which can target several packaging formats and must be installed on the host in order to build a package. 
  • Python should also be installed on the target platform. Several configuration and command line administration scripts use Python. 
  • The AP Bridge software assumes it's running on a Linux target and links against common runtime libraries available in Linux in addition to the libraries described on this page. 

The AP Bridge software links against several open-source libraries as described in Library Compilation below. 

You will need to install additional packages in order to build the AP Bridge software from source. For Ubuntu, the list of required packages is:

$ sudo apt-get install build-essential ruby-dev python2.7 scons libtool libtool-bin autoconf automake uuid-dev
$ sudo gem install fpm

The compiler (or cross-compiler) for the target platform must be added to your PATH. If you use the recommended tools layout described below, the following command would add the Raspberry Pi compiler to the PATH.

export PATH=$PATH:$HOME/dev/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin

Set up the development environment

The Raspberry Pi toolchain is hosted on Github.  You can use git to check out the toolchain on your build host. The repository contains both 32-bit and 64-bit versions of the ARM compilers - you should choose the version that matches your build host's OS.

$ cd ~/dev
$ git clone https://github.com/raspberrypi/tools.git

This will create a folder called "tools" in your ~/dev directory.  You can also download and unzip it directly from Github. Note that the folder name will be "tools-master" if you use the direct download.  Install the cross-compiler under ~/dev/arm-bcm2708

# from ~/dev
$ mv tools/arm-bcm2708 .


Next, download the apbridge source and the required libraries. Again you can use git or a direct download. A script for downloading the required library sources, get_library_sources.sh, is provided in the GitHub repository. 

# from ~/dev
$ git clone https://github.com/dustcloud/apbridge.git
$ sh apbridge/get_library_sources.sh


Verify that the SConstruct file contains the correct paths (these may differ if you a building for another target):

  • Set toolchain_dir to ~/dev/arm-bcm2708
  • Set tools_prefix to ~/dev/opt/dust-apc
  • Set boost_prefix to ~/dev/boost_1_60_0

Library compilation

The AP Bridge software links against the following libraries. While versions of these libraries other than those listed may function, the AP Bridge software has only been tested with the version listed.  

  • Boost version 1.60.0. The Boost libraries are used for a variety of C++ utility operations, including socket and serial I/O, program configuration and thread / synchronization management. 
  • Protocol Buffers (protobuf) version 2.5. Protocol Buffers is used as the serialization layer for structured data for inter-process communication with the APC Console. 
  • Zeromq version 3.2.5, czmq version 1.3.2. ZeroMQ is used as the messaging layer for inter-process communication with the APC Console. 
  • log4cxx version 0.10.0, which requires the apr and apr-utils libraries. The log4cxx library is used for event and diagnostic logging. The logging implementation is contained within the logging/ directory and can be replaced with another implementation.  
  • gpsd - gpsd is a daemon that monitors one or more attached GPS receivers. 
  • Standard runtime Linux libraries, such as pthread. These libraries are available with any Linux system and are provided with cross-compiling toolchains. 


These libraries, built for the target, and their headers must be available on the build host. In general, these libraries can be built by downloading the source and running a configure ; make ; make install sequence, but you will need additional configuration when cross-compiling, which will be explained in the sample build below.
 
The libraries also must be installed on the target - this will be done at the end of the build process. Since some of these libraries may be available as standard packages on the target distribution (and be different versions from those listed), the AP Bridge package is installed on the target in /opt/dust-apc and expects to find libraries there instead of the default /usr/local. 
 
The sections below only provide a simple overview of the steps for building and installing each of the libraries: refer to each library distribution for detailed instructions. Generally, the build processes allow for a configurable prefix, where the libraries will be installed, and a way to override the root of the install destination. The instructions show how to build the libraries for the Raspberry Pi target on a separate build host with the libraries required by the AP Bridge installed in the dev directory within the user's home directory. 


Boost

The build process for Boost uses its own build tool. The AP Bridge only uses a small set of the Boost libraries. Compiling only the necessary libraries is much faster than compiling the full Boost suite. 


$ cd ~/dev/boost_1_60_0
$ ./bootstrap.sh --with-libraries=atomic,chrono,filesystem,program_options,system,thread,timer
Building the Boost libraries with a cross compiler requires editing a build configuration file. If it does not already exist, create tools/build/src/user-config.jam (relative to the Boost sources) with the following content:
using gcc : arm : arm-linux-gnueabihf-g++ ;  # or specify the PATH to the cross-compiler to be used

The example assumes that the arm compiler is in your PATH. Otherwise you will need to specify the full path. Note that user-config.jam can also be in $HOME.

Build with:

# from ~/dev/boost_1_60_0
$ ./b2 toolset=gcc-arm

If successful, you will get a message informing you that the header files are in the boost source folder, and that the libraries are in the boost source folder under ./stage/lib.  You will need to copy the libraries to the appropriate location on your target (/opt/dust-apc/lib). 

Protocol Buffers

In addition to the runtime library and headers, the Protocol Buffers build also contains the protoc compiler, which is used in the AP Bridge build process. There are two stages: the protoc compiler must be built for the build host, and the libraries must be built for the target (if the target architecture is different from the build host).  In order to avoid polluting the protobuf source directory, we use a sub-directory to build the different platforms. 

The following steps first configures and builds protobuf for the build host and installs the protoc compiler in /usr/local/bin. 

$ cd ~/dev/protobuf-2.5.0
$ mkdir i686-linux
$ cd i686-linux
$ ../configure
$ make
$ sudo make install
$ sudo ldconfig

Then, cross-compile the protobuf libraries for the target.  To build the libraries so they can be installed in /opt/dust-apc, use the --prefix option in the configure step - this applies to both regular and cross-compilation. We also need to specify the location of the cross-compiler with the --host option, and specify the location of the protoc compiler we just built with --with-protoc option.  We use DESTDIR=$HOME/dev to install the libraries that the AP Bridge will be built against within the user's home directory.

$ cd ~/dev/protobuf-2.5.0
$ mkdir arm-linux
$ cd arm-linux
$ ../configure --host=arm-linux-gnueabihf --prefix=/opt/dust-apc --with-protoc=/usr/local/bin/protoc
$ make
$ make install DESTDIR=$HOME/dev

ZeroMQ

The ZeroMQ library and headers can be built for the target with the standard build process. 

$ cd ~/dev/zeromq-3.2.5
$ ./configure --host=arm-linux-gnueabihf --prefix=/opt/dust-apc 
$ make
$ make install DESTDIR=$HOME/dev

The czmq library and headers depend on the ZeroMQ library. The czmq build requires a pointer to where the ZeroMQ library is with the --with-libzmq option. 

$ cd ~/dev/czmq-1.3.2
$ LIBS=-lstdc++ ./configure --host=arm-linux-gnueabihf --prefix=/opt/dust-apc --with-libzmq=$HOME/dev/opt/dust-apc
$ make
$ make install DESTDIR=$HOME/dev

The AP Bridge software also uses an MIT licensed C++ wrapper for the ZeroMQ APIs, zmq.hpp. This header is included in the patches directory with the AP Bridge software. It must be copied alongside the installed ZeroMQ headers. 

$ cp ~/dev/apbridge/patches/zmq.hpp ~/dev/opt/dust-apc/include/

The ZeroMQ versions are out of date compared with the latest releases from the ZeroMQ project. The AP Bridge does not have an explicit dependency on the older version, but there may be API changes that have not been integrated into the AP Bridge software.

log4cxx

The log4cxx library is used for logging informational and diagnostic messages to a file. The log4cxx requires the apr and apr-util libraries to be able to build from source. Each of these libraries can be built with the standard build process.  

When building apr, the ./configure step may fail if the script tries to perform a test that requires running a test program on the target. These tests can be bypassed by setting the configure variable(s) explicitly.

When building the apr and apr-util packages, some toolchains do not properly include the required pthread library dependency, leading to build errors in log4cxx. A workaround is to define the LIBS environment variable to include the required library.

$ export LIBS=-lpthread


$ export LIBS=-lpthread
$ cd ~/dev/apr-1.4.6
$ ./configure --host=arm-linux-gnueabihf --prefix=/opt/dust-apc ac_cv_file__dev_zero=yes ac_cv_func_setpgrp_void=yes apr_cv_tcp_nodelay_with_cork=yes apr_cv_process_shared_works=yes ac_cv_sizeof_struct_iovec=8 apc_cv_mutex_robust_shared=no apr_cv_mutex_recursive=yes
$ make
$ make install DESTDIR=$HOME/dev

We need to tell apr-util configure where to find apr with the --with-apr option.

$ cd ../apr-util-1.5.1
$ ./configure --host=arm-linux-gnueabihf --prefix=/opt/dust-apc --with-apr=../apr-1.4.6
$ make
$ make install DESTDIR=$HOME/dev
The log4cxx source requires a patch to fix some header includes so that it builds with recent versions of gcc.   We also tell configure where to find apr and apr-util.
$ cd ~/dev/apache-log4cxx-0.10.0
$ patch -p1 < ~/dev/apbridge/patches/apache-log4cxx-0.10.0.patch
$ ./configure --host=arm-linux-gnueabihf --prefix=/opt/dust-apc --with-apr=../apr-1.4.6 --with-apr-util=../apr-util-1.5.1
$ make
$ make install DESTDIR=$HOME/dev

More recent releases of APR, such as 1.5.2, build a program to generate a header file. This step fails when cross-compiling. One workaround for this is to build APR for the build host and copy aside the generated file, then build the APR library for the target, copy the generated file into place when the make step fails and rerun make.


GPSd

The AP Bridge software links against the gpsd library to retrieve status from a connected GPS. The AP Bridge connects to the Linux gpsd daemon using the libgps API. The build host requires the gps library and headers to be present when building the AP Bridge. Install gpsd (version 3.11 is used in the pre-packaged Raspbian image) and the development files on the target (see Raspberry Pi example below). 

pi@raspberry$ sudo apt-get install gpsd libgps-dev

Copy the library and headers to the build host, e.g. using scp. 

$ scp pi@x.x.x.x:/PATH/libgps.so* $HOME/dev/opt/dust-apc/lib/
$ scp pi@x.x.x.x:/PATH/libgpsmm.h $HOME/dev/opt/dust-apc/include/

In the example above, we use the distribution's GPSd package to leverage the package's configuration of the gpsd daemon and to take advantage of any GPSd improvements. This approach could fail if the gpsd API changes, or if the cross-compiler installed on the build host uses a different version of the runtime C libraries than the compiler used by the target's distribution packages. The symptoms of this failure are compilation errors related the gpsd API, or unresolved symbols in the libgps library when building the AP Bridge.

Another approach is to build the GPSd daemon from source (using version 3.11 for compatibility) on the build host and to copy the headers and libraries to the appropriate directories. Then, install the gpsd daemon and libraries on the target. In this case, the gpsd daemon must be configured to run as a service on the target and connect to the GPS device if one is present.

AP Bridge compilation

The AP Bridge binary is built using SCons. The build process goes through several phases, first generating code, then compiling sources and linking into an executable that references shared libraries. The target option should be used to specify the target platform so the proper compiler configuration is used to build the AP Bridge software, e.g. for a RaspberryPI, the target is armpi.

$ scons apbridge target=armpi

SCons will print error messages if any of the build dependencies are not found. The most common source of errors in the compilation step is placing headers or libraries in the wrong directory on the build host.  

Packaging

The AP Bridge software includes several scripts to assist with configuration in the pkg directory. The AP Bridge binary, configuration scripts, and common configuration files are combined into a package, in this case a .deb file, that can be deployed on an Debian distribution. 

The fpm tool used for building the package can be used to build packages for other Linux distributions. The pkg/SConscript file contains the packaging configuration, which can be adjusted to generate a package appropriate to the integrator's platform. The details of packaging for a particular Linux distribution and target system are left to the integrator. 

To generate a package for the reference Raspberry Pi target:

$ scons apbridge_pkg target=armpi

The generated package can be found in the gen/<target>/pkg directory, named apbridge_<version>.deb; a symlink, apbridge_latest.deb, points to the current package build. 

The reference AP Bridge implementation is packaged as a .deb for Debian and Ubuntu-based distributions and the package files are installed in /opt/dust-apc on the target. The generated package for the reference implementation has several dependencies: 

  • The package declares dependencies on other system packages that are required for proper operation of the AP Bridge software. 
  • The package sets ownership of AP Bridge file to the dust user. The dust user and dust group must be created before installing the package. For certain configuration operations, the dust user should be allowed to use sudo commands. On the Raspbian distribution, this means the dust user should be a member of the sudo group. 
  • The AP Bridge binary runs as the dust user. To interact with the AP Mote, the AP Bridge software must be able to read and write to the AP Mote's serial devices. On the Raspbian distribution, this means the dust user must be a member of the dialout group. 

To install the generated .deb package, copy the .deb file to the target and run:

pi@raspberry$ sudo dpkg --install apbridge_VERSION.deb

In order to run the AP Bridge software, the runtime libraries must be copied into /opt/dust-apc/lib on the target.


Troubleshooting

The scons build prints failure messages if a required header or library can not be found. Typically, these failures can be corrected by editing the SConstruct file to match the layout of the development environment. The AP Bridge SConstruct details section contains additional description of the SConstruct file and how to modify it to reflect a specific build environment or build target. 

References