by cawan (cawan[at]ieee.org or chuiyewleong[at]hotmail.com)
on 11/11/2012
Cross compilation is a very important topic in embedded linux system design. It is
about to use a host machine to build a software which is going to be run in embedded
system. Besides, it is also about to harness the power of open source technology for
embedded applications. The compiler that is going to be used in cross compilation is
known as cross compiler. So, the cross compiler will be run in a standard x86 PC as
host machine, and generate binary for embedded platform such as ARM, MIPS, or PPC.
Let's illustrate with example.
Host:# cat hello.c
#include
main()
{
printf("hello world\n");
}
Host:# gcc -o hello-x86 hello.c
Host:# file hello-x86
hello-x86: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.15, dynamically linked (uses shared libs), not stripped
Host:# ./hello-x86
hello world
Host:#
Well, we have a hello.c and we compile it with default gcc to generate a x86
compliant elf executable which is able to run on x86 based platform, as shown.
Nothing special here. However, for cross compilation, we are going to use platform
dependent cross compiler instead of the default gcc. As demonstration here, we are
going to use a MIPS based cross compiler to generate a executable which is able to
run on MIPS platform. Let's start.
Host:# ./mips-linux-gnu-gcc -EL -o hello-mips hello.c
Host:# file hello-mips
hello-mips: ELF 32-bit LSB executable, MIPS, MIPS32 rel2 version 1, for GNU/Linux 2.6.12, dynamically linked (uses shared libs), not stripped
Host:# ./hello-mips
bash: ./hello-mips: cannot execute binary file
Host:#
In this case, the mips-linux-gnu-gcc is the MIPS based cross compiler. The -EL flag
is to tell cross compiler to generate code which is compliant to little endian MIPS
architecture. Without the -EL flag, the cross compiler will generate default big
endian MIPS binary. Of course, when we try to run it on our x86 based host machine,
it should fail. However, if we run the executable in a MIPS platform, it should work
as expected.
tango3[~]# ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: seq=0 ttl=64 time=1.942 ms
64 bytes from 192.168.1.1: seq=1 ttl=64 time=1.090 ms
64 bytes from 192.168.1.1: seq=2 ttl=64 time=1.049 ms
--- 192.168.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 1.049/1.360/1.942 ms
tango3[~]# mkdir ./cawan
tango3[~]# mount -o nolock 192.168.1.1:/home/smp383/mips-4.3/bin/ ./cawan
tango3[~]# cd cawan
tango3[cawan]# ls -l hello-mips
-rwxr-xr-x 1 root root 5870 Nov 10 2012 hello-mips*
tango3[cawan]# ./hello-mips
hello world
tango3[cawan]#
The 192.168.1.1 is host machine, when ping it from embedded system, it is alive.
Then, we create a local directory in the embedded system and mount it onto a
development directory at remote host machine by using NFS. The -o nolock is normally
added in embedded environment for compatibility issues. From development directory
of host machine, we found our hello-mips, and once we run it in MIPS console, it
works well as expected. How about the hello-x86 ? Let's verify.
tango3[cawan]# ./hello-x86
-sh: ./hello-x86: cannot execute binary file
tango3[cawan]#
Yes, it should not able to run in MIPS platform. How about big endian MIPS binary ?
Host:# ./mips-linux-gnu-gcc -o hello-mips-be hello.c
Host:# ls -l hello-mips-be
-rwxr-xr-x 1 root root 5870 2012-11-11 14:06 hello-mips-be
Host:# file hello-mips-be
hello-mips-be: ELF 32-bit MSB executable, MIPS, MIPS32 rel2 version 1, for GNU/Linux 2.6.12, dynamically linked (uses shared libs), not stripped
Host:#
tango3[cawan]# ./hello-mips-be
-sh: ./hello-mips-be: cannot execute binary file
tango3[cawan]#
Yes, the big endian MIPS binary should not able to run in little endian MIPS
platform. Well, that is simple. But, how to cross compile an open source project in
order to let it run in our MIPS platform ? For this issue, we need to know more about
make tool, and we need to instruct the make tool to invoke the appropriate cross
compiler instead of the default gcc. Let us demonstrate with an open source project
such as Lame MP3 Encoder.
Host:# pwd
/home/smp383/mips-4.3/bin/lame-3.99.5
Host:# make --help
Usage: make [options] [target] ...
Options:
-b, -m Ignored for compatibility.
-B, --always-make Unconditionally make all targets.
-C DIRECTORY, --directory=DIRECTORY
Change to DIRECTORY before doing anything.
-d Print lots of debugging information.
--debug[=FLAGS] Print various types of debugging information.
-e, --environment-overrides
Environment variables override makefiles.
-f FILE, --file=FILE, --makefile=FILE
Read FILE as a makefile.
-h, --help Print this message and exit.
-i, --ignore-errors Ignore errors from commands.
-I DIRECTORY, --include-dir=DIRECTORY
Search DIRECTORY for included makefiles.
-j [N], --jobs[=N] Allow N jobs at once; infinite jobs with no arg.
-k, --keep-going Keep going when some targets can't be made.
-l [N], --load-average[=N], --max-load[=N]
Don't start multiple jobs unless load is below N.
-L, --check-symlink-times Use the latest mtime between symlinks and target.
-n, --just-print, --dry-run, --recon
Don't actually run any commands; just print them.
-o FILE, --old-file=FILE, --assume-old=FILE
Consider FILE to be very old and don't remake it.
-p, --print-data-base Print make's internal database.
-q, --question Run no commands; exit status says if up to date.
-r, --no-builtin-rules Disable the built-in implicit rules.
-R, --no-builtin-variables Disable the built-in variable settings.
-s, --silent, --quiet Don't echo commands.
-S, --no-keep-going, --stop
Turns off -k.
-t, --touch Touch targets instead of remaking them.
-v, --version Print the version number of make and exit.
-w, --print-directory Print the current directory.
--no-print-directory Turn off -w, even if it was turned on implicitly.
-W FILE, --what-if=FILE, --new-file=FILE, --assume-new=FILE
Consider FILE to be infinitely new.
--warn-undefined-variables Warn when an undefined variable is referenced.
This program built for i486-pc-linux-gnu
Report bugs to
Host:#
Nothing related to cross compilation in make tool. But, the definition of cross
compiler should be in Makefile, which is generated by configuration tool. Let's
check now.
Host:# ./configure --help
`configure' configures lame 3.99.5 to adapt to many kinds of systems.
Usage: ./configure [OPTION]... [VAR=VALUE]...
To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE. See below for descriptions of some of the useful variables.
Defaults for the options are specified in brackets.
Configuration:
-h, --help display this help and exit
--help=short display options specific to this package
--help=recursive display the short help of all the included packages
-V, --version display version information and exit
-q, --quiet, --silent do not print `checking ...' messages
--cache-file=FILE cache test results in FILE [disabled]
-C, --config-cache alias for `--cache-file=config.cache'
-n, --no-create do not create output files
--srcdir=DIR find the sources in DIR [configure dir or `..']
Installation directories:
--prefix=PREFIX install architecture-independent files in PREFIX
[/usr/local]
--exec-prefix=EPREFIX install architecture-dependent files in EPREFIX
[PREFIX]
By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc. You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.
For better control, use the options below.
Fine tuning of the installation directories:
--bindir=DIR user executables [EPREFIX/bin]
--sbindir=DIR system admin executables [EPREFIX/sbin]
--libexecdir=DIR program executables [EPREFIX/libexec]
--sysconfdir=DIR read-only single-machine data [PREFIX/etc]
--sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]
--localstatedir=DIR modifiable single-machine data [PREFIX/var]
--libdir=DIR object code libraries [EPREFIX/lib]
--includedir=DIR C header files [PREFIX/include]
--oldincludedir=DIR C header files for non-gcc [/usr/include]
--datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
--datadir=DIR read-only architecture-independent data [DATAROOTDIR]
--infodir=DIR info documentation [DATAROOTDIR/info]
--localedir=DIR locale-dependent data [DATAROOTDIR/locale]
--mandir=DIR man documentation [DATAROOTDIR/man]
--docdir=DIR documentation root [DATAROOTDIR/doc/lame]
--htmldir=DIR html documentation [DOCDIR]
--dvidir=DIR dvi documentation [DOCDIR]
--pdfdir=DIR pdf documentation [DOCDIR]
--psdir=DIR ps documentation [DOCDIR]
Program names:
--program-prefix=PREFIX prepend PREFIX to installed program names
--program-suffix=SUFFIX append SUFFIX to installed program names
--program-transform-name=PROGRAM run sed PROGRAM on installed program names
System types:
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
Optional Features:
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
--enable-maintainer-mode enable make rules and dependencies not useful
(and sometimes confusing) to the casual installer
--disable-dependency-tracking speeds up one-time build
--enable-dependency-tracking do not reject slow dependency extractors
--enable-shared[=PKGS] build shared libraries [default=yes]
--enable-static[=PKGS] build static libraries [default=yes]
--enable-fast-install[=PKGS]
optimize for fast installation [default=yes]
--disable-libtool-lock avoid locking (might break parallel builds)
--disable-largefile omit support for large files
--enable-nasm Allow the use of nasm if available
--disable-rpath do not hardcode runtime library paths
--disable-cpml Do not use Compaq's fast Math Library
--disable-gtktest Do not try to compile and run a test GTK program
--enable-efence Use ElectricFence for malloc debugging
--disable-analyzer-hooks Exclude analyzer hooks
--disable-decoder Exclude mpg123 decoder
--disable-frontend Do not build the lame executable default=build
--enable-mp3x Build GTK frame analyzer default=no
--enable-mp3rtp Build mp3rtp default=no
--enable-dynamic-frontends Link frontends against shared libraries default=no
--enable-expopt=full,norm Whether to enable experimental optimizations
default=no
--enable-debug=alot,norm Enable debugging (disables optimizations)
default=no
Optional Packages:
--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
--with-pic try to use only PIC/non-PIC objects [default=use
both]
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-dmalloc use dmalloc, as in
http://www.dmalloc.com/dmalloc.tar.gz
--with-gnu-ld assume the C compiler uses GNU ld default=no
--with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib
--without-libiconv-prefix don't search for libiconv in includedir and libdir
--with-gtk-prefix=PFX Prefix where GTK is installed (optional)
--with-gtk-exec-prefix=PFX Exec prefix where GTK is installed (optional)
--with-fileio=lame Use lame's internal file io routines default
=sndfile Use Erik de Castro Lopo's libsndfile
(no stdin possible currently)
Some influential environment variables:
CC C compiler command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L
nonstandard directory
LIBS libraries to pass to the linker, e.g. -l
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I
you have headers in a nonstandard directory
CPP C preprocessor
PKG_CONFIG path to pkg-config utility
PKG_CONFIG_PATH
directories to add to pkg-config's search path
PKG_CONFIG_LIBDIR
path overriding pkg-config's built-in search path
SNDFILE_CFLAGS
C compiler flags for SNDFILE, overriding pkg-config
SNDFILE_LIBS
linker flags for SNDFILE, overriding pkg-config
Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.
Report bugs to
Host:#
Well, there are several items related to cross compilation here.
1. --host=HOST cross-compile to build programs to run on HOST [BUILD]
2. CFLAGS C compiler flags
3. --prefix=PREFIX install architecture-independent files in PREFIX
The --host is about to let us to specify the prefix of the cross compiler being
used. For our case here, we should specify it as "mips-linux-gnu". Regarding the
CFLAGS, of course we need to specify -EL for our little endian MIPS architecture.
For --prefix, it is about to redefine the installation directory which is other
than /usr/local. As personal experience, I really not recommend to use this option.
The reason is the binary being generated at the path that specified in --prefix is
not as "friendly" as the counterpart option in make install DESTDIR. Let us start
to cross compile this Lame MP3 Encoder now.
Host:# ./configure --host="mips-linux-gnu" CFLAGS="-EL"
configure: WARNING: if you wanted to set the --build type, don't use --host.
If a cross compiler is detected then cross compile mode will be used
checking build system type... i686-pc-linux-gnu
checking host system type... mips-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for mips-linux-gnu-strip... mips-linux-gnu-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for mips-linux-gnu-gcc... mips-linux-gnu-gcc
...
...
config.status: creating ACM/ddk/Makefile
config.status: creating ACM/tinyxml/Makefile
config.status: creating lame.spec
config.status: creating mac/Makefile
config.status: creating macosx/Makefile
config.status: creating macosx/English.lproj/Makefile
config.status: creating macosx/LAME.xcodeproj/Makefile
config.status: creating vc_solution/Makefile
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
Host:# make
make all-recursive
make[1]: Entering directory `/home/smp383/mips-4.3/bin/lame-3.99.5'
Making all in mpglib
...
...
collect2: ld returned 1 exit status
make[3]: *** [libmp3lame.la] Error 1
make[3]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5/libmp3lame'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5/libmp3lame'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5'
make: *** [all] Error 2
Host:#
Unfortunately, make error. It might because of some source files might be in .cpp
and since we don't specify the CPPFLAGS yet, error should be incurred. Let's check
for any .cpp file.
Host:# find . -iname "*.cpp"
./ACM/DecodeStream.cpp
./ACM/AEncodeProperties.cpp
./ACM/tinyxml/xmltest.cpp
./ACM/tinyxml/tinyxmlparser.cpp
./ACM/tinyxml/tinyxmlerror.cpp
./ACM/tinyxml/tinyxml.cpp
./ACM/main.cpp
./ACM/ACM.cpp
./ACM/ADbg/ADbg.cpp
./ACM/ACMStream.cpp
./Dll/Example.cpp
./dshow/REG.CPP
./dshow/Mpegac.cpp
./dshow/PropPage.cpp
./dshow/aboutprp.cpp
./dshow/PropPage_adv.cpp
./dshow/Encoder.cpp
Host:#
Yes, there are some .cpp files in certain subdirectories. Let's specify CPPFLAGS and
try again.
Host:# make distclean
Host:# ./configure --host="mips-linux-gnu" CFLAGS="-EL" CPPFLAGS="-EL"
Host:# make
However, the error still there. Now, we suspect we are missing LDFLAGS. Specify it
and try again.
Host:# make distclean
Host:# ./configure --host="mips-linux-gnu" CFLAGS="-EL" CPPFLAGS="-EL" LDFLAGS="-EL"
Host:# make
Unfortunately, the error is still. So, we doubt the Makefile being generated by
configure tool did not specify the options of cross compiler properly. In other
words, if we redefine the CFLAGS, CPPFLAGS, or LDFLAGS, the default options might
be overrode, and this might be the reason to cause the error. Let us ignore those
CFLAGS, CPPFLAGS, and LDFLAGS, and we specify the -EL option in CC now.
Host:# make distclean
Host:# ./configure --host="mips-linux-gnu" CC="mips-linux-gnu-gcc -EL"
Host:# make
Excellent, it is done. Now, let us make a comparison between the cross compilation
options being generated by those 2 configuration methods as aforementioned.
//////////////////////////////////////////////////////////////////////////////////
Host:# ./configure --host="mips-linux-gnu" CFLAGS="-EL" CPPFLAGS="-EL" LDFLAGS="-EL"
Makefile:
...
...
CC = mips-linux-gnu-gcc
CCDEPMODE = depmode=gcc3
CFLAGS = -Wall -pipe -EL
CONFIG_DEFS =
CONFIG_MATH_LIB = -lm
CPP = mips-linux-gnu-gcc -E
CPPFLAGS = -EL
...
...
//////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////
Host:# ./configure --host="mips-linux-gnu" CC="mips-linux-gnu-gcc -EL"
Makefile:
...
...
CC = mips-linux-gnu-gcc -EL
CCDEPMODE = depmode=gcc3
CFLAGS = -O3 -fomit-frame-pointer -ffast-math -Wall -pipe
CONFIG_DEFS =
CONFIG_MATH_LIB = -lm
CPP = mips-linux-gnu-gcc -EL -E
CPPFLAGS =
...
...
//////////////////////////////////////////////////////////////////////////////////
Apparently, some default options in CFLAGS have been overrode if we redefine the
CFLAGS ourself. So, it is important to note that we should avoid to redefine those
compilation and link flag in doing cross compilation. Now, the binaries are ready
for us, and we are not going to install it into /usr/local/. Because we are doing
cross compilation now, we expect the binaries will be concentrated in a special
directory in our development directory. Yes, for this purpose, we can make use of
DESTDIR option in doing make install. Let's check.
Host:# make install DESTDIR=$PWD/cawanlame
Making install in mpglib
make[1]: Entering directory `/home/smp383/mips-4.3/bin/lame-3.99.5/mpglib'
make[2]: Entering directory `/home/smp383/mips-4.3/bin/lame-3.99.5/mpglib'
make[2]: Nothing to be done for `install-exec-am'.
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5/mpglib'
make[1]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5/mpglib'
Making install in libmp3lame
...
...
make[2]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5'
make[1]: Leaving directory `/home/smp383/mips-4.3/bin/lame-3.99.5'
Host:# cd cawanlame
Host:# ls
usr
Host:# cd usr
Host:# ls
local
Host:# cd local
Host:# ls
bin include lib share
Host:#
Well, the lame executable and shared library should be in bin and lib folders,
respectively. Let's verify.
Host:# cd bin
Host:# ls -l male
ls: cannot access male: No such file or directory
Host:# cd ..
Host:# cd bin
Host:# ls -l lame
-rwxr-xr-x 1 root root 409167 2012-11-11 16:10 lame
Host:# cd ..
Host:# cd lib
Host:# ls -l
total 768
-rw-r--r-- 1 root root 429192 2012-11-11 16:10 libmp3lame.a
-rwxr-xr-x 1 root root 943 2012-11-11 16:10 libmp3lame.la
lrwxrwxrwx 1 root root 19 2012-11-11 16:10 libmp3lame.so -> libmp3lame.so.0.0.0
lrwxrwxrwx 1 root root 19 2012-11-11 16:10 libmp3lame.so.0 -> libmp3lame.so.0.0.0
-rwxr-xr-x 1 root root 343465 2012-11-11 16:10 libmp3lame.so.0.0.0
Host:#
Nice, we are at the right point now. Let us make sure the binaries are really for
little endian MIPS platform.
Host:# cd bin
Host:# file lame
lame: ELF 32-bit LSB executable, MIPS, MIPS32 rel2 version 1, for GNU/Linux 2.6.12, dynamically linked (uses shared libs), not stripped
Host:# cd ..
Host:# cd lib
Host:# ls -l
total 768
-rw-r--r-- 1 root root 429192 2012-11-11 16:10 libmp3lame.a
-rwxr-xr-x 1 root root 943 2012-11-11 16:10 libmp3lame.la
lrwxrwxrwx 1 root root 19 2012-11-11 16:10 libmp3lame.so -> libmp3lame.so.0.0.0
lrwxrwxrwx 1 root root 19 2012-11-11 16:10 libmp3lame.so.0 -> libmp3lame.so.0.0.0
-rwxr-xr-x 1 root root 343465 2012-11-11 16:10 libmp3lame.so.0.0.0
Host:# file libmp3lame.so.0.0.0
libmp3lame.so.0.0.0: ELF 32-bit LSB shared object, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, not stripped
Host:#
Yes, we got what we want. Let's run it in MIPS environment now.
tango3[cawan]# cd lame-3.99.5
tango3[lame-3.99.5]# cd cawanlame/
tango3[cawanlame]# cd usr/
tango3[usr]# cd local/
tango3[local]# cd bin
tango3[bin]# ./lame
LAME 32bits version 3.99.5 (http://lame.sf.net)
usage: ./lame [options]
Try:
"./lame --help" for general usage information
or:
"./lame --preset help" for information on suggested predefined settings
or:
"./lame --longhelp"
or "./lame -?" for a complete options list
tango3[bin]#
Good, the lame executable running well in our MIPS platform. In order to get better
experience in running executable in MIPS platform, let us use the lame executable to
compress an .wav file into .mp3 file now. Of course the process might take quite a
while because we are running the compression process in a resource constraint
embedded platform.
tango3[bin]# ./lame --help
LAME 32bits version 3.99.5 (http://lame.sf.net)
usage: ./lame [options]
RECOMMENDED:
lame -V2 input.wav output.mp3
OPTIONS:
-b bitrate set the bitrate, default 128 kbps
-h higher quality, but a little slower. Recommended.
-f fast mode (lower quality)
-V n quality setting for VBR. default n=4
0=high quality,bigger files. 9=smaller files
--preset type type must be "medium", "standard", "extreme", "insane",
or a value for an average desired bitrate and depending
on the value specified, appropriate quality settings will
be used.
"--preset help" gives more info on these
--help id3 ID3 tagging related options
--longhelp full list of options
--license print License information
tango3[bin]# ./lame -V2 luohua.wav luohua.mp3
LAME 3.99.5 32bits (http://lame.sf.net)
Using polyphase lowpass filter, transition band: 18671 Hz - 19205 Hz
Encoding luohua.wav to luohua.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=2)
Frame | CPU time/estim | REAL time/estim | play/CPU | ETA
6253/6253 (100%)| 1:52/ 1:52| 5:37/ 5:37| 1.4538x| 0:00
32 [ 81] **
40 [ 0]
48 [ 0]
56 [ 1] %
64 [ 2] %
80 [ 5] %
96 [ 2] %
112 [ 1] *
128 [ 73] %*
160 [2410] %%%%%************************************************
192 [3126] %%%%%%%%%%%%%%%%%%%%%%%%********************************************
224 [ 295] %%*****
256 [ 197] %%***
320 [ 60] %*
-------------------------------------------------------------------------------
kbps LR MS % long switch short %
181.4 23.4 76.6 95.6 2.4 2.0
Writing LAME Tag...done
ReplayGain: -8.0dB
tango3[bin]# ls
lame* luohua.mp3 luohua.wav
Well, the luohua.mp3 is generated. Let us play it with a player. Nice, it sounds
really good.
pdf version:
http://www.scribd.com/doc/112842652/What-is-Cross-Compilation-Cross-Compilation-Demystified
