Saturday, December 22, 2012

How To Cross Compile Asterisk and Run in Embedded System (Latest 1.8.19 Version)

How To Cross Compile Asterisk and Run in Embedded System (Latest 1.8.19 Version)

by cawan (cawan[at]ieee.org or chuiyewleong[at]hotmail.com)

on 12/12/2012

Asterisk is a very useful tool in constructing software based PABX server to manage
a VoIP system. From the perspective of embedded system designer, it is really meaningful
to run the asterisk in embedded platform to build a personal VoIP system in reducing
the cost being incurred by making international calls while traveling around the world.
In addition, by owning a personal VoIP system, the security and voice quality of our
private conversation can be improved significantly. On the other hand, from the
perspective of embedded hacker, the small box with asterisk running is an ideal
platform to perform spoofing of the real PABX server. Besides, with additional FXO
interface, it is possible to hack an automation system as well as the door access
system by abusing the DTMF signal. Well, those are all about the applications of
asterisk in embedded platform, but we should focus in cross compiling asterisk to run
in embedded system in this paper. The embedded platform that we are going to use is
MIPS32 24k, 500MHz. Let's start to download the most recent copy of asterisk source
code from http://downloads.asterisk.org/pub/telephony/asterisk/.

Host:# wget http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-1.8-current.tar.gz
...
...
Host:# wget http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-1.8-current-patch.gz
...
...
Host:# ls
asterisk-1.8-current-patch.gz  asterisk-1.8-current.tar.gz
Host:# tar xzvf asterisk-1.8-current.tar.gz
...
...
Host:# gunzip asterisk-1.8-current-patch.gz
Host:# ls
asterisk-1.8.19.0  asterisk-1.8-current-patch  asterisk-1.8-current.tar.gz
Host:# cd asterisk-1.8.19.0/
Host:# patch -p0 < ../asterisk-1.8-current-patch
The next patch would delete the file asterisk-1.8.18.0-summary.txt,
which does not exist!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored
The next patch would delete the file asterisk-1.8.18.0-summary.html,
which does not exist!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored
patching file build_tools/make_version
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
...
...
Apply anyway? [n]
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file contrib/scripts/ast_tls_cert.rej
patching file contrib/scripts/autosupport
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file contrib/scripts/autosupport.rej
Host:#

Well, it seems the patch has been applied to the current version of the asterisk source.
We just ignore it first. Let's start to cross compile it now.

Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu \
CC="mips-linux-gnu-gcc -EL" LD="mips-linux-gnu-ld -EL" AR="mips-linux-gnu-ar" \
CXX="mips-linux-gnu-g++ -EL"
...
...
checking for tgetent in -ltermcap... no
checking for tgetent in -ltinfo... no
checking for initscr in -lcurses... no
checking for initscr in -lncurses... no
configure: error: *** termcap support not found (on modern systems, this typically
means the ncurses development package is missing)
Host:#

Unfortunately, the libtermcap makes trouble again... Since we already have experience
in cross compiling libtermcap, we try to use libncurses this time. Let's start.

Host:# wget http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz
...
...
Host:# tar xzvf ncurses-5.9.tar.gz
...
...
Host:# cd ncurses-5.9
Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu \
CC="mips-linux-gnu-gcc -EL" LD="mips-linux-gnu-ld -EL" CXX="mips-linux-gnu-g++ -EL"
...
...
Host:# make
...
...
Host:# make install DESTDIR=$PWD/cawanmipsncurses
...
...
Host:# cd cawanmipsncurses/
Host:# ls
usr
Host:# cd usr/
Host:# ls
bin  include  lib  man  share
Host:# cd lib
Host:# ls
libcurses.a  libform_g.a  libmenu_g.a   libncurses++.a  libpanel.a    terminfo
libform.a    libmenu.a    libncurses.a  libncurses_g.a  libpanel_g.a
Host:# cd ..
Host:# pwd
/home/smp383/mips-4.3/bin/asterisk/testasterisk/ncurses-5.9/cawanmipsncurses/usr
Host:#

Nice, we have libncurses right now. Let's cross compile the asterisk again by specifying
the path of libncurses explicitly.

Host:# make distclean
...
...
Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu CC="mips-linux-gnu-gcc -EL" \
LD="mips-linux-gnu-ld -EL" AR="mips-linux-gnu-ar" CXX="mips-linux-gnu-g++ -EL" \
--with-ncurses=/home/smp383/mips-4.3/bin/asterisk/testasterisk/ncurses-5.9/cawanmipsncurses/usr
...
...
checking for xml2-config... no
configure: *** XML documentation will not be available because the 'libxml2' development package is missing.
configure: *** Please run the 'configure' script with the '--disable-xmldoc' parameter option
configure: *** or install the 'libxml2' development package.
Host:# make
make: -F.: Command not found
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
/bin/sh: Illegal option -
****
**** The configure script must be executed before running 'make'.
****               Please run "./configure".
****
make: *** [makeopts] Error 1

Well, it seems we should add --disable-xmldoc in configure tool. Let's try again.

Host:# make distclean
...
...
Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu CC="mips-linux-gnu-gcc -EL" \
LD="mips-linux-gnu-ld -EL" AR="mips-linux-gnu-ar" CXX="mips-linux-gnu-g++ -EL" \
--with-ncurses=/home/smp383/mips-4.3/bin/asterisk/testasterisk/ncurses-5.9/cawanmipsncurses/usr \
--disable-xmldoc
...
...
configure: Package configured for:
configure: OS type  : linux-gnu
configure: Host CPU : mips
configure: build-cpu:vendor:os: i686 : pc : linux-gnu :
configure: host-cpu:vendor:os: mips : unknown : linux-gnu :
configure: Cross Compilation = YES
Host:# make
...
...
checking for tgetent in -ltermcap... no
checking for tgetent in -ltinfo... no
checking for tgetent in -lcurses... no
checking for tgetent in -lncurses... no
configure: error: termcap support not found
make[1]: *** [editline/libedit.a] Error 1
make: *** [main] Error 2
Host:#

Unfortunately, error again in generating editline/libedit.a archive. Let's check.

Host:# find . -iname editline
./main/editline
Host:# cd main/editline/
Host:# nano configure

Let's have a look to the configure file. Since we already passed the libncurses path
to the main configure tool, so, we can try to bypass the check error directly and see
the libedit.a will be generated or not. At lines around 1282, we can see something
like this.

if { (eval echo configure:1282: \"$ac_link\") 1>&5; (eval $ac_link) 2>&5; } && \
test -s conftest${ac_exeext}; then
  rm -rf conftest*
  eval "ac_cv_lib_$ac_lib_var=yes"
else
  echo "configure: failed program was:" >&5
  cat conftest.$ac_ext >&5
  rm -rf conftest*
  eval "ac_cv_lib_$ac_lib_var=no"
fi

When libncurses is missing, the "ac_cv_lib_$ac_lib_var" will set to "no". Now, let
us change it to "yes". So, after the modification, it should look something like
this.

if { (eval echo configure:1282: \"$ac_link\") 1>&5; (eval $ac_link) 2>&5; } && \
test -s conftest${ac_exeext}; then
  rm -rf conftest*
  eval "ac_cv_lib_$ac_lib_var=yes"
else
  echo "configure: failed program was:" >&5
  cat conftest.$ac_ext >&5
  rm -rf conftest*
  eval "ac_cv_lib_$ac_lib_var=yes"
fi

Let us start the cross compilation again.

Host:# pwd
/home/smp383/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/main/editline
Host:# cd ..
Host:# cd ..
Host:# make
...
...
 +--------- Asterisk Build Complete ---------+
 + Asterisk has successfully been built, and +
 + can be installed by running:              +
 +                                           +
 +                make install               +
 +-------------------------------------------+
Host:#

Excellent, the cross compilation process completed. Let us verify the libedit.a being
generated.

Host:# mkdir ./test
Host:# cd test
Host:# cp ../main/editline/libedit.a .
Host:# mips-linux-gnu-ar t libedit.a
editline.o_a
fgetln.o_a
vis.o_a
unvis.o_a
strlcpy.o_a
strlcat.o_a
history.o_a
tokenizer.o_a
readline.o_a
Host:# mips-linux-gnu-ar x libedit.a
Host:# ls
editline.o_a  fgetln.o_a  history.o_a  libedit.a  readline.o_a  strlcat.o_a 
strlcpy.o_a  tokenizer.o_a  unvis.o_a  vis.o_a
Host:# file editline.o_a
editline.o_a: ELF 32-bit LSB relocatable, MIPS, MIPS32 rel2 version 1 (SYSV), not stripped
Host:#

Nice, the object file being generated are really those we are looking for. Let's continue
our cross compilation process.

Host:# cd ..
Host:# make install DESTDIR=$PWD/cawanasterisk
...
...
 +---- Asterisk Installation Complete -------+
 +                                           +
 +    YOU MUST READ THE SECURITY DOCUMENT    +
 +                                           +
 + Asterisk has successfully been installed. +
 + If you would like to install the sample   +
 + configuration files (overwriting any      +
 + existing config files), run:              +
 +                                           +
 +                make samples               +
 +                                           +
 +-----------------  or ---------------------+
 +                                           +
 + You can go ahead and install the asterisk +
 + program documentation now or later run:   +
 +                                           +
 +               make progdocs               +
 +                                           +
 + **Note** This requires that you have      +
 + doxygen installed on your local system    +
 +-------------------------------------------+
Host:# ls ./cawanasterisk/etc/asterisk/
Host:#

Well, still no any default configuration files found. Let's make samples.

Host:# make samples DESTDIR=$PWD/cawanasterisk
...
...
Installing file phoneprov/000000000000.cfg
Installing file phoneprov/000000000000-directory.xml
Installing file phoneprov/000000000000-phone.cfg
Installing file phoneprov/polycom_line.xml
Installing file phoneprov/polycom.xml
Installing file phoneprov/snom-mac.xml
Host:# ls -l ./cawanasterisk/etc/asterisk/
total 796
-rw-r--r-- 1 root root   140 2012-12-12 18:37 adsi.conf
-rw-r--r-- 1 root root  2636 2012-12-12 18:37 agents.conf
-rw-r--r-- 1 root root  2904 2012-12-12 18:37 ais.conf
-rw-r--r-- 1 root root  2084 2012-12-12 18:37 alarmreceiver.conf
-rw-r--r-- 1 root root  3498 2012-12-12 18:37 alsa.conf
-rw-r--r-- 1 root root   767 2012-12-12 18:37 amd.conf
-rw-r--r-- 1 root root  1044 2012-12-12 18:37 app_mysql.conf
-rw-r--r-- 1 root root  3254 2012-12-12 18:37 asterisk.adsi
-rw-r--r-- 1 root root  3461 2012-12-12 18:37 asterisk.conf
-rw-r--r-- 1 root root  4803 2012-12-12 18:37 calendar.conf
-rw-r--r-- 1 root root  6955 2012-12-12 18:37 ccss.conf
-rw-r--r-- 1 root root  2466 2012-12-12 18:37 cdr_adaptive_odbc.conf
-rw-r--r-- 1 root root  8381 2012-12-12 18:37 cdr.conf
-rw-r--r-- 1 root root  1617 2012-12-12 18:37 cdr_custom.conf
-rw-r--r-- 1 root root   418 2012-12-12 18:37 cdr_manager.conf
-rw-r--r-- 1 root root  2231 2012-12-12 18:37 cdr_mysql.conf
...
...
Host:#

Good, the make samples done successfully. We don't need to perform make config because
we are doing cross compilation but not going to install the asterisk locally. Now,
let's run the asterisk in MIPS environment.

tango3[~]# ifconfig eth0 192.168.1.198
eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
tango3[~]# mkdir cawan
tango3[~]# mount -o nolock 192.168.1.197:/home/smp383 ./cawan
tango3[~]# cd cawan
tango3[cawan]# cd mips-4.3/bin/asterisk/
tango3[asterisk]# cd testasterisk/asterisk-1.8.19.0/cawanasterisk/
tango3[cawanasterisk]# ls
etc/ usr/ var/
tango3[cawanasterisk]# cd usr/sbin/
tango3[sbin]# ls
astcanary*     astgenkey*     rasterisk@
asterisk*      autosupport*   safe_asterisk*
tango3[sbin]# ./asterisk -h
Asterisk 1.8.19.0, Copyright (C) 1999 - 2012, Digium, Inc. and others.
Usage: asterisk [OPTIONS]
Valid Options:
   -V              Display version number and exit
   -C Use an alternate configuration file
   -G       Run as a group other than the caller
   -U        Run as a user other than the caller
   -c              Provide console CLI
   -d              Enable extra debugging
   -f              Do not fork
   -F              Always fork
   -g              Dump core in case of a crash
   -h              This help screen
   -i              Initialize crypto keys at startup
   -I              Enable internal timing if DAHDI timer is available
   -L        Limit the maximum load average before rejecting new calls
   -M       Limit the maximum number of calls to the specified value
   -m              Mute debugging and console output on the console
   -n              Disable console colorization
   -p              Run as pseudo-realtime thread
   -q              Quiet mode (suppress output)
   -r              Connect to Asterisk on this machine
   -R              Same as -r, except attempt to reconnect if disconnected
   -s      Connect to Asterisk via socket (only valid with -r)
   -t              Record soundfiles in /var/tmp and move them where they
                   belong after they are done
   -T              Display the time in [Mmm dd hh:mm:ss] format for each line
                   of output to the CLI
   -v              Increase verbosity (multiple v's = more verbose)
   -x         Execute command (implies -r)
   -X              Execute includes by default (allows #exec in asterisk.conf)
   -W              Adjust terminal colors to compensate for a light background
tango3[sbin]# ./asterisk -V
Asterisk 1.8.19.0
tango3[sbin]# ./asterisk -c -v                                             
Asterisk 1.8.19.0, Copyright (C) 1999 - 2012 Digium, Inc. and others.
Created by Mark Spencer
Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type 'core show license' for details.
=========================================================================
No entry for terminal type "vt102";
using dumb terminal settings.
No 'modules.conf' found, no modules will be loaded.
Unable to open AMI configuration manager.conf, or configuration is invalid.
Asterisk management interface (AMI) disabled.
Can't find indications config file indications.conf.
 Asterisk Dynamic Loader Starting:
 Asterisk PBX Core Initializing
 Registering builtin applications:
 [Answer]
 [BackGround]
 [Busy]
 [Congestion]
 [ExecIfTime]
 [Goto]
 [GotoIf]
 [GotoIfTime]
 [ImportVar]
 [Hangup]
 [Incomplete]
 [NoOp]
 [Proceeding]
 [Progress]
 [RaiseException]
 [ResetCDR]
 [Ringing]
 [SayAlpha]
 [SayDigits]
 [SayNumber]
 [SayPhonetic]
 [Set]
 [MSet]
 [SetAMAFlags]
 [Wait]
 [WaitExten]
Could not load features.conf
Unable to open Asterisk database '/var/lib/asterisk/astdb': No such file or directory
Could not find valid ccss.conf file. Using cc_max_requests default
 Asterisk Dynamic Loader Starting:
No 'modules.conf' found, no modules will be loaded.
Asterisk Ready.
*CLI>

Well, we need to specify the path of asterisk.conf. Let's go.

tango3[sbin]# pwd            
/root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/usr/sbin
tango3[sbin]# ./asterisk -c -v -C /root/cawan/mips-4.3/bin/asterisk/testasterisk/\
asterisk-1.8.19.0/cawanasterisk/etc/asterisk/asterisk.conf
Asterisk 1.8.19.0, Copyright (C) 1999 - 2012 Digium, Inc. and others.
Created by Mark Spencer
Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type 'core show license' for details.
=========================================================================
No entry for terminal type "vt102";
using dumb terminal settings.
No 'modules.conf' found, no modules will be loaded.
Unable to open AMI configuration manager.conf, or configuration is invalid.
Asterisk management interface (AMI) disabled.
Can't find indications config file indications.conf.
 Asterisk Dynamic Loader Starting:
 Asterisk PBX Core Initializing
 Registering builtin applications:
 [Answer]
 [BackGround]
 [Busy]
 [Congestion]
 [ExecIfTime]
 [Goto]
 [GotoIf]
 [GotoIfTime]
 [ImportVar]
 [Hangup]
 [Incomplete]
 [NoOp]
 [Proceeding]
 [Progress]
 [RaiseException]
 [ResetCDR]
 [Ringing]
 [SayAlpha]
 [SayDigits]
 [SayNumber]
 [SayPhonetic]
 [Set]
 [MSet]
 [SetAMAFlags]
 [Wait]
 [WaitExten]
Could not load features.conf
Unable to open Asterisk database '/var/lib/asterisk/astdb': No such file or directory
Could not find valid ccss.conf file. Using cc_max_requests default
 Asterisk Dynamic Loader Starting:
No 'modules.conf' found, no modules will be loaded.
Asterisk Ready.
*CLI>

It seems the asterisk.conf needs to be modified. Let's do it.

Host:# nano asterisk.conf

[directories](!)
astetcdir => /etc/asterisk
astmoddir => /usr/lib/asterisk/modules
astvarlibdir => /var/lib/asterisk
astdbdir => /var/lib/asterisk
astkeydir => /var/lib/asterisk
astdatadir => /var/lib/asterisk
astagidir => /var/lib/asterisk/agi-bin
astspooldir => /var/spool/asterisk
astrundir => /var/run/asterisk
astlogdir => /var/log/asterisk

Let us change it to the correct path.

[directories]  
astetcdir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/etc/asterisk
astmoddir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/usr/lib/asterisk/modules
astvarlibdir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/lib/asterisk
astdbdir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/lib/asterisk
astkeydir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/lib/asterisk
astdatadir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/lib/asterisk
astagidir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/lib/asterisk/agi-bin
astspooldir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/spool/asterisk
astrundir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/run/asterisk
astlogdir => /root/cawan/mips-4.3/bin/asterisk/testasterisk/asterisk-1.8.19.0/cawanasterisk/var/log/asterisk

Remember to remove the (!) after [directories]

Let's start the asterisk again in MIPS environment.

tango3[sbin]# ./asterisk -c -v -C /root/cawan/mips-4.3/bin/asterisk/testasterisk/\
asterisk-1.8.19.0/cawanasterisk/etc/asterisk/asterisk.conf
Asterisk 1.8.19.0, Copyright (C) 1999 - 2012 Digium, Inc. and others.
Created by Mark Spencer
Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type 'core show license' for details.
=========================================================================
...
...
 pbx_ael.so => (Asterisk Extension Language Compiler)
 app_readexten.so => (Read and evaluate extension validity)
 func_channel.so => (Channel information dialplan functions)
 func_volume.so => (Technology independent volume control)
 func_sysinfo.so => (System information related functions)
 func_cdr.so => (Call Detail Record (CDR) dialplan function)
 res_mutestream.so => (Mute audio stream resources)
 app_directory.so => (Extension Directory)
 pbx_spool.so => (Outgoing Spool Support)
 func_pitchshift.so => (Audio Effects Dialplan Functions)
 func_sha1.so => (SHA-1 computation dialplan function)
 res_security_log.so => (Security Event Logging)
 func_frame_trace.so => (Frame Trace for internal ast_frame debugging.)
 codec_a_mu.so => (A-law and Mulaw direct Coder/Decoder)
 app_test.so => (Interface Test Application)
 func_iconv.so => (Charset conversions)
 chan_unistim.so => (UNISTIM Protocol (USTM))
 codec_gsm.so => (GSM Coder/Decoder)
 bridge_simple.so => (Simple two channel bridging module)
 app_senddtmf.so => (Send DTMF digits Application)
 app_milliwatt.so => (Digital Milliwatt (mu-law) Test Application)
 app_chanspy.so => (Listen to the audio of an active channel)
 func_callcompletion.so => (Call Control Configuration Function)
 codec_alaw.so => (A-law Coder/Decoder)
 app_exec.so => (Executes dialplan applications)
 codec_g726.so => (ITU G.726-32kbps G726 Transcoder)
 app_morsecode.so => (Morse code)
 app_originate.so => (Originate call)
 res_clioriginate.so => (Call origination and redirection from the CLI)
 func_timeout.so => (Channel timeout dialplan functions)
 pbx_loopback.so => (Loopback Switch)
 app_cdr.so => (Tell Asterisk to not maintain a CDR for the current call)
 app_disa.so => (DISA (Direct Inward System Access) Application)
 app_voicemail.so => (Comedian Mail (Voicemail System))
 app_while.so => (While Loops and Conditional Execution)
 func_callerid.so => (Party ID related dialplan functions (Caller-ID, Connected-line, Redirecting))
 app_dictate.so => (Virtual Dictation Machine)
 app_controlplayback.so => (Control Playback Application)
 app_image.so => (Image Transmission Application)
 app_playtones.so => (Playtones Application)
 app_playback.so => (Sound File Playback Application)
 app_adsiprog.so => (Asterisk ADSI Programming Application)
 func_vmcount.so => (Indicator for whether a voice mailbox has messages in a given folder.)
 func_md5.so => (MD5 digest dialplan functions)
 bridge_softmix.so => (Multi-party software based channel mixing)
 res_limit.so => (Resource limits)
 res_realtime.so => (Realtime Data Lookup/Rewrite)
 func_db.so => (Database (astdb) related dialplan functions)
 codec_ilbc.so => (iLBC Coder/Decoder)
 chan_phone.so => (Linux Telephony API Support)
 app_privacy.so => (Require phone number to be entered, if no CallerID sent)
 func_base64.so => (base64 encode/decode dialplan functions)
 func_uri.so => (URI encode/decode dialplan functions)
 app_authenticate.so => (Authentication Application)
 app_dial.so => (Dialing Application)
 app_speech_utils.so => (Dialplan Speech Applications)
 app_record.so => (Trivial Record Application)
 codec_lpc10.so => (LPC10 2.4kbps Coder/Decoder)
 res_phoneprov.so => (HTTP Phone Provisioning)
 app_system.so => (Generic System() application)
 app_directed_pickup.so => (Directed Call Pickup Application)
 chan_oss.so => (OSS Console Channel Driver)
 func_strings.so => (String handling dialplan functions)
 func_version.so => (Get Asterisk Version/Build Info)
 app_amd.so => (Answering Machine Detection Application)
 app_read.so => (Read Variable Application)
 app_sendtext.so => (Send Text Applications)
 codec_g722.so => (ITU G.722-64kbps G722 Transcoder)
 func_audiohookinherit.so => (Audiohook inheritance function)
 func_lock.so => (Dialplan mutexes)
 func_dialgroup.so => (Dialgroup dialplan function)
 func_blacklist.so => (Look up Caller*ID name/number from blacklist database)
 app_mixmonitor.so => (Mixed Audio Monitoring Application)
 app_url.so => (Send URL Applications)
 app_forkcdr.so => (Fork The CDR into 2 separate entities)
 func_module.so => (Checks if Asterisk module is loaded in memory)
 bridge_builtin_features.so => (Built in bridging features)
 codec_adpcm.so => (Adaptive Differential PCM Coder/Decoder)
 func_cut.so => (Cut out information from a string)
 codec_ulaw.so => (mu-Law Coder/Decoder)
 app_softhangup.so => (Hangs up the requested channel)
  == Aliased CLI command 'hangup request' to 'channel request hangup'
  == Aliased CLI command 'originate' to 'channel originate'
  == Aliased CLI command 'help' to 'core show help'
  == Aliased CLI command 'pri intense debug span' to 'pri set debug 2 span'
  == Aliased CLI command 'reload' to 'module reload'
 res_clialiases.so => (CLI Aliases)
 bridge_multiplexed.so => (Multiplexed two channel bridging module)
 res_convert.so => (File format conversion CLI command)
 app_waitforsilence.so => (Wait For Silence)
 app_festival.so => (Simple Festival Interface)
 app_sms.so => (SMS/PSTN handler)
 app_waituntil.so => (Wait until specified time)
 app_mp3.so => (Silly MP3 Application)
 func_aes.so => (AES dialplan functions)
 func_enum.so => (ENUM related dialplan functions)
 app_talkdetect.so => (Playback with Talk Detection)
 func_groupcount.so => (Channel group dialplan functions)
 app_readfile.so => (Stores output of file into a variable)
 app_minivm.so => (Mini VoiceMail (A minimal Voicemail e-mail System))
 func_rand.so => (Random number dialplan function)
 app_sayunixtime.so => (Say time)
 app_verbose.so => (Send verbose output)
 func_shell.so => (Collects the output generated by a command executed by the system shell)
 app_channelredirect.so => (Redirects a given channel to a dialplan target)
 func_env.so => (Environment/filesystem dialplan functions)
 app_setcallerid.so => (Set CallerID Presentation Application)
 app_dumpchan.so => (Dump Info About The Calling Channel)
 func_math.so => (Mathematical dialplan function)
 pbx_config.so => (Text Extension Configuration)
 func_global.so => (Variable dialplan functions)
 func_logic.so => (Logical dialplan functions)
 app_getcpeid.so => (Get ADSI CPE ID)
 app_ices.so => (Encode and Stream via icecast and ices)
 func_config.so => (Asterisk configuration file variable access)
 app_db.so => (Database Access Functions)
 app_userevent.so => (Custom User Event Application)
 app_celgenuserevent.so => (Generate an User-Defined CEL event)
 app_echo.so => (Simple Echo Application)
 app_queue.so => (True Call Queueing)
Asterisk Ready.
*CLI>

Nice, the asterisk is successfully running now. Let's check the sip info.

*CLI> sip show users
Username            Secret         Accountcode      Def.Context      ACL  ForcerPort
*CLI> sip show peers
Name/username        Host         Dyn Forcerport ACL Port     Status    
0 sip peers [Monitored: 0 online, 0 offline Unmonitored: 0 online, 0 offline]
*CLI> sip show channelstats
Peer    Call ID    Duration Recv: Pack  Lost  (%) Jitter Send: Pack  Lost  (%) Jitter
0 active SIP channels

Fine, there is no any sip extension yet. Let us create it now and restart the asterisk
again.

Host:# nano sip.conf

Go to the end of the file and append the following lines to create 2 extensions.

[101]
type=friend
secret=101
host=dynamic
context=internal
username=101
callgroup=1
pickupgroup=1

[102]
type=friend
secret=102
host=dynamic
context=internal
username=102
callgroup=1
pickupgroup=1

Then, we edit extensions.conf.

Host:# nano extensions.conf

Go to the end of the file and append the following lines.

[internal]
exten=>101,1,Dial(SIP/101,20)
exten=>102,1,Dial(SIP/102,20)

Now, let us restart the asterisk and check the sip extensions.

*CLI> sip show users
Username                   Secret           Accountcode      Def.Context      ACL  ForcerPort
101                        101                               internal         No   Yes      
102                        102                               internal         No   Yes      
*CLI> sip show peers
Name/username              Host                     Dyn Forcerport ACL Port     Status    
101/101                    (Unspecified)            D   N             0        Unmonitored
102/102                    (Unspecified)            D   N             0        Unmonitored
2 sip peers [Monitored: 0 online, 0 offline Unmonitored: 0 online, 2 offline]

Good, 2 sip extensions have been created. Let's test it by using 2 smart phones with
sip app (iSip or 3CXPhone) installed. Each smart phone registers to extension 101 and
102 respectively. Then, check the sip extensions in asterisk again.

*CLI> sip show peers
Name/username              Host                     Dyn Forcerport ACL Port     Status    
101/101                    192.168.1.3              D   N             5060     Unmonitored
102/102                    192.168.1.9              D   N             5065     Unmonitored
2 sip peers [Monitored: 0 online, 0 offline Unmonitored: 2 online, 0 offline]

Excellent, it seems both of the extensions are online right now. Let's make a call.
Yes, the callee rings and once the connection is accepted, it is ready to start a
conversation. While the conversation is running, let us check the channel statistic
in asterisk.

*CLI> sip show channelstats
Peer             Call ID      Duration Recv: Pack  Lost       (     %) Jitter Send: Pack  Lost       (     %) Jitter
192.168.1.9      .8zN5DtXR7-  00:00:22 0000000000  0000000000 ( 0.00%) 0.0000 0000000000  0000000000 ( 0.00%) 0.0000
1 active SIP channel
*CLI> sip show channels
Peer             User/ANR         Call ID          Format           Hold     Last Message    Expiry     Peer     
192.168.1.9      102              .8zN5DtXR7-g3dH  0x4 (ulaw)       No       Tx: ACK                    102      
192.168.1.3      (None)           r-9.sa-1gtndvpt  0x0 (nothing)    No       Rx: REGISTER                 
192.168.1.3      101              4d2aa9de449c20c  0x4 (ulaw)       No       Tx: ACK                    101      
3 active SIP dialogs

Nice, the conversation is shown accordingly.

pdf version:


http://www.scribd.com/doc/117698898/How-to-Cross-Compile-Asterisk-and-Run-in-Embedded-System

Monday, December 3, 2012

How To Cross Compile Python and Run in Embedded System (Latest Python 3.3.0 Version)

How To Cross Compile Python and Run in Embedded System (Latest Python 3.3.0 Version)

by cawan (cawan[at]ieee.org or chuiyewleong[at]hotmail.com)

on 1/12/2012

The cross compilation process of python 2.7.1 is very straight forward. There is a
simple guide about it in the following link,

http://www.cnx-software.com/2011/02/04/cross-compiling-python-for-mips-and-arm-platforms/

Let us do the same thing and wish to get the same result. The embedded platform to be
used is MIPS32 24k. First of all, let us download the source of python 2.7.1.

Host:# wget http://www.python.org/ftp/python/2.7.1/Python-2.7.1.tgz

Then get the patch accordingly.

Host:# wget http://www.cnx-software.com/patch/python-2.7.1-cross-compile.patch

So, we have 2 files now.

Host:# ls
python-2.7.1-cross-compile.patch  Python-2.7.1.tgz

Let us start.

Host:# tar xzvf Python-2.7.1.tgz
...
...
Host:# ls
Python-2.7.1  python-2.7.1-cross-compile.patch  Python-2.7.1.tgz
Host:# cd Python-2.7.1

Since we need python and pgen as assisting tools for our cross compilation process,
so we need to make a copy of them to be run in our host machine.

Host:# ./configure
...
...
Host:# make python Parser/pgen
...
...
Host:# mv python hostpython
Host:# mv Parser/pgen Parser/hostpgen

Well, we can later make use of the hostpython and hostpgen in our following cross
compilation process. Before that, let us revert the source to original and get
ready to cross compilation.

Host:# make distclean
...
...
Host:#

Let us apply the patch first.

Host:# patch -p1 < ../python-2.7.1-cross-compile.patch
(Stripping trailing CRs from patch.)
patching file configure
(Stripping trailing CRs from patch.)
patching file configure.in
(Stripping trailing CRs from patch.)
patching file Makefile.pre.in
(Stripping trailing CRs from patch.)
patching file setup.py
Host:#

Good, it seems the patch is fine. Now let us run the configure tool to generate
appropriate makefile.

Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu \
CC="mips-linux-gnu-gcc -EL" CXX="mips-linux-gnu-g++ -EL" AR=mips-linux-gnu-ar \
RANLIB=mips-linux-gnu-ranlib

We made some modification to the command line from the original one because the
cross compiler that we are using is different to the reference. For the same reason,
we will make some modification to the make command also, and run it accordingly.
Before that, let us make sure the variables BLDSHARED and HOSTPYTHON are really
being used in the make process.

Host:# grep BLDSHARED Makefile
BLDSHARED=    $(CC) -shared $(LDFLAGS)
...
...
Host:# grep HOSTPYTHON Makefile
HOSTPYTHON=    ./$(BUILDPYTHON)
...
...
Host:#

Good, the results are really made sense. Let us start the make process now.

Host:# make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes

However, we can't get the job done as what the reference did, and the error message
is stated as below.

  File "./setup.py", line 316
    self.announce('*** WARNING: renaming "%s" since importing it'
       ^
IndentationError: expected an indented block
make: *** [sharedmods] Error 1
Host:#

It seems this is just a simple indentation error of setup.py. Let us open the
setup.py and add some spaces as indentation accordingly.

if os.environ.get('CROSS_COMPILE_TARGET') != "yes":
self.announce('*** WARNING: renaming "%s" since importing it'
           ' failed: %s' % (ext.name, why), level=3)
          
After the modification, it supposes to look like this.

if os.environ.get('CROSS_COMPILE_TARGET') != "yes":
    self.announce('*** WARNING: renaming "%s" since importing it'
           ' failed: %s' % (ext.name, why), level=3)

Now, let us rerun the make process. After a while, the make process completed.
So, let us perform make install now.

Host:# make install HOSTPYTHON=./hostpython BLDSHARED="mips-linux-gnu-gcc -EL -shared" \
CROSS_COMPILE=mips-linux-gnu- CROSS_COMPILE_TARGET=yes DESTDIR=$PWD/cawanpython

However, we get error again, and the error message is stated as below.

File "/home/smp383/mips-4.3/bin/testpython/Python-2.7.1/cawanpython/usr/local/lib/
python2.7/struct.py", line 1, in
    from _struct import *
ImportError: /home/smp383/mips-4.3/bin/testpython/Python-2.7.1/build/lib.linux-i686-2.7/
_struct.so: cannot open shared object file: No such file or directory
make: *** [libinstall] Error 1
Host:#

Well, it seems the _struct.so can't be found. Let us check the shared library is
available or not.

Host:# ls -l ./build/lib.linux-i686-2.7/_struct.so
-rwxr-xr-x 1 root root 85558 2012-11-29 17:34 ./build/lib.linux-i686-2.7/_struct.so
Host:#

Strange, the _struct.so is in the path. Anyway, let us check the destination directory.

Host:# cd ./cawanpython/usr/local/lib/python2.7/
Host:# ls lib
lib2to3/ lib-old/ lib-tk/ 
Host:# ls lib

Unfortunately, the lib-dynload directory is even not generated yet. Let us try to
ignore the make error and let the make process to continue, and then observe the
destination directory again.

Host:# make -i install HOSTPYTHON=./hostpython BLDSHARED="mips-linux-gnu-gcc -EL -shared" \
CROSS_COMPILE=mips-linux-gnu- CROSS_COMPILE_TARGET=yes DESTDIR=$PWD/cawanpython

Please note that the -i option is added after the make command. After a while, the make
process done. Let's check the lib-dynload directory.

Host:# cd ./cawanpython/usr/local/
bin/     include/ lib/     share/  
Host:# cd ./cawanpython/usr/local/lib/python2.7/
Host:# ls lib-
lib-dynload/ lib-old/     lib-tk/     
Host:# ls lib-dynload/
array.so            _ctypes.so          linuxaudiodev.so             resource.so
audioop.so          _ctypes_test.so     _locale.so                   select.so
_bisect.so          datetime.so         _lsprof.so                   _sha256.so
cmath.so            dl.so               math.so                      _sha512.so
_codecs_cn.so       _elementtree.so     _md5.so                      _sha.so
_codecs_hk.so       fcntl.so            mmap.so                      _socket.so
_codecs_iso2022.so  _functools.so       _multibytecodec.so           spwd.so
_codecs_jp.so       future_builtins.so  _multiprocessing.so          strop.so
_codecs_kr.so       grp.so              nis.so                       _struct.so
_codecs_tw.so       _heapq.so           operator.so                  syslog.so
_collections.so     _hotshot.so         ossaudiodev.so               termios.so
cPickle.so          imageop.so          parser.so                    _testcapi.so
crypt.so            _io.so              pyexpat.so                   time.so
cStringIO.so        itertools.so        Python-2.7.1-py2.7.egg-info  unicodedata.so
_csv.so             _json.so            _random.so
Host:#

Excellent, all the shared libraries are built. Let's check the python executable.

Host:# cd cawanpython/usr/local/bin/
Host:# ls
2to3  idle  pydoc  python  python2.7  python2.7-config  python-config  smtpd.py
Host:# file python
python: ELF 32-bit LSB executable, MIPS, MIPS32 rel2 version 1, for GNU/Linux 2.6.12,
dynamically linked (uses shared libs), not stripped
Host:#

Well, it should work by ignoring the error message. Let's verify in MIPS environment.

tango3[Python-2.7.1]# cd ./cawanpython/usr/local/
bin/     include/ lib/     share/  
tango3[Python-2.7.1]# cd ./cawanpython/usr/local/bin/
tango3[bin]# ls
2to3*             pydoc*            python-config@    python2.7-config*
idle*             python*           python2.7*        smtpd.py*
tango3[bin]# ./python
Python 2.7.1 (r271:86832, Nov 29 2012, 17:22:08)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> pack
Traceback (most recent call last):
  File "", line 1, in
>>> from struct import *
>>> pack

>>>

Nice, it seems everything is fine now and modules can be imported and run properly.
Now, let us take some challenges to cross compile the latest version of python,
which is python 3.3.0. We will start by applying the same analogy.

Host:# wget http://www.python.org/ftp/python/3.3.0/Python-3.3.0.tgz

For the patch of cross compilation, we need py3k-20121004-CROSS.tgz.

Host:# wget http://bugs.python.org/file27444/py3k-20121004-CROSS.tgz

Let us start to apply the patch.

Host:# tar xzvf py3k-20121004-CROSS.tgz
0001-CROSS-fix-typo-in-thread-AC_CACHE_VAL.patch
0002-CROSS-restore-graminit.-to-source-directory.patch
0003-CROSS-restore-importlib-header-to-source-directory-a.patch
0004-CROSS-restore-AST-to-source-directory.patch
0005-CROSS-revert-issue13150-i.e.-python-solution-with-_s.patch
0006-CROSS-initialise-include-and-library-paths.patch
0007-CROSS-set-_PYTHON_PROJECT_BASE-to-current-build-dir.patch
0008-CROSS-use-_PYTHON_PROJECT_BASE-in-distutils-sysconfi.patch
0009-CROSS-pass-all-top-configure-arguments-to-libffi-con.patch
0010-CROSS-warn-only-if-readelf-is-not-in-host-triplet-fo.patch
0011-CROSS-append-gcc-library-search-paths-instead-to-pre.patch
0012-CROSS-avoid-ncursesw-include-path-hack.patch
0013-CROSS-properly-detect-WINDOW-_flags-for-different-nc.patch
0014-CROSS-use-build-directory-as-root-for-regression-tes.patch
0015-CROSS-test-tools-has-to-depend-only-from-location-of.patch
0016-CROSS-reload-may-fail-with-operation-on-closed-file-.patch
Host:#

There are 16 of them, let us try it one by one.

Host:# cd Python-3.3.0
Host:# patch -p1 < ../0001-CROSS-fix-typo-in-thread-AC_CACHE_VAL.patch
patching file configure.ac
Host:# patch -p1 < ../0002-CROSS-restore-graminit.-to-source-directory.patch
patching file Makefile.pre.in
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file Makefile.pre.in.rej
Host:# patch -p1 < ../0003-CROSS-restore-importlib-header-to-source-directory-a.patch
patching file Makefile.pre.in
Hunk #1 succeeded at 586 (offset -5 lines).
Hunk #2 succeeded at 1344 (offset -7 lines).
Host:# patch -p1 < ../0004-CROSS-restore-AST-to-source-directory.patch
patching file Makefile.pre.in
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file Makefile.pre.in.rej
Host:# patch -p1 < ../0005-CROSS-revert-issue13150-i.e.-python-solution-with-_s.patch
patching file Lib/sysconfig.py
patching file Makefile.pre.in
Hunk #1 succeeded at 470 with fuzz 1.
Host:# patch -p1 < ../0006-CROSS-initialise-include-and-library-paths.patch
patching file setup.py
Host:# patch -p1 < ../0007-CROSS-set-_PYTHON_PROJECT_BASE-to-current-build-dir.patch
patching file configure.ac
Host:# patch -p1 < ../0008-CROSS-use-_PYTHON_PROJECT_BASE-in-distutils-sysconfi.patch
patching file Lib/distutils/sysconfig.py
Host:# patch -p1 < ../0009-CROSS-pass-all-top-configure-arguments-to-libffi-con.patch
patching file setup.py
Hunk #1 FAILED at 1790.
1 out of 1 hunk FAILED -- saving rejects to file setup.py.rej
Host:# patch -p1 < ../0010-CROSS-warn-only-if-readelf-is-not-in-host-triplet-fo.patch
patching file configure.ac
Host:# patch -p1 < ../0011-CROSS-append-gcc-library-search-paths-instead-to-pre.patch
patching file setup.py
Host:# patch -p1 < ../0012-CROSS-avoid-ncursesw-include-path-hack.patch
patching file configure.ac
Host:# patch -p1 < ../0013-CROSS-properly-detect-WINDOW-_flags-for-different-nc.patch
patching file Include/py_curses.h
patching file configure.ac
patching file pyconfig.h.in
Host:# patch -p1 < ../0014-CROSS-use-build-directory-as-root-for-regression-tes.patch
patching file Lib/test/regrtest.py
Host:# patch -p1 < ../0015-CROSS-test-tools-has-to-depend-only-from-location-of.patch
patching file Lib/test/test_tools.py
Host:# patch -p1 < ../0016-CROSS-reload-may-fail-with-operation-on-closed-file-.patch
patching file Lib/imp.py
Host:#

Well, patch 0002, 0004, and 0009 are failed. Let's ignore it first and start our
cross compilation process now.

Host:# ./configure
Host:# make python Parser/pgen
Host:# mv python hostpython
Host:# mv Parser/pgen Parser/hostpgen
Host:# make distclean

Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu \
CC="mips-linux-gnu-gcc -EL" CXX="mips-linux-gnu-g++ -EL" AR=mips-linux-gnu-ar \
RANLIB=mips-linux-gnu-ranlib

However, we get an error.

configure: error: set ac_cv_file__dev_ptmx to yes/no in your CONFIG_SITE file when cross compiling

Let us try to solve it.

Host:# echo ac_cv_file__dev_ptmx=no > config.site
Host:# echo ac_cv_file__dev_ptc=no >> config.site
Host:# cat ./config.site
ac_cv_file__dev_ptmx=no
ac_cv_file__dev_ptc=no
Host:# export CONFIG_SITE=config.site
Host:# set | grep CONFIG_SITE
CONFIG_SITE=config.site
_=CONFIG_SITE
Host:#

Let's run the configure tool again. After a while, the process conpleted. Now, let us
try to start the make process.

Host:# make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes

After a while, we get an error.

case $MAKEFLAGS in *s*) quiet=-q; esac; \
     CC='mips-linux-gnu-gcc -EL' LDSHARED='mips-linux-gnu-gcc -EL -shared'
     OPT='-DNDEBUG -g  -O3 -Wall -Wstrict-prototypes' \
        ./python -E ./setup.py $quiet build
./python: 1: Syntax error: "(" unexpected
make: *** [sharedmods] Error 2

The make process should use ./hostpython instead of ./python. Must be something
wrong in Makefile, let's check.

Host:# nano Makefile

Go to line 497 and we should get something like this.

# Build the shared modules
sharedmods: $(BUILDPYTHON) $(SYSCONFIGDATA)
        case $$MAKEFLAGS in *s*) quiet=-q; esac; \
        $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' \
                $(PYTHON_FOR_BUILD) $(srcdir)/setup.py $$quiet build

Well, it seems the HOSTPYTHON has been replaced by PYTHON_FOR_BUILD. Let us verify
it by referring to the Makefile of python 2.7.1.

Host:# nano ../../Python-2.7.1/Makefile

Go to line 425 and we should get something like this.

# Build the shared modules
sharedmods: $(BUILDPYTHON)
        @case $$MAKEFLAGS in \
        *s*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)'  $(HOSTPYTHON) -E $($
        *) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)'  $(HOSTPYTHON) -E $(sr$
        esac

Good, we should change the HOSTPYTHON option into PYTHON_FOR_BUILD in python 3.3.0.
Let's run the make process again.

Host:# make PYTHON_FOR_BUILD=./hostpython HOSTPGEN=./Parser/hostpgen \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes

Fine, after a while, the make process completed. Let's go ahead for make install
process, and we should remember to change the HOSTPYTHON into PYTHON_FOR_BUILD.

Host:# make install PYTHON_FOR_BUILD=./hostpython BLDSHARED="mips-linux-gnu-gcc -EL -shared" \
CROSS_COMPILE=mips-linux-gnu- CROSS_COMPILE_TARGET=yes DESTDIR=$PWD/cawanpython

Nice, after a while, the make install process completed. Let's check the destination
directory.

Host:# cd ./cawanpython/usr/local/
Host:# ls
bin  include  lib  share
Host:# cd bin
Host:# ls
2to3      idle3    pydoc3    python3    python3.3-config  python3.3m-config  pyvenv
2to3-3.3  idle3.3  pydoc3.3  python3.3  python3.3m        python3-config     pyvenv-3.3
Host:# cd ../lib/
Host:# cd python3.3/
Host:# cd lib-dynload/
Host:# ls
array.cpython-33m_failed.so            math.cpython-33m_failed.so
atexit.cpython-33m_failed.so           _md5.cpython-33m_failed.so
audioop.cpython-33m_failed.so          mmap.cpython-33m_failed.so
_bisect.cpython-33m_failed.so          _multibytecodec.cpython-33m_failed.so
cmath.cpython-33m_failed.so            _multiprocessing.cpython-33m_failed.so
_codecs_cn.cpython-33m_failed.so       nis.cpython-33m_failed.so
_codecs_hk.cpython-33m_failed.so       parser.cpython-33m_failed.so
_codecs_iso2022.cpython-33m_failed.so  _pickle.cpython-33m_failed.so
_codecs_jp.cpython-33m_failed.so       _posixsubprocess.cpython-33m_failed.so
_codecs_kr.cpython-33m_failed.so       pyexpat.cpython-33m_failed.so
_codecs_tw.cpython-33m_failed.so       _random.cpython-33m_failed.so
_crypt.cpython-33m_failed.so           resource.cpython-33m_failed.so
_csv.cpython-33m_failed.so             _sha1.cpython-33m_failed.so
_ctypes.cpython-33m_failed.so          _sha256.cpython-33m_failed.so
_ctypes_test.cpython-33m_failed.so     _sha512.cpython-33m_failed.so
_datetime.cpython-33m_failed.so        spwd.cpython-33m_failed.so
_decimal.cpython-33m_failed.so         _struct.cpython-33m_failed.so
_elementtree.cpython-33m_failed.so     syslog.cpython-33m_failed.so
fcntl.cpython-33m_failed.so            termios.cpython-33m_failed.so
grp.cpython-33m_failed.so              _testbuffer.cpython-33m_failed.so
_heapq.cpython-33m_failed.so           _testcapi.cpython-33m_failed.so
_json.cpython-33m_failed.so            time.cpython-33m_failed.so
_lsprof.cpython-33m_failed.so          unicodedata.cpython-33m_failed.so
Host:#

The filenames of those shared libraries in lib-dynload directory are really strange.
Anyway, we ignore this issue first. Let's test the new python 3.3.0 in MIPS environment.

tango3[Python-3.3.0]# cd ./cawanpython/
tango3[cawanpython]# cd ..
tango3[Python-3.3.0]# cd ./cawanpython/usr/local/bin/
tango3[bin]# ls
2to3@              pydoc3@            python3.3*         pyvenv@
2to3-3.3*          pydoc3.3*          python3.3-config@  pyvenv-3.3*
idle3@             python3@           python3.3m*
idle3.3*           python3-config@    python3.3m-config*
tango3[bin]# ./python3
Python 3.3.0 (default, Nov 29 2012, 20:02:02)
[GCC 4.3.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> pack
Traceback (most recent call last):
  File "", line 1, in
NameError: name 'pack' is not defined
>>> from struct import *
Traceback (most recent call last):
  File "", line 1, in
  File "/root/cawan/mips-4.3/bin/testpython/Python-3.3.0/cawanpython/usr/local/lib/
  python3.3/struct.py", line 12, in
    from _struct import *
ImportError: No module named '_struct'
>>> pack
Traceback (most recent call last):
  File "", line 1, in
NameError: name 'pack' is not defined
>>>

Well, it seems the new python cannot import the struct library. Anyway, it is
expected. Let's change the strange filename of struct library into the proper one
and try again.

tango3[bin]# cd ../lib/python3.3/lib-dynload/
tango3[lib-dynload]# ls *struct*
_struct.cpython-33m_failed.so
tango3[lib-dynload]# cp _struct.cpython-33m_failed.so _struct.so
tango3[lib-dynload]# ls _struct.so
_struct.so
tango3[lib-dynload]# ls -l _struct.so
-rw-r--r--    1 root     root        86258 Nov 29  2012 _struct.so
tango3[lib-dynload]# cd ../../../bin/
tango3[bin]# ./python3
Python 3.3.0 (default, Nov 29 2012, 20:02:02)
[GCC 4.3.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> pack
Traceback (most recent call last):
  File "", line 1, in
NameError: name 'pack' is not defined
>>> from struct import *
>>> pack

>>>

Excellent, the python can import the struct library now and run the built-in command.
Let's try to import the socket library to double confirm.

tango3[Python-3.3.0]# cd ./cawanpython/usr/local/
tango3[local]# cd bin
tango3[bin]# ./python3
Python 3.3.0 (default, Nov 28 2012, 19:31:37)
[GCC 4.3.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import *
Traceback (most recent call last):
  File "", line 1, in
  File "/root/cawan/mips-4.3/bin/Python-3.3.0/cawanpython/usr/local/lib/
  python3.3/socket.py", line 47, in
    import _socket
ImportError: No module named '_socket'
>>>
tango3[bin]# ls ../lib/python3.3/lib-dynload/*socket*
ls: ../lib/python3.3/lib-dynload/*socket*: No such file or directory
tango3[bin]#

Unfortunately, the shared library of socket doesn't exist at all, so, we even can't
rename it from the strange filename. Let's check the make messages again in details.

...
...
...
Python build finished, but the necessary bits to build these modules were not found:
_bz2               _dbm               _gdbm          
_lzma              _sqlite3           _ssl           
_tkinter           readline                          
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_bisect            _codecs_cn         _codecs_hk     
_codecs_iso2022    _codecs_jp         _codecs_kr     
_codecs_tw         _crypt             _csv           
_ctypes            _ctypes_test       _curses        
_curses_panel      _datetime          _decimal       
_elementtree       _heapq             _json          
_lsprof            _md5               _multibytecodec
_multiprocessing   _pickle            _posixsubprocess
_random            _sha1              _sha256        
_sha512            _socket            _struct        
_testbuffer        _testcapi          array          
atexit             audioop            binascii       
cmath              fcntl              grp            
math               mmap               nis            
ossaudiodev        parser             pyexpat        
resource           select             spwd           
syslog             termios            time           
unicodedata        zlib                              
...
...
...

Well, that's the reason all the filenames of shared libraries in lib-dynload directory
are in strange format, they are even not being compiled properly... So, the cross
compilation patch for python 3.3.0 might be useless at all. Let us start everything
from the begining again, without applying the patch.

Host:# tar xzvf Python-3.3.0.tgz
Host:# cd Python-3.3.0
Host:# ./configure
Host:# make python Parser/pgen
Host:# mv python hostpython
Host:# mv Parser/pgen Parser/hostpgen
Host:# make distclean
Host:# echo ac_cv_file__dev_ptmx=yes > ./config.site
Host:# echo ac_cv_file__dev_ptc=yes >> ./config.site
Host:# export CONFIG_SITE=config.site
Host:# set | grep CONFIG_SITE
CONFIG_SITE=config.site
Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu \
CC="mips-linux-gnu-gcc -EL" CXX="mips-linux-gnu-g++ -EL" AR=mips-linux-gnu-ar \
RANLIB=mips-linux-gnu-ranlib LD="mips-linux-gnu-ld -EL"

Host:# make PYTHON_FOR_BUILD=./hostpython HOSTPGEN=./Parser/hostpgen \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes

Host:# make install PYTHON_FOR_BUILD=./hostpython \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes DESTDIR=$PWD/cawanpython

Good, the cross compilation process is completed without applying any patch. However,
we still get this message output.

Python build finished, but the necessary bits to build these modules were not found:
_bz2               _dbm               _gdbm          
_lzma              _sqlite3           _ssl           
_tkinter           readline                          
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Failed to build these modules:
_bisect            _codecs_cn         _codecs_hk     
_codecs_iso2022    _codecs_jp         _codecs_kr     
_codecs_tw         _crypt             _csv           
_ctypes            _ctypes_test       _curses        
_curses_panel      _datetime          _decimal       
_elementtree       _heapq             _json          
_lsprof            _md5               _multibytecodec
_multiprocessing   _pickle            _posixsubprocess
_random            _sha1              _sha256        
_sha512            _socket            _struct        
_testbuffer        _testcapi          array          
atexit             audioop            binascii       
cmath              fcntl              grp            
math               mmap               nis            
ossaudiodev        parser             pyexpat        
resource           select             spwd           
syslog             termios            time           
unicodedata        zlib                              

Anyway, ignore this and check the lib-dynload directory first.

Host:# ls -l ./cawanpython/usr/local/lib/python3.3/lib-dynload/
total 6200
-rw-r--r-- 1 root root 107510 2012-12-01 17:49 array.cpython-33m_failed.so
-rw-r--r-- 1 root root  23385 2012-12-01 17:50 atexit.cpython-33m_failed.so
-rw-r--r-- 1 root root  64169 2012-12-01 17:50 audioop.cpython-33m_failed.so
-rw-r--r-- 1 root root  23615 2012-12-01 17:50 _bisect.cpython-33m_failed.so
-rw-r--r-- 1 root root  70333 2012-12-01 17:50 cmath.cpython-33m_failed.so
-rw-r--r-- 1 root root 152927 2012-12-01 17:50 _codecs_cn.cpython-33m_failed.so
-rw-r--r-- 1 root root 160944 2012-12-01 17:50 _codecs_hk.cpython-33m_failed.so
-rw-r--r-- 1 root root  52443 2012-12-01 17:50 _codecs_iso2022.cpython-33m_failed.so
-rw-r--r-- 1 root root 253712 2012-12-01 17:50 _codecs_jp.cpython-33m_failed.so
-rw-r--r-- 1 root root 143311 2012-12-01 17:50 _codecs_kr.cpython-33m_failed.so
-rw-r--r-- 1 root root 111884 2012-12-01 17:50 _codecs_tw.cpython-33m_failed.so
-rw-r--r-- 1 root root  14570 2012-12-01 17:50 _crypt.cpython-33m_failed.so
...
...
...
-rw-r--r-- 1 root root  21492 2012-12-01 17:50 spwd.cpython-33m_failed.so
-rw-r--r-- 1 root root  86387 2012-12-01 17:49 _struct.cpython-33m_failed.so
-rw-r--r-- 1 root root  23632 2012-12-01 17:50 syslog.cpython-33m_failed.so
-rw-r--r-- 1 root root  32439 2012-12-01 17:50 termios.cpython-33m_failed.so
-rw-r--r-- 1 root root  97067 2012-12-01 17:50 _testbuffer.cpython-33m_failed.so
-rw-r--r-- 1 root root 133042 2012-12-01 17:50 _testcapi.cpython-33m_failed.so
-rw-r--r-- 1 root root  59832 2012-12-01 17:50 time.cpython-33m_failed.so
-rw-r--r-- 1 root root 800048 2012-12-01 17:50 unicodedata.cpython-33m_failed.so
Host:#

So, it seems the Makefile being generated is so far so good, the problem should be
in setup.py. After having a brief look to the setup.py, it seems the script can't
work in proper once the ssl lib doesn't exist. By referring to the cross compilation
patch of python 2.7.1, I created a patch for python 3.3.0, which I named it as
python-3.3.0-cross-compile-cawan.patch. Let's verify.

Host:# make distclean
Host:# patch -p1 < python-3.3.0-cross-compile-cawan.patch
patching file setup.py
Host:# ./configure --host=mips-linux-gnu --target=mips-linux-gnu \
CC="mips-linux-gnu-gcc -EL" CXX="mips-linux-gnu-g++ -EL" AR=mips-linux-gnu-ar \
RANLIB=mips-linux-gnu-ranlib LD="mips-linux-gnu-ld -EL"

Host:# make PYTHON_FOR_BUILD=./hostpython HOSTPGEN=./Parser/hostpgen \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes

Host:# make install PYTHON_FOR_BUILD=./hostpython \
BLDSHARED="mips-linux-gnu-gcc -EL -shared" CROSS_COMPILE=mips-linux-gnu- \
CROSS_COMPILE_TARGET=yes DESTDIR=$PWD/cawanpython

Excellent, the make message output become this.

Python build finished, but the necessary bits to build these modules were not found:
_bz2               _dbm               _gdbm          
_lzma              _sqlite3           _ssl           
_tkinter           readline                          
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_curses            _curses_panel      _md5           
_sha               binascii           zlib           

Let's check the generated lib-dynload directory.

Host:# ls -l ./cawanpython/usr/local/lib/python3.3/lib-dynload/
total 6136
-rwxr-xr-x 1 root root 107510 2012-12-01 18:22 array.cpython-33m.so
-rwxr-xr-x 1 root root  23385 2012-12-01 18:22 atexit.cpython-33m.so
-rwxr-xr-x 1 root root  64169 2012-12-01 18:22 audioop.cpython-33m.so
-rwxr-xr-x 1 root root  23615 2012-12-01 18:22 _bisect.cpython-33m.so
-rwxr-xr-x 1 root root  70333 2012-12-01 18:22 cmath.cpython-33m.so
-rwxr-xr-x 1 root root 152927 2012-12-01 18:22 _codecs_cn.cpython-33m.so
-rwxr-xr-x 1 root root 160944 2012-12-01 18:22 _codecs_hk.cpython-33m.so
-rwxr-xr-x 1 root root  52443 2012-12-01 18:22 _codecs_iso2022.cpython-33m.so
-rwxr-xr-x 1 root root 253712 2012-12-01 18:22 _codecs_jp.cpython-33m.so
-rwxr-xr-x 1 root root 143311 2012-12-01 18:22 _codecs_kr.cpython-33m.so
-rwxr-xr-x 1 root root 111884 2012-12-01 18:22 _codecs_tw.cpython-33m.so
-rwxr-xr-x 1 root root  14570 2012-12-01 18:22 _crypt.cpython-33m.so
-rwxr-xr-x 1 root root  67950 2012-12-01 18:22 _csv.cpython-33m.so
-rwxr-xr-x 1 root root 448007 2012-12-01 18:23 _ctypes.cpython-33m.so
-rwxr-xr-x 1 root root  47225 2012-12-01 18:22 _ctypes_test.cpython-33m.so
-rwxr-xr-x 1 root root 178708 2012-12-01 18:22 _datetime.cpython-33m.so
-rwxr-xr-x 1 root root 821247 2012-12-01 18:22 _decimal.cpython-33m.so
-rwxr-xr-x 1 root root 126793 2012-12-01 18:22 _elementtree.cpython-33m.so
-rwxr-xr-x 1 root root  35043 2012-12-01 18:22 fcntl.cpython-33m.so
-rwxr-xr-x 1 root root  22166 2012-12-01 18:22 grp.cpython-33m.so
-rwxr-xr-x 1 root root  36440 2012-12-01 18:22 _heapq.cpython-33m.so
-rwxr-xr-x 1 root root  92873 2012-12-01 18:22 _json.cpython-33m.so
-rwxr-xr-x 1 root root  47840 2012-12-01 18:22 _lsprof.cpython-33m.so
-rwxr-xr-x 1 root root  86758 2012-12-01 18:22 math.cpython-33m.so
-rwxr-xr-x 1 root root  55133 2012-12-01 18:22 mmap.cpython-33m.so
-rwxr-xr-x 1 root root  73487 2012-12-01 18:22 _multibytecodec.cpython-33m.so
-rwxr-xr-x 1 root root  36548 2012-12-01 18:22 _multiprocessing.cpython-33m.so
-rwxr-xr-x 1 root root  35569 2012-12-01 18:22 nis.cpython-33m.so
-rwxr-xr-x 1 root root  63724 2012-12-01 18:22 ossaudiodev.cpython-33m.so
-rwxr-xr-x 1 root root 156605 2012-12-01 18:22 parser.cpython-33m.so
-rwxr-xr-x 1 root root 233357 2012-12-01 18:22 _pickle.cpython-33m.so
-rwxr-xr-x 1 root root  39650 2012-12-01 18:22 _posixsubprocess.cpython-33m.so
-rwxr-xr-x 1 root root 552668 2012-12-01 18:22 pyexpat.cpython-33m.so
-rwxr-xr-x 1 root root  28805 2012-12-01 18:22 _random.cpython-33m.so
-rwxr-xr-x 1 root root  23111 2012-12-01 18:22 resource.cpython-33m.so
-rwxr-xr-x 1 root root  51393 2012-12-01 18:22 select.cpython-33m.so
-rwxr-xr-x 1 root root  40770 2012-12-01 18:22 _sha256.cpython-33m.so
-rwxr-xr-x 1 root root  71306 2012-12-01 18:22 _sha512.cpython-33m.so
-rwxr-xr-x 1 root root 174055 2012-12-01 18:22 _socket.cpython-33m.so
-rwxr-xr-x 1 root root  21492 2012-12-01 18:22 spwd.cpython-33m.so
-rwxr-xr-x 1 root root  86387 2012-12-01 18:22 _struct.cpython-33m.so
-rwxr-xr-x 1 root root  23632 2012-12-01 18:22 syslog.cpython-33m.so
-rwxr-xr-x 1 root root  32439 2012-12-01 18:22 termios.cpython-33m.so
-rwxr-xr-x 1 root root  97067 2012-12-01 18:22 _testbuffer.cpython-33m.so
-rwxr-xr-x 1 root root 133042 2012-12-01 18:22 _testcapi.cpython-33m.so
-rwxr-xr-x 1 root root  59832 2012-12-01 18:22 time.cpython-33m.so
-rwxr-xr-x 1 root root 800048 2012-12-01 18:22 unicodedata.cpython-33m.so
Host:#

Nice, the filenames of all shared libraries being generated are in proper format
right now. Let's verify in MIPS environment.

tango3[Python-3.3.0]# cd ./cawanpython/usr/local/bin
tango3[bin]# ./python3
Python 3.3.0 (default, Dec  1 2012, 18:22:07)
[GCC 4.3.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> pack
Traceback (most recent call last):
  File "", line 1, in
NameError: name 'pack' is not defined
>>> from struct import *
>>> pack

>>> from socket import *
>>>

As expected, the new python 3.3.0 can import both of the struct and socket modules
successfully now. Let's check the network features under python environment.

Host:# ifconfig eth9 | grep "inet addr"
          inet addr:192.168.1.197  Bcast:192.168.1.255  Mask:255.255.255.0
Host:# nc -l -vvv -p 8888
listening on [any] 8888 ...

Well, the host machine is listening to port 8888 now. Let's connect to it from the
python environment.

tango3[bin]# ./python3
Python 3.3.0 (default, Dec  1 2012, 18:22:07)
[GCC 4.3.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> cawansock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> cawansock.connect(('192.168.1.197', 8888))
>>> cawansock.send(b'hello cawan')
11
>>>

Let's check the host machine now.

connect to [192.168.1.197] from (UNKNOWN) [192.168.1.198] 57560
hello cawan

Good, the string has been sent from MIPS to host. Let's send something from host to
MIPS now. First, instruct MIPS to start receive.

>>> cawandata=cawansock.recv(32)

Then, from host machine, type something and press enter.

cawan reply

Let's check for anything received in MIPS.

>>> print(cawandata)
b'cawan reply\n'

Nice, the string received. Now, disconnect the socket.

>>> cawansock.close()
>>>

In host machine, it backs to the prompt.

cawan reply
 sent 24, rcvd 11
Host:#

So, the python 3.3.0 is running well in our MIPS platform now.

///////////////////python-3.3.0-cross-compile-cawan.patch///////////////////
begin-base64 644 .
H4sIAMLsuVAAA+1Y62+bSBDv18tfMZeoAmrDgV9xfJdT0rwaXfNQnFNVnSq0
xmubBu8idknq//5mFzsG7DzatD2dxHwIBJbfvH4zO+t4Jiec2U2n6bh2kHAh
7IBP4zCidkDuCHNiIoPJqxeJi9JptfQVpXT1vNZ265XXcNudZrvVauNzr9Nq
tl6B+zK1z5NUSJIAvEo4l4+te+r9/1Rs2wbym6AyjZ149kvD9Rq2u2M3dsDr
9Fy313YddyFQc7uuu1Gr1WBQ+kR95YHX7XmNXnN75ZO9PbCbzUa9AzV16cLe
3gbkRdBo5BDGeMoCahof9q/OT89PeiBuwjgO2RjCacwTCcGEBjcw4gkczMZ3
IbMHRNAhbL4Wm0YRUclroF+kw8iUWsWXCRqfsI3a4v9wBFw4lN2GCWfOmErT
OLi66Pf9g4uzy9P3R/71/tXJ0bVhwe4uGDMqjN7y4xzg/f+o1x9hESndsKvA
sY4mzmceMnON54M0jIZ+FA7qa16iOX4ezyw+TaNIP7131bKyeLe6GOhas+2W
4i2TWa+oBoPrRJwM/eEMEcLgHqtecMTK+xfQWMKpzspRkmBCiIC7CSLbKw6M
CAIMHRLHlA2Xdq5ZuWTAmzdv4J4FiVKvWKDyDCLEFXNCaGpIo5iM52fz113Y
xGxulrL5cnvKaDkxIItHD14LAymaCzbGz6pDRG9ptNssUZYIQbEAGJeZaSGL
IxLQ4iJVDRmURB054ok4CiUqMov51Dzp7Oi63PbqnrdSmAtBoIRO+S0117Ah
x4h9KZNwkEqqSdFbBSvFNWVkEFGQHDJ4UPACzHDMeEKHlmEVU0MjQZ/O1n2m
dIKyeC8zVAc0EO4oRpqgxlCCsYpYStqABiQVVH1FEgr5nRIRMY9PISxpXyuX
UW8lkr6cxRkfMJMDzKOYIaHxechG3LQe7Z0Fpi5ZifopEyFnWUwMnfuu59a3
odb1tvGSTz2y2+dYrkJE/i1N0AL3i+vuuNvY05erJhg+n7DZYqkyFD/AehAQ
Cs3Vc84wYmyoX2CDK7woIaVCcSEHZq4oQKRi21gjebv/3C27Yj2VqCe+184c
E6QhpmHpAPacsrHlJjta52Uvy0Njp+61MBHNTr3ZKGRiCz5gAKM7MhMwn8xA
TihSEdsEQ4rgnwvE6vffq9CSWyS7LikzFCKlsOW1OjtNy8kjnkpDoDHJFEtN
aIcQEfjgMw1wi+VDqpCQNTMwW67dduGvtxCj61M+TCNaz0NxFs1A7R1YYNoW
EsiURPgQq2WIau+zhQQUiz0AjhZsNA1fTEij3THq8I+R3WZqnMD4VH8y2UOq
AMXuP8aEiAkSzJkYnyywnqe37TUWevH2J+mdDttaKV5/nqfewk/vu+isLato
69sSu/z+q3U/S/PDqf2hmh9I7o/2dn16X6C1MBvP56ZD/8Pp9Tv/8uPh0du/
TwBnPtXI1za1Qo/dgmtsL8rMeQdR21JEp5RJoTtP/92+B8oCbHRjnoRyMnVK
2/4jvi9cX3peiFvOAkzKAxZc9ffhkEgCfRqkaADuu6cscODssF1GOsOeScbU
PgzHVODAs7C4jtEUAQ4/atRgcHV8AF6z4TmgdJdBGA0UTDKbzztomRPoTqzu
JnrGwG00SlVfndCEPjcemn+Pb3GCpwlqx/21SNM6GNqMAnMeYc0CYA1n1J6p
Sn7tDNFVM8Sz6GVeJ7iD4U0e5o+16NYK5xab4pBTwQyJ1/m+mTKJs7Hr7Dhd
jAVOdEYUwSBRQxJGBvgd01x8PgEfanIrNPyWfpVFFvJ77pl+qYhLJMQJvw1x
x8b9WYQCIdHTMJCIi+yyo/CG4oA4JehzIJCLH3kKd2EU5fEQYoDlO4M7XKZG
cZIkhI2pPmejChyTkehqkOMjFcOpWjOguVmDMz3GeNvdjjpzel3Pq+88eJZY
K6NR6AecjcKxqghrzclhlPAp+iZkivkTzjBMfHW3+GlgeqNOOqufZc9Nha+P
2fidtbrxZap9kow1r/GaeY9XLGYcv7MF+tg9X3tLEnPz4OL8+PTExwNlf9PK
jlnmGviSIOlN07DtCRdy11AaUJGlKY9PtZnLx9an1XLc+r72ruJ/b4OL9n6l
gZuwWRi2l1Ze4rEY6DSWMzg4fr9/0oflQU2V9ecUmZtVnO7zCRVppM5D68Dm
CEjuS/2rqCF+B3usnLQvGnokntOeY8ENnY3/+re7SiqppJJKKqmkkkoqqaSS
SiqppJJKKnlK/gUxgM3OACgAAA==
====
///////////////////python-3.3.0-cross-compile-cawan.patch///////////////////

To retrieve the patch file, simply copy the content from "begin-base64 644 ."
to "====" and paste it in a file then name it as cawan. After that, under the
same directory, run the following commands.

Host:# uudecode cawan -o cawan.tgz
Host:# tar xzvf cawan.tgz

////////////////////////////////////////////////////////////////////////////////////////////////.


pdf version:

http://www.scribd.com/doc/115120171/How-to-Cross-Compile-Python-and-Run-in-Embedded-System

Tuesday, November 27, 2012

The Essential Of Hacking Methodology From The Perspective Of Embedded Hacker

The Essential Of Hacking Methodology From The Perspective Of Embedded Hacker

by cawan (cawan[at]ieee.org or chuiyewleong[at]hotmail.com)

on 27/11/2012

Hacking is about an action to a system. The system can be a single box or machine,
or can be a group of boxes or machines to be interconnected via a center communication
bus, which is usually known as network. For the case of single box or machine,
in fact, it can be assumed as a group of peripherals which are managed by a center
processing unit, over the system communication bus, such as PCI, PCIe, I2C, SPI, USB,
and etc. The group of peripherals can be in chip form or card form. In chip form, it
can be sound interface, network interface, video interface, or RAM interface, which is
defined as the lowest level of interface within the system. So, their communication
bus should be I2C, SPI, USB, or some variation of such standard communication bus. On
the other hand, in card form, it can be enhanced version of sound interface, video
interface, network interface, or some forms of extended features such as I/O card,
optical interface, or RF interface. Besides, there are also some custom communication
interface such as Cobranet, Ethersound, LonWorks, ARCNET, and etc. Well, they are
connected via PCI or PCIe. Regarding to the storage access such as hard disk, optical
driver, or SSD, they are connected via PATA (IDE), SATA, or SCSI bus. Since there are
so many types of buses in the same box or machine, the processor needs a chipset to
manage those buses in performing multiple access or multiplexing in order to allow
those peripherals can be run accordingly. In fact, the chipset is only necessary for
those systems which their processors are in von neumann architecture, where its
address bus and data bus are shared to the same physical bus. On the other hand, for
harvard architecture based processor, its address bus and data bus are separated, and
hence the chipset can be skipped.

For the case of a group of boxes or machines being interconnected via the network,
each of them must comply to a standard protocol among each others. In link layer, they
might need to comply ethernet, wifi, pppoe, or mpls, which is depending to the
physical medium of the associated communication bus. In higher layer, let's say the
network layer, almost all of them comply to TCP/IP. So, the TCP/IP is undoubtedly the
protocol glue among all the boxes or machines. From here onwards, the rest of the
issues are about application layer such as HTTP, HTTPS, FTP, TFTP, Telnet, SNMP, SMTP,
POP3, IMAP, and etc... which are the most common names in networking. On the other
hand, for the case of industrial networking, there are a huge variety of them, which
are dependent to the application domains. For example, in building automation industry,
there are bacnet, modbus, lontalk, profibus, and some others which are manufacturer
proprietary. Besides, in scada industry, there are modbus, iec60870, iec61850, and a
lot of others proprietary implemented items.

In order to run the hardware, we need software. So, we need something to manage the
hardware in overall, and this is kernel. The kernel will control various types of
hardware with appropriate kernel module or device driver. The kernel executive will
ensure the interoperability among the hardware resources. From application point of
view, the hardware resources are operated in transparent way via appropriate system
call. Of course, there are some protections imposed such as the kernel space is
prohibited to be accessed from user mode. Besides, each process space is isolated
among each other to avoid data collision or overwrite intentionally or unintentionally.

From hacker perspective, there are a lot of exploitation points can be considered to
launch the attacks. In general hacking methodology, majority of the attacks are about
exploiting the buffer to overwrite some system structures or registers, and causes
the system run unexpectedly. The buffer exploitation can be about the stack or the
heap. Besides, it is possible to exploit the kernel stack or heap from user mode via
some vulnerable kernel modules or device drivers. In order to control the system
once it is exploited, we need a piece of shellcode. The shellcode is something highly
customized which is dependent to the processor platform, OS platform, operation mode
(kernel/user), intentional action, avoidance of illegal characters, and etc. So, all
the attacks need to start from a target application. If the application runs locally,
then the malform local user input is the root cause to incur exploitation. In addition,
if the application is network savvy, then the attack can be launched from anywhere
within the network. Before launching any attack on the system at another part of the
same network, it is important to understand the protocol being used by the system.
So, a sniffer is necessary in this case to study the protocol based on the network
packets being captured. If the protocol is proprietary, then it needs some times to
analyze the packets sequence in understanding the protocol in details. However, if
the protocol is open source, then the analysis process will getting much easier by
referring the specification from time to time. On the other hand, it is possible to
perform fuzzing process to the system. With some level of understanding to the
protocol, it is possible to manipulate the data in different fields in the network
packet automatically and observe the reaction of the system. Yes, the fuzzing process
is some kind of trial and error or brute-force approach, but it is really effective.
Besides, it is possible to perform code analysis to the application to find any
vulnerability to be exploited, but it is really time-consuming.

From embedded hacker perspective, the hacking methodology suppose to be much simpler.
Due to the reduced hardware resources in embedded system, it is impossible to implement
full security protection which is really resouce-intensive. In fact, a large portion
of embedded linux system even didn't implement NX and ASLR all together. So, in this
case, ROP can be skipped when we are designing the exploit and shellcode for such
systems. Besides, for some very special cases, some network applications are even
don't have sanity check to the network packets. Thus, it is simpler to be exploited.
However, due to the nature of embedded system which is highly customized, it is a
need to master the skill of shellcode design for RISC processors such as ARM, MIPS,
and PPC. So, it is important to make those different types of instruction sets at
your own disposal. Besides, it is important to note that the peripherals being used
in embedded system are not something in standard or in generic. Instead, they are
something special with customized device driver. From the angle of embedded enginners
who designing the device driver, they just need to ensure the device driver can run
in the most stable way but not in the most secure way. Hence, it is most probably the
device driver is vulnerable to be exploited. Besides, the security concern to the
embedded system is really less as compared to the proper computer system. The reason
is simple, it is really not many people can imagine a headless embedded system is
hackable and the value behind to abuse the hacked embedded system. It is really hard
to convince somebody about an embedded system product can be changed into a proprietary
sniffer to perform dedicated MITM attack to the network. On the other hand, regarding
to the physical attack, it is really not hard to duplicate the data image from flash
chip. Besides, from higher level perspective, the ramdisk file can be manipulated
which is a remarkable security breach. In addition, the simple implementation of
bootloader does not has good security implication to block unauthorized access to
the system. In other words, once the configuration interface of the bootloader is
getting accessed, then it is just a few of commands to duplicate certain partitions in
flash to a network drive as data image. It is crucial to remember the term of readonly
in certain file system for embedded system is really nothing to do with the security.
It just means we can't simply add or remove a file from the file system, but we can
definitely overwrite the whole partition by using the device file in /dev. Nothing
special here.

Now, what is the essential of hacking methodology from the perspective of embedded
hacker ? Well, we already know hacking is just an action to a box or to a system.
Then what is the core of embedded system hacking ? In order to have a substantial
idea to start any meaningful attack to an embedded system, it is necessary to
understand the system and protocol internals. So, reverse engineering should be the
core of embedded system hacking methodology. When talking about the reversing of
binary, it comprises static and dynamic approaches. In static approach, it is about
to perform disassembly of the binary with any type of disassemblers. However, IDA Pro
should be the one with highest ranking to master with. Yes, it is really good in
generating comprehensive output in assembly language level with proper formatting.
But, in most of the times, we are only interested to some portions of the codes. So,
it is a little bit hard to find those little portions of codes from the big pool of
assembly instructions in IDA Pro enviroment. Of course, we can based on some special
pattern of string to do the job, but the existence of such string is not always true.
So, we need to cooperate with the dynamic approach. By using a good debugger such as
gdb, it is possible to set a breakpoint at an interesting address of symbol, and let
the program counter hit on the breakpoint. Then, it is easy to backtrace or to dump
some meaningful information from memory as guideline in locating the code portion
that we are interested with from the big pool of assembly instructions under the
IDA Pro environment. Yes, both of the static and dynamic approaches should be worked
together in proper way, and this is the fundamental skill of an embedded hacker.
On the other hand, the architecture of RISC processors should be in good
understanding. So, the concepts of calling conventions, registers, addressing modes,
operation modes, and memory management for each of the RISC processors should be in
our disposal. In addition, it is important to understand the linux kernel in detail.
Hence, the internals of runtime, loader, shared library, kernel module, and system
call should be with good understanding. Besides, it is also important to understand
the communication bus of the peripherals such as I2C, SPI, PCI, USB, and etc. By
understanding them, it is really helpful to interpret the communication data being
captured in signal level. Well, it is crucial in doing injection to find vulnerability
in signal level or in bus level. The possibility is only limited by our imagination.
As conclusion, reversing techniques in terms of hardware and software should be the
core of hacking methodology for a serious embedded hacker.

pdf version:

http://www.scribd.com/doc/114591513/The-Essential-of-Hacking-Methodology-From-the-Perspective-of-Embedded-Hacker