Better support for sipdroid on Android

For a while I've wanted to use my fancy Asterisk setup with my HTC Evo 4G, but i've always been getting horrible performance from sipdroid as well as any other clients I was trying on the phone.

It seems that i've identified the problem as actually being a codec problem.

I looked a little closer at the verbose logs in asterisk and saw messages like this:

[Nov 19 02:57:44] WARNING[9373] chan_gtalk.c: Asked to transmit frame type alaw, while native formats is 0x4 (ulaw) (read/write = ulaw/ulaw)

It seems that whenever I was making calls, the formats kept on flipping from ulaw to alaw and vice versa.  When it did work, only one end could hear the other.  To make matters worse, I couldn't get G722 to work whenever I tried to force it.

Comparing the OpenWRT Asterisk 1.8.x install to a standard install on Ubuntu, it looks like the codec files for G722 weren't even getting installed.  I modified the packaging and republished additional packages at my repository for Asterisk on arctangent.net (http://arctangent.net/~superm1/openwrt)

Now there is two more packages available, one for alaw and the other for g722.  You should be able to update your opkg list and install them:

# opkg install asterisk18-codec-alaw asterisk18-codec-g722

After they're installed, modify sip.conf to make sure you explicitly mention that you now support G722:

[101]

username=101

secret=101

type=friend

callerid="Mario"

host=dynamic

context=outgoing

outgoinglimit=1

incominglimit=1

canreinvite=no

disallow=all

allow=g722

allow=alaw

allow=ulaw

allow=gsm

Now restart Asterisk and Sipdroid should be able to use G722 for some much better audio.

Caller ID with Google Voice and Asterisk 1.8

Previously it was possible to do a lookup of your contact's name against the numbers in your GMail/Google Voice contact list and apply that to your CallerID via John Baab's method outlined on his blog.

I've since then discovered that since updating to Asterisk 1.8 with native Google Voice this no longer functions.  This is because the format of the CallerID string that is sent from incoming calls is now different.

The entire string is sent in the "Name" field of the caller ID.
For Voice Calls, this is what it looks like:
+18475551212@voice.google.com/srvres-MTAuMTIuMTM4LjI2OjD4NdM=
For calls coming in from Talk or GMail, this is what it looks like:
user@gmail.com/gmail.424BF12F

Fortunately, this entire field can be processed using built in asterisk functions to extract the data we need, restoring the functionality of John Baab's script.

Here's all the steps that i've taken to add Caller ID support:

1) Install python, python-expat, python-openssl
opkg install python python-expat python-openssl

2) Fetch John Baab's script and place it somewhere useful.
wget http://www.baablogic.net/googlecontacts.py


3) Download gdata-python from http://code.google.com/p/gdata-python-client/
wget http://gdata-python-client.googlecode.com/files/gdata-2.0.12.tar.gz


4) Manually install it.  The setup.py script doesn't work on OpenWRT, so this method can be used:

tar xzzf gdata-2.0.12.tar.gz
cp -r gdata-2.0.12/src/* /usr/lib/python2.6/site-packages/
rm -rf gdata-2.0.12*

5) Modify John's script to include your gmail login information.  Run it once manually to populate your database
./google-contacts.py

6) Modify your [google-in] context to process the number properly and do a database lookup:

[google-in]
exten => superm1@gmail.com, 1, NoOp(Callerid  ${CALLERID(name)})
exten => superm1@gmail.com, n, Set(CALLERID(num)=${SHIFT(CALLERID(name),@)})
exten => superm1@gmail.com, n, Set(CALLERID(name)=${DB(cidname/${CALLERID(num)})})
exten => superm1@gmail.com, n, Dial(SIP/101, 180, D(:1))

7) (optionally) Add this to your crontab so that it will sync new numbers on a regular basis.  John's script will actually redo the database on every run, so it will also get updates then for every removal too.

crontab -e

This will only take effect for calls routed in through google voice or talk because we only modify the google-in context.  If there is no data available, the name field will just remain empty and the number will be shown on Caller ID instead.

Updating the squashfs with an external rootfs

If it ain't broke don't fix it

We've all heard the phrase at least a dozen times in our life. Yet knowingly it's disobeyed over and over because of shiny new features that we don't need.

It was brought to my attention that there have been a ton of ath9k fixes pulled into the 10.03.1-rc3 interim candidate that should improve wifi performance.  I've had some wifi problems, but it's hard to assess if they'd really be fixed by these patches.  I figured I didn't have much to lose and decided to svn up and rebuild my squashfs image to upgrade on the router.

Normally, this is supposed to be a rather straightforward process (http://wiki.openwrt.org/toh/d-link/dir-825#upgrading.openwrt)

1) svn up

2) make

3) scp bin/ar71xx/openwrt-ar71xx-dir-825-b1-squashfs-sysupgrade.bin root@192.168.1.1:/tmp

4) ssh root@192.168.1.1

5) sysupgrade -v /tmp/openwrt-ar71xx-dir-825-b1-squashfs-sysupgrade.bin

6) Profit

Unfortunately I ran into lots of problems along the way, that eventually stemmed to the contents of the external rootfs being out of sync with what was on the jffs2 partition.  Running an opkg upgrade caused the router to not boot anymore with the flash drive plugged in.

Consequently, I've discovered when you have an external rootfs, you need to follow a bit of a different process when flashing new ROMs.

1) Update svn

svn up

2) Refresh config

make menuconfig

3) Examine the contents of /overlay/etc/ on the router.  In your source tree, you will want to make a files/etc directory and copy in all the relevant configs that you need to persist after you flash.  They'll be baked directly into the ROM image itself.  Over time you should eventually not even need much of overlay configs in /overlay/etc as they'll be in your home rolled roms.

4) Record a list of all packages you have installed that you care about.

4) Power off the router and plug the flash drive into a computer.  Format it.

5) build the new ROM

make

6) follow the regular steps to scp the ROM over and load it.  Don't plug in your flash drive yet.

7) Reboot the router and make sure it comes up

8) Now insert the flash drive and reboot

9) Lastly, install any relevant packages you previously recorded a list of.

All of your configuration should already be on the ROM, so you won't need to reconfigure anything.

Asterisk 1.8 and native Google Voice support

Last week the Asterisk development team announced Asterisk 1.8's release with native support for Google Talk / Gmail calling.  By doing so, a lot of the hackery that was previously done with bridging and AGI dialers in my earlier posts can be axed.

Even though Asterisk 1.8 was released, I didn't see any developers jumping on creating OpenWRT packages, so I decided to do so myself.  I've produced packages for both ar71xx devices (like the one I detailed in earlier posts) as well as brcm-2.4 devices that the market is flooded with.

These packages can be found at http://www.arctangent.net/~superm1/openwrt/.
I've also submitted the packaging changes to a ticket for inclusion in a future OpenWRT release at https://dev.openwrt.org/ticket/8132.

So for an ar71xx device, you would add this line to your /etc/opkg.conf:

  src/gz asterisk1.8 http://www.arctangent.net/~superm1/openwrt/ar71xx

If you've got asterisk 1.6 installed, you'll need to remove all of the associated packages for it before proceeding.

opkg remove asterisk16-sounds asterisk16-res-musiconhold asterisk16-res-agi asterisk16-func-db asterisk16-format-g726 asterisk16-form
at-g729 asterisk16-codec-g726 asterisk16-codec-a-mu asterisk16-app-system asterisk16

For Asterisk 1.8, I think these are the only packages you'll need:

opkg install asterisk18 asterisk18-func-db asterisk18-res-musiconhold asterisk18-chan-gtalk asterisk18-res-rtp-asterisk

Configuration Files

Once installed, you need to add your google information to /etc/asterisk/gtalk.conf.  This configuration will dump my incoming gtalk calls directly to a context that will be defined as 'google-in'.


[general]
context=google-in ; Context to dump call into

[guest] ; special account for options on guest account

disallow=all

allow=ulaw

;context=guest

[superm1-gtalk]

username=superm1@gmail.com

disallow=all

allow=ulaw

context=google-in

connection=superm1

Next configure jabber information in /etc/asterisk/jabber.conf.  This setup will cause you to always be signed into jabber, so make sure to set yourself away and give an informative message to know that it's a computer signed in, not a human.


[superm1] 
type=client
serverhost=talk.google.com
username=superm1@gmail.com
secret=(your password)
statusmessage=I am asterisk [*]
status=xaway

Lastly configure your /etc/asterisk/extensions.conf dialplan for the new inbound and outbound features.  This is built upon my previous dialplan.  It will support incoming sip or google talk/voice as well as outgoing SIP, toll free, and GV.

; --
; Inbound Calls
; --


; * This extension is where any external SIP calls should route to
[default]
exten => s,1,Set(CALLERID(name)=${DB(cidname/${CALLERID(num)})})
exten => s,n,Dial(SIP/101,10)
exten => s,n, Hangup
exten => 101, 1, Dial(SIP/101, 10)


; * From Google (superm1)
[google-in]
exten => superm1@gmail.com, 1, Dial(SIP/101, 180, D(:1))


; --
; Outbound Calls
; --


; * Default starting context for internal SIP devices
[mario-default]
include => local-devices
include => tollfree
include => gv-outbound
include => dial-uri


; * These are for any local extensions we should be supporting
[local-devices]
exten => _1, 1, Dial(SIP/101,10)


; * Toll free numbers (don't use GV for these)
[tollfree]
exten => _411, 1, Dial(SIP/18004664411@proxy.ideasip.com,60)
exten => _1800NXXXXXX,1,Dial(SIP/${EXTEN}@proxy.ideasip.com,60) 
exten => _1888NXXXXXX,1,Dial(SIP/${EXTEN}@proxy.ideasip.com,60) 
exten => _1877NXXXXXX,1,Dial(SIP/${EXTEN}@proxy.ideasip.com,60) 
exten => _1866NXXXXXX,1,Dial(SIP/${EXTEN}@proxy.ideasip.com,60)


; * Route the call using the google voice bridge
[gv-outbound]
;append an area code if necessary
exten => _NXXXXXX,1,Set(CALLERID(dnid)=1512${CALLERID(dnid)})
exten => _NXXXXXX,n,Goto(1512${EXTEN},1)
;append a 1 if necessary
exten => _NXXNXXXXXX,1,Set(CALLERID(dnid)=1${CALLERID(dnid)})
exten => _NXXNXXXXXX,n,Goto(1${EXTEN},1)
;do our real dialing
exten => _1NXXNXXXXXX,1,Dial(Gtalk/superm1/${EXTEN}@voice.google.com)
exten => _+1NXXNXXXXXX,1,Dial(Gtalk/superm1/${EXTEN}@voice.google.com)


; * Dialing by SIP URL eg foo@domain.com
[dial-uri] 
exten => _[a-z].,1,Dial(SIP/${EXTEN}@${SIPDOMAIN},120,tr)
exten => _[A-Z].,1,Dial(SIP/${EXTEN}@${SIPDOMAIN},120,tr)
exten => _X.,1,Dial(SIP/${EXTEN}@${SIPDOMAIN},120,tr)

Advantages / Disadvantages

Advantages

• This configuration is far quicker and more like a real phone.  When you dial a number, it will be connected within seconds rather than 30's of seconds.
• The quality seems better than the old way (and in theory should be less hops, and less latency)

Disadvantages

• You have to always be signed into Google Talk/Chat with this solution.  If people IM you and you aren't on a computer, then the IM is lost in your gmail history.
• If you are signed into Gmail Chat and the computer supports video (has the plugin installed) then your SIP phone won't ring.

Thanks

Thanks to Paul Jennings for helping to provide a quite functional configuration to get started here for me, as well as AST/Calling+using+Google for detailing all the options actually available to configure.

Updates
11/23:
This post seems to be the post that people are linking to and reading mostly when setting up Asterisk 1.8.  Just want to make sure I mention, there have been improvements since this post (as detailed in other future blog postings).  To avoid having to relive the learning experience I did with my other improvements, my (scrubbed for passwords) configs are available at http://www.arctangent.net/~superm1/gv_configs/