...a clever blog name here...

I just added French langauge support to my forked version of Anagramarama. This entails creating a new background image with localized strings on it but the French equivalents take up a bit more room compared to the English words in use. So I added a config file to the internationalization folder to permit re-defining the position and size of the active click region for each action word (the new, solve, quit and shuffle words). This should also make it possible to create completely re-designed game screens if this is extended to re-position the main word boxes and the clock region as well.

The French langauge word file is a trimmed down version of the scrabble dictionary again. This time the French dictionary is ODS5 and again I removed all the words too big or too small to be used in the game. It seems to be working fine.

Extending the NSIS installer was frustrating. Adding some accented characters I saved the NSIS file as utf-8 but this is not supported in stock NSIS. While using European languages we can manage using cp1252 but if I was to extend this to Russian it would be completely messed up. Seems stupid. Anyway the Anagramarama03.exe installer now contains Brazilian Portugese and French along with the British English game. If you want to test a language other than your own, set the LANG environment variable (eg: set LANG=fr) before running the program.

The additional source commits are all in the git repository.

Back in 2006 I found a nice SDL based game called Anagramarama which my daughter and I had some fun playing. I was playing this on my laptop and noticed that the CPU was maxed out the entire time. This is highly undesirable on a battery powered device so I looked into this and produced a patch. Unfortunately this has remained ignored since 2006 so recently I had cause to revisit this code (my daughter wanted the game again) so I've published this Git fork. The primary focus is a cleanup of the C code, fix the CPU usage and fix the memory leaks.

Hopefully in the future these patches can be merged into the main source which is currently available at googlecode. However, the primary project seems pretty moribund at this time.

Since my daughter tends to use Windows, I've added some build support for MSVC6 and an NSIS installer script. The current installer can be downloaded from Anagramarama03.exe although it might not be produced from the most recent commit on this repository (check the date)

I have been meaning to compile Android and get it running on my HTC Magic device ever since I got the phone. I finally got around to actually doing this now so here are some notes I made on the way. My primary concern was obviously that I didn't turn my phone into an expensive brick - but avoiding attempting to flash the SPL should prevent that. The second concern was that I should be able to revert back to exactly what firmware I had before I started to mess about. I can confirm that Nandroid lets you do exactly that.

When I originally attempted to build Android I just followed the instructions provided by Google but this is just a basic build the generic image and you need a kernel customised for the device hardware. You can get this but it is much easier to just go with CyanogenMod as this is reasonably close to generic AOSP but with various fixes and improvements.

Fetch the source, configure and build

mkdir /opt/src/droid
cd /opt/src/droid
repo init -u git://github.com/cyanogen/android.git -b eclair
repo sync
# go and drink a couple of pots of coffee
source build/envsetup.sh 
make -j4 adb
lunch cyanogen_dream_sapphire-eng

# make sure your phone is connected via USB
cd vendor/htc/dream_sapphire
./extract-files.sh
cd ../../cyanogen
./extract-htc-keyboard.sh ;# (failed for me running cyanogenmod)
./extract-google-files.sh ;# (missed a few that dont exist on sapphire)

cd /opt/src/droid
make -j4 droid otapackage ;# takes 1.5 to 2 hours for me

At this point you can test this build in the emulator by just typing emulator. The emulator loads the firmware files from your build output directory (defined in ANDROID_PRODUCT_OUT)

Build the update.zip file

To make a CyanogenMod update.zip package:

• ensure you have already installed pngcrush
• ensure you have already installed squashfs-tools

cd /opt/src/droid
source build/envsetup.sh
lunch cyanogen_dream_sapphire-eng
TARGET_NO_RADIOIMAGE=true make -j4 droid otapackage
vendor/cyanogen/tools/squisher

This creates an update-cm-VERSION-LOGNAME.zip file that can be flashed. Copy the update file to the sdcard, boot to the AmonRA recovery shell and flash the newly built zip. From CM5.0.7 you don't need to wipe. The first boot is a bit slow as it regenerates the cached dalvik code from new set of APK files.

I had some trouble making an OTA update package the first few times. I eventually worked out that I needed to have the squashfs-tools installed or else you must define NO_SQUASHFS before running the squisher script. Without either of these the system image was missing the xbin/ directory and would hang up on booting.

This is log of what happened as I installed cyanogenmod onto my UK HTC Magic Android phone. It basically all went fine and now I can run android-2.1 without waiting eons for an official update.

My thanks go to all the developers responsible for making this work. They have done a great job.

Fortunately for me the UK HTC Magic supports fastboot which simplifies matters greatly by allowing me to boot any image downloaded from my PC over the USB cable. So we can get the AmonRA image running to get things going.

Update session

Boot to the fastboot screen by holding Back and pressing Power.
Put in clean 2GB sdcard with:

• DRC83_base_defanged.zip
• update-cm-5.0.7-DS-signed.zip
• RA170G.img (recovery-RA-sapphire-v1.7.0G.img)

Booted phone to fastboot screen.
fastboot boot recovery-RA-sapphire-v1.7.0G.img
Selected Nandroid+ext and performed backup.
Returned 'Error : Run 'nandroid-mobile.sh' via adb!'
So adb shell (yields a root shell)
/sbin/nandroid-mobile.sh --backup
dumped boot.img, recovery.img, misc.img, system.img, data.img, cache.img to nandroid folder, generated md5sums and unmounted system, data, sdcard and says it was successful.

Rebooted and fastbooted again after copying the nadroid files to PC.
adb push recovery-RA-sapphire-v1.7.0G.img /sdcard/RA170G.img
(note, this is NOT the actual Sdcard location as its not mounted yet)
adb shell flash_image recovery /sdcard/RA170G.img
Rebooted the new recovery image by holding Home while booting. Worked!.

Now factory wiped.
flash zip DRC83_base_defanged.zip
flashed zip update-cm-5.0.7-DS-signed.zip
All completed ok.
Rebooted using menu item. Immediately got buzz + vodaphone screen.
After one minute got cyanogen's animated boot graphic.
A couple more minutes and I have the cyanogen screen.
The new firmware is running ok but is missing all the google apps and the market.

It seems the directions I had were not quite right - we now have to either extract our own google application images and make our own image or our own overlay update.zip. Someone has provided one though so in the meantime we can overlay this.

Rebooted to recovery screen and installed gapps-ds-ERE36B-signed.zip.

Now we have a proper googley phone.

I previously saved a number of apps to sdcard using Astro and I can now reinstall them using 'adb install'. Everything is working!

Restoring application data

The factory wipe removes all saved data except that on the sdcard. A number of application can let you save their own data to the sdcard. Astro lets you save applications to the card. Astrid can export its tasks. You can download "SMS Backup & Restore" to preserve your SMS messages if you care about those.

I just fired off yet another request to get access to the Coverity Scan information for the Tk project. I fully expect another round of resounding silence for this request as that will match every other attempt I've made. It seems that if you missed making such a request just after Coverity's original public announcement of the Open Source scan results you just get ignored forever. Thats not that suprising really as managing such access must have some cost and they are providing this for free. On the other hand it look pretty shoddy from where I sit. They made the offer to permit maintainers to access these results as a PR exercise and then ignore anyone that tries to follow up on this. The Tcl project got access and fixed a few issues and moved to 'Rung 2' along with a small number of other fairly high profile projects. But I also know that the Tcl maintainer who had access to this no longer does so and his requests have since been ignored too. I cannot say this encourages me to recommend this service for use in my commercial development. An all-round PR fail as far as I am concerned.

Following a lot of discussion on an msysGit issue and having researched this quite a bit myself I decided the state of proxying git on Windows could use some summarizing. In brief, it can be done for some circumstances and not for others. We will need some git patches to let everyone be able to operate over proxies.

Proxying the git protocol over an http proxy

The git protocol is a simple transport used over stream connections that is used when the repository URL begins with git://. This transport can be made to use a proxy by providing a command to handle the proxy connection. This is done by setting GIT_PROXY_CONNECTION or the core.gitproxy variable to point to some program that will make a connection to the git server on the target site. The connect program can be used for this purpose and creating a Windows command script like that below will use your http_proxy environment variable settings to open a connection through your http proxy and operate the git protocol over this connection.

@connect -h %*

There is a problem with this however. The way to make a direct connection through an http proxy is to issue the CONNECT command with the hostname and port of the target site. In this case we will be issuing CONNECT git.example.com:9418. Generally http proxies are configured to deny making connections to anything other than port 443 (the secure http port) and perhaps a limited set of additional ports (imaps is also common). So there is a good chance that your proxy will refuse this request and quite often it seems it will not bother to tell you either. If this is the case then you are restricted to using some other transport method unless your proxy admins will permit the git port.

Incidentally, it seems that curl cannot be used to make such a proxy tunnel. Curl provides good support for using proxies with and without authentication and even has a --proxytunnel option that will make it use the CONNECT method with the proxy to open a tunnel to the remote site. However it then always issues a GET request or some other request as specified by the --request option. It cannot be instructed just to silently pass data over this link once it is connected. For this reason we use the connect program as described above.

Proxying the git protocol over a SOCKS proxy

See the previous article for this but the essentials are that you set the SOCKS_SERVER environment variable to point to your SOCKS5 server and then use connect -s %* instead of using the -h option as above.

Proxying git using the http protocol

Git supports using the http protocol when the repository url uses the http:// scheme identifier. In this case it uses standard HTTP for the transport. HTTP can obviously be proxied so all that is necessary to use a proxy is to set the git variables or http_proxy environment variable. If authentication is required then this can be included in the proxy url (eg: http://userid:password@proxy.example.com:3128/. Alternatively curl supports the use of a .netrc file where authentication details can be recorded on a per hostname basis. On Windows this is called _netrc and needs to live in your HOME directory (a HOME environment variable may need to be created). The layout of the file is as below.

machine git.example.com login username password secret

Unfortunately it appears that curl will always use Basic authentication with the proxy. If your proxy needs something else, perhaps NTLM for a Windows network, then you have a problem. Curl is used to handle all the http transport details and this does support the NTLM authentication method but I know of no method to pass the necessary options to curl. Git makes use of curl via its library binding so it is not enought just to replace the curl executable with a wrapper script. It might be sufficient to compile the curl library with SSPI support. On windows this can be used to pass the logged in user authentication details to a program and usually works for NT domains with proxies that use NTLM. Otherwise it seems the http transport in git needs to have some variables added to control this.

One of the problems with running services on a unix box is that port numbers below 1024 are restricted. A process cannot bind to a low numbered port unless it is owned by the super-user. In the context of tclhttpd this means we have to launch the tcl process using the root account and then drop privileges as soon as we have bound our listening sockets by switching to another user identity. This can be done quite handily using the TclX extension and tclhttpd has support for this. But I have an aversion to running script interpreters as root.

One solution that is often used with scripted services like tclhttpd is to operate on a non-restricted port and have firewall rules rewrite the incoming traffic to send it from port 80 to your service port (perhaps 8080). With firewalls supporting NAT redirection this is a simple solution but once we consider ipv6 we have a problem. IPv6 has no need for NAT and the Linux ip6tables firewall does not provide a means to redirect packets to a different port.

An ideal solution is offered by the new Linux capabilities that is now available since kernel 2.6.24 and can be found in the latest Karmic Koala Ubuntu Server versions. Capabilities permits much more finely grained permissions assignments than the traditional mechanism of 'setuid root'. In particular we can assign our process exactly the permission required: permission to bind restricted ports - and no other permissions. So now we can set this capability and run the executable image using the correct user id from start to finish.

As a specific example, I use a tclkit-cli executable to run my tclhttpd service. I copied it into the tclhttpd tree and named it tclhttpd as this helps the init.d daemon tools to manage the process. I then used the setcap utility from the libcap2-bin package to set the correct capability. In this case we want to be able to bind the restricted ports and that is the CAP_NET_BIND_SERVICE capability.

cp /opt/bin/tclkit-cli tclhttpd
setcap 'cap_net_bind_service=+ep' tclhttpd

The capability flags above raise the effective and permitted flags for the cap_net_bind_service capability. capabilities(7) explains what all this means but basically as we did not raise the inherited flag any child processes will not have this capability.

Now I have my server listening on ipv4 and ipv6 without messing with the firewall and without requiring to be started by root. With the installation is a suitable file to automate the restart of the service it is all up and running.

It is interesting to note that this method can be used to remove the setuid bit from other programs. For instance, the ping utility requires the CAP_NET_RAW capability and does not need root access if this is provided. Once capabilities spread through the linux distributions this should help make systems just that little bit more secure.

git-gui is a graphical interface to git written in Tk that is particularly useful as a commit tool. However, it doesn't look all that native when used on Windows XP or Vista. So I have spent a few hours applying ttk magic to the application to bring it up to date for Tk 8.5/8.6. Mostly this just means the usual selection of ttk widgets over the standard ones or replacing tk_optionMenu with ttk::combobox. I left in some of the coloured backgrounds that are being used as headers but did the appropriate ttk::style stuff to make it work with the style engine. I also added support for the 8.6 tk fontchooser which shows the stock font selection dialog on MacOS and Windows. So now it looks like this on Windows 7

I actually did most of this on linux testing out the tile-qt theme so on Ubuntu it looks pretty native too if that theme is available. Under X11 there is no 'native' theme to select but you can choose one by defining TkTheme in the X resource database. Something like echo '*TkTheme: tileqt' | xrdb -merge - will select the Qt theming engine. Another option is clam which is built in to Tk 8.5.

There are still some issues to deal with. The menus and tooltips ping a bit with Ubuntu's compiz window manager. I've done some work on this recently to get the correct extended window manager hints applied. We have patched 8.5.8+ already for the menu hint but Tk needs a generic way to let scripts specify the window type using EMWH attributes. At the moment I have a wm attributes $win -type command option to do this and this works pretty well. At least the tooltips animate the right way with this and the combobox dropdown acts more normally.

I recently had cause to try and merge two branches from separate git repositories that had related content but disconnected ancestry. The situation was caused by creating a git repository from a CVS conversion. I then wanted to join work from a separate git repository that had been started from the same CVS tree. However, the developer who made this git repository had begun his git tree having already made changes to the content.

So I ran git cvsimport to convert all the CVS history into one git repository. I can now pull in the new work by adding another git repository as a new remote and fetching the master branch from this remote. However, what I now have is a disconnected chain of commits. There are now two separate chains in my repository. What I need to do is merge the two together. I can do this if I can take the first commit of the new work and inject it into the cvs converted repository as the next commit. I don't want to apply a patch, I need the state of the repository for this commit to be exactly that of the first commit of the new work. For this there is git read-tree. This command takes a git tree and reads it into the index. This is where git stages your work when you do git add and git rm and where it assembles the new tree to be committed by git commit. So by setting the index to a given tree, we are setting up a for a new commit that is exactly the tree provided. Just what I need. Having staged this tree, I can go ahead and commit it and then cherry-pick or merge the rest of the tree. In my case I cherry-picked so that I could fix the author name and tidy up the commit messages.

Below are the commands I used to fix up the tile-qt repository. First adding the remote and fetching its commits. Checking the correct tree and loading it into the index and then updating the working files from the index.

git remote add jdc file:///opt/src/tile-qt-jdc
git fetch jdc
git ls-tree 1519481 
git read-tree -m 1519481
git checkout-index -f -u -a

The final results can then be pushed to some public location like github or in this case the tktable sourceforge project.