Thursday, January 29, 2009

Comparison of Boston, San Francisco and Seattle on weather

I'm toying with moving to Boston from SF area and nature is a big factor. Here's some data I'm looking at:

Monday, January 19, 2009

Using jgit with socks5 proxy on Ubuntu

I use jgit to store my git repositories on Amazon S3. On my laptop I also periodically connect to the internet through tetherbot, which uses a socks proxy (see earlier post for more details there). I finally figured out how to get jgit, and I suppose other java applications working with the proxy. Jgit is java based and you need to pass the right flags to java, as described at http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html. The flags are:-DsocksProxyHost=localhost -DsocksProxyPort=1080. I inserted the following code into the jgit program, before the exec .... $java_args part:
if (netstat -pl --numeric-ports 2>&1 | grep ':1080.*adb' > /dev/null); then
echo "Using Java socks5 proxy args"
java_args="$java_args -DsocksProxyHost=localhost -DsocksProxyPort=1080"
fi

I inserted the lines using emacs. For some reason, xemacs and nano garbled the file when I tried editing it with them.

Saturday, January 17, 2009

Transparent proxy in Ubuntu

I have a socks5 proxy on my machine (actually on my Android phone, but port-forwarded through adb), and want non-socks5 aware applications to be able to use it. I tried tsocks, but couldn't get it to work with wget - it wouldn't proxy the DNS resolution, even if I set the compile time flags. I then switched to proxychains, which works great with wget and other commandline utils. The only issue is that proxychains does not seem to work with stand-alone java applications, failing in the connection library like:


Caused by: java.net.SocketException: Network is unreachable
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
at sun.net.www.http.HttpClient.(HttpClient.java:233)
at sun.net.www.http.HttpClient.New(HttpClient.java:306)
at sun.net.www.http.HttpClient.New(HttpClient.java:323)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:837)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:816)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:703)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1026)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)


Update: got this working with shell-wrapped jar files. See next post

options for git on Amazon S3

Here's the options for backing up git on S3 as I see them now:

  • jgit - standalone java app that does git fetch and push to S3. I currently use this one, but it doesn't work with transparent proxy apps such as proxychains. Occasionally I use an old laptop with no wifi or bluetooth, and so want to use a USB-tethered Android G1 for internet access.
  • S3-backed Filesystems

  • gits3 - a git implementation that can push to s3. Pretty new. Is it stable yet?

I'm tempted to try the filesystem options, but I'm concerned about reliability and transparency (I want more visibility into how the S3 ops are going).

Friday, January 2, 2009

Store git repository on Amazon S3

I keep my home dir and some other dirs in a few git repositories. I used to back them up to slicehost, which I've been very happy with. Since S3 has much lower storage charges, I modified my scripts to backup the repositories to S3 as well using jgit from here. Figure I'll play around with it and see if the repositories get out of sync.

Bluetooth tethering from Ubuntu to Android G1

Update: Looks like this does not work with the new Cupcake OS :(

Yeah! I finally got this working. I got my Thinkpad T500 running Ubuntu Intrepid to connect via bluetooth to my Android G1. So the laptop connects to the internet through whatever internet connection the phone has (Edge, 3G, or Wifi). I'm writing this at a starbucks on my laptop, connected through my G1 to a TMobile hotspot (install the hotspot app on your G1 and reboot the phone. You'll see a bullseye in the notification bar when you wifi connect to the hotspot on the phone) Roughly the steps I did to tether were:

  1. Get 'adb' installed from the Android dev tools. I downloaded the whole sdk.
  2. The main starting point is the blog entry at: http://www.gotontheinter.net/content/second-g1-story-proper-bluetooth-tethering-how-short
  3. From there I got the files pand, iptables and bnep.ko and copy to /data/local/bin/, creating the dir if necessary.
  4. Get busybox from http://benno.id.au/blog/2007/11/14/android-busybox and install in /data/busybox (it looks like the directory is hard-coded in that binary somewhere)
  5. I did a bit of shell code to move to a subdirectory any binaries in busybox for which another version existed outside of busybox, except for ps. I figured I trust the android ones more, but Android's ps was missing the all important 'w' option
  6. Generate a file called /data/local/bin/tether.sh on the phone with the following contents:
    #!/system/bin/sh
    # first copy all files to /data/local/bin

    if ! `echo $PATH | /data/busybox/grep /data/busybox > /dev/null`; then
    export PATH=/data/busybox:$PATH
    fi

    pand_pidfile=/data/local/bin/pand.pid
    PAND_PID=0
    if [ -e $pand_pidfile ]; then
    PAND_PID=`cat $pand_pidfile`
    if [ "x" = "x$PAND_PID" ]; then
    PAND_PID=0
    else
    if ! (ps | grep /data/local/bin/pand | grep -v grep > /dev/null); then
    PAND_PID=0
    fi
    fi
    fi

    logfile=/data/local/bin/tether-log

    case "$1" in
    start)
    if ! lsmod | grep bnep > /dev/null; then
    echo "Adding bnep kernel module"
    insmod /data/local/bin/bnep.ko || exit
    fi
    if [ 0 -eq $PAND_PID ]; then
    echo "Starting pand"
    rm $pand_pidfile > /dev/null 2>&1
    PAND_PID=0
    rm $logfile > /dev/null 2>&1 # if it exists
    /data/local/bin/pand --listen --role NAP --devup /data/local/bin/blue-up.sh --devdown /data/local/bin/blue-down.sh --pidfile $pand_pidfile || exit
    fi
    echo "Success"
    ;;
    stop)
    if [ 0 -ne $PAND_PID ]; then
    echo "Killing pand at pid $PAND_PID"
    /data/local/bin/pand -K || exit
    kill $PAND_PID || exit
    rm $pand_pidfile > /dev/null 2>&1 # Looks like pand may have removed it for us
    sleep 1
    fi
    if lsmod | grep bnep > /dev/null; then
    echo "Removing bnep module"
    rmmod bnep || exit
    fi
    echo "Success"
    ;;
    *)
    echo "Usage: /data/local/bin/tether.sh {start|stop}"
    exit 1
    esac
    exit 0

  7. Then put the following into /data/local/bin/blue-up.sh:

    #!/system/bin/sh
    # blue-up.sh
    ifconfig bnep0 10.0.1.1 netmask 255.255.255.0 up
    /data/local/bin/iptables -F
    /data/local/bin/iptables -t nat -F
    /data/local/bin/iptables -t nat -A POSTROUTING -s 10.0.1.5 -j MASQUERADE
    /data/local/bin/iptables -t nat -A POSTROUTING -j ACCEPT
    echo 1 > /proc/sys/net/ipv4/ip_forward
    /data/busybox/udhcpd /data/local/bin/udhcpd.conf
    echo "Interface up " >> /data/local/bin/tether-log

  8. And the following goes into /data/local/bin/blue-down.sh:

    #!/system/bin/sh
    # blue-down.sh

    udhcpd_pidfile=/data/local/bin/udhcpd.pid
    UDHCPD_PID=0
    if [ -e $udhcpd_pidfile ]; then
    UDHCPD_PID=`cat $udhcpd_pidfile`
    if [ "x" = "x$UDHCPD_PID" ]; then
    UDHCPD_PID=0
    fi
    fi
    echo "udhcpd at pid $UDHCPD_PID" >> /data/local/bin/tether-log
    if [ 0 -ne $UDHCPD_PID ]; then
    echo "Trying to kill udhcpd at pid $UDHCPD_PID" >> /data/local/bin/tether-log
    kill -1 $UDHCPD_PID # 1 is SIGHUP
    rm $udhcpd_pidfile
    fi
    ifconfig bnep0 down
    echo 0 > /proc/sys/net/ipv4/ip_forward
    /data/local/bin/iptables -F
    /data/local/bin/iptables -F -t nat
    echo "Interface down " >> /data/local/bin/tether-log

  9. Make sure all '.sh' files and binaries in /data/local/bin have execute permissions. E.g., do:
    su
    chmod 755 /data/local/bin/*.sh
    chmod 755 /data/local/bin/pand
    chmod 755 /data/local/bin/iptables

    I think you can alternatively set execute permissions on your laptop before you copy the files over.
  10. And the following in /data/local/bin/udhcpd.conf:
    start 10.0.1.5
    end 10.0.1.5
    max_leases 1
    interface bnep0
    pidfile /data/local/bin/udhcpd.pid
    option dns 208.67.222.222 208.67.220.220 # Freedns dns servers
    option router 10.0.1.1
    option subnet 255.255.255.0
    option domain local
    option lease 1440 # 4 hours


  11. Next, on your laptop machine, add a line to /etc/network/interfaces to tell Ubuntu about the bnep0 interface:
    iface bnep0 inet dhcp

  12. After modifying /etc/network/interfaces, restart networking to pick up the change by executing:
    $ sudo /etc/init.d/networking restart

  13. Install the pand utility by adding bluez-compat:
    $ sudo aptitude install bluez-compat

  14. On my laptop, I have a script to start and stop things:
    #!/bin/bash
    case "$1" in
    start)
    if ps ax | grep sbin/NetworkManager | grep -v grep > /dev/null; then
    echo "Halting NetworkManager"
    /etc/init.d/NetworkManager stop || exit
    fi
    if ! (pand -l | grep bnep0 > /dev/null); then
    echo "Establishing bluetooth PAN connection"
    pand --connect **:**:**:**:**:** -n || exit
    if ! (pand -l | grep bnep0 > /dev/null); then
    echo "pand returned success but it looks like the connection was not made"
    exit 1
    fi
    sleep 1
    fi
    if ! (ifconfig | grep bnep0 > /dev/null); then
    echo "Bringing up interface bnep0 with dhcp"
    if cat /var/run/network/ifstate | grep bnep0; then
    echo "ifup/down thinks interface is still up. Down it first"
    ifdown bnep0
    fi
    ifup bnep0 || exit
    fi
    echo "Success"
    ;;
    stop)
    if (ifconfig | grep bnep0 > /dev/null); then
    echo "Taking down bnep0"
    ifdown bnep0 || exit
    fi
    pand -K || exit # MUST be run as sudo, though it does not error out otherwise.
    if ! ps ax | grep sbin/NetworkManager | grep -v grep > /dev/null; then
    echo "Restarting NetworkManager"
    /etc/init.d/NetworkManager start || exit
    fi
    echo "Success"
    ;;
    *)
    echo "Usage: sudo laptop-connect-tether {start|stop}"
    exit 1
    esac
    exit 0

    Where you replace the '**:**...' with your phones hardware bluetooth address. [ Update: 'thwarted' says You can get your phone's bluetooth address in Settings | About Phone (bottom) | Status | Bluetooth Address (3rd from bottom) ]


So to use this, run /data/local/bin/tether.sh start on your phone and then run the above script on your laptop with 'start' or 'stop' as desired. Note that to start tether.sh:
  • tether.sh file must be executable (see above)
  • you need to su before running
  • You must invoke the binary with an explicit pathname, e.g. './tether.sh' or '/data/local/bin/tether.sh'. Just cd'ing into the dir and typing 'tether.sh' does not work.


If you get a not found error when invoking tether.sh and you swear it really is there and you are trying this from windows, it may be that pasting the code to a file appended '^M' to the end of each line. Consider using dos2unix to remove the extra characters.



Sorry this is a bit rough, but let me know if you find this useful or find bugs.