Comparison of Boston, San Francisco and Seattle on weather

I'm toying with moving to Boston from SF area and nature is a big factor. Here's some data I'm looking at:

• Oregon Climate Service precipitation maps of the area. Looks like Boston gets about as much rain as Seattle, and almost double SF area.
  
• Climate projects has data looking forward, page 890 has climate and rainfall projections. Looks like Boston will get a bit warmer and more rain. CA and Seattle dry out a bit.
  
• Air quality summary. Looks like Boston is worse than Seattle and SF, though the SF counties include gobs of costal area which biases the measurement. Monthly breakdown
  
• And browse some census data data such as median age
  

Using jgit with socks5 proxy on Ubuntu

I use jgit to store my git repositories on Amazon S3. On my laptop I also periodically connect to the internet through tetherbot, which uses a socks proxy (see earlier post for more details there). I finally figured out how to get jgit, and I suppose other java applications working with the proxy. Jgit is java based and you need to pass the right flags to java, as described at http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html. The flags are:-DsocksProxyHost=localhost -DsocksProxyPort=1080. I inserted the following code into the jgit program, before the exec .... $java_args part:

if (netstat -pl --numeric-ports 2>&1 | grep ':1080.*adb' > /dev/null); then
    echo "Using Java socks5 proxy args"
    java_args="$java_args -DsocksProxyHost=localhost -DsocksProxyPort=1080"
fi

I inserted the lines using emacs. For some reason, xemacs and nano garbled the file when I tried editing it with them.

Transparent proxy in Ubuntu

I have a socks5 proxy on my machine (actually on my Android phone, but port-forwarded through adb), and want non-socks5 aware applications to be able to use it. I tried tsocks, but couldn't get it to work with wget - it wouldn't proxy the DNS resolution, even if I set the compile time flags. I then switched to proxychains, which works great with wget and other commandline utils. The only issue is that proxychains does not seem to work with stand-alone java applications, failing in the connection library like:

Caused by: java.net.SocketException: Network is unreachable 
at java.net.PlainSocketImpl.socketConnect(Native Method) 
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) 
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) 
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) 
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) 
at java.net.Socket.connect(Socket.java:519) 
at java.net.Socket.connect(Socket.java:469) 
at sun.net.NetworkClient.doConnect(NetworkClient.java:163) 
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) 
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529) 
at sun.net.www.http.HttpClient.(HttpClient.java:233) 
at sun.net.www.http.HttpClient.New(HttpClient.java:306) 
at sun.net.www.http.HttpClient.New(HttpClient.java:323) 
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:837) 
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:816) 
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:703) 
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1026) 
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)

Update: got this working with shell-wrapped jar files. See next post

options for git on Amazon S3

Here's the options for backing up git on S3 as I see them now:

• jgit - standalone java app that does git fetch and push to S3. I currently use this one, but it doesn't work with transparent proxy apps such as proxychains. Occasionally I use an old laptop with no wifi or bluetooth, and so want to use a USB-tethered Android G1 for internet access.
  
• S3-backed Filesystems
  
  • Jungledisk
    
  • s3fs
    
  • FuzeOverAmazon(also called s3fs)
    
  • s3fs-fuse
    
  
  
• gits3 - a git implementation that can push to s3. Pretty new. Is it stable yet?
  

I'm tempted to try the filesystem options, but I'm concerned about reliability and transparency (I want more visibility into how the S3 ops are going).

Store git repository on Amazon S3

I keep my home dir and some other dirs in a few git repositories. I used to back them up to slicehost, which I've been very happy with. Since S3 has much lower storage charges, I modified my scripts to backup the repositories to S3 as well using jgit from here. Figure I'll play around with it and see if the repositories get out of sync.

Bluetooth tethering from Ubuntu to Android G1

Update: Looks like this does not work with the new Cupcake OS :(

Yeah! I finally got this working. I got my Thinkpad T500 running Ubuntu Intrepid to connect via bluetooth to my Android G1. So the laptop connects to the internet through whatever internet connection the phone has (Edge, 3G, or Wifi). I'm writing this at a starbucks on my laptop, connected through my G1 to a TMobile hotspot (install the hotspot app on your G1 and reboot the phone. You'll see a bullseye in the notification bar when you wifi connect to the hotspot on the phone) Roughly the steps I did to tether were:

1. Get 'adb' installed from the Android dev tools. I downloaded the whole sdk.
   
2. The main starting point is the blog entry at: http://www.gotontheinter.net/content/second-g1-story-proper-bluetooth-tethering-how-short
   
3. From there I got the files pand, iptables and bnep.ko and copy to /data/local/bin/, creating the dir if necessary.
   
4. Get busybox from http://benno.id.au/blog/2007/11/14/android-busybox and install in /data/busybox (it looks like the directory is hard-coded in that binary somewhere)
   
5. I did a bit of shell code to move to a subdirectory any binaries in busybox for which another version existed outside of busybox, except for ps. I figured I trust the android ones more, but Android's ps was missing the all important 'w' option
   
6. Generate a file called /data/local/bin/tether.sh on the phone with the following contents:
   

   #!/system/bin/sh
   # first copy all files to /data/local/bin
   
   if ! `echo $PATH | /data/busybox/grep /data/busybox > /dev/null`; then
       export PATH=/data/busybox:$PATH
   fi
   
   pand_pidfile=/data/local/bin/pand.pid
   PAND_PID=0
   if [ -e $pand_pidfile ]; then
       PAND_PID=`cat $pand_pidfile`
       if [ "x" = "x$PAND_PID" ]; then
           PAND_PID=0
       else
           if ! (ps | grep /data/local/bin/pand | grep -v grep > /dev/null); then
               PAND_PID=0
           fi
       fi
   fi
   
   logfile=/data/local/bin/tether-log
   
   case "$1" in
     start)
       if ! lsmod | grep bnep > /dev/null; then
    echo "Adding bnep kernel module"
    insmod /data/local/bin/bnep.ko || exit
       fi
       if [ 0 -eq $PAND_PID ]; then
    echo "Starting pand"
    rm $pand_pidfile > /dev/null 2>&1
    PAND_PID=0
    rm $logfile > /dev/null 2>&1 # if it exists
    /data/local/bin/pand --listen --role NAP --devup /data/local/bin/blue-up.sh --devdown /data/local/bin/blue-down.sh --pidfile $pand_pidfile || exit
       fi
       echo "Success"
       ;;
     stop)
        if [ 0 -ne $PAND_PID ]; then
     echo "Killing pand at pid $PAND_PID"
     /data/local/bin/pand -K || exit
     kill $PAND_PID || exit
     rm $pand_pidfile > /dev/null 2>&1 # Looks like pand may have removed it for us
     sleep 1
        fi
        if lsmod | grep bnep > /dev/null; then
     echo "Removing bnep module"
     rmmod bnep || exit
        fi
        echo "Success"
        ;;
     *)
       echo "Usage: /data/local/bin/tether.sh {start|stop}"
       exit 1
   esac
   exit 0
   

   
   
7. Then put the following into /data/local/bin/blue-up.sh:
   
   

   #!/system/bin/sh
   # blue-up.sh
   ifconfig bnep0 10.0.1.1 netmask 255.255.255.0 up
   /data/local/bin/iptables -F
   /data/local/bin/iptables -t nat -F
   /data/local/bin/iptables -t nat -A POSTROUTING -s 10.0.1.5 -j MASQUERADE
   /data/local/bin/iptables -t nat -A POSTROUTING -j ACCEPT
   echo 1 > /proc/sys/net/ipv4/ip_forward
   /data/busybox/udhcpd /data/local/bin/udhcpd.conf
   echo "Interface up " >> /data/local/bin/tether-log
   

   
   
8. And the following goes into /data/local/bin/blue-down.sh:
   
   

   #!/system/bin/sh
   # blue-down.sh
   
   udhcpd_pidfile=/data/local/bin/udhcpd.pid
   UDHCPD_PID=0
   if [ -e $udhcpd_pidfile ]; then
       UDHCPD_PID=`cat $udhcpd_pidfile`
       if [ "x" = "x$UDHCPD_PID" ]; then
           UDHCPD_PID=0
       fi
   fi
   echo "udhcpd at pid $UDHCPD_PID" >> /data/local/bin/tether-log
   if [ 0 -ne $UDHCPD_PID ]; then
       echo "Trying to kill udhcpd at pid $UDHCPD_PID" >> /data/local/bin/tether-log
       kill -1 $UDHCPD_PID  # 1 is SIGHUP
       rm $udhcpd_pidfile
   fi
   ifconfig bnep0 down
   echo 0 > /proc/sys/net/ipv4/ip_forward
   /data/local/bin/iptables -F
   /data/local/bin/iptables -F -t nat
   echo "Interface down " >> /data/local/bin/tether-log
   

   
   
9. Make sure all '.sh' files and binaries in /data/local/bin have execute permissions. E.g., do:
   

   su
   chmod 755 /data/local/bin/*.sh
   chmod 755 /data/local/bin/pand
   chmod 755 /data/local/bin/iptables
   

   
   I think you can alternatively set execute permissions on your laptop before you copy the files over.
   
10. And the following in /data/local/bin/udhcpd.conf:
    

    start 10.0.1.5
    end   10.0.1.5
    max_leases 1
    interface bnep0
    pidfile /data/local/bin/udhcpd.pid
    option dns 208.67.222.222 208.67.220.220  # Freedns dns servers
    option router 10.0.1.1
    option subnet 255.255.255.0
    option domain local
    option lease 1440 # 4 hours
    

    
    
    
11. Next, on your laptop machine, add a line to /etc/network/interfaces to tell Ubuntu about the bnep0 interface:
    

    iface bnep0 inet dhcp
    

    
    
12. After modifying /etc/network/interfaces, restart networking to pick up the change by executing:
    

    $ sudo /etc/init.d/networking restart
    

    
    
13. Install the pand utility by adding bluez-compat:
    

    $ sudo aptitude install bluez-compat
    

    
    
14. On my laptop, I have a script to start and stop things:
    

    #!/bin/bash
    case "$1" in
        start)
          if ps ax | grep sbin/NetworkManager | grep -v grep > /dev/null; then
       echo "Halting NetworkManager"
       /etc/init.d/NetworkManager stop || exit
          fi
          if ! (pand -l | grep bnep0 > /dev/null); then
       echo "Establishing bluetooth PAN connection"
       pand --connect **:**:**:**:**:** -n || exit
       if ! (pand -l | grep bnep0 > /dev/null); then
           echo "pand returned success but it looks like the connection was not made"
           exit 1
       fi
       sleep 1
          fi
          if ! (ifconfig | grep bnep0 > /dev/null); then
       echo "Bringing up interface bnep0 with dhcp"
       if cat /var/run/network/ifstate | grep bnep0; then
           echo "ifup/down thinks interface is still up.  Down it first"
           ifdown bnep0
       fi
       ifup bnep0 || exit
          fi
          echo "Success"
          ;;
        stop)
          if (ifconfig | grep bnep0 > /dev/null); then
       echo "Taking down bnep0"
       ifdown bnep0 || exit
          fi
          pand -K || exit # MUST be run as sudo, though it does not error out otherwise.
          if ! ps ax | grep sbin/NetworkManager | grep -v grep > /dev/null; then
       echo "Restarting NetworkManager"
       /etc/init.d/NetworkManager start || exit
          fi
          echo "Success"
          ;;
        *)
          echo "Usage: sudo laptop-connect-tether {start|stop}"
          exit 1
    esac
    exit 0
    

    
    Where you replace the '**:**...' with your phones hardware bluetooth address. [ Update: 'thwarted' says You can get your phone's bluetooth address in Settings | About Phone (bottom) | Status | Bluetooth Address (3rd from bottom) ]
    

So to use this, run /data/local/bin/tether.sh start on your phone and then run the above script on your laptop with 'start' or 'stop' as desired. Note that to start tether.sh:

• tether.sh file must be executable (see above)
  
• you need to su before running
  
• You must invoke the binary with an explicit pathname, e.g. './tether.sh' or '/data/local/bin/tether.sh'. Just cd'ing into the dir and typing 'tether.sh' does not work.
  

If you get a not found error when invoking tether.sh and you swear it really is there and you are trying this from windows, it may be that pasting the code to a file appended '^M' to the end of each line. Consider using dos2unix to remove the extra characters.

Sorry this is a bit rough, but let me know if you find this useful or find bugs.