Joshua Redstone's Blog

How to stream audio from mog.com on Ubuntu laptop to remote linux box

2011-08-21T21:52:00.000-07:00

The problem: I recently got a linux set-top box that I have connected to my stereo in my living room. I use the internet music service mog.com and I'd like to be able to play it on my stereo and control it from my linux laptop.

I've tried lots of approaches that didn't work, like using vnc/nx/etc and setting up pulseaudio with module-tunnel-sink.

The way that seems to work the best for me is to first, setup pulseaudio on the settop box to install the module-native-protocol-tcp by adding the following line to /etc/pulse/default.pa:

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;192.168.0.0/16

and restart pulseaudio with:

pulseaudio --kill ; pulseaudio --start

This tells pulseaudio to accept network connections from other machines on the local network.

Then, on my laptop, I run google-chrome as follows:

PULSE_SERVER=tcp:[settop box hostname] google-chrome --user-data-dir=/home/redstone/tmp/chrome-mog --app=http://mog.com

This starts a separate chrome session that will stream audio to the settop box. Note: you are free to simultaneously start google-chrome without the flags and start a google chrome sessions that will use your local laptops speakers as well. The reason for the user-data flags is that you can only specify the pulse server to use when chrome starts a new session, and if you don't start chrome with the user-data flags, it will just open a new window in your existing chrome session rather than start a new one.

You may want to run:
PULSE_SERVER=tcp:[settop box hostname] pavucontrol
to fiddle with the audio settings on the settop box.

I suspect that this approach should work with any other internet music streaming service like spotify/pandora/etc.

Back and internal pictures of Niles Audio PS-1 Phono/Aux A-B Switcher

2011-08-20T21:52:00.001-07:00

For those of you who are considering buying a Niles Audio PS-1 and are wondering what it looks like on the back and inside, I took a few pics. Here they are:

Subscribing to a twitter feed in Google reader

2011-08-20T12:21:00.000-07:00

Google reader is a bit picky about the types of feed URLs you can paste into it to subscribe to feeds. I found that the following works:

http://api.twitter.com/1/statuses/user_timeline/[username].atom

where you substitute [username] for the actual usename. For example, to subscribe in google reader to Facebook's twitter feed (http://twitter.com/#!/facebook) you click on 'Add a subscription' in reader.google.com and paste in:

http://api.twitter.com/1/statuses/user_timeline/facebook.atom

There is more documentation on this API at https://dev.twitter.com/docs/api/1/get/statuses/user_timeline. It's listed at the bottom in the 'Extended description' section with the words 'not recommended' :). The 'recommended' API, using URL parameters, doesn't work in google reader. Further note: I initially tried getting RSS feed format to work and didn't have any luck.

Plotting the ratio of two stocks

2011-08-20T09:22:00.000-07:00

I found a website, stockcharts.com, that let's you plot the ratio of two stocks. In the 'symbol' field, you specify the two stocks separated by a ':'. For example, "goog:$SPX" to plot the ratio of Google and the S&P 500. As a side note, there is a link partway down, 'linkable version' that let's you bookmark the chart. A recent one I was interested in:

GOOG versus S&P 500 over the past year

Hot-water shower in my car for after surfing

2011-03-07T19:24:00.000-08:00

One of the pain points to surfing in the winter is showering off in the ice-cold showers that are common in beach parking lots. So I built a hot-water shower contraption that I can load in the trunk of my car to provide a hot shower after surfing.

Basically, I bought a 12V water pump and attached it to a cigarette lighter power plug. I use the pump to pump water from a 5 gallon storage tank to a hand-held shower head. The idea is that I fill the tank with hot water from my bathtub before I head out surfing, I put the tank and pump in my car trunk, and, after I'm done surfing, I plug in the pump and shower off with the water from the tank. By the time I'm done surfing, the water should have cooled to a pleasant temperature.

I've tested the electrical work and full system test to come soon!
Here's a pic of the work area while I was putting it together:

Update 3/12/2011:
Surf shower v1.5. The water system test was a success! The pump has quick-connect plugs to which you attach the hoses. Mechanically, they are rather fragile. To protect the quick connects, I've mounted the pump inside a rubbermaid tub and used electric conduit clamps padded with a bit of neoprene to guide the hoses so that the hoses can flex and move around without straining the connectors. I also mounted a lighted power switch on the other side of the tub. I used a copy of the disappointing 'The World is Flat' book as a backstop for the drill and cutting board for the neoprene.

Update 7/12/2011:
I've been using the shower regularly after surfing and it works wonderfully. The 5-gallon tank has turned out to be a good size. Most full-flow shower heads flow at a rate of 2 to 2.5 gallons-per-minute. So a 5-gallon tank (which weighs 40 pounds, when full) will last a bit over 2 minutes of continuous usage. I find that by turning off the water switch on the shower head while I soap, I end up using about 3 to 4 gallons of water, and usually have plenty left over at the end.

Here's the final setup:

The water flows out of the resevoir, through the strainer (the black circular device attached to the tube), to the pump, then from the pump to the hand-held shower head. There is a lighted rocker switch mounted on the right side of the tub.

Making slipper-socks from socks

2010-12-24T13:32:00.000-08:00

I decided to try my hand at making slipper socks. I wasn't satisfied with just wearing through athletic socks around the house and thought slippers were too hot and didn't cover the ankles. To make slipper socks:

Start with a choice sock
Put them on and trace out the outline of your feet on a piece of cardboard.
Cut out the shapes
Use the outlines to cut out a piece of fabric that has non-slip surface on one side (got it from a fabric store)
Put the cardboard into the sock to hold the shape, and then glue the fabric to the sock, using fabric glue.
For good measure, I then added a felling stitch around the edges to help hold the fabric on in case the glue doesn't hold.

And the results:

Quiet an exceptional piece of work, if I do say so myself :). One refinement I want to try next time around: When glueing the fabric to the sock, rather than pressing it against the sock with the cardboard inside, instead press it against the sock with your foot inside. I introduced a small registration problem with I pressed the fabric - I didn't quite center it to where my foot naturally rests in the sock. It's possible you could get away without using the cardboard at all.

Full custom neoprene jacket for glass water bottle

2010-12-16T19:39:00.001-08:00

On vacation with some time on my hands, and figured what better time to construct a protective jacket for the perfect water bottle (see my post earlier on the bottle). Pictured here is the result of a sheet of neoprene laminated with a cloth-like layer, 8 coat hooks, 2 yards of 1mm yellow line and a few hours sewing sprinkled with the occasional curse.

The intent is that the neoprene jacket will cushion the bottle if it falls on a hard surface so that it hopefully won't break. The important areas protected are the corners because they are what is likely to hit a hard surface if the bottle falls on the ground.

The reason for the separate pieces and line connecting is so that it is easy to remove and clean, and also so that you can see the contents of the bottle.

Update: As an alternative, I also tried a rough cut at a version using eyelets instead of stitching. In the image below, I haven't glued the sides or anything, so it appears a bit flappy. I like having a lower-profile to the protective case, but th eyelets pop out too easily. Also, not sure how the glue will work to glue the edges of each piece together...

A candidate for best water bottle

2010-11-21T12:43:00.001-08:00

My quest to find the best bottle to drink water out of has ended, at least for now. Read on... results at the bottom.

Background:

I used to drink water out in disposable plastic bottles. Then I started reusing the plastic bottles till I heard that disposable plastic bottles leach chemicals into the water and that reusing them is even worse. I switched to plastic bottles (e.g., Nalgene), then heard about the BPA issues. I started doing research and it seems like the next best thing was stainless steel bottles like Kleen Kanteen. Stainless steel doesn't leach plastic chemicals, though it does affect the taste of the water a bit. Kleen Kanteen is made out of food-grade stainless steel.

To take it to the next level, there's a stainless-steel bottle originally made by Guyot out of surgical-grade stainless-steel that I also started using. Surgical-grade sounds like it just has to be better than food-grade, doesn't it? I couldn't help myself. I think technically, the main difference between the two is that surgical-grade steel (Guyot is 18/10, also known as 316 stainless steel) is a bit more resistant to corrosion in salt-water. Yeah, I know that we're not using it to store ocean water, but it's the principle, right? Maybe there's a couple of grains of salt in your water on occasion... On the other hand, there is a concern that the nickel in the stainless steel isn't good for you.

So what's the ideal bottle that doesn't leach chemicals or interact with the water in any way? The gold standard for this is glass. Turns out there are some consumer glass water bottle makers, like Lifefactory and Takeya. I had two issues with these bottles. First, the largest size is 22oz, which is a bit small for me. Second, the bottles apparently may break if you put boiling water into them. This is an issue because I use boiling water to clean my stainless steel water bottles. I like cleaning with boiling water because you don't have to worry about soap residue taste and you know any bad stuff is killed.

There's gotta be a better bottle! Turns out that there is, though it's not for consumers. Remember high-school chemistry class? Glass is often used to contain chemicals. If you look at chemistry supply websites, there is a class of glassware called "media storage bottles". These bottles are intended to store potentially nasty chemicals, so the ability to store the chemical without leaching junk from the bottle is a priority. I looked around for the best media bottle. Some makers are Corning, Schott/Duran and Wheaton.

The best bottle I could find is.... [ drum roll please ]:

Duran Pressure-Plus Laboratory Bottle

Cool properties of this bottle:

Made from Duran glass, which I think is basically Pyrex. Check out the link for more info. It can handle boiling water (and apparently contents up to 500 degrees Celsius). It's super resistant to all kinds of chemicals and very inert. It can handle thermal shock up to 100 degrees Celsius (e.g., boiling water on one side of the glass and an ice-cube on the other). Also, the glass is pretty thick and sturdy, though I haven't dropped it yet.
The Pressure-plus bottle in particular can handle pressure between -1 bar (a vacuum inside of the bottle) and +1.5 bar (1.5 atmospheres of pressure pushing outward). One caveat is that if there is a pressure differential between the inside/outside of the bottle, the thermal shock resistance is lower (they claim up to 30 degrees Celsius difference maximum when at maximum rated pressure).

The main downside to this bottle is that it's bulky and that it it doesn't come with any consumer niceties like a holding strap, a protective silicon case or a cap. It takes a standard cap size called GL-45 - I use the cap from a runner-up bottle not described here from Corning.

Getting MAME working on Ubuntu 9.10 (Karmic)

2010-05-06T11:23:00.000-07:00

Getting Mame (the arcade game emulator) working on my Thinkpad T500 took the following steps:

Install mame and the pulse audio driver:

sudo aptitude install mame libsdl1.2debian-pulseaudio

At this point, sounds was crackling, but enabling multithreading fixed that:
```
mame -multithreading
```

And that's it. I modified my /etc/sdlmame/mame.ini to enable multithreading, lower the volume, and have it run in a window rather than fullscreen so I don't need to specify those options on the command line. Try 'man mame' for explanations of other options. When starting mame, 'TAB' pops up the mame option menu including all the keyboard command mappings. 'ESC' quits the menu or the game. '1' starts game play.

Also, Mame seems to be the best supported emulator. I tried to get a few others (nestra, fceu) working without success.

Sprint mobile broadband (Sierra 598) on Ubuntu 9.10

2010-04-15T19:57:00.000-07:00

I was pleasantly surprised to find that plugging my Sprint mobile broadband dongle (it says Sierra Wireless USB 598 on the back) into my laptop running Ubuntu 9.10 worked out of the box! After plugging in, go to the network icon in the toolbar and there appeared an entry for 'Mobile Broadband'. I checked the checkbox underneath it, which took me through an easy setup wizard (selected 'USA' and 'Sprint') and it worked! Warms my heart when Ubuntu works so well.

Replacing power-steering vacuum suction hose on 2001 Honda Civic

2009-12-22T20:52:00.000-08:00

On a non-computer note, I had the misfortune of not replacing the power-steering reservoir in its bracket after changing the left headlight on my civic. The hose connecting it to the power-steering pump rubber one of the belts, eventually puncturing the hose. The fluid ran out of the system and my steering started making unhappy noises. Then followed seven hours of me figuring out how to fix this on a Sunday when shops were closed. The upshot was that I learned that the name of the hose is the power-steering suction hose, and it is a part available only at the dealer (costs < $5). It is a curved, molded tube about a foot long. It looks a bit like a question mark. It is distinct from the high-pressure hose and the return hose. Here are the steps I took to replace it (after my temporary hack of duck-tape and epoxy):

Turn off engine
Suck as much fluid as you can out of the power-steering reservoir with a turkey baster (literally) or some other such device. I used a turkey baster with a straw on the end to reach a bit further in. Deposit this fluid in a container that you can bring to an auto parts shop for proper disposal.
Take a pair of pliers, grab the spring-clamp attaching the suction tube to the reservoir and lower the clamp down the tube so it is no longer clamping the tube onto the reservoir.
Place a receptacle underneath the hose and reservoir to catch any fluid that spills out when you remove the hose from the reservoir. I used a disposable plastic cup.
Remove the hose from the reservoir. It comes off moderately easily. A fair bit of fluid will drain out of the reservoir.
Place reservoir on a towel or something to catch any remaining drips.
Empty plastic cup
Repeat procedure for other end of hose. Less fluid will spill out this time, but it may spill on the alternator, which probably is not good.
Remove hose from car, remove clamps from hose, and slide clamps onto replacement hose.
Reverse detachment procedure to reattach hose to pump and reservoir and move clamps back into place clamping the hose at each end.
Place reservoir back into holding bracket.
Fill up reservoir to min fill line with honda-compatible power steering fluid.
Start engine and turn off after a few seconds. The fluid level in the reservoir will have dropped a bunch.
Add more fluid to reservoir to bring it back up to somewhere between min and max level.
Start engine and turn steering wheel stop to stop a few times. Don't hold at a stop too long - this strains the pump. The purpose of turning wheel a lot is to work out any air bubbles from the system. I could not tell if this was effective.
Evaluate fluid level with engine off.

Tether via USB from Android G1 to Ubuntu

2009-11-25T21:05:00.000-08:00

I installed Cyanogen's mod version 4.2.5 on my Android G1 phone, and it's been working great. On feature that was added since version 4.2.3 is USB networking. Turns out it's simple to use with Ubuntu 9.10 (Karmic). It may work with earlier version as well. The steps:

Check the box for 'Internet tethering' under Settings->Wireless controls
Plug G1 into Ubuntu box with USB cable. Btw, looks like the order of these first two steps doesn't matter.
Click on the networking icon in your toolbar on the Ubuntu box. In the menu that appears, I have an item entitled 'Wired Network (HTC Android Phone)'. Underneath it is a choice 'ifupdown (bnep0)'.
Click on 'ifupdown (bnep0)'.
Now you should be connected to internet.

So this is pretty straightforward, and in particular, it doesn't seem that the Android application, Wired Tether for Root Users, is necessary.

Howto transfer photos from Android G1 to computer via USB

2009-10-05T21:34:00.001-07:00

I'm not sure why it was so hard to to figure this one out. I wanted to transfer photos from my Android G1 to my linux laptop. Basically, you follow the following steps, many of which are from, http://forums.t-mobile.com/tmbl/board/message?board.id=AndroidFAQs&message.id=74:

Plug in USB cable to connect the phone and laptop
On the phone, go the notifications (drag down the top of the screen). There should be a 'USB connected' notification. Click on it.
Click on 'Mount'
On my Ubuntu desktop, under 'Places', a new option now shows up titled '1.0 GB Media'. That is the connection to the Android phone.
For me, the option to open 'F-Spot' photo tool popped up. I declined and used the built in file browser. In either case, the photos are located in the folder dcim/Camera.
When finished manipulating the files, unmount the volume:
1. Go to Places -> '1.0 GB Media' and opening up the file browser
2. Go to File and select 'Unmount Volume'
3. On the phone, go to the notifications pull down again and click on 'Turn off USB storage' and click on 'Turn Off'
Unplug USB cable from phone and computer.

Jaunty and 2.6.30 kernel with Nvidia drivers

2009-06-14T09:20:00.000-07:00

I recently upgraded to 2.6.30 on my Jaunty laptop and desktop. Turns out there are some issues with the nvidia drivers that come with Jaunty and the newer 2.6.30 kernel. The following upgrade process worked for me (based on the solution found at https://bugs.launchpad.net/ubuntu/+source/nvidia-common/+bug/384639/comments/8):

If you have Nvidia drivers then:

Go to some temp directory (~/tmp for me)

wget -c http://archive.ubuntu.com/ubuntu/pool/restricted/n/nvidia-graphics-drivers-180/nvidia-180-kernel-source_180.60-0ubuntu1_i386.deb http://archive.ubuntu.com/ubuntu/pool/restricted/n/nvidia-graphics-drivers-180/nvidia-180-libvdpau_180.60-0ubuntu1_i386.deb http://archive.ubuntu.com/ubuntu/pool/restricted/n/nvidia-graphics-drivers-180/nvidia-glx-180_180.60-0ubuntu1_i386.deb

sudo dpkg -i nvidia-180-kernel-source_180.60-0ubuntu1_i386.deb nvidia-180-libvdpau_180.60-0ubuntu1_i386.deb nvidia-glx-180_180.60-0ubuntu1_i386.deb

wget -c http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.30/linux-headers-2.6.30-020630-generic_2.6.30-020630_i386.deb http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.30/linux-headers-2.6.30-020630_2.6.30-020630_all.deb http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.30/linux-image-2.6.30-020630-generic_2.6.30-020630_i386.deb

sudo dpkg -i linux-headers-2.6.30-020630-generic_2.6.30-020630_i386.deb linux-headers-2.6.30-020630_2.6.30-020630_all.deb linux-image-2.6.30-020630-generic_2.6.30-020630_i386.deb

[product-research] wireless headset for pc and cell

2009-05-25T11:28:00.000-07:00

I'm going to experiment with keeping notes on various product research expeditions I go on to see if it's useful to others (or myself). This is the first one.

Goal: I want a wireless stereo headset and microphone with good audio quality and long range that I can wear around the house. Bluetooth isn't sufficient because of range. The idea is that I can leave my cell phone and computer in my bedroom and wander around the house while on a cell call or VOIP call. I was hoping that this headset would seamlessly work with my cellphone and computer and also have good enough quality that it could be used for music as well as voice. It'd also be great if I could end calls from the headset.

Result: I couldn't find any product that quite does this, but 100m Class-1 bluetooth comes darn close. All I need is a headset that has decent audio quality, and perhaps a repeater so my class-2 cellphone can get decent range. Here are some options I'm considering:

Callpod Dragon looks pretty darn close. Class-1 Bluetooth (extended range), simultaneously can connect to PC and phone. For max range, the PC and phone have to have Class-1 bluetooth connectivity.
- Class-1 dongle: http://www.x-micro.com/main-product-dongle20plus.html
- http://www.startech.com/item/USBBT1EDR-USB-Bluetooth-Adapter-Class-1.aspx
- Or search for [class-1 bluetooth] or [100 meter bluetooth]
Creative labs HS-1200. 72 foot range, RF, USB, good audio quality. Range maybe insufficient and not sure how it would hook up to cell phone.
Plantronics CS361N. 300 foot range, RF, analog RJ-9 connector. no audio quality documented. Could buy RJ-9 to 2.5mm connector.

Some other options I was toying with:

Get a bluetooth headset and get some sort of bluetooth repeater/antenna to extend range. Just came across the CRAZY: http://www.aircable.net/host-xr.html

Android G1 Cupcake hacking links/summary

2009-05-25T09:44:00.000-07:00

I have the G1 dev phone, but with the recent cupcake update, it got reverted to T-Mobile's stock OS. In particular I lost root on the phone. I ended up installing the JesusFreke OS on the phone, which is working well. Here's what I figured out along the way.

There are a few important concepts in G1 modding (please correct me where I'm wrong):

SPL: The Secondary-Program-Loader is a piece of firmware that is basically the bootloader - it loads the OS. You want to replace this with a more fully functional version. I use the HardSPL loader. This loader, as well as some of the other ones out there, gives you the option to use fastboot (here or here), which lets you install on partitions on the phone and boot.
Recovery partition: There is a partition on the phone that has a kernel that is used only when the phone is booted into recovery mode (turn on holding power and home at the same time, then press alt+l to get to the menu). Installing a custom image here, like JFv1.43_Recovery.zip from android-roms, gives you an extra option in the recovery menu to use nandroid to backup your entire system (not sure if it includes the sdcard). Usefull if you want to fiddle with your firmware or wiping your phone and want to be able to restore.
System ROM: This is the system that runs during normal booting and operation. I replaced this with JesusFreke JFv1.51_CRB43-ADP.zip using instructions here. It gives you a kernel set up for wifi-tether, and also includes a couple of extra neat things like the ability to multi-touch pinch zoom in and out on the browser. If you're just interested in wifi-teher, you probably don't need JesusFreke's ROM. The wifi-tether link has an alternate kernel that is a smaller mod. Note that installing the JesusFreke ROM recommends wiping the data on the phone.

Android-roms at http://code.google.com/p/android-roms/ is a useful site that pulls together most of this info. JesusFreke's blog has details on each ROM release and what was changed from the stock ROM.

Finally, if you have an android dev phone and lost root with the cupcake update. The directions at http://forum.xda-developers.com/showthread.php?t=518251 worked for me to regain it. I think these directions require the custom SPL, such as HardSPL.

Ubuntu Intrepid, Skype, and Alsa Settings

2009-02-01T11:59:00.000-08:00

I finally figured out the magic incantations to get skype working with Ubuntu Intrepid 8.10 and pulseaudio, at least on my hardware. I have 82801I (ICH9 Family) HD Audio Controller according to lshw and lspci. I have a microphone connected to the front mic input. The key was to ensure the alsa settings were correct. Here were the steps:

Configure alsa using alsamixer, part of the Ubuntu alsa-utils package. I think there are also gui interfaces to do this. Here are some notes on settings:
- In the 'recording' section (navigate with TAB), there are settings for both capture and capture 1. The key for me was to set capture to front mic. I think this has changed from Hardy. In Hardy, I had to set capture 1 to front mic. For good measure I set both to front mic.
- The mic and mic boost settings didn't seem to affect anything on my system.
- The front mic settings in 'playback' section apply to monitoring the mic input. A good initial sanity check is to bump up the front mic settings and make sure you hear the mic out of your speakers.
Install the libasound2-plugins package. apt-cache show libasound2-plugins shows the version as 1.0.17-0ubuntu5 for me.
Make sure the microphone works in pulseaudio. For this, the Ubuntu padevchooser package was handy. Run it and it adds a tray icon. I chose 'volume meter (recording)' from the drop-down menu. This basically invoked pavumeter with the right args to show the mic level that pulseaudio reads. At this point, when you tap your front mic, the vumeter window should show a bump in signal.
Start skype (I think I use the one from the medibuntu repository) and go to options -> sound devices. There are three main settings:
```
Sound in: pulse
Sound out: pulse
Ringing: pulse
```
I set all to pulse, but my guess is that default is fine and defaults to pulse anyway.
At this point the skype test call worked for me!

Comparison of Boston, San Francisco and Seattle on weather

2009-01-29T20:55:00.001-08:00

I'm toying with moving to Boston from SF area and nature is a big factor. Here's some data I'm looking at:

Oregon Climate Service precipitation maps of the area. Looks like Boston gets about as much rain as Seattle, and almost double SF area.
Climate projects has data looking forward, page 890 has climate and rainfall projections. Looks like Boston will get a bit warmer and more rain. CA and Seattle dry out a bit.
Air quality summary. Looks like Boston is worse than Seattle and SF, though the SF counties include gobs of costal area which biases the measurement. Monthly breakdown
And browse some census data data such as median age

Using jgit with socks5 proxy on Ubuntu

2009-01-19T15:44:00.001-08:00

I use jgit to store my git repositories on Amazon S3. On my laptop I also periodically connect to the internet through tetherbot, which uses a socks proxy (see earlier post for more details there). I finally figured out how to get jgit, and I suppose other java applications working with the proxy. Jgit is java based and you need to pass the right flags to java, as described at http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html. The flags are:-DsocksProxyHost=localhost -DsocksProxyPort=1080. I inserted the following code into the jgit program, before the exec .... $java_args part:

if (netstat -pl --numeric-ports 2>&1 | grep ':1080.*adb' > /dev/null); then
    echo "Using Java socks5 proxy args"
    java_args="$java_args -DsocksProxyHost=localhost -DsocksProxyPort=1080"
fi

I inserted the lines using emacs. For some reason, xemacs and nano garbled the file when I tried editing it with them.

Transparent proxy in Ubuntu

2009-01-17T09:47:00.001-08:00

I have a socks5 proxy on my machine (actually on my Android phone, but port-forwarded through adb), and want non-socks5 aware applications to be able to use it. I tried tsocks, but couldn't get it to work with wget - it wouldn't proxy the DNS resolution, even if I set the compile time flags. I then switched to proxychains, which works great with wget and other commandline utils. The only issue is that proxychains does not seem to work with stand-alone java applications, failing in the connection library like:


Caused by: java.net.SocketException: Network is unreachable 
at java.net.PlainSocketImpl.socketConnect(Native Method) 
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) 
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) 
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) 
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) 
at java.net.Socket.connect(Socket.java:519) 
at java.net.Socket.connect(Socket.java:469) 
at sun.net.NetworkClient.doConnect(NetworkClient.java:163) 
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) 
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529) 
at sun.net.www.http.HttpClient.(HttpClient.java:233) 
at sun.net.www.http.HttpClient.New(HttpClient.java:306) 
at sun.net.www.http.HttpClient.New(HttpClient.java:323) 
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:837) 
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:816) 
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:703) 
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1026) 
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)

Update: got this working with shell-wrapped jar files. See next post

options for git on Amazon S3

2009-01-17T09:26:00.000-08:00

Here's the options for backing up git on S3 as I see them now:

jgit - standalone java app that does git fetch and push to S3. I currently use this one, but it doesn't work with transparent proxy apps such as proxychains. Occasionally I use an old laptop with no wifi or bluetooth, and so want to use a USB-tethered Android G1 for internet access.
S3-backed Filesystems
- Jungledisk
- s3fs
- FuzeOverAmazon(also called s3fs)
- s3fs-fuse
gits3 - a git implementation that can push to s3. Pretty new. Is it stable yet?

I'm tempted to try the filesystem options, but I'm concerned about reliability and transparency (I want more visibility into how the S3 ops are going).

Store git repository on Amazon S3

2009-01-02T19:14:00.001-08:00

I keep my home dir and some other dirs in a few git repositories. I used to back them up to slicehost, which I've been very happy with. Since S3 has much lower storage charges, I modified my scripts to backup the repositories to S3 as well using jgit from here. Figure I'll play around with it and see if the repositories get out of sync.

Bluetooth tethering from Ubuntu to Android G1

2009-01-02T18:42:00.000-08:00

Update: Looks like this does not work with the new Cupcake OS :(

Yeah! I finally got this working. I got my Thinkpad T500 running Ubuntu Intrepid to connect via bluetooth to my Android G1. So the laptop connects to the internet through whatever internet connection the phone has (Edge, 3G, or Wifi). I'm writing this at a starbucks on my laptop, connected through my G1 to a TMobile hotspot (install the hotspot app on your G1 and reboot the phone. You'll see a bullseye in the notification bar when you wifi connect to the hotspot on the phone) Roughly the steps I did to tether were:

Get 'adb' installed from the Android dev tools. I downloaded the whole sdk.
The main starting point is the blog entry at: http://www.gotontheinter.net/content/second-g1-story-proper-bluetooth-tethering-how-short
From there I got the files pand, iptables and bnep.ko and copy to /data/local/bin/, creating the dir if necessary.
Get busybox from http://benno.id.au/blog/2007/11/14/android-busybox and install in /data/busybox (it looks like the directory is hard-coded in that binary somewhere)
I did a bit of shell code to move to a subdirectory any binaries in busybox for which another version existed outside of busybox, except for ps. I figured I trust the android ones more, but Android's ps was missing the all important 'w' option

Generate a file called /data/local/bin/tether.sh on the phone with the following contents:

#!/system/bin/sh
# first copy all files to /data/local/bin

if ! `echo $PATH | /data/busybox/grep /data/busybox > /dev/null`; then
    export PATH=/data/busybox:$PATH
fi

pand_pidfile=/data/local/bin/pand.pid
PAND_PID=0
if [ -e $pand_pidfile ]; then
    PAND_PID=`cat $pand_pidfile`
    if [ "x" = "x$PAND_PID" ]; then
        PAND_PID=0
    else
        if ! (ps | grep /data/local/bin/pand | grep -v grep > /dev/null); then
            PAND_PID=0
        fi
    fi
fi

logfile=/data/local/bin/tether-log

case "$1" in
  start)
    if ! lsmod | grep bnep > /dev/null; then
 echo "Adding bnep kernel module"
 insmod /data/local/bin/bnep.ko || exit
    fi
    if [ 0 -eq $PAND_PID ]; then
 echo "Starting pand"
 rm $pand_pidfile > /dev/null 2>&1
 PAND_PID=0
 rm $logfile > /dev/null 2>&1 # if it exists
 /data/local/bin/pand --listen --role NAP --devup /data/local/bin/blue-up.sh --devdown /data/local/bin/blue-down.sh --pidfile $pand_pidfile || exit
    fi
    echo "Success"
    ;;
  stop)
     if [ 0 -ne $PAND_PID ]; then
  echo "Killing pand at pid $PAND_PID"
  /data/local/bin/pand -K || exit
  kill $PAND_PID || exit
  rm $pand_pidfile > /dev/null 2>&1 # Looks like pand may have removed it for us
  sleep 1
     fi
     if lsmod | grep bnep > /dev/null; then
  echo "Removing bnep module"
  rmmod bnep || exit
     fi
     echo "Success"
     ;;
  *)
    echo "Usage: /data/local/bin/tether.sh {start|stop}"
    exit 1
esac
exit 0

Then put the following into /data/local/bin/blue-up.sh:

#!/system/bin/sh
# blue-up.sh
ifconfig bnep0 10.0.1.1 netmask 255.255.255.0 up
/data/local/bin/iptables -F
/data/local/bin/iptables -t nat -F
/data/local/bin/iptables -t nat -A POSTROUTING -s 10.0.1.5 -j MASQUERADE
/data/local/bin/iptables -t nat -A POSTROUTING -j ACCEPT
echo 1 > /proc/sys/net/ipv4/ip_forward
/data/busybox/udhcpd /data/local/bin/udhcpd.conf
echo "Interface up " >> /data/local/bin/tether-log

And the following goes into /data/local/bin/blue-down.sh:

#!/system/bin/sh
# blue-down.sh

udhcpd_pidfile=/data/local/bin/udhcpd.pid
UDHCPD_PID=0
if [ -e $udhcpd_pidfile ]; then
    UDHCPD_PID=`cat $udhcpd_pidfile`
    if [ "x" = "x$UDHCPD_PID" ]; then
        UDHCPD_PID=0
    fi
fi
echo "udhcpd at pid $UDHCPD_PID" >> /data/local/bin/tether-log
if [ 0 -ne $UDHCPD_PID ]; then
    echo "Trying to kill udhcpd at pid $UDHCPD_PID" >> /data/local/bin/tether-log
    kill -1 $UDHCPD_PID  # 1 is SIGHUP
    rm $udhcpd_pidfile
fi
ifconfig bnep0 down
echo 0 > /proc/sys/net/ipv4/ip_forward
/data/local/bin/iptables -F
/data/local/bin/iptables -F -t nat
echo "Interface down " >> /data/local/bin/tether-log

Make sure all '.sh' files and binaries in /data/local/bin have execute permissions. E.g., do:
```
su
chmod 755 /data/local/bin/*.sh
chmod 755 /data/local/bin/pand
chmod 755 /data/local/bin/iptables
```
I think you can alternatively set execute permissions on your laptop before you copy the files over.

And the following in /data/local/bin/udhcpd.conf:

start 10.0.1.5
end   10.0.1.5
max_leases 1
interface bnep0
pidfile /data/local/bin/udhcpd.pid
option dns 208.67.222.222 208.67.220.220  # Freedns dns servers
option router 10.0.1.1
option subnet 255.255.255.0
option domain local
option lease 1440 # 4 hours

Next, on your laptop machine, add a line to /etc/network/interfaces to tell Ubuntu about the bnep0 interface:
```
iface bnep0 inet dhcp
```
After modifying /etc/network/interfaces, restart networking to pick up the change by executing:
```
$ sudo /etc/init.d/networking restart
```
Install the pand utility by adding bluez-compat:
```
$ sudo aptitude install bluez-compat
```

On my laptop, I have a script to start and stop things:

#!/bin/bash
case "$1" in
    start)
      if ps ax | grep sbin/NetworkManager | grep -v grep > /dev/null; then
   echo "Halting NetworkManager"
   /etc/init.d/NetworkManager stop || exit
      fi
      if ! (pand -l | grep bnep0 > /dev/null); then
   echo "Establishing bluetooth PAN connection"
   pand --connect **:**:**:**:**:** -n || exit
   if ! (pand -l | grep bnep0 > /dev/null); then
       echo "pand returned success but it looks like the connection was not made"
       exit 1
   fi
   sleep 1
      fi
      if ! (ifconfig | grep bnep0 > /dev/null); then
   echo "Bringing up interface bnep0 with dhcp"
   if cat /var/run/network/ifstate | grep bnep0; then
       echo "ifup/down thinks interface is still up.  Down it first"
       ifdown bnep0
   fi
   ifup bnep0 || exit
      fi
      echo "Success"
      ;;
    stop)
      if (ifconfig | grep bnep0 > /dev/null); then
   echo "Taking down bnep0"
   ifdown bnep0 || exit
      fi
      pand -K || exit # MUST be run as sudo, though it does not error out otherwise.
      if ! ps ax | grep sbin/NetworkManager | grep -v grep > /dev/null; then
   echo "Restarting NetworkManager"
   /etc/init.d/NetworkManager start || exit
      fi
      echo "Success"
      ;;
    *)
      echo "Usage: sudo laptop-connect-tether {start|stop}"
      exit 1
esac
exit 0

Where you replace the '**:**...' with your phones hardware bluetooth address. [ Update: 'thwarted' says You can get your phone's bluetooth address in Settings | About Phone (bottom) | Status | Bluetooth Address (3rd from bottom) ]

So to use this, run /data/local/bin/tether.sh start on your phone and then run the above script on your laptop with 'start' or 'stop' as desired. Note that to start tether.sh:

tether.sh file must be executable (see above)
you need to su before running
You must invoke the binary with an explicit pathname, e.g. './tether.sh' or '/data/local/bin/tether.sh'. Just cd'ing into the dir and typing 'tether.sh' does not work.

If you get a not found error when invoking tether.sh and you swear it really is there and you are trying this from windows, it may be that pasting the code to a file appended '^M' to the end of each line. Consider using dos2unix to remove the extra characters.

Sorry this is a bit rough, but let me know if you find this useful or find bugs.

T-mobile G1 new phone setup

2008-12-25T18:09:00.000-08:00

I've been working on getting my G1 up and running productively. Some notes on my experience:

The contact syncing mechanism had a problem in which when I deleted 200 contacts from my gmail list, I could not get the G1 to also delete the contacts. I tried clearing the 'data' from the Contacts app (under settings->applications), but that didn't work. Ended up doing a factory reset of the phone and starting over.
Visual voicemail would be great, but Phonefusion requires an SMS for each voicemail, and also it reroutes voicemail to its own server. :(
Considered apps:
- chompsms
- missed call
- any cut
- astro (file manager)
- toggle settings
- quickpedia
- chompsms
Installed apps:
- Xtremelabs Speedtest
- Tetherbot (from here) - update: actually, I may go with this or this
- t-mobile hotspot
- sms popup
- power manager

Howto create custom csv file to upload into gmail contacts

2008-12-25T16:48:00.000-08:00

I recently got the T-Mobile G1 phone and wanted to transfer my contacts from my old phone to the G1. The G1 syncs its contacts with gmail contacts, so I added them there. How I pulled my contacts from my old Motorola E815 phone into a CSV is a tale for another blog, but it was painful. Gmail has a page: http://mail.google.com/support/bin/answer.py?answer=12119 that's supposed to specify how to create a csv file suitable for upload to gmail, but I found it useless because it didn't clearly define the set of acceptable field names. In the end what worked for me was to first export a contact from gmail into a csv file in Outlook format. The first line of that file has all of the field names. I found that the gmail importer was robust to reordering of the field names and omitting field names. It looks like there isn't an easy way to merge these contacts uploaded with email contact entries that already exist.

For the record, here are the field names for Outlook's CSV format that gmail understands:

Name,E-mail Address,Notes,E-mail 2 Address,E-mail 3 Address,Mobile Phone,Pager,Company,Job Title,Home Phone,Home Phone 2,Home Fax,Home Address,Business Phone,Business Phone 2,Business Fax,Business Address,Other Phone,Other Fax,Other Address

Comparison of historical financial data accuracy among three providers

2008-12-07T11:08:00.000-08:00

I had been using Yahoo finance's historical data for some quantitative modeling I have been dabbling with. At one point, I noticed that that for two assets, Yahoo did not report closing prices on the same set of days. That is, there was a day in which Yahoo reported a closing price for one but not the other. So I started investigating the extent of data inconsistencies and here's the upshot:

I compared two historical data sources and used a third to resolve disaggreements. The first is
Yahoo's historical data, which is free and is based on Commodity Systems, Inc. (CSI). The second data source is Xignite, which costs about $160/month and is based on some combination of DDF, Reuters, and Zacks. When the two sources disagreed, I resolved it by consulting a third, marketwatch.com, which is free and based on Interactive Data Pricing and Reference Data. The comparison was of market days as well as closing prices.

Here is the abbreviated raw output of my comparison script for two assets: VEURX and VBISX (Vanguard funds)


desktop:~/finance$ ./st.py veurx
  Xignite/VEURX(2008-11-20 - 1994-01-04) vs Yahoo/VEURX(2008-11-13 - 1990-11-01)
  WARNING: Xignite/VEURX is missing date 2006-01-13, present in Yahoo/VEURX and marketwatch.
  WARNING: Xignite/VEURX is missing date 2004-05-26, present in Yahoo/VEURX and marketwatch.   
  WARNING: Xignite/VEURX is missing date 2004-02-25, present in Yahoo/VEURX and marketwatch.   
  WARNING: Xignite/VEURX is missing date 1995-07-12, present in Yahoo/VEURX and marketwatch.   
  Xignite/VEURX leads with 5 extra points
  Yahoo/VEURX goes further back 802 points
  BAD DATA on 1994-02-08
    Yahoo= 0.00350 Xignite=-0.00083 1994-02-08 1994-02-08          8.60000 8.57000 12.04000 12.05000
    Xignite is the outlier
  BAD DATA on 1994-02-09
    Yahoo=-0.00465 Xignite= 0.00000 1994-02-09 1994-02-09          8.56000 8.60000 12.04000 12.04000
    Xignite is the outlier
  BAD DATA on 1994-03-01
    Yahoo=-0.01175 Xignite=-0.02843 1994-03-01 1994-03-01          8.41000 8.51000 11.62000 11.96000
    Xignite is the outlier
  BAD DATA on 1994-03-02
    Yahoo=-0.01665 Xignite= 0.00000 1994-03-02 1994-03-02          8.27000 8.41000 11.62000 11.62000
    Xignite is the outlier
  BAD DATA on 1994-03-04
    Yahoo= 0.00963 Xignite= 0.00000 1994-03-04 1994-03-04          8.39000 8.31000 11.68000 11.68000
    Xignite is the outlier
  BAD DATA on 1994-03-07
    Yahoo= 0.01192 Xignite= 0.02226 1994-03-07 1994-03-07          8.49000 8.39000 11.94000 11.68000
    Xignite is the outlier
  BAD DATA on 1994-05-26
    Yahoo= 0.00239 Xignite=-0.01954 1994-05-26 1994-05-26          8.39000 8.37000 11.56936 11.79995
    Xignite is the outlier
  BAD DATA on 1994-05-27
    Yahoo=-0.01073 Xignite= 0.01127 1994-05-27 1994-05-27          8.30000 8.39000 11.69969 11.56936
    Xignite is the outlier
  BAD DATA on 1994-06-27
    Yahoo= 0.00000 Xignite= 0.01317 1994-06-27 1994-06-27          8.10000 8.10000 11.56936 11.41898
    Yahoo is the outlier
  BAD DATA on 1994-06-28
    Yahoo= 0.01605 Xignite= 0.00260 1994-06-28 1994-06-28          8.23000 8.10000 11.59944 11.56936
    Yahoo is the outlier
  BAD DATA on 1994-07-18
    Yahoo= 0.00000 Xignite= 0.00993 1994-07-18 1994-07-18          8.59000 8.59000 12.23104 12.11074
    Yahoo is the outlier
  BAD DATA on 1994-07-19
    Yahoo= 0.00815 Xignite=-0.00164 1994-07-19 1994-07-19          8.66000 8.59000 12.21099 12.23104
    Yahoo is the outlier
  BAD DATA on 1996-02-20
    Yahoo=-0.01214 Xignite= 0.00000 1996-02-20 1996-02-20          10.58000 10.71000 15.09505 15.09505
    Xignite is the outlier
  BAD DATA on 1996-02-21
    Yahoo= 0.00567 Xignite=-0.00699 1996-02-21 1996-02-21          10.64000 10.58000 14.98949 15.09505
    Xignite is the outlier
  BAD DATA on 1998-05-19
    Yahoo= 0.01495 Xignite= 0.00000 1998-05-19 1998-05-19          19.69000 19.40000 27.32290 27.32290
    Xignite is the outlier
  BAD DATA on 1998-05-20
    Yahoo= 0.01524 Xignite= 0.03048 1998-05-20 1998-05-20          19.99000 19.69000 28.15558 27.32290
    Xignite is the outlier
  [ ...... omitted rest for brevity ....... ]
  TOTAL: Yahoo=1.51066 Xignite=1.51084
desktop:~/finance$ ./st.py vbisx
  WARNING: Fabricating data for Yahoo/VBISX on 2003-11-28
  Xignite/VBISX(2008-11-19 - 1994-03-10) vs Yahoo/VBISX(2008-11-10 - 1996-06-20)
  WARNING: Xignite/VBISX is missing date 2004-05-26, present in Yahoo/VBISX and marketwatch.   
  WARNING: Xignite/VBISX is missing date 2004-02-25, present in Yahoo/VBISX and marketwatch.   
  Xignite/VBISX leads with 7 extra points
  Xignite/VBISX goes further back 575 points
  BAD DATA on 2004-04-20
    Yahoo= 0.00000 Xignite=-0.00391 2004-04-20 2004-04-20          8.58000 8.58000 15.61512 15.67636
    Yahoo is the outlier
  BAD DATA on 2004-04-21
    Yahoo=-0.00233 Xignite= 0.00098 2004-04-21 2004-04-21          8.56000 8.58000 15.63043 15.61512
    Yahoo is the outlier
  BAD DATA on 2008-10-31
    Yahoo= 0.00101 Xignite= 0.00412 2008-10-31 2008-10-31          9.95000 9.94000 18.22287 18.14801
    WARNING:  All prices agree with Marketwatch
  TOTAL: Yahoo=0.88037 Xignite=0.88195

In general, over a bunch of assets, it seemed like Xignite's data was more buggy than Yahoo's. The caveat here of course is that marketwatch is an independent data source. I think it is because it sometimes agrees with Yahoo and sometimes with Xignite, though that's not conclusive.

Lenovo T500 noise issue

2008-12-07T10:56:00.000-08:00

So as many others have reported with other laptops, the Lenovo T500 makes a high-frequency whining noise. After going through at least 6 cases with Lenovo tech-support (they had no idea what to make of it and kept thinking it was the fan), including replacing the fan, it got escalated, and tech support returned the laptop with 'CPU power management' disabled in BIOS. Investigating further, and it looks like alternating the CPU to low power states C4 onwards causes it to make the noise. Thinkwiki has a bunch of suggestions on reducing the noise, but the only one that reliably worked is to prevent the CPU from entering those states. I found this is done either by the BIOS setting change, or I found that plugging in a USB mouse also prevented the CPU from entering the power saving state - as reported by powertop. I'm running Ubuntu on it. I've heard elsewhere people had tricks on windows to tell the OS how to schedule the power cycling, but I haven't tried them.

Laptop power consumption

2008-12-07T10:41:00.000-08:00

I recently got a Lenovo T500 based on the new Intel 4-series chipset. It's pretty solidly built, except that the display has a dead pixel. My first action was to get it to dual-boot Ubuntu and Vista - the hardest part was getting Vista not to hog the whole disk. Maybe I'll write on that procedure sometime. Also, the laptop makes a high-pitched noise that, after 4 tech-support services, was finally diagnosed as being due to CPU power management. Apparently this happens on lots of laptops. As the CPU goes into and out of low power states (C4 - C6), it emits the noise. Lenovo's 'solution' was to disable CPU power management in BIOS. I installed powertop, a nifty utility to measure power usage (though not sure if it captures ALL power consumed or just a subset) and did the following measurements. The setup was the laptop running on battery power, with gmail running in firefox. I wasn't interacting with it.

Mode	Power reported by powertop
CPU power management disabled: LCD maximum brightness	19.8W
CPU power management disabled: LCD minimum brightness	15.2W
CPU power management enabled: LCD maximum brightness	18.7W
CPU power management enabled: LCD minimum brightness	14.0W
CPU power management enabled: LCD minimum brightness, RF Kill switch on	13.0W

I also tried a bunch of the optimizations suggested at thinkwiki, such as disabling pcmcia and usb 1.1, and reducing disk writeback, but it didn't seem to make a difference. Of course, that could be because powertop power reporting does not include those power drains in its numbers.