You CAN still get a pre-paid data SIM card in Japan

I read a lot of blogs stating you couldn't get a pre-paid SIM card in Japan. It's true that the big players (NTT Docomo, Softbank, AU) don't sell these anymore but turns out that there is still at least one way to do this.

A company called b-Mobile sells pre-paid SIM cards valid for 1 month (about $USD32) and 3 month (about $USD100) with 1GB of data each. Yes, I wonder if anyone goes with the 3 month option...

The cards don't support calls but they run on NTT Docomo's LTE, HSDPA, 3G networks and data coverage seems very good.

You can only get them from big retailers. In Osaka, that means BIC Camera in Namba or, in my case, Yodobashi Camera in Umeda. You also need to activate them in Japanese - with a Japanese mobile apparently. So it definitely helps if you speak Japanese, have a Japanese friend, or if you're super convincing, maybe you can persuade the sales person to do it for you. From what I could tell, the U300 SIM they used to offer in English, pre-activated 24 hours after mail order purchase is no longer on offer. :(

In my case, I've been running on a 1GB, 1 month MicroSIM for a week now and using SkypeIn for receiving incoming calls. Calls are very high latency (almost unusable) but it has served its purpose so far - if I get a call that isn't working well, I just fall back to contact via email.

Best of luck Japanese travellers!

The worlds ugliest webm streaming webserver?

I was trying to find an easy way to get low latency video from a webcam to a remote browser today. Requirements:

1. Quick deployment
2. Runs on Raspberry Pi
3. Runs with out-of-the-box debs (see quick deployment)

My hacky solution was a Django python app that uses the StreamingHttpResponse class, gstreamer and a pipe. Disguisting, but works well. Sadly, latency is about 10 seconds to localhost so its not exactly live... 

import pygst
pygst.require("0.10")
import gst

def grab(request):
  """ Return a webm live stream from the first attached webcam. """
  class VideoStreamer():
    def __init__(self):
      self.pipein, self.pipeout = os.pipe()
      self.player = gst.parse_launch ("v4l2src ! video/x-raw-yuv,width=640,height=480,framerate=10/1 ! ffmpegcolorspace ! vp8enc max-latency=1 lag-in-frames=1 ! webmmux name='m' streamable=true ! fdsink fd=%d" % self.pipeout)
      self.player.set_state(gst.STATE_PLAYING)
    def start(self):
      fd = os.fdopen(self.pipein)
      try:
        while True:
          yield fd.read(4096)
      except Exception, e:
        print "Exception was ", e
    def __del__(self):
      self.player.set_state(gst.STATE_NULL)
      os.close(self.pipeout)
  return StreamingHttpResponse(VideoStreamer().start(), content_type="video/webm")

Requires pygst and django 1.5 (use pip).

Bricked Netgear Stora? Arduino to the rescue!

My latest attempt at getting offsite backups working for me in the most convenient manner possible involves scattering storage devices around at places I frequently visit such as relatives places, etc. I thought I'd re-purpose a Netgear Stora device I had lying about (P.S. Don't ever buy one of these if you value your privacy) by modding the firmware on it. Turns out, an arduino makes a great TTL serial adapter if you short the reset pin to GND. :) 

I'm also particularly proud of my ghetto MacGyver-like pin connectors. They were created with PVC tape rolled around the leg of a resistor and then cut into thirds and slipped over each pin to hold them in place. (Yes, I need to get myself some more electronics gear...)

Pin outs here in case anyone stumbles across this wanting to do something similar. :)

 

DataMule - My first Android app. :)

I've been toying with various backup solutions for months now. I've tried tahoe-lafs. I tried my own hacky scripts and before that I even had the audacious idea of writing my own P2P backup software from scratch, starting at the low-level RPC layer.. My current approach is to use duplicity and throw the files on Google's cloud storage. The biggest problem with this latest approach is that full backups of my 100GB of data take about 12.5 days of maxing out my ADSL uplink. QoS eases some of the frustrations trying to use the internet while this is happening but doesn't completely remove them.

So DataMule is going to be a simple Android app that includes an SFTP/SSH client and configuration for pairs of WIFI SSIDs.

When you come into range of your "source" SSID, the SD card partition on your phone gets filled up with data copied via SFTP from your source server (until full). When you come into range of your "destination" SSID, the process is reversed and data is copied to the destination machine via SFTP. To complete the data migration, rsync is kicked off on the destination host with flag to delete after copying. The rsync will connect directly back to the source and copy (from the source) to the new file. This will just kick off a file hash verification and either delete the file (if its OK) or fix it if its not (with hopefully minimal bandwidth).

A HDD at work, a HDD at my parents place and an Android app and I'm all set. So far I've got a skeleton app working but its very very rough.

More updates to come!

Hacking at heat-maps and Google Maps API

I've been hacking away at a map visualization that I'm surprised hasn't appeared elsewhere already - a heatmap that interpolates between sample points. 

Every other heat map I've come across approximates interpolation by rendering circles with fading alpha at sample points and relies on the density of points to achieve a smooth result.

Below is an early screenshot of a canvas-based heatmap overlay I'm writing that is built from the Delaunay triangulation of the underlying data points (that's quite a mouthful!). 

I'm using a scanline-based Gouraud shading algorithm to render the triangles to the canvas myself having tried and failed to get WebGL, SVG and CSS to do the job for me (none of these can draw triangles with three different coloured vertices with any serious speed).

Prices for 2BR units across Sydney's inner west.

If there is interest, I will clean up and release the code once I have it working reasonably well and post it somewhere. For now, I'm rethinking my choice of Gouraud shading because with under-sampled data like I have here, it gives some very ugly, sharp lines.

I'm currently using it to display property values but hopefully others can think up some more good uses for it. 

GNU SDR and EZCap

Another nerdy post. I seem to be getting worse! :P

My ezcap USB dongle (RTL1832U/E4000) arrived from dealextreme yesterday and it took a whole hour of tweaking settings and wikipedia browsing to go from fuzzy crackle to working (and I think I understand the basics of it) software defined radio FM decoding. I started with someone elses grc file (although I can't remember who's) and tweaked it until I got to this setup.

In the top FFT you can see the station I'm tuned to in the middle and two stations on either side (they seem to be spaced every 800khz apart in Sydney). The bottom shows the filtered signal for the station I'm tuned to.

An amateur radio expert would probably laugh at me as I arrived at these settings through nothing but a bit of a rough direction and a lot of sheer luck but it seems to sound pretty good! Now I need to think of a legitimate use for this. :)

An experiment in Van Emde Boas layouts

Given an arbitrarily large sorted, balanced binary tree of values, the obvious way to search for elements is binary search. In undergraduate classes I learned that a binary search requires log(n) operations and is thus as efficient as you can get without throwing additional storage space at the problem. One thing my lecturer didn't dive into though was the effect of CPU cache on this otherwise simple little log(n) algorithm.