Wednesday, 18 April 2018

Slingbox Alternative: Build your own!

I've been trying to find ways to watch Japanese TV in Australia for a while now. It appears I can't pay for this no matter how much I want to - it just doesn't seem offered anywhere (I specifically want NHK Eテレ). Sadly the only two off-the-shelf options that seemed to present themselves were a Slingbox (and the annoyance of remote controlling a friend/families TV set from overseas) or something with a dubious streaming source like the UBOX that I assume may just stop working at some stage in the future.

I'd repeatedly written off both ideas. That was, until I came across @shotasano's blog post! It seems it is possible to decode Japanese digital TV with a valid B-CAS card, reader and the right software. So that's exactly what I did!

  • Like @shotasano, I also used a Raspberry Pi 3 B running Raspbian.
  • Mirakurun handles the DVB device IO (spawning recdvb mostly) and manages tuners, channels and the slowly polls raw EPG data with idle tuners.
  • Chinachu provides a Web UI with EPG and PVR features as well as a background "operator" process that manages free space, cleans up old recordings, schedules new ones according to rules and updates the EPG.
  • I'm using Kodi as a frontend. There is a dedicated PVR module for it, but I didn't fancy building APK's from scratch so I opted for the lazy approach with fewer features and installed a deprecated Simple IPTV PVR library for Chinachu instead and forgoing the PVR functionality.
The only slight challenge was obtaining a B-CAS card. In the end I ripped a red B-CAS card out of an old unused 地上デジタルチューナー box. 

All this together ends up fitting in a box only slightly larger than the Raspberry Pi itself and lives at a friends house in Japan, letting me stream to Sydney. Success!

I made one small code change so that I could visit and with chrome and not get a bunch of 文字化け.

--- a/app-wui.js
+++ b/app-wui.js
@@ -508,6 +508,8 @@ function httpServerMain(req, res, query) {
                if (ext === 'asf') { type = 'video/x-ms-asf'; }
                if (ext === 'json') { type = 'application/json; charset=utf-8'; }
                if (ext === 'xspf') { type = 'application/xspf+xml'; }
+               if (ext === 'm3u8') { type = 'text/plain; charset=utf-8'; }
+               if (ext === 'xml') { type = 'text/plain+xml; charset=utf-8'; }

I also under-clocked everything and turned off wifi, bluetooth and audio to try to avoid under-voltage issues with my cheap power supply by adding this to /boot/config.txt:
# Default is 1200 for Pi3.

# Lower GPU frequency. We'll turn of HDMI later, but this might help.

# Lower RAM frequency (from 450Mhz) too. We don't stress it too much.

# Save power

In /etc/rc.conf, I turned off HDMI which supposedly saves a tiny bit more (around 10mW):

# Disable HDMI
/usr/bin/tvservice -o

That it! Looking forward to kicking the tires of this thing over the coming months!

Tuesday, 11 July 2017

Ethereum Mining

All the kids are doing it. I've been doing it too for a while with 3 cards. And it's a total waste of money. Don't do it! Don't bother! Stick your money in something else, BUY Ethereum if you must!

The problem is that we're all fighting over the same piece of pie. The more miners, the smaller the share. And before you shout "You just want it all for yourself!" and pull out the pitchforks, let me explain why it's too late, and why I'll shortly be stopping.

Rewind back to late May 2017 when the price shot up to $200. I assume a larger contributing force behind this were investors in Japan and elsewhere in Asia realizing that bitcoin was technically less capable than it's newer cousin, Ethereum. Several Asian exchanges started supporting it. Japan introduced a law legitimizing crypto-currencies and the rest was greed feeding greed.

Of course, I joined the bandwagon. I got in around here and shortly after the price shot up to $400. At the peak, my little 3 card setup was making me almost $20 USD a day. It's around this point that I imagine everyone doing this pats themselves on the back, tells themselves how clever they are and does an entirely unrealistic projection of how much money they are going to make.

Since then, the value has dropped back to ~$220 USD but the miners haven't stopped. They've been piling on like crazy. News of GPU shortages have only piqued interest and fueled the fire further. What these punters are forgetting to factor in is not just the price correction, but the difficulty increase. I thought I was clever. I thought I had factored it in but my projections were off. It turns out the world is far sillier and greedier than I expected.

Ethereum, like all proof-of-work crypto-currencies, targets a certain "block rate". This is effectively the rate at which blocks of the transaction ledger get signed off on. To avoid any old person declaring their ledger is the ledger, miners spend a lot of time working on solving a complex computing problem. The one that solves it gets their name attached to the ledger and 5 ETH for their trouble. The network is designed so that this happened with a median time of 15 seconds.

To simply that even further: Every hour, roughly 1200 ETH at $225 each ($270kUSD), is up for grabs for miners.

It's easy to see why a gold rush was inevitable. But....

See that little red box on the difficulty evolution. Thats a bump of over 100 trillion in difficulty. At 33MH/s hashing rate (a single GTX1070 graphics card), that's equivalent to 30 million cards jumping on the mining bandwagon... overnight! Yes, there is some statistical error in there, but in any case it's a big increase!

Current returns are only a few dollars a day and electricity needs to be factored into that. Paying back your $500 graphics cards is going to take you an eternity if current trends continue.

What happened, I suspect, is that some early big investors bought ETH and then realized the markup they were paying and have decided to mine instead. Meanwhile, ICOs rode the hype, sucking money out of the system by driving up demand and then cashing out. There have also been exchange hacking and network capacity issues that have scared people off.

I have seen photos of shops that churn out hundreds of rigs with thousands of cards at inflated prices. These are making the news and gullible first timers are jumping in without understanding the ecosystem. The clever money at the moment is in selling the shovel to the gold miner I assume...

I imagine there are a lot of people out there that have invested in this with cards on back-order so there is a delay in adding capacity to the network but already we're dangerously cost to returns dipping below the cost of electricity.

For me, I was going to burn the electricity on heating anyway so I'll continue through the winter and then sell it all off. It was an interesting flash in the pan, but still just a flash.

Wednesday, 27 April 2016


I noticed by chance in early April that Google Public DNS (the people) now offer DNS-over-HTTPS. I thought this would a nice little addition for privacy but it seems so new that nothing out there supports it! A few weekends later, I now have a little proxy daemon and for the past week I've been running it on my OpenWRT router without issue! It's not perfect but I've uploaded the code here if anyone is interested.

Cleanups, some security auditing and test coverage work needed but I feel it's working well enough to release it to others at this stage. Hope it's helpful to someone else!

Friday, 4 December 2015

Old Mac Mini runs old OpenSSH with broken cipher set

I couldn't find much about this on the interwebs but I happen to know an OpenSSH developer so I went straight to the experts here.

Basically, if you run OpenSSH on a mac mini, you shouldn't trust Apple to give you updates and you're implementation may well be full of holes. The latest OpenSSH version is 7.1 and my 2009 mac mini with all the updates was reporting
$ ssh -V
OpenSSH_5.5p1, OpenSSL 0.9.8n 24 Mar 2010
Wow... That's not good.

If you MUST run with this version and you want to be security conscious (and NSA paranoid) so you've restricted your allowed cipher list on your client machines, note that aes128-gcm is advertised by this broken Apple build but not actually supported by the binary. This will look like an immediate disconnection after connecting. You have this problem if your system.log contains:

$ tail -n 100 /var/log/system.log| grep fatal
Dec 4 11:16:09 macmini.lan sshd[27028]: fatal: matching cipher is not supported: [preauth]

The quick fix (you should probably upgrade OpenSSH anyway, somehow..) is to add a cipher line to /etc/sshd_config as follows:
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour

Hope that saves someone else some confusion. :D

Wednesday, 18 February 2015

Playing with the ESP8266

We live in amazing times. For $2.50 each I got some ESP8266 on AliExpress and flashed nodemcu on them with minimal fuss. I've now got kilobytes of flash spare on a WiFi device with 10 digital IO, 1 analog input and a standby current draw of 78 micro amps!

Thanks to the author of this for the instructions. Basically just put the FTDI adapter in 3.3v mode and get breadboarding with the magic pinouts described on the esptool github page and flashed the latest firmware binary blob. Reboot and I get a Lua prompt via minicom. Worked flawlessly.

Next up, DS18B20 temperature sensor, an I2C humidity sensor and some soldering. :)

Saturday, 10 January 2015

Reverse-engineering Efergy's internet-connected smart power meter

I just got myself one of these Efergy Home Hub Online things. I'm super impressed with the physical build quality. Sadly though, nothing's perfect. With this unit two fundamental things bothered me, both which are addressable to some degree.

1. Problematic DHCP?

I plugged it in and the network port came alive. Data LEDs blinked but the solid red light indicating the device was booting stayed on forever. I changed the Ethernet cable, port, lots of reboots, turned my 1Gbit ports down to 100Mbit... Nothing. It turned out whatever DHCP client they've implemented doesn't seem to work with a Fritz Box 7390. I've never seen this issue before. I returned the first unit thinking it was bad. When the second unit did the same thing I tried shoving the Ethernet cable into the Ethernet port of my desktop and fired up udhcp. To my surprise, that did the trick. Now I need to figure out some way to get this on my network without forwarding traffic through my desktop that isn't always on.. Urgh!

2. The all-your-data-are-belong-to-us cloudy thing.

Everything these days seems to want to provide an app. That alone isn't a problem, but in order to serve data to the app, your data generally lives in the cloud and this is where I reach for my tin-foil hat and begin the fun little process of reverse engineering the protocol this thing uses. :D

I don't think there is anything sinister going on here. I just want control of my data and would like my device to continue to work if (or rather when) the manufacturer decides to stop hosting it. The web interface this thing provides is pretty slick and there is both an iPhone and Android app but when it comes to my personal data, I'd rather hold it myself, thanks! So on that note...

The solution: Reverse Engineering!

The hardware I presume was actually produced by Efergy but it seems that the software for the "hub" device was provided by a company called Hildebrand based out of London and it's Hildebrand that actually receive the raw meter readings and store them on their servers. The Efergy website hosts the interface but the raw data goes to a domain that is registered to Hildebrand.

Boot Sequence

I have a box stamped with HK 1.1 Firmware AU on the bottom of it. I can't guarantee other regions behave the same way but here goes.
  1. DHCP request is broadcast to network. Device waits for a repsonse.
  2. DHCP response received. Device starts resolving using DHCP-provided DNS server the hostnames "" and "" where "aabbccddeeff" is the MAC address of the device.
  3. Device requests from the resovled sensornet host IP the URL https://<ip>/get_key.html. Response is HTTP/200 of the form "TT|ALPHANUMERIC"
  4. Device requests from the same host IP the URL "https://<ip>/check_key.html?p=TT&ts=
    0000019D&h=<some hash>. HTTP/200 with an empty response is returned.
  5. Device starts posting periodically to https://<ip>/h2 with "Content-Type: application/eh-data" and content of the form: "123456|1|EFCT|P1,0.00.".
  6. Device periodically also sends requests to https://>ip>/h2 with "Content-Type: application/eh-ping" and an empty body, presumably just to let the service know it's alive.
Some other nice tidbits:
  • SSL is used but certificates are never checked so MITM is very easy to do to read the data.
  • The "ts" GET argument in check_key.html is not a timestamp. I've seen it go backwards and always seems close to zero. I suspect it's irrelevant as we can return "success" on any data. We don't really care about authentication here.
  • The data format seems to be "<Sensor ID> | 1 | <sensor type> | <INPUT>,<Reading>."
  • I suspect unit of measurement to be in milliamps but haven't yet confirmed this. The user will have to take voltage and power factor into account to work out kW and kW/h.

Update: A fake cloudy thing.

I had a Raspberry Pi that I wasn't doing much with so I turned it into my data logger by running my own DHCP, DNS and HTTPS servers, each pointing the device to the rPi instead of the hildebrand servers. I have a USB to Ethernet dongle to talk to the hub and the rPi ethernet adapter to talk to my LAN. Win! Source code is on github here.

Saturday, 20 September 2014

Graphing HDD health with smartctl

I proudly built myself a front door for my TV cabinet recently - the very same TV cabinet that houses my NAS. Two weeks later and two crashes of my NAS box (that coincidentally uses my drives for swap), I discover 2 of my 4 HDD's had started giving errors, one had completely died. Turns out this thing called "ventilation" is important after all! *shrugs*