epoll_threadpool

I set out a few months back to build a distributed filesystem with features similar to dropbox but with 100% encrypted storage that is shared amongst friends in a "dark-net" of sorts. In the process, I found myself wanting a fast, light-weight RPC system and, in turn, a fast, light-weight event queuing system. After fighting with libevent and msgpack-rpc, I eventually decided to write my own and epoll_threadpool was born.

The library is still in its infancy but I don't expect it to grow much (if at all) in size. Its Linux only (epoll-based), dependencies are light, speed should be reasonable and all tests are passing so I thought I'd throw it out to the world to see if anyone finds it useful.

Features:

• epoll-based
• runs a thread pool, executing events on the first available thread.
• easy-to-use IOBuffer class for streaming data.
• TCP client and server support.

Available on github as epoll_threadpool.

Qantas Rant

Qantas rant! I also object to Mr Joyce's pay hike but lets put things in real numbers with respect to union demands. Pilots already get paid (in my opinion) WAY more than they should thanks to strong unionism and after a 17% hike - way more than inflation - a second hike seems plain greedy.

A senior pilot's wage is around $500k according to this and presumably a senior co-pilot would be around $350k. According to this most pilots work 215hrs a month (75 hours of that flying). In hourly rates, that's $193/hr for pilots and $135 for co-pilots.

If a plane flies 80% of the time, there are 86.4 of the 108 qantas grounded planes in the air at any given time with a pilot and copilot each. That's an operational cost of $28339.20 an hour (24/7) or $248 million a year. A 2.5% pay increase for pilots will this cost the airline $6.2 million in flight time alone. That doesn't cover ground-based preparation and other duties.

There are no parties here that are NOT being greedy but the union is trying to tarnish the brand in order to push around management. Strike action is their biggest hammer and they've been swinging it around with way too little regard for the people they hit with it for way too long. If pilots are not happy with their conditions they should walk with their feet (out the door). If their salaries are average as they claim for senior pilots, they shouldn't have trouble finding work elsewhere.

As for the CEO's salary, keep separate issues separate. That's a topic for another rant.

Can OpenWRT save my Netgear DGN2000?

I made the mistake of buying a Netgear DGN2000 ADSL2+ modem when I first got ADSL. Not only does it not support IPv6, its WiFi range is pathetic, bridging between WiFi and ethernet seems to die after several days of operation, and under heavy load the device hard-crashes, requiring a reboot. I suspect the crashing might be due to poor thermal design and perhaps my specific device but clearly the software is not also without some blame. Given my experiences, I would NEVER recommend this device. But, now that I have one that I can't return, I'm going to document the process of installing OpenWRT to see if I can give this thing a second lease on life.

Python's Global Lock

For years I've heard (and occasionally been part of) religious Perl vs Python language debates. I've always been firmly on the Python side of the fence. While I have made some great use of perl in my time, I love being able to actually re-read my code without having to decipher it first. Python has always seemed like a much easier language to work in to me.

So, given my firm stance in this debate, I'm a little shocked after seeing
Carl Karsten's video. Want to know why Ctrl-C doesn't work in threaded python apps? Why threading in python can actually slow you down? Why not all Python C bindings are not born equal? If you have even a general interest in python, I greatly recommend checking out this talk. It'll shake your faith. :)

Blog moved

Maintaining my own wordpress instance was getting tiresome so I've switched to blogger. Article information should still work just fine but comments are missing. My apologies!

Building Python2.5 for Ubuntu 11.04

I run a few appengine instances and so I was quite disappointed after upgrading to Ubuntu 11.04 to find no pre-packaged versions of python2.5 (the version on which appengine is currently based). Python 2.7 is not compatible with various parts of the appengine dev server. After a bit of playing around this is the easiest way I found to get everything working that I needed. YMMV.

edit the top of dev_appengine.py and replace

#!/usr/bin/env python

with

#!/usr/bin/env python2.6

Python 2.6 is not ideal but it seems to do the job. If you want to go the extra mile (as I did) and install Python 2.5.5, you need to manually add ssl and address some bad assumptions python makes about sqlite3's library locations...