Monday, December 21, 2009

Static files with Grails AppEngine plugin

Its been quite a while since my last post, but here is a quick tip:

For my latest project I've decided to use Grails and deploy onto Google AppEngine. There is a nice looking Grails plugin specifically for deploying onto AppEngine ( Unfortunately the plugin has a bug which means that static files don't get reloaded when running in development mode. This is a real pain when most of this project is being written in JavaScript - I needed to restart the development web server after every change!

As a workaround I've installed nginx and configured it to serve my static files directly from the file system (so they get reloaded when changed) and proxy any other request to the development web server.

On MacOS this was pretty simple:
  1. Install nginx from MacPorts (sudo port install nginx)
  2. Edit the example config with the following relevant sections:
    location / {
    #root share/nginx/html;
    #index index.html index.htm;

    location ~* ^.+\.(js|css|png|jpg|gif)$ {
    root /path/to/your/application/web-app;

Wednesday, April 22, 2009

Google Reader mobile client with Rhodes

I've been an avid user of Google Reader for many years now, but I've always felt it was missing a good mobile client. Sure, it has a mobile optimised web interface ( but I've always found this pretty ordinary due to the long waits moving from story to story (even on a decent 3G network). What I've always wanted is a mobile client that caches the stories locally on the mobile device and allows me to skim through them quickly without waiting for a page load for each one.

I recently came across the Rhodes mobile framework which allows "building locally executing, device-optimized native mobile applications" in Ruby. There are a couple of nice things about Rhodes that I really like:
  • Once you've written your application it can be built for a variety of mobile applications - iPhone, BlackBerry, Symbian, Android and Windows Mobile.
  • Development is done in Ruby with a Rails-like structure - the mobile GUI is written in HTML but designed to retain the look and feel of each mobile devices native interface.
  • It provides Rhosync - the back-end server component to help syncronise data between the data source and the local cache on the mobile device.
So I finally decided to attempt to write a Google Reader mobile client using the Rhodes framework. I originally tried this a while ago when version 0.3 was out but found it a bit hit and miss and eventually gave up. When 1.0 was recently released I decided to try again.

The Rhomobile wiki is a good place to start, with a fair bit of information on how to get started. The biggest thing that tripped me up was when generating the model in Rhodes - its not really clear, but the source_id parameter must match the id of the source on the rhosync backend.

The only work I really had to do on the Rhosync server was to create a source adapter for the Google Reader items with the following methods:
  • query - this logs into Google Reader using the user's credentials (the Rhosync server allows users to register themselves, subscribe to various sources and provide their credentials for each), and retreives the most recent set of items from Google Reader which are stored in the @result instance variable
  • sync - the sync method takes the @results from query and simply twists them into the object-value format the Rhosync uses (this is so that it can sync each attribute of each object individually)
  • update - this is called by Rhosync when a mobile client has made a change to the data locally that now needs to be propogated to the backend. In my case the only property I cared about was the read state.
One issue I encountered writing the source adapter was that Rhodes/Rhosync can not handle arbitrary characters in the object id, which meant I had to strip any special characters out of the Google Reader id. However I needed to have access to the original Google Reader id for when I was propogating the read state back to Google Reader. In the end I had to store the original Google Reader id as an attribute in each object, and look it up in the update method (it isn't available automatically because the client only passes the modified attributes).

The actual Rhodes client was fairly simple to implement - it is mostly just editing HTML in ERb. Although, I did a couple of things to clean up the generic application:
  • Edit /config.rb to set the page that the app starts on (by default it will start with a list of all your models).
  • In the controller for the start page I added the following to redirect to the login page when required:

if SyncEngine::logged_in > 0
@items = Item.find(:all).sort {|a,b| b.timestamp <=> a.timestamp}
redirect :controller=>"Settings", :action=>"login"

  • By default, the login view has the password field as type "text" meaning the password is visible as it is being typed, I changed it to "password".
So finally, this is what it looks like on the iPhone:
After all this ... I have decided that I'm going to release the source for this, but I am not going to release it as an application. The reason is that I would need to host the Rhosync server, which is not a problem in itself - after a couple of quick changes it seems to work fine on Heroku (although I think I would need to make some changes to restrict users to just subscribing to apps, not creating them). However, the Rhosync server would need to store the Google Reader password for all of the users which is not something I want to deal with. Perhaps one day Google will deign to add OAuth to Google Reader which would mean that Rhosync just needs to store its OAuth token ... maybe at that time I'll revisit this. For now I'll probably just use this myself and release the source. (Note that Rhomobile have announced a hosted version of Rhosync called Rhohub which would make this a bit easier, but I still don't want to be responsible for people's Google password)

If you're interested the Rhosync and client source is hosted in my GitHub account:

Monday, March 23, 2009

MythTV on Early 2009 Mac Mini

I've been running MythTV for a few years now on an old early dual core machine, however I've always been annoyed by how noisy and ugly it was. It has always struck me that the Mac Mini would be the perfect form factor for a MythTV box but the hardware has been a bit long in the tooth ... until now. In early March 2009, Apple released an updated version of the Mac Mini, so I finally got around to buying one. This post will serve to document my MythTV setup on the Mac Mini.

(Note - this isn't intended to be a HOWTO, I assume your comfortable working out exact commands on your own. Also, I didn't take notes or anything while setting this up so any commands in here are from memory and may not be exactly correct)

Basically, I bought the lowest-end Mini from the available choices, this is what I got:
  • 2.0GHz Intel Core 2 Duo (I didn't feel it was worth the AU$270 for the 2.26GHz)
  • 1GB RAM (I'll probably upgrade this later from a 3rd party)
  • 120GB Hard drive (even maxing this out to 320GB wouldn't be big enough - see below)
The internal hard drive on the Mini is available up to 320GB from Apple, but I don't really consider this to be big enough for MythTV, so I stuck to the smallest and purchased an external USB2.0 3.5" SATA enclosure to use for recording storage. Ideally, I would have liked to use the Firewire-800 connection available on the Mini for this as I was a bit concerned about the throughput of USB2.0 but I couldn't find a Firewire-800 enclosure for a decent price. The USB enclosure seems to be going OK so far.

For a TV tuner, I plan on getting a HD HomeRun DVB-T but it hasn't been fully tested on MythTV yet so for now I got a Leadtek WinFast DTV Dongle Gold which is a simple USB based DVB-T tuner. (Note that there seem to be many different variants of this dongle with different chipsets, so your experience may differ).

For a remote, I reused the same Microsoft MCE remote (model 1039) that I had from my old MythTV box.

Initial Setup
To get started, I followed the excellent procedure for installing Ubuntu 8.10 outlined by Victor Costan here. (Don't make my mistake and think that you can just format the entire disk and install Ubuntu :-) ... it doesn't boot and you'll need to reinstall OS X from the DVD)

Victor has also kindly added some additional comments I made regarding sound and configuration at the bottom of his post. Make sure to follow those if you want sound and video acceleration.

I also installed all updates before continuing.

At this point its probably a good idea to ensure all the hardware has been detected by the kernel - use dmesg for this. In my case most things were detected automatically (remote, keyboard, network, etc). For sound and video see above. The USB tuner took a bit more work ...

First, install mercurial ("sudo apt-get install mercurial"). Then grab the V4L sources ("hg clone"), compile them ("cd v4l-dvb; make") and install them ("sudo make install"). If you now unplug/plug the USB tuner and look in dmesg you should see that the driver was loaded. Most likely it will still be missing a firmware for the tuner but the dmesg output should give some indication of the filename - Google should be able to find it for you.

I installed MythTV from the Mythbuntu website. It should automatically install MySQL, and any other requirements. I also really like the Mythbuntu Control Centre. I enabled my Mini as both a Primary back-end and a front-end, and use the control centre to enable automatic login as my myth front-end user.

After enabling the Myth components run the MythTV back-end setup to configure your tuner card, channels, storage, etc.

I don't stress too much about the guide data configuration in the MythTV Backend setup, as I install shepherd straight away (it is without a doubt the best way to get guide data for Australia - really easy to configure, grabs data from multiple sources and reconciles them all together). Follow the installation instructions to get it going.

That should pretty much be it .. fire up the front-end and give it a try.

After getting this all going and starting to use it, I have been noticing some random segfaults in most of the MythTV processes (front-end, back-end, and mythfilldatabase). I can't work out if these are related to the USB tuner, the USB HDD, or something else. I've just upgraded to the SVN weekly builds of MythTV 0.21-fixes from mythbuntu (follow the instructions here), and haven't seen the problem again yet, but I will keep monitoring it and update this post if it happens again.

I usually add a fair bit of other functionality to my MythTV box, and am slowly doing the same to the Mini - currently it is running apache (for MythWeb, but also a couple of other things), samba (to share out my iTunes and video libraries) and the Vuze bitorrent client (so that I don't need to leave my desktop PC on all the time).

Hopefully this post has some useful pointers for using a new MacMini as a MythTV box. If you try this and have any issues (or not) leave me a comment and I'll try to help you out.

Tuesday, March 3, 2009

REST Client

Recently I've wanted to access some content from Google Reader. Previously for this I've used both Net::HTTP and Open-URI, both of which work fine. This time, however, I needed to authenticate and grab content from a user account. Open-URI doesn't support the POST method and Net::HTTP seemed like a bit too much work. Enter REST Client:

require 'rest_client''', :service=>'reader',:Email=>email,:Passwd=>password,:source=>'mreader',:continue=>'')
if authresponse=~/SID=(.*)/

xml=RestClient.get("", {:Cookie=>"SID=#{sid}"})

As you can see REST Client makes the REST methods GET and POST (also DELETE and PUT) really easy.

Monday, March 2, 2009

First post

So this is the first post on my new blog.

The hardest thing about starting this blog was coming up with a name for it, because … well there really isn’t a point to this. It will probably end up being a random collection of bits and pieces, so I thought it would be a good idea to pick a random word for the name. After a few minutes of trying to think of a random word without success I decided (in typical nerd fashion) to try and automate the process using bash. With a little mucking around, this is the command I came up with to pick a random word from the dictionary (let me know if you can think of a better way):

awk “FNR == $(( $RANDOM * `wc -l /usr/share/dict/words| awk ‘{print $1}’` / 32768 ))” /usr/share/dict/words

And sure enough, the first word that came out of this was nonspeaking, which I thought was pretty cool … and here we are. Stay tuned for (hopefully) interesting tidbits and learnings from my various half finished projects...