mrblack.co.uk weblog

Prelude
This post is just my head throwing ideas about. Non of which are fully formed, nor fully investigated.

Intro
Today Google labs spitted out an early “plugin” for Internet explorer and firefox to make websites viewable and interactive offline. This isn’t entirely new as Apollo offers something similar. However, this is interesting considering Google is one of the kings of webapps. With Gmail, Google Calender and Google Docs all obviously the target for this technology I doubt that it would fail spectacularly.

Very brief and rubbish Gears overview
First off, there are two main components to Gears. The webserver and the database. The webserver is pretty simple to get our heads round, it essentailly caches pages that us as developers tell it to. For me the most interesting parts is the database.

The Gears Plugin that the user installs includes a fully relational SQLite database. As developers we can interacted with it by passing SQL commands via javascript. Yes thats right… we interact with it by using javascript code such as

var rs = db.execute('select * from Demo order by Timestamp desc');

The benifits of this are pretty interesting. For example webservers technically don’t have to store information at their end, ever, just relying upon the users own store.

I feel however the most use of such an app would be for something like gmail, where when you are offline you write and send an email with gears facilitating the actual sending of the email when the internet becomes available.

So how would Rails fit into all of this?
I love writing webapps in rails. Its quick, easy and Ruby is lovely to write. So could developing a Rails app with gears functionality be a possibility without having to drop into insane javascripty-SQL-type goo every time?

The answer in my opinion is yes, but it won’t be easy.

Here are the major barriers in the way to making a my ideal acts_as_gears type plugin.

1. The plugin should be able to convert Ruby into javascript for offline controllers and model interactions. A massive undertaking (possibly recreating active record in javascript.. shudder).
2. Creating the user’s local database and linking each column with the live database columns would be essential if we wanted to do any sort of syncing.
3. The automatic creation of database syncing methods in javascript and ruby.
4. Creating a seamlessness of data origin in views. For example, a table of information should be able to be printed out on one page irrespective if data came via the live or local database.
5. The views should only have to be writen once, with of cause different javascript payloads.
6. Loading and unloading of data needs and state management.

Anyway that’s what’s on my mind right now, a half baked whirlpool of rails, javascript and gears.

Reading
Google gears
Adobe Apollo
Firefox offline info

I’m currently making an app that will spit out pdf reports all over the shop. These reports will get kinda big, doing loads of database calls and rendering images. As such it could take 15 seconds or more to render. Making the reports on the fly therefore could be a horrid user experience with unexpected waits and halts (and a nightmare for a server admin).

So my first attempt was batch the process, as the majority of the data that the pdfs render only changes about once a day. The batch process would go through our database and render all the reports and dump them out to file. User access then would be instantaneous. Wonderful!

The problem is that some bits of data can change at any time. For example these reports display a contact name and email address at the top. So if that changed, the pdf wouldn’t reflect this. Not very professional.

To combat this I added a spooling table, where if some data changed a request was added to a table. A script would then run every minute that would process the pending requests.

This is a slightly dumb approach, I know. What I need is two threads. So when the user updates the contact information, one thread would render the html page as usual and the other to render the pdf. The benefit being that the user doesn’t have to wait for the pdf thread to complete.

Ruby can deal with threads but finding information about doing a task like this is tough, let alone with Rails. Any help out there?

Readings
Rails wiki on threads
Ruby Reliable Messaging Gem
Reliable Messaging Rails example

Well Spam karma 2 is really doing the job at keeping spam comments at bay. So far it has killed spams 100% successfully ie, non got through the net and no real comments were killed. So for anybody running Wordpress I wholeheartedly recommend it.

Wordpress is cool. It offers a simple way for anybody to play around with blogging on their site, however, as with any piece of software which is widely used it can be open to exploitation. Wordpress, due to its popularity, has attracted comment Spammers, very annoying. I am trying to work out if this type of spam is spawned form virus infected “zombie” machines, not sure yet. Anyhow the most effective thing to overcome it is WP-Hashcash, which requires a browser with javascript working to make a comment, something that Spam bots usually lack. So far it has eliminated all my spam, of which I was getting about 20 comments a day.

Update: Well it seems that it isnt working anymore, recently i am get about 40 to 100 spam comments a day.

Update 2: Giving Spam Karma 2 a go.

I have finally set up a section of my site which shows how many times I play each artist. I wasn’t happy with other solutions so I built my own based on the NowPlaying iTunes plug-in and a whole load of scripty goodness. It can be found here and is linked to below the currently playing album art to the right of this post. (for those reading the RSS/Atom feed).

This is great news, you can send a email containing a photo directly to flickr from an O2 phone. This means that my previous work should become redundant. However flickr still doesn’t support the attached text message that you may slap with your photo. In my script I have implemented this, and here is how.

Essentially this is what needs to be done.

1) Use wget as before to grab
http://139.2.165.14/mms2legacy/showMessage2.do?encMmsId= ID NUMBER
2) Run through the downloaded page and look for the line containing the string ‘div style=”color: #000000;” ‘
3) When you get to that line, the next line will be blank, skip through to next line.
4) you are now on the line with the text message on, copy that to a variable.
5) remove the tabs and spaces from before the message text

Now you have the message attached to the photo. I would just add this to the email text, flickr would add this to the photo description. Here is a link to the code I used to extract the message written in php. Sorry for the shitness in code, but it works. I hope this is of use to somebody.

Ok I have the mobile to flickr thing almost sorted, email me if you want to be a tester by the way, O2 UK only (andy DOT shearer AT gmail.com). I will leave that to run for a few weeks before I set up an automated service where people can sign up for it.

A guy on b3ta asked me why did I do this considering flickr has a great mobile upload facility on its mobile site? Well to be honest I should have searched for this sooner, flickr’s mobile site is pretty good. If I hadn’t already made my little work around I would have no doubt used this, but having said that I would point out the clicking of ‘send’ on a picture then selecting the email address is much simpler and quicker than playing around with WAP. Plus later on I might add to it so you can send the photo to multiple address, not just to flickr.

Leading on from this post, I have implemented a way of sending an email from my phone to Flickr containing a full sized image. With thanks to James at Purple Cloud (my host) who installed a newer version of wget for me, everything is now working a-ok. I can now send an email to a pop3 account (based here at mrblack.co.uk) and every minute that account is checked, if everything is fine the email is read, photo extracted, and sent on to flickr as a proper email with a full sized photo attached. Yay!

I am currently thinking about making this available for everybody on O2 who wants to send full sized photos emails from their phone to people or to flickr directly rather than sending them an O2 advert. I might also see if I can set this up for any other carriers who fail to offer full sized email photo attachments. As for the code, I will make it available when I have fully tested the system, and cleaned up all the filthy comments.

My flickr stream