mrblack.co.uk weblog

I would love to have a single place where all my music and videos are stored rather than having a bunch of stuff hanging around on laptops, PCs and iPods. So I bit the bullet and began Le Grande Media Server project.

First thing I wanted to do was to set up a music server, so this post is basically stage one. Now, as specs go, we all use iTunes at home and I use it at work, so ideally my music should be served to that piece of software. That about it for specs..

The server I’ll be using is my old desktop running a P4, 512mb ram and about 130gig of hard drive space. Of cause I could throw more hard drives in if need be. This is more than enough power for this task, but meh it was just sitting there.

Step 1 : Install Ubuntu
I chose to install the latest beta of Ubuntu, feisty fawn, as I was rather eager but I’m sure all this will work with Dapper Drake. Feisty really is an impressive operating system, much better than the XP it replaces on my old machine, and, thankfully, makes this task relatively simple.

Installation is pretty straight forward, and documented best over at www.ubuntu.com/

Step 2 : Install mt-daapd
iTunes uses Digital Audio Access Protocol (DAAP) to share content. Now this is ideal for us, as some clever chaps have reverse engineered the protocol and made mt-daapd, an open program that uses this protocol to share a media directory to our local network. So in theory our little Linux box can be seen by any copy of iTunes on our local network.

First of all install ID3 tag support (so mt-daapd can read mp3 files)

sudo apt-get install libid3tag0

Then for mt-daap itself. Its best to grab the latest debian package rather than apt-get to be sure of a working copy (iTunes changes so often).

wget http://prdownloads.sourceforge.net/mt-daapd/mt-daapd_0.2.4-1_i386.deb?download

and then install it

sudo dpkg -i mt-daapd_0.2.4-1_i386.deb

Then, once installed, you need to edit /etc/mt-daapd.conf with whatever you use (vi, nano, gedit, I don’t care). Within this file, point the server to your media directory then restart mt-daapd.

sudo /etc/init.d/mt-daapd restart

Now on another machine you should see your share appear in iTunes and will be able to play music! If not, check to see your Ubuntu box has tcp port 3689 and udp port 5959 open and that iTunes is actually listening for shared music.

Thats sorted me and my family out at home. Hurrah.

Step 3: But I wanna listen at work! *throws tantrum*
DAAP was never supposed to work over the internet. There is a way though, and below are instructions on how I did it for a mac. For windows, it will be possible with putty, but I’m not sure how to complete the entire process.

However, for Ubuntu make sure ssh is installed on your server.

sudo apt-get install ssh

Once thats out of the way check to see if port 22 (the ssh port) is open on your Ubuntu box’s firewall (firestarter is a nice tool for this). Now we want to be able to get to this port from outside our network for that on-the-go-music-server-accessness. So set up port 22 to forward to your server on your modem/router.

If all goes well you should be able to type the following on your Mac and get to your Ubuntu server.

ssh your_user_name@your_external_ip

If not, check your router/modem firewall (or that your pipes aren’t clogged) .

Now we want to set up the tunnel between the server and your shinny mac so the music can flow to you as if you were on your home network.

ssh your_user_name@your_external_ip -L 3689:localhost:3689

Once you have put your password in and agreed to accepting keys, the tunnel is now set up. This is not it though, for iTunes to see the share you need to broadcast its presence. To do this I used Network Beacon, which is set up like in this picture.

Once that chap is up and going, you should be able to see your media server in iTunes.

Hope that’s of use. For me its ace, all that music, at work and at home. Bliss.

Reading
I must point out that most of this was thanks to the following so if you are lost visit these.
Matt’s Blog : Setting up mt-daapd on Ubuntu 5.10
Firefly media server
mt-daapd wiki - ssh tunnel
Ubuntu
Firestarter linux firewall
Network Beacon
iTunes

The Apple TV has not been out for a week and people have opened up the little fella and have had a play with its underbelly.

So far they have ..

• installed a Xvid codec
• installed samba
• installed ssh (and opened up the ports)
• installed a 120gb hard drive

For me this is sweet. I’m thinking about setting up a media server in a wardrobe somewhere to serve daap to the family’s computers and broadcast music to me when I’m at work (through a ssh tunnel.. possibly). This little baby will be running Ubuntu on a low level PC and a few massive hard drives full of content.

Now if the Apple TV could connect via samba to this server and play my ripped DVDs, that would be so nice. It might be even better if it could connect via daap and stream video content from an non iTunes source, then it would make the most of apple tvs already buffering and caching system.

So please hack on!

Readings
Macdevcenter article
appletv hacker
appletvhacks.net
Somethingawful thread on the hacking
Gizmondo covers diy 120gig upgrade
Digg’s “OMG LOLZ H4XXORS”

I’m currently making an app that will spit out pdf reports all over the shop. These reports will get kinda big, doing loads of database calls and rendering images. As such it could take 15 seconds or more to render. Making the reports on the fly therefore could be a horrid user experience with unexpected waits and halts (and a nightmare for a server admin).

So my first attempt was batch the process, as the majority of the data that the pdfs render only changes about once a day. The batch process would go through our database and render all the reports and dump them out to file. User access then would be instantaneous. Wonderful!

The problem is that some bits of data can change at any time. For example these reports display a contact name and email address at the top. So if that changed, the pdf wouldn’t reflect this. Not very professional.

To combat this I added a spooling table, where if some data changed a request was added to a table. A script would then run every minute that would process the pending requests.

This is a slightly dumb approach, I know. What I need is two threads. So when the user updates the contact information, one thread would render the html page as usual and the other to render the pdf. The benefit being that the user doesn’t have to wait for the pdf thread to complete.

Ruby can deal with threads but finding information about doing a task like this is tough, let alone with Rails. Any help out there?

Readings
Rails wiki on threads
Ruby Reliable Messaging Gem
Reliable Messaging Rails example

Amusingly if you downloaded Muse - Knights of Cydonia music video from iTunes a few days ago, you would have received “Hondo” , a western starring John Wayne. So I grabbed it, downloaded about a 1gig of “music video” and chuckled at the cock up.

Today, Apple sent me the actual music video alas not once, not twice but three times. For such generosity I would like to thank Matt, Amy and Lilly from iTunes support . Oh what a lark.

I just hope iTunes will sell these sort of movies at this price (£2) when they finally strike a deal to distribute films in the UK.

Update 22/3/07 12:15 Thanks Kate!! I now have my 4th copy of the music video.
Update 22/3/07 14:16 Thanks Amy!! I now have my 5th copy of the music video.
Update 22/3/07 16:16 Thanks Brian!! Number 6 is on its way.

Readings
Whale Salad blog post
Digg’s “insightful comments”

Modest mouse’s (miceses?) latest album is simply amazing. And how come nobody told me that Johnny Marr has joined them? Well anyhow, Dashboard is the latest single and the new album will be out April 2nd in the UK joyfully titled ‘We Were Dead Before the Ship Even Sank”. Tracks I’m singing on the M6 recently are “Florida”, “We’ve got everything” and “Missed the boat”.

Readings
Official homepage
My-”my eyes bleed due to bad design”-space page
Modest mouse artist Wikipedia entry
“Dashboard” video
Guardian interview

The beeb is splattering its content all over the web utilising the tactic of an ape flinging shit against a wall and seeing what sticks. Not a bad approach if you have the cash, but tech and blog sites seem to have gone crazy over the latest BBC turd winging its way to the internet wall. The forthcoming iPlayer.

The iPlayer is an online TV catch up service and might well be implementing Microsoft’s Digital Rights Management (DRM) technology. This will mean it won’t work on a Mac, Linux or whatever, you need Windows and/or a device vetted by Microsoft. I’ll be a little peeved if this doesn’t change after the public consultation.

However, its worth pointing out that the ape that is the BBC has had, so far, a very successful podcast trial and are now putting some effort into video podcasts. At the moment these video podcasts seem to be condensed versions of current shows, such as the BBC Breakfast takeaway, Heaven and Earth extra or the Blue Peter podcast.

These podcasts have no DRM associated with them.* They will play on anything from macs to zunes, without any ridiculous licensing, phoning home or any such silliness that DRM suffers from . I can even play around with the feeds and create a crazy new website if i keep to their rules. Hopefully this turd will stick, as its a much better way of getting BBC video and audio on the net compared to the crippled iPlayer.

So how come I actually think the iPlayer isn’t the such a bad thing considering video podcast are so much better?

Well the iPlayer might well have all that DRM silliness, but this sillyness can act as a security blanket for content providers. The beeb might be able to use this blanket to coax its content providers to put their content online, which is better than nothing at all.

Then, I hope, the BBC will continue developing its video podcasts. If done right people will want to have BBC podcasts on their phones, televisions, mobile video players and computers. If thats a big success (big ‘if’ I know), perhaps the other content providers will be able to see the light and want a piece of that yummy DRMless pie with all of its platform independent fillings.

I therefore hope the BBC can lead the way here. Get the content providers to use the iPlayer to test the internet water with arm bands on. Whilst the BBC allows their own video content, that without licensing or other legal issues, to be released as podcasts and not give up (like it did with the ogg streams).

*podcast is an odd word. I could replace this line with “The video files referenced as an enclosure in an RSS feed (which when looked as a whole become known as a podcast) have no DRM associated with them” I suppose.

At work I develop Ruby on Rails applications on my wonderful little Ubuntu servers using apache2, fcgid and mysql5. They work a treat, fast responsive and is basically an easy stage to bounce off new apps. So when I was thinking about my own apps I knew I wouldn’t be able to get the same spec machine and redundancy for what I can afford, but I thought I would give dreamhost a try, I should be able to get something near the same speed for test purposes. They were cheap, seemed to offer a massive range of options and copious amounts of bandwidth.

How wrong I was. As I was developing my still unreleased super cool app I noticed that the dreamhost email servers were going up and down faster that a 10 year old with a yo yo. Odd, very odd. Then I uploaded my app tonight and holy shit was it slow to upload. Then I began to use her and yes.. it sucked utter balls, too slow.

There seems to be a few factors responsible for the shit speeds. First of all is FastCGI. FastCGI should make my app wiz around like mad, however compared to CGI its basically the same on dreamhost, which suggests a hardware or priority shitness on the servers.

Then there could be database access speed slowing everything down. The database seems to be located on a different server altogether, again noted by others.

I thought it was just RoR on dreamhost that was slow, but no it wasn’t just that. Even if you access via ssh it all seems very sluggish, even doing a “rm -r” on a directory that contained about 20 files took about 10 seconds. Thats just stupid and should be sorted.

Am I expecting too much? A page that takes less than 10 seconds to load?

Perhaps dreamhost is fine for static pages, images and nothing else. Sigh.

The other night I popped along to the North West Ruby user group, which with some others we bounded around the idea of teaching Ruby in schools and universities over a few beers. Its an interesting thought worthy of exploration.

I remember learning my first programming language at university, it was Java and took so long to build anything. Hours and hours building applets using odd tools to plot a graph and what have you. It was tough but only through my dogged determination I got through it. If we compare this to Ruby on rails where running “scaffold” you can have a working, albeit, simple website engaging with a database. Such speed and ease into programming could prove to be a decisive revelation to the timid and fickle fresher under pressure to use their student loan on the only subject they can be confident in succeeding in. i perhaps would have spent more time working rather than drinking beers and experimenting in .. errr.. stuff if I was taught Ruby.

Its not just the speed and the confidence that this brings but the syntax of Ruby is more ‘user friendly’ than java. As was noted at the group meeting when chatting about Java and “who remembers what public static void main actually means?”. Keeping the syntax simple could leave lecturers and students to concentrate on more concrete and universal ideas of computer science such as OO design and usability something which we all could benefit from.

As for how rails may fit into this, well, it could be used to demonstrate structured design. Notably the Controller, Model, View structure and how beneficial this is to stability, security, design, extendibility and maintenance of an application. As somebody who has worked on too many PHP projects made by others this idea of separating the functional, database and visual elements of an application needs to be taken up by many more people (yes moodle this means you).

I know some of you might be thinking “What? replacing Java with a scripting language, this is a dumbing down of computer science”. An interesting point but as we all know, languages can be learned very quickly when we know the concepts of programming. My argument is that Ruby is syntactically simpler and quicker which in turn make it more engaging for our student to actively learn as compared with Java whilst still offering the same broad range of conceptual ideas.

Of cause this is all arguable, Ruby is still very new and may, as with java, may fall by the weigh side. But it must be clear the UK needs more people engaging in computer science to sustain our dependence on technology. So its important to make the step from novice html builder to programmer an easy an exciting one, java failed to do that with me, but I feel Ruby can.