Last.fm – the Blog

UPDATE 25/01/11: After a weekend of work Last.fm is now back up to full capacity. If you’re continuing to experience problems then leave a comment below and let us know. Thank you for your patience.

—————————————————————————————————

As many of you will be aware, Last.fm has been experiencing an extended period of downtime in all user-facing services. A hardware failure has led to one of the most serious system outages we have experienced for a long time, and we are very sorry about the inconvenience caused to our listeners. At this moment everything should be on its way back to normal, but it could take some time for all services to return to a fully stable state.

We want to apologise for this outage, and explain the problems that have led to the difficulties you may be experiencing now.

Yesterday afternoon a fault in a blade chassis in one of our datacentres caused it to break, taking the power supply for its rack with it.

On site teams were unable resolve the fault with this chassis, but were able to restore power to the rest of the rack. Unfortunately, this chassis contained several critical components of the top-level load balancing systems we use to evenly distribute traffic across all of our datacentres.

Load balancing has been redistributed among the remaining centres. As these are running under a higher than usual load intermittent service outages may have resulted, leading to problems across all parts of Last.fm. As a result you might be experiencing difficulty with radio, the website or with your scrobbles.

Because some of our top level DNS services were impacted by this outage it has taken longer than usual for fresh DNS information to propagate. Many of you may have been presented with incorrect DNS information due to caching, despite the workarounds we have been putting in place. Eventually correct information will propagate in its place.

Last.fm’s Operations team has been working since it was first reported to provide listeners with short-term workarounds while the affected hardware is replaced or repaired. Our first priority has been to ensure that user-data is preserved wherever possible.

We’re doing everything we can to ensure that the site is operating for as many people as possible while the hardware is restored.

Some of you are worried about the status of your scrobbles; all scrobbles that are making it through are safe, and client caching should ensure that any that aren’t are queued and will be submitted correctly once the service is fully restored. These will appear as normal once the faults have been repaired.

We’d also like to apologise for not communicating more about the problem – as I’m sure you can appreciate our priority has been getting the issues fixed.

In the meantime we will be updating status.last.fm and the twitter account @lastfm as soon as we have more information.

The moment many of you have been waiting for arrived yesterday: Last.fm revealed the top ten of our Best of 2010 list.

This year Ke$ha crashed into the number one spot, stealing the crown from 2009’s winner Lady Gaga. She’s proven practically untouchable in the race to the top too, clocking up an astonishing 15.9 million scrobbles over the year with her January 2010 release Animal.

She took over 4 million more than the runner-up, Mumford & Sons, and if you look at the stream graph of the most-played music month-by-month you’ll see she’s been a permanent fixture.

In my opinion, however, the even more exciting bit is that we’re also making the underlying data available for anyone to download.

I was one of the privileged people here at Last.fm who had to shed lots of blood, sweat and tears to get the data for the Best of 2010 list generated, so I’d really hate for it to only be used by us. There’s now plenty of opportunity for you to create different views on this data by visualising it in new and clever ways, combining it with other information obtained via our API or even from entirely different sources. I simply can’t wait to see what other people will do with it.

To get you started, we quickly whipped up a balloon race style data visualisation. The “Best of 2010 Balloon Race“ is based on the Best of 2010 data, showing you how much the top artists in our chart have risen (or fallen) in popularity since last year, as well as contrasting the overall number of listeners for each artist against their rank (number of listeners for their 2010 release).

We even went all the way and made a personalised version of this visualisation available to subscribers on Playground. As usual, we invite you all to send us feedback or join the Playground Group to discuss.

If you’re interested in working with data and consider yourself capable of doing useful things with large amounts of it, then you should come work for us. Maybe you’ll be the one showing off the Best of 2011 data next year!

Today we reveal the first part of Best of 2010, a countdown of the year’s most-played music, all based on the tracks you’ve been scrobbling.

Over the next three weeks you’re going to be surprised, delighted and horrified in equal measure by the names in the top forty, whatever you’ve been listening to.

The technical bits. Like last year we aren’t counting re-releases, live albums, compilations or greatest hits collections; just new albums released this year (so the hardcore Sgt. Pepper’s fans out there are going to be disappointed, sorry). We look at releases between October 2009 and October 2010, but include plays right up until November: that’s so we give everything a fair wiggle room before locking the chart down.

Finally, we base it solely on your scrobbles – no ‘Love’ or ‘Ban’ bias – and we don’t take deletions into account; it might be a guilty pleasure you don’t want people to know about, but that still means you scrobbled it.

As with last year, we’ll be releasing the data and a few more fun features in the final week, so you can really dig into your own “Best of”. In the meantime we’d love to find out what your top ten looks like, so you can let us know over on the group page.

We reveal #20 – #11 next Wednesday, and you can find out who the ten most scrobbled artists of the year are on 15th December.

Big thanks to everyone on the Last.fm Team, including guest writer Chal Ravens and our friends at creative studio Rehab.

Find out who’s at #40 – #21 right now…

You might just remember that back in the summer we launched a new recommendations page dedicated to full-length previews offered by up-and-coming and independent artists on Last.fm. While lots of you have been enjoying this as a way to connect with brand new artists, we’ve been working hard to build a much cooler version.

Our new Recsplorer (recommendation explorer) lets you listen to full-length previews by a mix of new artists that we think you might like, based on your scrobbles, loves, tags, etc.

If you hear something you like, you can ask Recsplorer to find you more tracks like the one you’re listening to. Or, if you fancy a change, Recsplorer will give you a new set of varied recommendations.

Last but not least, you also have the option just to click once, lean back, and listen to a sequence of full-length tracks recommended for you, direct from the artists who made them.

Enjoy exploring your new recommendations!

————————————————————————————————

EDIT 16:35 Due to demand you might find that it’s not working on launch – we’re just fixing that now. Make a cup of tea and check back in a little bit.

EDIT 16:41 It should be fixed now. Sorry about that.

First up, an introduction: I’m Matt, Last.fm’s Data Griot. My job is to flag up some of the stuff Last.fm can do, both with our data and as a music discovery service.

Something we all loved seeing a couple of weeks back was the Office Music Democratizer from the folks at BREAKFAST. I got in touch with one of the creators – Zolty – and asked him to write a few words about how it came to be built…

BREAKFAST is a crew of engineers, designers, coders, inventors and all around creative folk. It’s this mish-mash of skills that makes us unique in that you don’t usually find this mix of people under one roof. What it means is that we can create crazy products and experiences that easily span from online to the real world.

Sometimes it comes in the form of a Kinect-like experience, and other times it means building a bike that can share its thoughts and feelings online.

Our latest toy – the Office Music Democratizer – is an example of how we keep our tools sharp. We’re always looking for annoying little problems that we can solve quickly in our extra hours.

Like many design offices, we explored a slew of options to solve the enjoyed-by-all office jukebox. Last.fm seemed to be the answer, but going over to a computer to rate a song felt a bit un-inviting. So, we thought “wouldn’t it be great if anyone could just smack a pretty button on the wall instead.”

The Democratizer is a fully working prototype that hangs in our New York office – as seen in the video. We’ve had a good deal of purchase requests, but aren’t planning mass production anytime too soon. Rather than a big production shop, we think of ourselves more like the tailors on Savile Row – hand-made, custom things for those who appreciate them most. We’re much more excited about making our next great toy rather than dealing with mass production… at least for now.

If you’ve seen something else that does cool stuff with Last.fm, be it with the API or with plastic and glue, then drop us a comment below.

A few months ago I walked into a meeting with Microsoft and came face to face with a little ET-shaped camera. It tilted its head up to look at me and I did what anyone in my position would have done: I waved at it.

Not quite believing it my hand started moving across the screen behind Kinect, and things started moving. Within a few moments half of the office were crowding in behind me, wanting to have a go. Fast forward to today and now it’s your turn.

What’s Kinect, you might ask?

While everyone is comparing Kinect to Minority Report, we’d rather bring up Total Recall. Not just because it has the best Arnie line ever, or because it’s part of our Laserdisc collection, but because there’s a scene that fairly accurately depicts how Kinect works:

Kinect bounces an infrared beam around the room, captures this with a camera, separates your body from the background and converts this data into a skeleton. This is the basis for your avatar. (What’s even cooler is its skeletal recognition is smart enough to tell you apart from someone else, even if you try and fool it with a mask. Trust us – Jonty tried)

Using gestures, you can control what’s happening on-screen.

How the Last.fm app works

Last.fm for Kinect is a light-weight version of the radio app that uses gestures to control the player. It lets you browse and play Last.fm Radio with little waves of your hand. While more intensive actions such as scrolling through artist info are still best suited to a controller, it’s really cool to have a new way to interact with Last.fm.

But the voice commands are the best. Want to skip the track? Just say “Xbox, Next!” It’s probably the closest I’ll come to being on the Enterprise, or having Johnny 5 as a DJ.

If you’ve got a Kinect then start playing and let us know what you think!

If you listen to Last.fm radio through the Last.fm desktop client or on an Android phone then you may have noticed that we have just launched a new station: Mix Radio. Mix Radio is inspired by the idea that the best music discoveries can sometimes be made close to home.

While the pure science of music recommendation puts a heavy emphasis on novelty, Last.fm’s incomparable store of data about real listening preferences – as well as our own experience as music lovers – convinced us that it would be interesting to try a different approach. We noticed that listening to all-new music can be a bit heavy going. Similarly, just listening to your old favourites sometimes isn’t what you want either! A few shakes of the test tube in Last.fm’s radio and recommendations laboratory (known internally as the MIR or Music Information Retrieval team), and Mix Radio was born – a station that’s exactly that: a mix of the music you already know + some new recommendations!

The tracks you’ll hear on Mix Radio have been selected in three different ways: some are brand new recommended tracks; others are tracks that you haven’t scrobbled before, but by artists that you know already; and the rest are simply tracks that you know already. We think that the combination makes a really enjoyable new way to explore Last.fm’s recommendations, based as ever on your scrobbles, tags, loves and so on, in the context of more familiar music. Please let us know what you think – we want your feedback as we add Mix Radio to the website and the other Last.fm apps in the near future.

Last but not least, if you’re interested in building the next generation of Last.fm radio and recommendation services, we’re hiring!

Hello people. I’m cms, and my job here at Last.fm is looking after the databases. Much of the time I’m involved with operational running of database servers, designing and optimising SQL queries, and scaling work on our relational database clusters. Every now and then though, I do get an opportunity to poke around in the Last.fm dataset and explore some of the interesting relations.

I recently re-discovered the seminal album ‘Spirit Of Eden’ by ‘Talk Talk’ (haven’t tried it? You really should, it’s magical), and I’d been giving it quite heavy rotation. This prompted a comment on my profile by one of our lovely users, who suggested making a playlist from artists whose names consisted of repeating word patterns. This idea appealed to me, but off the top of my head I could only come up with a paltry half-dozen candidates. Surely there were many, many more. If only there was some kind of database nearby I could query…

We keep our main catalogue data in a PostgreSQL database. PostgreSQL has a nice set of extended string operators, including quite comprehensive regular expressions support, which would be useful for an ad-hoc query like this.

Here’s what I came up with initially off the top of my head

select name from artist where name ~* E'^(\\w+\\M)\\s+\\y\\1$' ;

Using the case insensitive regular expression match operator ~* and matching against a string that begins with a sequence of word characters leading up to a word boundary, which I’m capturing as a group, then a sequence of whitespace, then the start of a word boundary followed by the original captured match.

This query worked really well at defining the pattern for repeating names. I was matching well over 10,000 distinct strings. The problem was that we store all the submitted data for artists, and this includes data from a broad range of unverifiable sources. I was getting lots of great artist names in my set, but many of them were bogus; typos, mis-taggings, spelling corrections, and that was just the obvious mistakes.

I needed to come up with a way of filtering the set further. My first iteration was to use track information. Incorrect artist attributions seemed unlikely to have relations over tracks in the catalogue, and I could extend my query relatively easily to take account of prolificness like so.

select count(1), a.name from artist a, track t where a.name ~* E'^(\\w+\\M)\\s+\\y\\1$' and t.artist = a.id group by 2 order by 1 desc;

This got me a shorter set of artists (8000 odd), with some ordering. I could see that recognisable artist names (hello Duran Duran !) were sorting towards the top. However, ordering by catalogue volume still wasn’t quite right. Ideally I needed some kind of popularity weighting. Unfortunately we don’t store any scrobble data in the PostgreSQL catalogue schemas.

However we do store scrobbles, alongside exported catalogue information in our Hadoop cluster. Although I have been known to write Java code in the past, I’m mildly allergic to it. Luckily for me we have a Hive interface to Hadoop. Hive offers an interactive query language over Hadoop that is closely modelled on SQL. The only stumbling block remaining was porting my regular expression over to use Java syntax.

Here’s what I ended up with as a hive query:

select meta_artist.name, overallplayreach_artist.reach from meta_artist join overallplayreach_artist on meta_artist.id = overallplayreach_artist.id where meta_artist.name RLIKE '^(.+?\\b)\\s+\\b\\1$' and meta_artist.correctid IS NULL and overallplayreach_artist.reach > 50 order by overallplayreach_artist.reach desc ;

Joining against some “playreach” data to give a weighting according to rough popularity. My original SQL query took 17 minutes to run, on a fairly beefy database server. The hive query took less than 100 seconds to return, running across the entire Hadoop cluster. Awesome.

Without any further ado, here’s the top 10 results, roughly ordered by artist popularity.

I’ve created a tag artistartist, and tagged some of the entries already.

The full list is available here. There might well still be some rough data in there, I haven’t particularly sanity checked it by eye.

If you too would like the chance to play with Last.fm’s vast amounts of data and join our team, check out our job openings.

Tweet

About 6 weeks ago I started a short internship at Last.fm. For my project I wanted to explore Last.fm’s data to learn how listening preferences vary with the listener’s age and gender. Apart from the science, the most important thing I found is that you can make awesome plots with this information.

I started by making a chart to show what kind of music you “should” be listening to if you really want to fit in with the most common artists in your age range and gender:

The sizes of the artists’ names indicate how popular they are, while their position shows the gender mix and average age of their listeners. Based on the positions of the larger names, it’s already obvious which age category is most common amongst Last.fm users.

So, you can now use this plot to decide which music you might want to listen to. For example, if you are a healthy young male in your early twenties, you probably should listen to bands such as Iron Maiden and Metallica. Gorillaz and Radiohead might just be acceptable. If you get older you can then switch to artists like Neil Young and Genesis. It’s all quite obvious really.

Of course, when I realized what nice plots I could make, I tried it on several other types of data as well. Tags for example:

You can use it in the same way as the previous plot. Apparently females like using band names as tags (Super junior, McFly), while males prefer finding lots of ways to say the same thing (metal, jazz). Most importantly we have just used science to prove that men don’t listen to much k-pop.

Obviously music is the most important data that’s available at Last.fm, but there are some other profile items that can be interesting too. The words used in the ‘About Me’ section on users’ profile pages might even lead to the most interesting plot of them all:

There are actually so many fun facts about this plot that it’s just best to check it out yourself. The most obvious one is which hobbies you “should” have depending on your gender. Or you can find out at what age you should retire.

I used all of this to create a fun new playground demo that enables all Last.fm users to compare themselves with their friends. This is the plot for the data and recommendations team for example:

We’ve even thought of those of you who like to print their visualisations as a poster by providing a bigger PDF version that has more artist names on it.

Hopefully you’ll enjoy this demo as much as we did. In any case, we’d love you all to let us know what you think.

A bit less than a year ago we launched the VIP zone on our Playground, with the promise that we would keep adding fancy visualizations to it as a special treat for our loyal subscribers. We already delivered on this promise with the personalised Listening Trends and Music Universe visualisations, and today we’re delivering some more.

This time around we got inspired by the WOMRAD 2010 paper Rocking around the clock eight days a week: An exploration of temporal patterns of music listening. By applying some nifty circular statistics formulas, we managed to create an interesting new visualisation that shows at what times of the day a given Last.fm subscriber has been listening to music over a certain time period. Here’s an example:

In this case we’re looking at Norman‘s listening behaviour for the past 90 days. Red and green represent weekdays and weekends, respectively, and the longer the hand the more the listening was focused around the time to which it points. Generally speaking, Norman seems to listen to music at later times of the day in weekends than on weekdays, and his listening seems to be less restricted to certain hours in the weekend. It’s also quite clear that he tends to listen to music from 10AM to 7PM on weekdays, which isn’t that much of a surprise since those are our working hours here at Last.fm. He accidentally left his radio playing overnight a few times though, as indicated by the smaller red bars from 8PM until 9AM.

Our beloved LAST.HQ‘s listening clock for the same time period is a more extreme example:

Since we use this account for the reception radio in our offices — which plays pretty much 24/7 — the listening is spread out across all times of the day, leading to two hands that are extremely tiny and cute.

We very much hope you’ll enjoy playing around with this new visualisation, and that some of you will point to particularly interesting listening clocks or discuss potential improvements in our Playground forums. Meanwhile, we’ll start working on the next one!