Last.fm – the Blog

Last week I attended The Great Escape which is a fantastic festival and conference in Brighton. In a previous life I built the Great Escape website for the Mama group and I’ve been to the festival for the last two years. I love it.

If you have never been to TGE, then I’d highly recommend you check it out next year. Spread out over 30 venues it is a great place to listen to new and upcoming bands as well as some more established artists. The vibe of the festival is chilled and when the sun shines, there’s no better place to be than Brighton.

On Friday morning I attended a conference about New Music Radio and listened intently about how curated radio is changing with an interesting panel of podcasters, radio entrepreneurs and DJs, one of which kindly let me crash at his family’s house (thanks Darren!) for the duration my stay.

When the debate was opened up for questions from the audience Duncan Geere from Wired asked how the panel see services like Last.fm. The response from Matt Young who runs Song By Toad was interesting and although I know he has a vested interest as a ‘radio presenter’, what he said was simply not correct. And I quote:
“I really think Last.fm and Pandora are fucking pointless. Whenever I log in to Last.fm, it just plays me 20 songs I already know. There’s no way to listen to anything new”.

Well Mr Young.. have you not checked out your personalised recommendations? Have you missed the new release recommendations, have you checked out Discover? Ever tried Your Recommended radio? browsed music by tag or tried multi-tag radio? There are SO many ways to find new music via Last.fm that the potential to discover tracks and artists you like but haven’t yet listened to is enormous. Better yet, we can offer a choice to account for the needs of the people you seem to have overlooked… people who WANT to listen to music they know and like.

The point was well made by Mr Trick – which thankfully added a reality check to the statements from others on that panel – and that is… not everyone has a desire to have a DJ ‘throw them a curve-ball’. They actually want to listen to, and be recommended, music that we know they will like.

The fact that new, upcoming and independent artists have uploaded almost 4 million of their tracks to be discovered in and amongst the entire catalogue of almost 15 million streamable tracks and the fact that our Last.fm users have scrobbled and added rich data for more than 50 million artists should tell you that the scope to discover new music with Last.fm is huge.

It is interesting to note that the poll running on Wired’s site currently shows that 70% of people (at time of writing) are listening to less traditional radio because of Last.fm.

So, Matt Young, please come on over to Last.fm anytime and join us for a techmosis session, perhaps one Friday. We’d love to show you how good Last.fm really is and why millions of people all around the world know for a fact that we are doing here is far from “fucking pointless”.

About 6 weeks ago I started a short internship at Last.fm. For my project I wanted to explore Last.fm’s data to learn how listening preferences vary with the listener’s age and gender. Apart from the science, the most important thing I found is that you can make awesome plots with this information.

I started by making a chart to show what kind of music you “should” be listening to if you really want to fit in with the most common artists in your age range and gender:

The sizes of the artists’ names indicate how popular they are, while their position shows the gender mix and average age of their listeners. Based on the positions of the larger names, it’s already obvious which age category is most common amongst Last.fm users.

So, you can now use this plot to decide which music you might want to listen to. For example, if you are a healthy young male in your early twenties, you probably should listen to bands such as Iron Maiden and Metallica. Gorillaz and Radiohead might just be acceptable. If you get older you can then switch to artists like Neil Young and Genesis. It’s all quite obvious really.

Of course, when I realized what nice plots I could make, I tried it on several other types of data as well. Tags for example:

You can use it in the same way as the previous plot. Apparently females like using band names as tags (Super junior, McFly), while males prefer finding lots of ways to say the same thing (metal, jazz). Most importantly we have just used science to prove that men don’t listen to much k-pop.

Obviously music is the most important data that’s available at Last.fm, but there are some other profile items that can be interesting too. The words used in the ‘About Me’ section on users’ profile pages might even lead to the most interesting plot of them all:

There are actually so many fun facts about this plot that it’s just best to check it out yourself. The most obvious one is which hobbies you “should” have depending on your gender. Or you can find out at what age you should retire.

I used all of this to create a fun new playground demo that enables all Last.fm users to compare themselves with their friends. This is the plot for the data and recommendations team for example:

We’ve even thought of those of you who like to print their visualisations as a poster by providing a bigger PDF version that has more artist names on it.

Hopefully you’ll enjoy this demo as much as we did. In any case, we’d love you all to let us know what you think.

A bit less than a year ago we launched the VIP zone on our Playground, with the promise that we would keep adding fancy visualizations to it as a special treat for our loyal subscribers. We already delivered on this promise with the personalised Listening Trends and Music Universe visualisations, and today we’re delivering some more.

This time around we got inspired by the WOMRAD 2010 paper Rocking around the clock eight days a week: An exploration of temporal patterns of music listening. By applying some nifty circular statistics formulas, we managed to create an interesting new visualisation that shows at what times of the day a given Last.fm subscriber has been listening to music over a certain time period. Here’s an example:

In this case we’re looking at Norman‘s listening behaviour for the past 90 days. Red and green represent weekdays and weekends, respectively, and the longer the hand the more the listening was focused around the time to which it points. Generally speaking, Norman seems to listen to music at later times of the day in weekends than on weekdays, and his listening seems to be less restricted to certain hours in the weekend. It’s also quite clear that he tends to listen to music from 10AM to 7PM on weekdays, which isn’t that much of a surprise since those are our working hours here at Last.fm. He accidentally left his radio playing overnight a few times though, as indicated by the smaller red bars from 8PM until 9AM.

Our beloved LAST.HQ‘s listening clock for the same time period is a more extreme example:

Since we use this account for the reception radio in our offices — which plays pretty much 24/7 — the listening is spread out across all times of the day, leading to two hands that are extremely tiny and cute.

We very much hope you’ll enjoy playing around with this new visualisation, and that some of you will point to particularly interesting listening clocks or discuss potential improvements in our Playground forums. Meanwhile, we’ll start working on the next one!

And that’s a wrap!

Last week we revealed the final top ten in our Best of 2009 list, with — yup, as some of you guessed on the group shoutbox three weeks ago — Lady GaGa in top place with her album The Fame. With roughly 6 million more scrobbles than #2 artist The Killers had with Day & Age, she definitely earned her spot! Perhaps unsurprisingly, she’s also top of the chart for most unwanted scrobbles thanks to the love-it-or-hate-it “Poker Face.”

We also put up a data download for those of you interested in remixing or visualising the 2009 data. We’d love to hear about your creation too, so don’t forget to post about it in the web services forum when you’re done.

All the scrobbles that’re fit to print

To have a bit more fun with the dataset ourselves, we traded pixels for picas at Last.HQ for a week and created a newspaper edition of the list! In addition to the Top 40, the newspaper includes some local data visualisations for London and New York based on listening in those cities. We were able to make this thanks to the lovely folks over at Newspaper Club, a new London start-up dedicated to helping people print their own newspapers.

If you’re out of our newspaper delivery squad’s range, have no fear: here are downloadable A3 poster versions of the newspaper centrefold visualising month-by-month listening trends for both New York and London. Or, if those cities don’t mean much to you, why not grab the activity page instead? It’s got comics, a music crossword, and our favourite — a crabcore connect-the-dots.

Scrobbling in the name of

Moving from one set of charts to another, the race for UK Christmas #1 has been a hotly debated one this year. For the last 4 years the #1 spot has been taken by the winner of X-Factor, a reality show much like Pop Idol, driven by the infamous Simon Cowell. Bored with the years of “X-Factor monotony”, a husband and wife team set up a Facebook campaign to get Rage Against the Machines’s Killing in the Name to number one. People started buying the single en masse, and thanks to our exciting modern world of downloads and live updating charts on iTunes, Amazon and other retailers, people realised that the competition was actually achievable. Hundreds of thousands of people who had before ignored the UK official charts were suddenly inundating the BBC Radio 1’s website to get the chart scores, and tuned in to the radio to hear the results first.

Much to everyone’s disbelief, Rage managed to pull way out in front of X-Factor winner Joe McElderry, bringing in 50,000 more copies sold than Joe, and totalling 500,000 downloads. The betting industry allegedly lost an estimated £1million, and the single has risen 127 places on our own Top Tracks chart to come in at number 7 for this week.

Last.fm users joined in the campaign:

Pretty cool.

Happy New Year

And that’s all for 2009! Thanks for continuing to support Last.fm seven years on; we’ll be back in 2010 with more. Right now, it’s time for a little Christmas tag radio.

Jingling all the way,
– Team Last.fm

We’re thrilled to announce that the Last.fm Playground now has a brand new VIP zone. In this subscriber-only area we’ll showcase some of our fanciest ideas, visualizations, or plain weird projects as a treat to our loyal subscribers! Here’s an overview of the new stuff we’re releasing today.

Tube Tags

Which genre were you into last summer? How have your listening habits changed over time? Is there a correlation between the music you listen to and important events in your life? We’ve built a unique visualization, the Tube Tags map, to help you answer all these questions at glance, and to marvel at all the twists and turns your music taste has taken through the passage of time. Here’s some details from mine (and some of our intermediary sketches):

Each line is a tag, moving north or south depending on how much you listened to music described by this tag. Your most popular artists for each tag are also shown. Of course, the longer you’ve been scrobbling, the better it looks! I’m really proud that I’ve been scrobbling so regularly over the past years — and that I’ve left this trail of data that allows me to revisit today, through music, past moments of my life.

While only subscribers can currently generate a Tube Tags map from their listening history, the map is visible to anybody — we thought you might want to share it with your friends. If you prefer printing this as a poster, we recommend Diginate’s online poster printing.

At Last.fm, we enjoy being mad scientists, playing with data and infographics — stay tuned for more in the visualization department!

New toys

The new VIP zone also contains a few other toys: Image Chart creates a collage of your top artists’ images, History Chart summarizes your listening activity for your top artists in a neat visualization, and Artist Connections is a musical equivalent to the Six Degrees of Kevin Bacon game; try it out to check if there’s a chain of similar artists linking Paris Hilton to Metallica. Finally, we also added a new World Chart demonstration that shows you in which countries a given artist has been listened to most often.

We’d love to hear more from you about these features – please send us feedback or join the Playground Group to discuss them with others.

Update: We have added an option to let you render the image for your entire listening history (you’ve been warned: some pdf viewers might choke on the large image size, and there might be meatballs lurking between all those lines…). Just add &full to the download url: http://playground.last.fm/demo/tagstube/map?user=flaneur&full)

One night at the pub we discussed whether one could replace Hadoop (a massive and comprehensive implementation of Mapreduce) with a single bash script, an awk command, sort, and a sprinkling of netcat. This turned into a weekend project dubbed bashreduce.

To be fair, Hadoop probably does a few more things than bashreduce. But we’ve managed to cover a few key concepts in our script:

• Task coordination (kind of! sort of!)
• Mapping/Partitioning
• Reducing
• Merging
• Distributed file system (sort of! if you squint just right)

More than just a toy project, bashreduce lets us address a common scenario around these parts: we have a few analysis machines lying around, and we have data from various systems that are not in Hadoop. Rather than go through the rigmarole of sending it to our Hadoop cluster and writing yet another one-off Java or Dumbo program, we instead fire off a one-liner bashreduce using tools we already know in our reducer: sort, awk, grep, join, and so on.

I think it’s a neat idea! If you think it’s a neat idea, and you look at this gnarly bash code and think of ways to improve it, to make it more useful or more elegant, you would enjoy working for us. We’re looking for a clever C++ developer to help us tackle data mining and scale problems. My favorite line in the job posting is Interested in – we do all those things save one, which you can probably guess.

We’ve collected a few of our developer’s blogs here as well – more fodder for those of you interested in what we do.

There are plenty of great groups on Last.fm. However, some of them can be a bit hard to find. We thought one way to make it easier to find groups would be to make it easier to find similar groups. What do you think?

Here are some of my personal favourites and their similar groups:

• Bossa Nova: they have a great group radio, if you are into that kind of music
• Addicted to Last.fm: the title says it all
• Converse: this special sponsored group has a really cool feature you won’t find elsewhere on the site
• The Special Interest Tag Radio Collective: some of the best tag radio stations I know of are listed there
• People with no social lives that listen to more music than is healthy who …: a wonderful example of how a long group name totally makes sense
• Last.fm Book Club: a great example that there is sometimes more to life than just music
• …

Btw, what are your favourite groups?

Last.fm groups are a great way to find people that share common interests not necessarily related to music, such as ear-chopped-off painter fans, car enthusiasts, and people that clearly like meticulous descriptions of their behaviour.

Group aficionados have probably already noticed that it’s possible to listen to group radio, but unfortunately this has sometimes been a disappointing experience. The reason is simple: the playlist was generated by averaging the tastes of the group members, and this can result in predictable charts.

We have recently been experimenting with ways to emphasize what makes groups unique. By considering the average listener’s behaviour, we can show that members of the Saxophonists group listen to far more Sonny Rollins than others. Even though they listen to The Beatles a lot, it’s less special, because everyone listens to them a lot!

So, next time you tune into group radio, expect a much better experience!

Here’s a new list of artists you’re likely to listen to on these group radio stations:

Soundtrack geeks

• Hans Zimmer
• John Williams

• Danny Elfman

People with no social lives, etc..

• Motion City Soundtrack
• Cobra Starship

• Hellogoodbye

Switzerland

• Züri West
• Die Ärzte

• Patent Ochsner

Since I was a teenager I have always loved music and at the same time have also had an affinity for numbers. Put the two together and what more could a young boy want? I’m talking about charts. I was addicted to them. Still am. I fervently refresh my last.fm profile page on Sunday nights and Monday mornings to see my personal weekly charts. In the past I used to listen to the South African charts live on the radio to see how my predictions for the movements of certain tracks had panned out. I used to buy British pop magazines to read the UK charts. I would stay up late on the weekends to watch US video charts. I got pen pals in Morocco to put the North African charts on postcards and send them to me. You get the picture.

Now most of these charts have something in common – they generally feature new music and there is a lot of change from week to week. A hot new tune could shake things up in the space of a few days. This is where the last.fm top tracks chart differs. Top tracks summarises what our users have listened to in the past week and while this often does include new music, if a whole bunch of people decided to go retro and relive the 80s by playing Public Enemy constantly, then Public Enemy would show up, even though they haven’t released anything (good) in years. Now, this has a certain charm to it and it can be interesting to see when an artist generates a lot of publicity (by dying for example) that they leap up the charts without there being any new music from them during that period. However it can also be really boring when the charts just stay the same for months on end.

Case in point. If you go back to Top Tracks for the week ending Sunday 14 October 2007 you will see the 10 songs on Radiohead’s “Rainbows” album (which was released that week) come out of nowhere, blasting Kanye West from number 1 and taking over the entire top 10. At the time I thought, wow, cool, this really is proof of the power of a new model of releasing music over the internet directly to your fans. It also shows that the whole album is popular, not just a few “singles” which would be released with a bunch of wack filler tracks around them. So far so good. Christmas came and went without anybody managing to get a word in on the top 10. After 18 or so weeks, the top 10 tracks were still all Rainbows. Yawn. I’m not a big Radiohead fan at the best of times, so this was really starting to get to me. Please, somebody, anybody, release something awesome and save the charts from Radiohead!

And then two weeks ago it happened, someone else managed to nudge their way not only into the top 10, but into the number one spot no less. It wasn’t a new song though, it was Amy Winehouse’s “Rehab” which saw a resurgence of popularity in the wake of her Grammy winnings. “Back to black” also made it in at number 9. Now, I’m not a big Amy fan either, but hey, at least here was something shaking things up a bit. Was this beginnng of the end for Rainbows in the Top 10? As much as a I would like to think so, last week saw them snatch number 1 back and drop Amy down to number 5. Ho hum.

Am I the only person who finds the Top Tracks chart boring? Surely not? Being a chart addict a few possible solutions come to mind – for example, adding more charts. Charts by tag could be nice, especially if more people were as anal as me when it comes to tagging music by genre. More detailed hype charts which would focus on new music? Charts which only cover music that was recently released? A worldwide Radiohead boycott? We are actually working on some of these at the moment (not the boycott!) and they will hopefully see the light of day in the coming months. If you’ve got ideas, we’d love to hear ‘em.

It is surprisingly hard to type when nestled in a pool of balls. You type, you sink, you get attacked from the east by an enterprising young programmer. Settled now at the bottom of the pit you clear a small gap and realise, actually, you can touch type. You haven’t just been claiming you can for the last five years. It’s really true.

Our excuse that this is a relaxing harbour for meetings may not hold water. Nobody can stay still for thirty seconds before they dive to the depths of the pool rooting around for someone to pull under. Erupting magnificently, balls flying everywhere, before everything collapses into a game of who can hit Jonty on the head the most times in thirty seconds.

23 thousand balls goes surprisingly little distance. It was in the local, last night, over a feisty pint of best that the idea emerged. “Ball pool”. An idea so splendid, so ludicrous – it quickly became the only topic of conversation. “London is a big city. Someone must have balls to sell” postulated RJ. So we started the research. We got numbers. We booked the van. There was planning. Surprisingly, I turned up the next day to find RJ bouncing around all excited. Tony was getting the balls. We were building the pool.

We had fun.

It was awesome.

More pictures can be found on flickr

It’s been decided, in future, all interviews will be held in the ballpit. But we don’t plan on stopping there. The meeting room next door is begging to be converted into an ice-rink. The kitchen is lacking for a climbing frame. The toilets are not yet complete without a waterslide.

Fancy a job?