Last.fm – the Blog

For the past five months I have had the honour of being the next data team intern at Last.fm, building software and trying to make sense of what people now call Big Data™. In particular during my time here I looked at biographical data for artists, i.e. the place and the year a band was formed. This data is generated by Last.fm’s users and attached to artists’ wiki pages (see the factbox on the right of the page). There’s a nice number of artists where this type of data is available, so I was wondering what kind of analyses we could do with it.

When did this genre take off?

One thing that I was looking for in the data was empirical evidence of when certain genres became popular. Since we have a massive amount of user tag data available we can easily correlate tags and years and measure “popularity” of a genre by counting the number of artists formed in a specific year. Even with this data being skewed a bit towards the more popular artists, you can definitely see spikes of popularity for certain genres where you’d expect them:

Click for a larger version

Props to our users getting punk and post-punk in the right order!

If you’re a fan of metal music maybe the following chart, showing the progression of metal subgenres from hard rock to death metal, will be of interest:

Click for a larger version

Distinctive lyrics for cities

Andrew did a fantastic job a while ago generating distinctive lyrics for certain genres. I was wondering if we could generate distinctive lyrics for cities as well. By taking about 75.000 song lyrics, matching them to artist’s location metadata from our wikis and applying a simple term frequency function to each word, we can generate a list of words that occur in some cities more often than in others. Please take these results with a grain of salt as they are skewed by several factors, especially towards the more popular artists:

Click to open full images in a new window. Warning: they contain lyrics you may find offensive. Not safe for work.	London	Atlanta
Los Angeles	New York	Seattle

I really like that “sorry” is in London’s top 10

In internships you’ll often find that you’re given pointless work just to occupy yourself. This is not the case at Last.fm. You’ll be able to work on in-production code and be given plenty of time to do things on your own, whatever interests you. So even though the ball pit is no more (turns out they have to be cleaned once in a while), if you enjoy working on backend software and exploring immense data sets then this is the right place to do it.