Genre Timelines and More Distinctive Lyrics

Thursday, 6 September 2012
by Janni Kovacs
filed under About Us
Comments: 10

For the past five months I have had the honour of being the next data team intern at Last.fm, building software and trying to make sense of what people now call Big Data™. In particular during my time here I looked at biographical data for artists, i.e. the place and the year a band was formed. This data is generated by Last.fm’s users and attached to artists’ wiki pages (see the factbox on the right of the page). There’s a nice number of artists where this type of data is available, so I was wondering what kind of analyses we could do with it.

When did this genre take off?

One thing that I was looking for in the data was empirical evidence of when certain genres became popular. Since we have a massive amount of user tag data available we can easily correlate tags and years and measure “popularity” of a genre by counting the number of artists formed in a specific year. Even with this data being skewed a bit towards the more popular artists, you can definitely see spikes of popularity for certain genres where you’d expect them:

Click for a larger version

Props to our users getting punk and post-punk in the right order!

If you’re a fan of metal music maybe the following chart, showing the progression of metal subgenres from hard rock to death metal, will be of interest:

Click for a larger version

Distinctive lyrics for cities

Andrew did a fantastic job a while ago generating distinctive lyrics for certain genres. I was wondering if we could generate distinctive lyrics for cities as well. By taking about 75.000 song lyrics, matching them to artist’s location metadata from our wikis and applying a simple term frequency function to each word, we can generate a list of words that occur in some cities more often than in others. Please take these results with a grain of salt as they are skewed by several factors, especially towards the more popular artists:

Click to open full images in a new window.

Warning: they contain lyrics you may find offensive. Not safe for work.

London

Atlanta
 

Los Angeles

New York

Seattle

I really like that “sorry” is in London’s top 10

In internships you’ll often find that you’re given pointless work just to occupy yourself. This is not the case at Last.fm. You’ll be able to work on in-production code and be given plenty of time to do things on your own, whatever interests you. So even though the ball pit is no more (turns out they have to be cleaned once in a while), if you enjoy working on backend software and exploring immense data sets then this is the right place to do it.

Comments

  1. Christopher Sutton
    6 September, 15:31

    I have indeed spent hours full of fun walking in London!

    Great work, this is a fun idea for analysis done well.

    Christopher Sutton – 6 September, 15:31
  2. Chris
    6 September, 15:49

    Pretty cool stuff. I wonder if you were to add Disco if it would fill in the gap between Progressive Rock and Punk!

    Chris – 6 September, 15:49
  3. Zack
    6 September, 18:00

    Hey, this is pretty cool. I did something along similar lines using the last.fm public data. In case you’re interested, it’s here:
    http://datagardenblog.net/2012/05/6/mined-pop-history/

    Let me know if you want to play around with any of the data or code I put together for this ;)

    Zack – 6 September, 18:00
  4. johntmcneill
    7 September, 03:37

    I’d be happy if you could just keep the dang thing up and running 24/7

    johntmcneill – 7 September, 03:37
  5. Graham
    7 September, 14:38

    Said the actress to the Bishop

    Graham – 7 September, 14:38
  6. Ilia
    7 September, 16:14

    Well Done!.. Zachary Nichols ;) I add your data Garden In My RSS feed

    Ilia – 7 September, 16:14
  7. Dan
    13 September, 11:07

    I just love your blog, keep up the excellent work!

    Dan – 13 September, 11:07
  8. Dmitryi
    14 September, 13:58

    Hello. The first time at you on the blog, such idle time, but at the same time with taste, anything superfluous.

    Dmitryi – 14 September, 13:58
  9. christianlouboutinioutlet
    19 September, 07:09

    The first time at you on the blog,

    christianlouboutinioutlet – 19 September, 07:09
  10. serdar
    15 October, 05:07

    http://youtu.be/z40dyVU_eAY
    ı made this animation but ı cant find same music..what is the name this music

    serdar – 15 October, 05:07

Comments are closed for this entry.