Last.fm – the Blog

Today is pretty exciting; we’re launching a new Last.fm/presents video channel. We have ten new videos that give you some pretty cool insights into what shaped the musical life of such artists as Blood On The Wall, Santogold or Moby.

While you now get to enjoy these exclusive bites of wisdom from the comfort of your own laptop, we thought it would be fun to give some context into how these interviews came into existence. After all this is our first time doing these on such a scale… and they were all filmed in an insane 3 days during SXSW.

On the first day, Teniel, Hannah and I sit down at the Magnolia café over some breakfast tacos to finalize the interview questions. Our goal: get the artists to open up about unusual unique experiences. Stuff no one has heard before.

First stop is the Parish for the Hotel Café tour. Teniel shows up early and shoots Josh Radin backstage before his show. Easy. He even sings us a little tune.

Joshua Radin interview

It’s about midnight and Steve Aoki is waiting for us across the Lake. We hop in a cab and head to some obscure address in the middle of the woods. Turns out to be the Elk Lodge. Three senior citizens, two 5-day membership and twenty minutes later, we’re in. Danny Masterson is DJing and Steve Aoki is about to go on but he spends 30 minutes with us anyway.

Steve Aoki interview

It’s 2 am, we steal Does It Offend You, Yeah? ’s cab and call it a night.

Next morning, Hannah fills in for me, and films while Teniel interviews Moby in the garden in front of the Four Seasons. We were promised only 5 minutes but Moby seems to be enjoying himself and they chat for 20.

Moby interview

Next we meet up with Boom Bip and Gruff Rhys of Neon Neon, the two most laid back musicians we met the whole trip. We sit down at a café terrace and have a nice conversation. What did I learn from this interview?

1. Don’t stand in the sun while filming (or bring sunscreen)
2. Austin North/South streets are loud as f**k

Neon Neon interview

In the afternoon we head over to the Fader Fort to meet with White Rabbits. The line goes around the block; this is going to suck. Luckily we spot Tad and Howie from Low Vs Diamond walking down the street and we quickly grab them for an interview at a nearby café.

Low Vs Diamond interview

I resist my natural urge as a Frenchman to cut in line and we resume our queuing. Luckily, Steve and Greg from White Rabbits are still waiting for us inside and have time for a chat.

White Rabbits interview

The following day, we meet up with Brooklyn lo-fi rockers Blood On The Wall on top of the American Apparel store. These guys made me laugh so hard, thank god I had a tripod. I stood in the shade too.

Blood on the Wall interview

We take a quick detour by Brush Park to catch The Virgins rehearsing before their set

The Virgins interview

and find ourselves once again facing the endless queue at Fader Fort to interview Santogold. Harnessing my inner Frenchman, we head to the front of the line and magically get in. Just as we start the interview in a quiet back alley, a semi decides it’s the perfect time to see if he can fit inside the alley, which is only about 6 inches wider than he is. Luckily, we manage to squeeze in to a garage door to make way for him and narrowly avoid being transformed into crepes.

Santogold interview

With the festival almost over, we call up Britt from Spoon and set up our most chilled out interview in his hotel room. If only all interviews could be this easy.

Spoon interview

See all ten interviews here: Last.fm/presents

Our data center is losing power tonight for 8 hours, and we are in the process of shutting down all non-essential systems (and some semi-essential ones) to try and keep the site alive on a bare minimum of servers.

Please bear with us over the weekend – things might break in interesting and mysterious ways, and we’ll be trying to hold things together while DC/AC inverters explode all around us because we’re drawing too much power :)

Your scrobbles will be safe, but this blog post might self destruct in 19 hours….

Update 1
As usual, you can check status.last.fm for more information

Update 2
After painstakingly moving machines over to the backup power supply, the backup power supply flipped out (apparently it won’t happen again).

Things are going to be really broken / slow for a while whilst we bring a couple of hundred servers back online again, and wait for 300GB of memcache to fill up. What a day…

Update 3
The power outage is over now, and we’re slowly powering everything back up. It’ll be a day or three before things are completely back to normal though.

Update
We have now filled all the slots for the internship. Thanks to everyone who applied!

Our lovely back end team is looking for fresh meat. Specifically Java programming students that want to work with huge datasets and clusters of yellow elephants. So if you are interested in hanging out in East London and getting your hands dirty hacking some of the most exciting music-related software on the ‘net, then read on for more information on our summer internship programme.

You’ll be spending your days bathing in the ball pit and getting back rubs from our support department. Occasionally we’ll ask you to do some work too, for example developing new features for the open source projects below or improving our own internal systems. You will be mentored by experienced Last fm developers and stand to learn loads about making the software that is used daily by millions of people across the globe.

Potential areas to work in:

• The open source distributed data processing project Apache Hadoop.
• The distributed data storage project Apache HBase.
• Processing our listening data into interesting funky stuff we can use on the site.
• Improving the Java support in (soon to be Apache) Thrift.
• Using X-Trace to find and resolve bottlenecks in our distributed systems.
• We’re also open for student suggestions! If it’s useful and/or cool, include any ideas you may have in your application.

If you’re allowed to work in the UK please e-mail your applications including a CV to jobs@last.fm, put “java summer intern” in the subject line. The internship will last for the coming summer (we are quite flexible with the exact dates). If things go really well, you might even end up joining our team when your studies are over.

More information on our jobs page.

Last week Johan and I were fortunate to be sent to the Hadoop Summit 2008, the first conference focused on the Hadoop open source project and its surrounding ecosystem. Great talks, conversations with a lot of interesting and talented people, and loads of food for thought.

Big Data Startups

Last.fm is a prominent representative of a growing class of Internet startups: smaller companies whose business entails storing and processing huge amounts of data with a low amount of overhead. Our project teams are comparably tiny and we rely mostly on open source infrastructure; and while the Big Data problems of big corporations have been catered to for decades (in the form of an entire industry of integrated infrastructures, expensive hardware, consulting fees, training seminars, …) we’re not really in that market. As a result we often have to create our own, low-overhead solutions.

Throughout the Hadoop Summit it was very apparent that we’re seeing the dawn of a new culture of data teams inside Internet startups and corporations, people who manage larger and larger data sets and want better mechanisms for storage, offline processing and analysis. Many are unhappy with existing solutions; because they solve the wrong problems, are too expensive, or based on unsuitable infrastructure designs.

The ideal computing model in this context is a distributed architecture: if your current system is at its limits you can just add more machines. Google was one of the first Internet companies to not only recognise that, but to find great solutions; their papers on MapReduce (for distributed data processing) and their Google File System (for distributed storage) have had great impact on how modern data processing systems are built.

Doug Cutting’s Hadoop project incorporates core principles from both of them, and on top of that is very easy to develop for. It’s a system for the offline processing of massive amounts of data; where you write data once, then never change it, where you don’t care about transactions or latency, but want to be able to scale up easily and cheaply.

Distributed Computing, Structured Data and Higher Levels of Abstraction

One current trend we’re seeing within the Hadoop community is the emergence of MapReduce processing frameworks on a higher level of abstraction; these usually incorporate a unified model to manage schemas/data structures, and data flow query languages that often bear a striking resemblance to SQL. But they’re not trying to imitate relational databases — as mentioned above, nobody is interested in transactions or low latency. These are offline processing systems.

My personal favourite among these projects is Facebook’s Hive, which could be described as their approach to a data warehousing model on top of MapReduce; according to the developers it will see an open source release this year. Then there’s Pig, Jaql, and others; and Microsoft’s research project Dryad, which implements an execution engine for distributed processing systems that self-optimises by transforming a data flow graph, and that integrates nicely with their existing (commercial and closed-source) development infrastructure.

Another increasingly prominent project is HBase, a distributed cell store for structured data, which in turn implements another Google paper (”Bigtable“). HBase uses Hadoop’s distributed file system for storage, and we’re already evaluating it for use inside the Last.fm infrastructure.

But despite all this activity it’s still very apparent that this is a young field, and there are at least as many unsolved problems as there are solved ones. This is only a faint indicator of what’s to come…

If You Just Got Interested

Are you a software developer with a background in distributed computing, large databases, statistics, or data warehousing? Want to gain first-hand experience inside an emerging industry? Then apply for our Java Developer and Data Warehouse Developer positions!

Those of you who’ve been keeping an eye on the blog will have noticed we are working on some audio fingerprinting technology to assist us in cleaning up the mess that is music metadata.

Fingerprint Metrics

So far our fingerprint server identified 23 million unique tracks, from the 650 million fingerprint requests you’ve thrown at it. Who knows how many unique tracks there are out there.. We have a couple of hundred million tracks based on spelling alone – but not all of them are spelt correctly. We’re getting closer to an answer though.

Order from Chaos

Erik has developed an ingenious method of extracting the correct names from the millions of fingerprinted tracks. This turns out to be a decidedly tricky problem – the most popular spelling is not necessarily the correct one. There is also the issue of foreign language artist names, and popular non-misspellings such as name-changes, abbreviations or acronyms (think RATM, TAFKAP, GNR, ATWKUBTTOD etc). Oh, and there are plenty of interesting corner cases, like our old friend “Various Artists” – yes, I’m talking about Torsten Pröfrock

We’d also like to take this opportunity to big up the Musicbrainz and Discogs massive – without the free availability of high-quality datasets such as these, we’d be having a much harder time checking our results. We are planning to feed some useful data back to them in the form of artists we think exist, but don’t show up in their respective catalogues.

Based on all this, we have an initial data set which maps fingerprints to what we think the correct metadata might be. We are using this data in two ways right now: tech-demo fingerprint client, and artist alias voting on the site.

Command-line fingerprinter demo

The tech-demo for this is a command-line fingerprint client that can convert an mp3 file into a fingerprint id, and lookup the metadata.

Grab the command-line fingerprint tool here:

• for Windows
• for Mac OS X
• for Linux

• Source code (GPL2): svn://svn.audioscrobbler.net/recommendation/MusicID/lastfm_fplib

We’re interested in any feedback on the accuracy of the metadata lookup feature, especially if you find things that are horribly wrong/broken.

Making sense of it all with artist corrections

So based on all this fingerprint data, we have a new artist alias list. We’re not doing any auto-corrections until we’re sure how sure we are, but in the meantime if you stumble across an artist page that has potential corrections, you are given a chance to vote on the correct one:

You will only see this if you are logged in.

This basic voting system will help us evaluate how good the current data set is. We have a track dataset too, but let’s start with artists and see how it goes.

The Guns N’ Roses Issue

Back in December I used Guns N’ Roses to illustrate the metadata problem by asking:

• Just how many ways to write “Guns N’ Roses – Knockin’ on Heaven’s Door” are there?

We still don’t have a concrete answer for that question, but here are the Top 100 ways to write: Guns N’ Roses – Knockin’ on Heaven’s Door

And just for fun, here are some of the artist names used on Guns N’ Roses MP3s in descending order of popularity – yes people really do mistag things this badly :)

What the future holds

Fixing up music metadata remains a hot topic for us this year, there is plenty more to come:

• Disambiguation to properly display the nine different artists that all have the name: Mono.
• Correct data displayed when scrobbling, even if your tags suck.
• Maybe the ability to correct your crappy tags, if it’s accurate enough, and there is demand for it.
• Return other identifiers from fingerprints, such as Musicbrainz IDs and Discogs URLs.
• Consolidate the stats on Last.fm so that Artist top tracks, and ultimately user charts and all other charts are corrected.
• Implicitly improve Last.fm recommendations and radio due to less noise in the data.

• More formal collaboration with other music metadata crusaders.

Conclusion

If you are using the official Last.fm software, every time you play a track you are contributing to the metadata cleanup effort by scrobbling and fingerprinting what you play.

In the meantime, we are working to refine the data, and will publish updated artist corrections, track corrections and some analysis of the feedback we receive from this round of voting soon.

While most of us stick around to keep the site running (lucky for you!), some fortunate folks among us will soon be heading to Austin for SXSW. If you’re also going, say hi to Hannah, Matt, Jonas, Jeff, Lynn or Kevin.

It’s probably every music enthusiast’s dream to dive into the vast amount of live shows at a festival like SXSW, but with over 1500 performers this year how do you plan 4 packed days (and nights) and not miss what could be your next favorite band?

Head over to the SXSW Group page, type in your username and get your very own Band Aid — a list of all the bands you should see (plus some recommendations) based on your music taste.

Join the group for info updates by Austin native, mr_orange_ringo, and don’t forget to check back after the festival to see all the ‘Tag SXSW’ pics that have been uploaded. What’s this? Last.fm is tagging up the real world — with stickers! Get them from the last.fm team in Austin.

Enjoy!

I started building webpages at Last.fm in early 2005. In the days of the old Audioscrobbler site, from our glamorous Whitechapel headquarters, the social music revolution was starting to take shape. And I thought I was the one of the lucky few shaping it.

I was wrong.

The first time I sat next to RJ watching our real-time access logs, I didn’t recognize half the URLs scrolling past. Instead of profile pages and forum posts, it was screenfuls of recenttracks.rss and artisttags.xml requests. Huh?

“Yeah, those are the webservices. Could be anything, really…”

Anything (and Everything)

Since 2003, we had been operating under a pretty simple premise: Being able to get data out of Last.fm –- whether one person’s recently played tracks or community-driven artist and tag info — was as important as being able to put it in.

Five years on, traffic from the incredible wealth of mashups, widgets, and services that leverage Last.fm data matches traffic to our website itself. These applications — contributed by fans, companies, and partners around the world — have literally built the social music revolution. I can take my music profile wherever I go on the net; I can sort my friends by musical compatibility; I can explore interactive graphs and stats; I can scrobble tracks from platforms and devices whose names I can’t even pronounce.

build.last.fm

Today we’re pleased to introduce build.last.fm, a gallery of apps and services that take slices of the Last.fm experience into tons of cool new directions. We’ve only just started adding to it, but — with your help, of course — we expect it to grow quickly! (In features too; stay tuned for search, RSS feeds, user comments, and more.)

This is just a small piece of our much larger goal for 2008, which will include a re-launch of our developer site, along with new and improved APIs for some of our most important functionality.

We hope you enjoy checking out all the things you can do with a Last.fm profile. But we’re even more excited to see where you’ll take the music next. :)

Have you built something too?
We’ve already added some popular apps to the gallery, but if you can’t find your creation, please tell us all about it.

Last.fm groups are a great way to find people that share common interests not necessarily related to music, such as ear-chopped-off painter fans, car enthusiasts, and people that clearly like meticulous descriptions of their behaviour.

Group aficionados have probably already noticed that it’s possible to listen to group radio, but unfortunately this has sometimes been a disappointing experience. The reason is simple: the playlist was generated by averaging the tastes of the group members, and this can result in predictable charts.

We have recently been experimenting with ways to emphasize what makes groups unique. By considering the average listener’s behaviour, we can show that members of the Saxophonists group listen to far more Sonny Rollins than others. Even though they listen to The Beatles a lot, it’s less special, because everyone listens to them a lot!

So, next time you tune into group radio, expect a much better experience!

Here’s a new list of artists you’re likely to listen to on these group radio stations:

Soundtrack geeks

• Hans Zimmer
• John Williams

• Danny Elfman

People with no social lives, etc..

• Motion City Soundtrack
• Cobra Starship

• Hellogoodbye

Switzerland

• Züri West
• Die Ärzte

• Patent Ochsner

Since I was a teenager I have always loved music and at the same time have also had an affinity for numbers. Put the two together and what more could a young boy want? I’m talking about charts. I was addicted to them. Still am. I fervently refresh my last.fm profile page on Sunday nights and Monday mornings to see my personal weekly charts. In the past I used to listen to the South African charts live on the radio to see how my predictions for the movements of certain tracks had panned out. I used to buy British pop magazines to read the UK charts. I would stay up late on the weekends to watch US video charts. I got pen pals in Morocco to put the North African charts on postcards and send them to me. You get the picture.

Now most of these charts have something in common – they generally feature new music and there is a lot of change from week to week. A hot new tune could shake things up in the space of a few days. This is where the last.fm top tracks chart differs. Top tracks summarises what our users have listened to in the past week and while this often does include new music, if a whole bunch of people decided to go retro and relive the 80s by playing Public Enemy constantly, then Public Enemy would show up, even though they haven’t released anything (good) in years. Now, this has a certain charm to it and it can be interesting to see when an artist generates a lot of publicity (by dying for example) that they leap up the charts without there being any new music from them during that period. However it can also be really boring when the charts just stay the same for months on end.

Case in point. If you go back to Top Tracks for the week ending Sunday 14 October 2007 you will see the 10 songs on Radiohead’s “Rainbows” album (which was released that week) come out of nowhere, blasting Kanye West from number 1 and taking over the entire top 10. At the time I thought, wow, cool, this really is proof of the power of a new model of releasing music over the internet directly to your fans. It also shows that the whole album is popular, not just a few “singles” which would be released with a bunch of wack filler tracks around them. So far so good. Christmas came and went without anybody managing to get a word in on the top 10. After 18 or so weeks, the top 10 tracks were still all Rainbows. Yawn. I’m not a big Radiohead fan at the best of times, so this was really starting to get to me. Please, somebody, anybody, release something awesome and save the charts from Radiohead!

And then two weeks ago it happened, someone else managed to nudge their way not only into the top 10, but into the number one spot no less. It wasn’t a new song though, it was Amy Winehouse’s “Rehab” which saw a resurgence of popularity in the wake of her Grammy winnings. “Back to black” also made it in at number 9. Now, I’m not a big Amy fan either, but hey, at least here was something shaking things up a bit. Was this beginnng of the end for Rainbows in the Top 10? As much as a I would like to think so, last week saw them snatch number 1 back and drop Amy down to number 5. Ho hum.

Am I the only person who finds the Top Tracks chart boring? Surely not? Being a chart addict a few possible solutions come to mind – for example, adding more charts. Charts by tag could be nice, especially if more people were as anal as me when it comes to tagging music by genre. More detailed hype charts which would focus on new music? Charts which only cover music that was recently released? A worldwide Radiohead boycott? We are actually working on some of these at the moment (not the boycott!) and they will hopefully see the light of day in the coming months. If you’ve got ideas, we’d love to hear ‘em.

It’s Saturday night and the first Last.fm Presents of the year is upon us. Somewhere amid the packed room is Lachlann Rattray of Gay Against You. On stage, his normally bespectacled partner-in-crime Joe Howe fears he has lost him forever. “If you’re not me, please sit-down” Lachlann requests (after locating his microphone) in a last ditch attempt to be reunited with his band mate.

It’s hot in here. And shambolic. And this is almost what we expected when we decided to ask Last.fm chart toppers Gay Against You to headline our February show. Almost, only better.

The rest of the line-up are equally as impressive, Pagan Wanderer Lu‘s cynical songs of everything mundane and Look Look (Dancing Boys)‘s insanity-fueled punk mean that tonight is free of drummers and bands of more than two members. In their place we’re offered a menu full of bleeps, shouting, dancing and, well, swearing.

The crowd at times don’t seem to know how to react to any of it, expressions ranging from bewilderment through to excitement and from awkwardness to all-out enthusiasm fill the sauna-like Old Blue Last.

Gay Against You however are the real stars, loud, messy, energetic and all over the place. They prowl the crowd, turning indifference into grins and getting us to sing along. At times, I fear for my life as the floor feels like it might give way under the weight of however-many dancing (and sweating) bodies struggling to stay upright.

As they finish the crowd barely thins, as people opt to stay, dancing their way into the early hours of the morning to the White Heat DJ‘s indie disco and Tack! Tack! Tack!‘s Swedish pop.

Thanks a lot to everyone who came down and made this such a successful night and apologies to anyone whose clothes were ruined by sitting on the floor.