Making my playlist processing be concurrent.

Read Time:2 Minute, 17 Second

This may come as a shock to some people, but I have this itch to continually tweak my music playlist. It started with me having a set of songs I liked… And [un?]fortunately, this collection grew, and grew. Soon, I wanted to have a way to organize my music and ensure that I’m listening to a certain set of songs. Enter the iTunes Smart Playlists, which let me define, in Set Theory-like notation, multiple playlists each following a different set of rules, that I could combine together and randomize to produce a “daily playlist”, which I loaded into my Gen 1 iPhone. This worked fairly well.

Well… eventually I replaced that phone, and chose a less technologically-advanced phone (read: not a smart phone), and had to come up with some other way to get my music. So I took my music, and used a plethora of different streaming servers, building the same kind of functionality as I had with the iTunes smart playlists, building a daily playlist to stream to me at work (or wherever I can get the stream from). I eventually got fed up with most of the streaming servers, and settled onto Music Player Daemon (MPD). I also found a nice python library to interface with the MPD server, and control it (python-mpd2). This meant I could basically write my own playlist generator, how I wanted it, and have my music streamed through MPD, without all of the annoyances that the proprietary streaming servers were doing.

Now, several generations and revisions later, I have my own music database (the backend is unimporant, it will work with any database backend that SQLAlchemy can use) for my playlist generator, which has transformed into a playlist controller (it continually runs, and updates as songs are played through the day, eventually loading in a new batch of songs as the internal playlist gets used up). I have rebuilt it so that the database updating, and the playlist generating is all done concurrently, with multiple “worker processes” that drive handles chunks of the processing of each independently of the other processes. The beauty of this is that I can leverage multi-core computers for the playlist controller, and be able to have the processes spread around the different cores, which gives a nice performance boost.

Now, I am trying to add in new features to it. I want it to scrobble to last.fm. I want my holiday music playlist modifications to work again. I want it to play stuff from podcasts, I want to be able to mix in music from Pandora. These will keep m busy, now that I think I have the core processing working in a way I am comfortable with.