Sessions were organised spontaneously around various topics, and one of which, proposed by Amarok2’s Lydia/nightrose, was “Problems Audio Players face today”. Which resulted in 15-20 people from various projects (incl. Amarok2, Rockbox, Maemo, XMMS2 & others) talking about solving problems encountered by all music player developers (e.g. lyrics and cover art fetching), as well as features on our beloved users’ wishlists (portable player support, tags, musicbrainz, etc).

And if you weren’t there, don’t despair, for I just (finally) posted minutes for this session on the GSoC wiki. Some interesting ideas in there, so go and have a look, and get to work!

Quizz: What do Winamp, Foobar2000, Emacs, Eclipse and Firefox have in common?

(Vim users, we love you too, but shut up for now please.)

They are very popular and they highly promote extensibility.

Now, we all know that correlation != causation. What is certain, however, is that all those projects boast many fan(boy)s, some of whom even get quite religious about it.

This enthusiasm reflects itself in the emergence of strong communities: people who share content, tips, modules, extensions, themes, configurations, scripts, etc. And those people usually aren’t even core developers of the software, sometimes not developers at all; just users or hobbyists who like the project.

Paradoxically, most of what they share is aimed at personalizing the software, i.e. making it the user’s own, fitting the user’s wishes and needs. This creates a feeling of ownership and satisfaction, because the user can bend it the way she likes and master it like a pro.

In essence, the idea here is give power to people to shape the software the way they want, and they will extend it in creative ways and get together to share these extensions. And I think that encouraging this creative freedom is one of the many factors which contribute to popularity.

Obviously, each of the projects mentioned above have their own way of allowing user extensions.

• Classic Winamp 2.* had (lots of) theme and some plugins (mostly effects, visualization), but remained otherwise quite static in terms of interaction or features.
• foobar2000 goes much further with theming and interface customization, allowing varied usage and user experience.
• Eclipse focuses mainly on extension modules (to interact with the VCS, manage project workflow, etc) and a jungle of configuration options.
• Firefox boasts rich installable extensions which add powerful new features, two levels of configurability (standard Preferences and advanced about:config), and recently themes (Personas).
• emacs allows essentially everything through scripting: altering and extending looks, interaction, features, as long as you have a high enough Elisp-fu (or enough curiosity to install other people’s modes).

More to the point, extensions can be mapped along three mostly orthogonal axes:

• Themeable appearance (look)
• Configurable interaction (feel)
• Scriptable features (personal usage)

Let’s take each of them and see what it means for our beloved imaginary music player.

Themeable appearance

Not an easy one, especially when one wants to rely on advanced features and widgets from a toolkit, which doesn’t necessarily allow for heavy reskinning.

There are more and more ways to draw fancy stuff in a window (vectorial canvas, HTML view, etc) but those often come at the price of limited interactivity with the rest of the interface (drag and drop, event handling, interoperability with the MVC architecture).

Using Qt StyleSheets (possibly with the help of QtScript) to style native widgets might be a better option, though I haven’t yet investigated in depth how much can be achieved with them.

Anyone with a better clue is welcome to step up.

While the whole layout of the player could benefit from imaginative and beautiful theming, one of the oft talked about use of styling was for the rendering of the playlist (usually out of jealousy for gorgeous foobar2000 screenshots), with album grouping, sexy styling of information instead of boring columns, etc.

Viewing and browsing the medialib could definitely benefit from a rich, themeable design, which would ideally (and optionally?) smoothen navigation with animations (see a hastily put together demo I made with jQuery, a while ago).

My foobar2000 – 6-28-08, by BestTechie

Customizable interaction

Power-users get easily nervous about GUIs, as they fear that their personal preferences might not be respected:

Will there be a STOP button, or only PAUSE?

The best way to approach such religious questions is to set a default reasonable with regards to the rest of the default settings (i.e. coherent with the “default experience”), and let the user change it if she wants to.

Note that I put as much emphasis here on configurability as on picking a sensible default. In parallel, fine-grained options should be hidden away from the main configuration panel to keep it usable — a standard configuration pane similar to Mozilla’s about:config can do the trick.

Will it require me to reach for the mouse?

I believe that a complete GUI player should be usable either exclusively with the mouse, exclusively with the keyboard, or with a combination of both.

The keyboard access is often assimilated to keyboard shortcuts, and indeed advanced users often rely on shortcuts to work quickly with an interface. They should even be able to define their own shortcuts bound to arbitrary actions (jump to the playing song, enqueue the album for the track I’m viewing, etc).

However, another underused keyboard-based interface has been making a comeback into GUIs lately. You might have heard of it, it’s called the command-line.

Apart from the venerable emacs and vim, command-lines have been creeping more or less discretely into various Mozilla projects. Bespin, the web-based text editor has attempted at allowing users to run smart commands directly in the editor. And of course, the Firefox 3 Awesome Bar is just a step away from a command-line; a step taken by the TaskFox (see demo), which brings Ubiquity into the browser address bar.

I had experimented with the idea of a CLI-based GUI (sounds strange doesn’t it?) with my unfinished lindalë prototype, and I think it could bring a lot of interesting power to our new graphical player.

More on this idea in a future post.

Scriptable features

As usual, the level of scriptability is a trade-off decision, somewhere between the horror of Excel inline formulas (a sad ad-hoc hack most of the time) and the universal meta-ness of emacs (which allows the editor to rewrite itself when you’re not looking). The latter is naturally attractive to us geeks, but it’s probably excessive in terms of work required.

On the other hand, if we pursue our idea of using a high-level
language for the GUI (possibly QtScript, which is being worked on), it should be relatively easier to import self-contained extensions into the player.

In terms of design, it would be preferable to think of such “extensions” as features of the player, rather than external add-ons bolted onto it. The player itself would therefore be nothing more than a platform hosting extensions/features, and even the default interface would be implemented as extensions, indistinguishable from third-party ones.

This would for instance allow anyone to tweak the widget that displays search results, or replace it with something completely different. Even better, entirely new features could be imported, e.g. a rich interface exposing a bookmarking system (managing a generic bookmarking service client under the scene), or new widgets to explore one’s music library.

Possibilities are endless, and our time to work on them isn’t.

I have been discussing some of those ideas with greafine, our student working on this project for this year’s Summer of Code, and we’ll work together to determine what is feasible technically and temporally. In any case, I hope this post hints at various ways we shall explore to achieve high extensibility!

starwars_rock, by Clint M Chilcott

After all, we have a brand new korving CLI (nycli), isn’t that enough? In a sense it is, but it fills a different niche. GUI applications are good at things that CLI applications aren’t, and vice-versa. So the goal is to exploit the specific advantages of graphical music players.

For instance, even the most hardcore fans of the command-line will admit that the following tasks are easier with a graphical player:

• Edit a playlist, using mouse selection and drag-and-drop.
• Browse albums by cover.
• Organize music manually into playlists or using dynamic collections.

But these are just simple examples that are now expected of any standard graphical music player.

Can’t we get something more exciting?

As Obama taught us, “yes, we can!”

Three main aspects usually poorly supported and under-exploited in music players are powerful tools to:

• Browse
• Organize
• Explore/discover

Browsing

Graphical applications provide a rich visual experience that can be exploited to navigate large amounts of information, in particular using the spatial aspect and our ability to recognize images quickly.

Typically, users have become familiar with widgets like iTunes Cover Flow, which exploit the visual clue of album covers to quickly flick through releases. Several XMMS2 developers have expressed their interest in a more “album-centric” client, which essentially means supporting album entities in the interface, as opposed to just tracks.

An extra step would be to also promote the artists, and possibly other properties (e.g. genre, year, label), as premium entities. So rather than “album-centric”, the client would be “entity-centric” (in the sense that most existing music players are track-centric). Each entity could have its own fullscreen pane (or “page”), with corresponding information (more below) and links to browse the related entities. This would lead to a web-like navigation, where for instance each artist would get a page with the list of his releases (plus a photo, bio, etc), and clicking on a release would bring the page for that release, with the list of tracks.

However, browsing doesn’t have to follow a rigid path: we’re used to browsing inside categories (e.g. releases) alphabetically, and to jump between categories using the explicit hierarchy from the data (e.g. from an artist to its releases to their tracks), but that’s just one of many possibilities.

The user may want to browse a subset of her media library, for instance filtering by a range of release years and genre (e.g. “70’s rock”), or a custom collection she assembled herself. She may want to follow connections from an artist page to pages of “related artists”, to use tags to jump from a track to a list of albums, to find all the music she added the same week as a given release.

It’s time to think beyond the simple local data and harvest The Cloud to enrich the user experience; services like Last.Fm already provide an API to retrieve Similar Artists, social Tags, etc. And with our Collections API, it’s just a whole lot of power waiting to be unleashed!

Organizing

While browsing is the passive process of visiting what’s there, organizing is the active process of applying your own order on the content.

Playlists are the most common organizational tool, usually directly tied to playback, and the usual editing features should naturally be supported (insert, enqueue, move, remove). Special playlist behavior (queue, party shuffle, random, etc) should also be configurable easily. (Note: fancy playlist formatting is outside the scope of this post.)

The second main organizational tool is collections. A collection is akin to a “themed-bucket”, i.e. a set of music that the user has put together in order to reuse it later. But rather than focusing on the underlying nature of a collection (a graph of operators), the interface should emphasize the organic process of someone creating a custom group of music. Any search, or essentially any “view” of music, should be recordable as a collection; and it should always be possible to refine or further filter a collection, as well as add custom content to it. It should be as easy as typing a search or dropping content in a folder, rather than as complex as setting up a network of mail filters.

Finally, tags should be at the user’s disposal for applying a minimal description to content. There is a subtle difference between tags and collections (and how they play together), and I don’t have much in mind regarding tags so far, but I think they will definitely be a powerful addition.

Exploring/Discovering

In the browsing section above, different new ways to navigate one’s music have been evoked. The next logical step after that, however, is to help the user discover new music he doesn’t yet know about, by giving pointers to music and information outside of his media library.

Many online services offer to make you discover new music, but this feature remains unusual in desktop music players, except Spotify (with a tab of related artists) and recent versions of Banshee (with custom recommendations, perhaps based off Last.Fm?).

Those illustrate two interesting directions.

First, providing pointers from a certain point in the user’s media library to complementary information and new music. For instance, on an artist pane, show the list of all its releases, including those missing from the user’s music files, show related artists, both present in the user’s media library or not, show the artist’s news feed, etc. Or show reviews for albums, or lyrics for individual tracks. Basically, gather information from external sources about the user’s music, and invite him to discover new music as well.

The second direction is more general: given the user’s music profile and playback history, suggest new artists or genres he might be interested in exploring. For instance, using all the artists present in the media library, infer what will be the user’s next favorite band. Or suggest popular releases, based on the kind of music recently played. Here, suggestions are made spontaneously, based on the behavior of the user.

The goal of these two features is to get the user excited about not just his current music files, but the whole portion of the music world they span. They should invite the user to be curious about his or new music.

One place to put these suggestions on would be the music player’s “home screen”. Without entering into too much details yet, the player could provide the user with ideas on what he might want to listen, based on what he played recently, his collections, tags, etc. Rather than a long table of “all the tracks”, the entry screen could be a richer, custom view of different ways he could start playing his music.

Obviously, this post hinted at a lot of potentially complex features, which would take a lot of time and effort to all put together perfectly. The main point, however, was to point at various ways of making the experience of this music player a little special. In particular, most existing players are still nothing more than a fancy dressing over music tracks (i.e. files).

But to make a really rich and complete experience, I believe that one must embrace music as a culture; promoting entities (e.g. artists, genre, releases, etc) to key navigation points and tying it to all the information available on the web would be a good start in that direction.

Darth Cee-Lo, by Ethan Hein

All graphical application projects face the dreaded perspective of endless flamewars about technical choices, and XMMS2 clients haven’t been spared:

What language to write the client in? What toolkit to use? What platform to support? Where to put the opening curly brace?

In fact, this freedom of choice seems to always have been one of the major obstacles to the creation of an official GUI client.

At FOSDEM ‘09, perhaps helped by the virtues of IRL discussions (and all the great food & beer), we had a debate about the pros and cons of all possible combinations and came up with a reasonable proposition that should hopefully suit most interested developers in the XMMS2 community.

Because we don’t want to require users to install unusual graphical toolkit libraries (good morning Rasterman), GTK+ and Qt are the only two obvious choices, with GTK+ having the slight advantage of being more widely installed in the GNU/Linux world. However, nobody really contests the claim that Qt is a superior API in terms of design.

One important argument brought into the discussion was platform support. In line with the XMMS2 Vision, we want to respect the users’ freedom to choose their OS by making the official GUI client run on a wide selection of systems, at least GNU/Linux, MacOS X and Windows. In the current state, GTK+ integration in MacOS X simply isn’t good enough, and given the new Qt licensing as fully Free Software, the Qt toolkit is the most appropriate choice.

The other touchy subject is obviously the programming language. Most people agreed that using high-level interpreted languages would make for a more dynamic and simple development process. The most popular contestants for the code throne are Python and Ruby, with Python more widely used and installed on GNU/Linux.

Either choice would require bundling the runtime with the client on proprietary platforms. A recent article on Ars Technica analyzed the deployment of PyQt apps on Windows and OS X, and while it seems reasonably good on Windows, the OS X support was quite lacking according to the author.

An alternative solution suggested by DraX was to develop the client QtScript, which is the ECMAScript (think JavaScript) implementation embedded in the newer Qt. It remains uncertain, however, how much can be done purely in QtScript and where are the limits, both in terms of performance and design limitations. tru hinted at it, but I think it’d still be worth investigating and experimenting before we draw final conclusions.

If QtScript was too limited, we’d have to rely on more C++ code (which is somewhat annoying for rapid development) or switch to Python (which isn’t particularly my language of preference, as some may know). In case of a draw, it might also simply be up to the people who start working on the project, including a potential Google Summer of Code student with XMMS2!

As tru suggested, it’d probably be best to use the native C++/Qt XMMS2 bindings, and stabilizing them could be part of the preparatory work for this project (and a good motivation).

In short, the current idea is to write the new XMMS2 client using Qt and C++/QtScript or possibly PyQt, and to have it available at least on the main desktop platforms (GNU/Linux, MacOS X, Windows). Discussions still open, but avoid feeding the trolls please.

Darth Vader on the violin, by Houston Marsh

We need a dream!

I have a dream that one day this community will rise up and live out the true meaning of its creed: “when the music is end xmms power of the pc.”

I have a dream that one day on #xmms2, the sons of former GTK+ developers and the sons of former Qt developers will be able to work together on the code of a GUI client.

I have a dream that one day even the state of Microsoft, a state sweltering with the heat of instability, sweltering with the heat of dull interfaces, will be transformed into an oasis of freedom and music.

I have a dream that XMMS2 clients will one day live in a community where they will not be judged by their GUI toolkit but by the content of their user experience.

I have a dream today!

(Kudos to Martin Luther King for the original draft.)

In the end, all we want is an awesome music player.

But do we all share the same definition of what an awesome music player is? Probably not. In fact, I don’t think most of us even knows what an awesome music player is; if there were one, we wouldn’t have to build one, right?

So the goal here is to gather qualities that we expect of such a project, and refine them into a common vision. I won’t start dropping design mockups or fancy feature ideas until we have established what we all want, conceptually.

What I expect of this GUI client, and the vision for the project, is that it should be:

• Exciting
• Original
• Expandable
• Clearly focused

Nobody wants to work on boring project — at least nobody in the FOSS community. Elsewhere, people do code for money and dream of larger cars and larger breasts, but in the XMMS2 community, all we dream of are exciting coding projects. It should be exciting enough to make developers drop their own projects to work on it, and to make users fret about it. It should be exciting enough to compensate compromises by the quality of the end result.

One way to make it exciting is to make sure it is original. It’s way more thriving to build something new and unique than to try to replicate something everyone has seen before. Harder, yes, but more exciting! For the users, it will also help it stand apart from the variety of existing music players, either as a grand fiasco, or as a sexy newcomer.

Because people love to experiment and to make things their own, it should be expandable: rather than a static monolith, it should let developers and users play with it and customize it and adapt it to their needs. There will always be limits of course, but to remain true to the XMMS2 spirit, we should favor a modular design and bundle freedom inside.

Finally, it is primordial to establish a clear focus on what problem this client is meant to solve. The worst usability often comes from a blurry focus, or the wish to solve too many (or all) different problems.

I will come back to the first three qualities in future posts, and elaborate on the clear focus in the rest of this post.

Before anything else, we need to define what we want the target audience to be: newbies? your mum? the average random user? hardcore music fans? “everyone”?

It’s a tough call, but clearly, by trying to content everyone, we couldn’t provide the best solution for each group of users. But if we look at the XMMS2 demographics and, more importantly, the (brand new) Vision, it’s easy to see that we already target a particular niche of music listeners: demanding audiophiles, passionate fans who care about music. Which is very different from “everyone” or an average user.

Concretely, they might tend to have larger music libraries and more complete releases than scattered tracks. They might be more keen on browsing and organizing their music (using tags, folders, or their own semantics), on fine-tuning their audio setup (soundcard, equalizer, gapless playback), on joining music networks (e.g. Last.Fm) and discovering new music. They are the people who spend multiple nights getting complicated plugins and fancy themes working in foobar2000, to provide a full experience for their music; not the people who gaze in wonder at the atrocious WMP fullscreen visualization.

We can expect slightly more patience and curiosity from them but in return, we must provide them with powerful tools, with a great user experience and ways to make it their own.

Now, it doesn’t mean that the player should be unusable by anybody outside that niche. Simply, it should focus on filling it as best as possible, before anything else.

Therefore, my suggestion is to make this XMMS2 GUI client a great music player for people who care about and love music, and make it a rich experience for playing, browsing, searching, organizing, discovering and enjoying music.

It is an ambitious goal, but I believe it is one that is exciting, original and expandable!

Pearl, by Camila…

Addendum: FLACvest just posted a
very flattering post about XMMS2 and mentions similar attributes
that would benefit XMMS2: Totally Fresh and Unique, Beautiful,
Cross-Platform, some “magic” ingredient!

But, like, all your GUI clients suck lol!

You needn’t have punched the poor guy in the face, because I also believe it is quite true.

We have quite a few clients (you will have to check out the matrix by yourself), but none that can really be called polished, mature or attractive enough to motivate people to switch. No disrespect to the authors (I’m even one of them), some cool stuff has been done, but graphical clients aren’t really “ready for the desktop” yet. Which is really a pity, given the awesome framework we have in place to create a great music player.

I believe that the reason for this isn’t that we don’t have competent developers, but rather that all of these projects are one- or two-men effort and not really thrived by the community. Because of the freedom you have to write your own client, well, people do: they follow their personal vision of what they want their music player to be like and start coding. Unfortunately, this isn’t the most effective strategy.

Perhaps thanks to the physicality of our meeting at FOSDEM, or to all the Belgium beer, we acknowledged that and decided that, now that the new official command-line interface (AKA nycli) has been merged and is being worked on by various people (thanks greafine, AnthonyG, nesciens!), it was time to get serious about a graphical client.

The best way to get people to work together on a new project is to establish a clear and common vision of what we’re aiming at, what are the ideas that structure the project, and get people excited about it! The vision should be compatible with the XMMS2 vision we’re currently discussing (thanks to Debian’s Bdale Garbee for the inspiration).

I want to make it clear that even if members of the XMMS2 community start working together on a common client, it is in no way incompatible with other people writing their own client if they so wish. Simply, we hope to gather people working on similar clients and focus the effort to build something really, really cool!

So I’m going to start posting things on this blog to propose directions for a vision to follow, and everyone is welcome to comment, reply, blog and debate the ideas so we solidify the basis we will start from. There will be a lot of things to discuss, from technical choices to interface challenges, so let’s get started!

Just send me a blurb and selected screenshots, and you’ll get instant fame.

The first client to be announced is Konfetka, a Qt GUI interface recently released by afrol and BatteryCell. Try it out, it’s quite original and feature-complete!

Ding dong. New client?
Yesness.But… but… what is it?

Konfetka — It’s the abandoned kid of Amarok and XMMS1 who spent too much time around Yakuake when growing up and now is powered by XMMS2 and kryptonite, if those aren’t the same thing.

Or, in practical terms:
We got this little thing that we’ll be binding to KDE 4 when that comes out. However, even right now, it has wiki, album art, lyrics and playlist/medialib interfaces. The idea was to have a player that would appear and hide like Yakuake, rolling down from the top of the screen and hiding back up. Code’s messy, bugs are plentiful, but hey, its our first release and it is usable; so give it a try.

Here are some screenshots of version 0 alpha 1 (click to enlarge):

We think it’s rather cool.

Konfetka. Come visit us at http://code.google.com/p/konfetka/.

~Konfetka devs. A. Frolenkov, Joey Lynch.

The contest page on the official website isn’t quite as verbose as one would have hoped, though, and the link to the rules points to a missing webpage. Some more intel is however available in a blog post on Michael Robertson’s blog. Yes, Michael MP3.Com/Linspire Robertson, who is also Michael MP3Tunes Robertson, obviously.

The deal: you have until November 5th, 2007, to submit an interface to MP3Tunes in one of the ten predefined categories (Desktop Player, Game Console, WebUI, PDA, etc). The winner in each category will receive a cosy $1000. I haven’t determined whether it’s a public vote, a closed jury, or some other semi-random algorithm involving car-driving girafes.

I haven’t yet checked what the API offers, either, but I sure wonder whether we could hack something together with XMMS2. If it could be hooked into the daemon, and apart from the obvious ever satisfying benefit of being neat and fun, it might allow using any of our interfaces (AKA clients) to play music from MP3Tunes.

From what I see, MP3Tunes isn’t much more than an online storage for music files, that you either upload or retrieve from wherever on the web. (Let’s leave the obvious legal issues to Michael’s lawyers, shall we?) This means that you have a whole medialib on their server, apparently indexed and searchable and, naturally, playable. If I’m not mistaken, the “streaming” seems to be nothing more than retrieving the music file (original or transcoded to mp3) over HTTP, which XMMS2 already supports.

In the current scheme, our medialib assumes to contain all the media and their metadata, so unless this is extended to support distributed media libraries (hello, Summer of Code 2008?), it means we would have to sync the remote MP3Tunes medialib to the local XMMS2 medialib. Sounds a bit tedious and expensive and ugly, so suggestions are welcome with better solutions!

These are the few implementation considerations I can think of off the top of my head, but overall it doesn’t sound so complicated, unless we want the support to be very smart (async, allow upload/download media, etc).

Of course, the whole contest looks like a call for devs to code cool interfaces to a money-making online service (mostly) for free, more than anything else, but let’s just consider the idea for a minute and whether it could justify spending some energy on writing one or several polished clients (which I think most of us agree we need).

And transparently sharing music through a (currently) free, password protected, remote service can’t be such a useless idea, can it?

Wasted place all over the place, too small text placed within too large widgets, inconsistent style, a festival of duplicated information in the playlist view on the right, the ever-dreaded tabs on the left (yes, tabs are usually evil).

Seriously.

Someone has to wake up and fix this but hey, guess what, it’s just a mockup, so they might!

What is more interesting, however, is the shift of focus in the way users are supposed to interact with the player. The developers make it clear that they want to make the “context browser” central, and reduce the importance of the playlist. I’ve been advocating something similar since the early days of Collections (already in my initial manifesto), but I find it bold of them to push this change in an already widely popular program.

Users will be shocked and disturbed, so it better be polished enough to prove how superior this approach is. In particular, the interaction between the context view and the playlist needs careful attention, and the edition of playlists should be facilitated.

In the current mockup, I find it worrying that they don’t seem to find it useful to give easy access to multiple saved playlists. Right now, you need to go from the playlist to the other side of the screen, and hunt for the (two?) playlist tabs (which seems to require cocking one’s head sideways by 90° unless you’re Mad-eye Moody). I certainly hope they will optimize the interaction flow as much as the interface.

Whether the idea and its implementation are good isn’t even as relevant as the observation that you cannot satisfy all users with a single program. User interaction and preferences is a multi-dimensional space, and you can never span it in every dimension.

An interface is a compromise. Usability defines how large the compromise is, and how much the application satisfies the goals it aims for, in the user’s experience.

I believe that this is one of the qualities of XMMS2, which solves the technical problems for the end-developers and lets them focus on providing a wide variety of complementary interfaces, all backed by the same daemon. Of course, this modularity is a performance tradeoff, and the daemon isn’t optimized for every possible use. Still, it calls for new experiments and ideas. And with the approaching end of the Summer of Code, new possibilities (GenIPC, service clients, visualization, test framework) are at the door!

The number of music players on Linux has been steadily increasing lately, but while these projects have been getting more and more polished, we have yet to see revolutionary improvements in terms of user experience. Indeed, the trend has privileged borrowing as many features as possible from other projects, rather than questioning the reasons behind their design.

This article describes XMMS2’s attempt to address long-standing limitations of music players, through its new support for Collections. First, I will summarize the rationale behind this feature, then present its concept and implementation. A conclusion about the current state and future directions of Collections will close the article.

Design rationale

I have been concerned with the state of music players for a long time. Two years ago, I wrote a Manifesto for a Better Music Player. Although my ideas have evolved since then, the general conclusions of that article still hold.

One important argument I made is that the design of a music player should focus on the users’ needs, rather than on a list of well-known features. All the traditional features (playlist, media library, cover browsing, etc) and hacks (play queue, random mode, etc) stem from the needs users have for:

• playing music non-linearly
• searching for specific media
• browsing their media library
• organizing their music

Non-linear playback was first introduced in a crude form as the “random mode”, directly inspired from legacy CD players. iTunes later popularized its “Party Shuffle” mode, which solved the unpredictability of playback by maintaining a queue of randomly selected songs. What we are still waiting for, though, is a smarter mode that would also take into account beat, artist similarity, or other semantic information.

All music players based on a media library provide a search feature. Unfortunately, its power is often hindered by annoyingly complex forms used to choose the fields to query. Few developers seem to have noticed the success of Google’s search interface: minimalistic, but enriched by rating heuristics and a rich syntax for advanced users.

The other axis required by our ever-growing music libraries is browsing. Media library browsing is always present in some form, although mostly simplistic and uninspired. When they are not cloning iTunes genre/artist/album filters or cover art browsing, most music players simply present the users with the list of all their media in a plain multi-column layout. Easy to implement, but hard on the eyes for the users. Interestingly, Foobar2000 (freeware) is the only popular player to allow a rich customization of the layout, which greatly improves readability.

The lack of features that help users organize their media library contributes to the difficulty of addressing the two previous issues. In the physical world, users can arrange their CDs spatially in their own personal way (by artist, date of release, mood, etc), set a couple of albums aside for playing at a party, or highlight their latest acquisitions on a shelf. This lets them build a cognitive map of the location of items. On computer-based music players, however, they are barely provided with the possibility to create playlists, possibly dynamic, but seldom integrated well enough to be used powerfully. Even bare files have richer organizational possibilities, using directories!

The reason behind these limitations is not that they are inherently unsolvable. The truth is that a lot of effort is required to implement new approaches in any of these fields. Experimentation, either conceptual or in terms of interface, is expensive.

The Collections concept

The goal of Collections is to address this problem by creating a common abstraction layer. Search, browsing and organization all share one property: they act on subsets of the media library. Computers are especially good at handling sets, but music players haven’t really exploited that fact yet.

A collection is defined as a subset of the media library.

This set of media (songs) can be dynamic, for instance “All media by Kraftwerk released prior to 1980″ or “All media added to the media library last week, except those by Justin Timberlake”. A static set, for instance hand-picked media selected for parties, is just a special case of dynamic sets.

Note that a collection is not merely what some players call a “Smart Playlist” (or “Dynamic Playlist”). A “Smart Playlist” is only used to play an arbitrary list of media, while a collection is a generic representation of a set of media. For instance, this includes the results of a search, a filtered view of the media library, the list of tracks from a given album, etc.

Because a collection is an abstract representation, it can be used ubiquitously throughout all the features of the music player: browsing, searching in the media library or the playlist, enqueuing, jumping, etc. A collection can also be saved on the server, thus allowing the users to organize their music and reuse their selection in homogeneous and flexible ways.

Collections for the XMMS2 player

The XMMS2 project turned out to be the perfect ground to implement collections. Unlike its popular predecessor XMMS, XMMS2 hasn’t gathered much attention yet. However, it features all that you would expect from a recent music player: a media library, support for many audio formats and multiple platforms (Linux, *BSD, OS X, Windows, etc), bindings for many languages (C, C++, Ruby, Python, Perl, Java), and a friendly community open to innovation.

In addition, the player was designed according to a client-server architecture, so that the server is responsible for all the boring work (audio decoding, media library management, tag extraction, etc), while any flavor of user interface can be implemented as a client connected to the server, possibly across the network.

Collections have been implemented in XMMS2 as a student project during the Google Summer of Code 2006, and finally merged into the stable tree on May 20, as part of the DrJekyll release.

Support for collections was implemented on the server, as a layer above the media library and playlists (more about that in the implementation details), and exposed to the clients through a collections API. This API allows clients to save collections on the server, query the media library, enqueue the content of a collection, etc. Thus, although the user interface depends on the client, the server and the clients all share the same abstract representation.

Clients are also freed from the need to generate complex SQL queries themselves; instead, they can easily build a (DBMS-agnostic) collection and the tedious query is performed by the server. In addition, a parser is provided to generate a collection from a string with an enriched search syntax.

Collections make it essentially trivial to browse and search the media library. Moreover, advanced features are either natively available or very easy to implement: iTunes-like Party Shuffle, recursive filtering (e.g. search inside the playlist), display Top 10 or never played songs, changing the equalizer settings if the playing song is in a particular collection (e.g. “Jazz Vinyl rips”), etc.

Implementation

Strictly speaking, collections are implemented as a directed acyclic graph (DAG), each node of which is a collection operator. In fact, because the structure is recursive, each node of the graph corresponds to a collection. This model was chosen to emphasize the aggregated nature of users’ music collections.

Collection operators come in four different flavors:

• set operators
• filter operators
• list operators
• reference operator

The set operators take an arbitrary number of operands and returns the collection obtained by applying the corresponding set operation to them. For instance, “any music by The Beatles or any music by The Rolling Stones”. Available set operators: union, intersection, complement.

The filter operators enforce conditions on properties of the media; the resulting collection only contains the media that match the filtering attributes. For instance, “all the songs with ’stairway’ in their title“. Available filter operators: equals, match (partial matching of strings using wildcards), larger/smaller (for numbers), has (checks whether a property is present).

The list operators are a bit special. The basic list operator (called “idlist”) does not accept any operands; instead, it simply generates the collection corresponding to the custom list of media it contains. Because list operators store static, ordered lists of media, they are used as playlists in XMMS2! Available list operators: list, queue (pop songs once they have been played), Party Shuffle (takes an operand, used to randomly feed the list with new entries).

The reference operator is simply used to refer to the content of a saved collection or playlist. For instance, “all the songs released in 2007 in the Foo playlist“. A reference operator is also used to refer to the whole media library (all media).

Now, let’s illustrate all this with a sample collection structure:

The nodes represent collection operators, while edges simply connect operands to operators.

Here, “All Media” is a reference to the whole media library, and we use a Match operator to only keep media for which the artist has a name starting by “A” (1). We then take the union (3) of this and the content of the “Rock 90’s” saved collection (2). The result is passed as an operand to a Party Shuffle operator (4), which we save under the name “Interesting” (5).

When the user plays the “Interesting” playlist, songs are popped from the list as soon as they are finished, and new songs matching the operand collection (3) are automatically enqueued, so that the list always contains at least 20 items. This is specified by the “size” attribute of the Party Shuffle. Of course, the user can also edit the playlist and add tracks to it manually.

This is only one example of collections among many. As you can see, the modular structure of collections allows virtually unlimited possibilities. As such, they have been tightly integrated both on the server and in the client API.

On the server, a dedicated module is responsible for handling collection features. When a collection is queried, it serializes the structure into an SQL query, runs it in the media library and returns the matching media, either as a list of media ids or hashes containing the requested media properties. When a collection is saved on the server, it is added to the collection DAG and kept in memory while the server is running. On shutdown, the whole DAG is serialized into the database. Note that playlists are nothing but collections, albeit restricted to list operators and saved into a dedicated namespace.

In the client API, collections introduced many important changes. First, executing raw SQL queries has been deprecated; all queries are now to be performed using collections. Collection data structures can be built either using a set of dedicated functions, or by calling the collection parser on a string given by the user. Finally, many XMMS2 methods have been extended to support collections (e.g. to enqueue media) and new methods allow clients to query, save and retrieve collections from the server.

If you want to learn more about the concept of collections, please have a look at the collections concept page on the XMMS2 wiki. For more details about the implementation, check the collections design page and the API documentation.

Adoption and future directions

Several XMMS2 clients have started offering features based on collections, including Abraca (GTK2 client) and gntxmms2 (console client). Other clients have ported search and browsing to the collections API: Esperanza (Qt4 client), gxmms2 (GTK2 client) and the official command-line interface.

Hopefully, client developers will start exploring new directions now that collections are in the main release. The xmms2 CLI client has already been scheduled for a full rewrite.

Several improvements are also expected to address current limitations of the collections implementation. One limitation is that all collections are treated equally as media sets; if a filter is applied on a playlist, the order and duplicated items will be lost. A smarter internal distinction between lists and sets inside the DAG is in the works. An ordering collection operator could then be introduced to transform a set into an ordered list, as well as an operator to select subsequences of such lists, similarly to SQL LIMIT operation. They could be used to create a collection containing the “list of the 20 most recently added media”. The SQL query generator could also be further optimized, unless we decide to replace the database backend completely.

Collections have just made it into the official XMMS2 distribution, but people already use them through features like search, Party Shuffle or groups of songs saved in the media library. They are a powerful toy for developing new features in the clients and hopefully helping users organize and use their music library.

It’s an exciting time to come up with fresh ideas in the XMMS2 world, and I hope the rest of the developers in the music player community will take the time to reflect on and discuss all these questions earnestly!