insomnia bytes

Or why and how the new XMMS2 GUI client should be extensible (and what I mean by extensible anyway).

Quizz: What do Winamp, Foobar2000, Emacs, Eclipse and Firefox have in common?

(Vim users, we love you too, but shut up for now please.)

They are very popular and they highly promote extensibility.

Now, we all know that correlation != causation. What is certain, however, is that all those projects boast many fan(boy)s, some of whom even get quite religious about it.

This enthusiasm reflects itself in the emergence of strong communities: people who share content, tips, modules, extensions, themes, configurations, scripts, etc. And those people usually aren’t even core developers of the software, sometimes not developers at all; just users or hobbyists who like the project.

Paradoxically, most of what they share is aimed at personalizing the software, i.e. making it the user’s own, fitting the user’s wishes and needs. This creates a feeling of ownership and satisfaction, because the user can bend it the way she likes and master it like a pro.

In essence, the idea here is give power to people to shape the software the way they want, and they will extend it in creative ways and get together to share these extensions. And I think that encouraging this creative freedom is one of the many factors which contribute to popularity.

Obviously, each of the projects mentioned above have their own way of allowing user extensions.

• Classic Winamp 2.* had (lots of) theme and some plugins (mostly effects, visualization), but remained otherwise quite static in terms of interaction or features.
• foobar2000 goes much further with theming and interface customization, allowing varied usage and user experience.
• Eclipse focuses mainly on extension modules (to interact with the VCS, manage project workflow, etc) and a jungle of configuration options.
• Firefox boasts rich installable extensions which add powerful new features, two levels of configurability (standard Preferences and advanced about:config), and recently themes (Personas).
• emacs allows essentially everything through scripting: altering and extending looks, interaction, features, as long as you have a high enough Elisp-fu (or enough curiosity to install other people’s modes).

More to the point, extensions can be mapped along three mostly orthogonal axes:

• Themeable appearance (look)
• Configurable interaction (feel)
• Scriptable features (personal usage)

Let’s take each of them and see what it means for our beloved imaginary music player.

Themeable appearance

Not an easy one, especially when one wants to rely on advanced features and widgets from a toolkit, which doesn’t necessarily allow for heavy reskinning.

There are more and more ways to draw fancy stuff in a window (vectorial canvas, HTML view, etc) but those often come at the price of limited interactivity with the rest of the interface (drag and drop, event handling, interoperability with the MVC architecture).

Using Qt StyleSheets (possibly with the help of QtScript) to style native widgets might be a better option, though I haven’t yet investigated in depth how much can be achieved with them.

Anyone with a better clue is welcome to step up.

While the whole layout of the player could benefit from imaginative and beautiful theming, one of the oft talked about use of styling was for the rendering of the playlist (usually out of jealousy for gorgeous foobar2000 screenshots), with album grouping, sexy styling of information instead of boring columns, etc.

Viewing and browsing the medialib could definitely benefit from a rich, themeable design, which would ideally (and optionally?) smoothen navigation with animations (see a hastily put together demo I made with jQuery, a while ago).

My foobar2000 – 6-28-08, by BestTechie

Customizable interaction

Power-users get easily nervous about GUIs, as they fear that their personal preferences might not be respected:

Will there be a STOP button, or only PAUSE?

The best way to approach such religious questions is to set a default reasonable with regards to the rest of the default settings (i.e. coherent with the “default experience”), and let the user change it if she wants to.

Note that I put as much emphasis here on configurability as on picking a sensible default. In parallel, fine-grained options should be hidden away from the main configuration panel to keep it usable — a standard configuration pane similar to Mozilla’s

can do the trick.

Will it require me to reach for the mouse?

I believe that a complete GUI player should be usable either exclusively with the mouse, exclusively with the keyboard, or with a combination of both.

The keyboard access is often assimilated to keyboard shortcuts, and indeed advanced users often rely on shortcuts to work quickly with an interface. They should even be able to define their own shortcuts bound to arbitrary actions (jump to the playing song, enqueue the album for the track I’m viewing, etc).

However, another underused keyboard-based interface has been making a comeback into GUIs lately. You might have heard of it, it’s called the command-line.

Apart from the venerable emacs and vim, command-lines have been creeping more or less discretely into various Mozilla projects. Bespin, the web-based text editor has attempted at allowing users to run smart commands directly in the editor. And of course, the Firefox 3 Awesome Bar is just a step away from a command-line; a step taken by the TaskFox (see demo), which brings Ubiquity into the browser address bar.

I had experimented with the idea of a CLI-based GUI (sounds strange doesn’t it?) with my unfinished lindalë prototype, and I think it could bring a lot of interesting power to our new graphical player.

More on this idea in a future post.

Scriptable features

As usual, the level of scriptability is a trade-off decision, somewhere between the horror of Excel inline formulas (a sad ad-hoc hack most of the time) and the universal meta-ness of emacs (which allows the editor to rewrite itself when you’re not looking). The latter is naturally attractive to us geeks, but it’s probably excessive in terms of work required.

On the other hand, if we pursue our idea of using a high-level
language for the GUI (possibly QtScript, which is being worked on), it should be relatively easier to import self-contained extensions into the player.

In terms of design, it would be preferable to think of such “extensions” as features of the player, rather than external add-ons bolted onto it. The player itself would therefore be nothing more than a platform hosting extensions/features, and even the default interface would be implemented as extensions, indistinguishable from third-party ones.

This would for instance allow anyone to tweak the widget that displays search results, or replace it with something completely different. Even better, entirely new features could be imported, e.g. a rich interface exposing a bookmarking system (managing a generic bookmarking service client under the scene), or new widgets to explore one’s music library.

Possibilities are endless, and our time to work on them isn’t.

I have been discussing some of those ideas with greafine, our student working on this project for this year’s Summer of Code, and we’ll work together to determine what is feasible technically and temporally. In any case, I hope this post hints at various ways we shall explore to achieve high extensibility!

starwars_rock, by Clint M Chilcott

Now that we have defined what/whose problem we’re trying to solve, and debated about the implementation details, it would be worth asking why a graphical XMMS2 client would be a good fit.

After all, we have a brand new korving CLI (nycli), isn’t that enough? In a sense it is, but it fills a different niche. GUI applications are good at things that CLI applications aren’t, and vice-versa. So the goal is to exploit the specific advantages of graphical music players.

For instance, even the most hardcore fans of the command-line will admit that the following tasks are easier with a graphical player:

• Edit a playlist, using mouse selection and drag-and-drop.
• Browse albums by cover.
• Organize music manually into playlists or using dynamic collections.

But these are just simple examples that are now expected of any standard graphical music player.

Can’t we get something more exciting?

As Obama taught us, “yes, we can!”

Three main aspects usually poorly supported and under-exploited in music players are powerful tools to:

• Browse
• Organize
• Explore/discover

Browsing

Graphical applications provide a rich visual experience that can be exploited to navigate large amounts of information, in particular using the spatial aspect and our ability to recognize images quickly.

Typically, users have become familiar with widgets like iTunes Cover Flow, which exploit the visual clue of album covers to quickly flick through releases. Several XMMS2 developers have expressed their interest in a more “album-centric” client, which essentially means supporting album entities in the interface, as opposed to just tracks.

An extra step would be to also promote the artists, and possibly other properties (e.g. genre, year, label), as premium entities. So rather than “album-centric”, the client would be “entity-centric” (in the sense that most existing music players are track-centric). Each entity could have its own fullscreen pane (or “page”), with corresponding information (more below) and links to browse the related entities. This would lead to a web-like navigation, where for instance each artist would get a page with the list of his releases (plus a photo, bio, etc), and clicking on a release would bring the page for that release, with the list of tracks.

However, browsing doesn’t have to follow a rigid path: we’re used to browsing inside categories (e.g. releases) alphabetically, and to jump between categories using the explicit hierarchy from the data (e.g. from an artist to its releases to their tracks), but that’s just one of many possibilities.

The user may want to browse a subset of her media library, for instance filtering by a range of release years and genre (e.g. “70’s rock”), or a custom collection she assembled herself. She may want to follow connections from an artist page to pages of “related artists”, to use tags to jump from a track to a list of albums, to find all the music she added the same week as a given release.

It’s time to think beyond the simple local data and harvest The Cloud to enrich the user experience; services like Last.Fm already provide an API to retrieve Similar Artists, social Tags, etc. And with our Collections API, it’s just a whole lot of power waiting to be unleashed!

Organizing

While browsing is the passive process of visiting what’s there, organizing is the active process of applying your own order on the content.

Playlists are the most common organizational tool, usually directly tied to playback, and the usual editing features should naturally be supported (insert, enqueue, move, remove). Special playlist behavior (queue, party shuffle, random, etc) should also be configurable easily. (Note: fancy playlist formatting is outside the scope of this post.)

The second main organizational tool is collections. A collection is akin to a “themed-bucket”, i.e. a set of music that the user has put together in order to reuse it later. But rather than focusing on the underlying nature of a collection (a graph of operators), the interface should emphasize the organic process of someone creating a custom group of music. Any search, or essentially any “view” of music, should be recordable as a collection; and it should always be possible to refine or further filter a collection, as well as add custom content to it. It should be as easy as typing a search or dropping content in a folder, rather than as complex as setting up a network of mail filters.

Finally, tags should be at the user’s disposal for applying a minimal description to content. There is a subtle difference between tags and collections (and how they play together), and I don’t have much in mind regarding tags so far, but I think they will definitely be a powerful addition.

Exploring/Discovering

In the browsing section above, different new ways to navigate one’s music have been evoked. The next logical step after that, however, is to help the user discover new music he doesn’t yet know about, by giving pointers to music and information outside of his media library.

Many online services offer to make you discover new music, but this feature remains unusual in desktop music players, except Spotify (with a tab of related artists) and recent versions of Banshee (with custom recommendations, perhaps based off Last.Fm?).

Those illustrate two interesting directions.

First, providing pointers from a certain point in the user’s media library to complementary information and new music. For instance, on an artist pane, show the list of all its releases, including those missing from the user’s music files, show related artists, both present in the user’s media library or not, show the artist’s news feed, etc. Or show reviews for albums, or lyrics for individual tracks. Basically, gather information from external sources about the user’s music, and invite him to discover new music as well.

The second direction is more general: given the user’s music profile and playback history, suggest new artists or genres he might be interested in exploring. For instance, using all the artists present in the media library, infer what will be the user’s next favorite band. Or suggest popular releases, based on the kind of music recently played. Here, suggestions are made spontaneously, based on the behavior of the user.

The goal of these two features is to get the user excited about not just his current music files, but the whole portion of the music world they span. They should invite the user to be curious about his or new music.

One place to put these suggestions on would be the music player’s “home screen”. Without entering into too much details yet, the player could provide the user with ideas on what he might want to listen, based on what he played recently, his collections, tags, etc. Rather than a long table of “all the tracks”, the entry screen could be a richer, custom view of different ways he could start playing his music.

Obviously, this post hinted at a lot of potentially complex features, which would take a lot of time and effort to all put together perfectly. The main point, however, was to point at various ways of making the experience of this music player a little special. In particular, most existing players are still nothing more than a fancy dressing over music tracks (i.e. files).

But to make a really rich and complete experience, I believe that one must embrace music as a culture; promoting entities (e.g. artists, genre, releases, etc) to key navigation points and tying it to all the information available on the web would be a good start in that direction.

Darth Cee-Lo, by Ethan Hein

Note: tru just blogged about this subject, so I’m adapting my original draft and replying to tru’s post.

All graphical application projects face the dreaded perspective of endless flamewars about technical choices, and XMMS2 clients haven’t been spared:

What language to write the client in? What toolkit to use? What platform to support? Where to put the opening curly brace?

In fact, this freedom of choice seems to always have been one of the major obstacles to the creation of an official GUI client.

At FOSDEM ‘09, perhaps helped by the virtues of IRL discussions (and all the great food & beer), we had a debate about the pros and cons of all possible combinations and came up with a reasonable proposition that should hopefully suit most interested developers in the XMMS2 community.

Because we don’t want to require users to install unusual graphical toolkit libraries (good morning Rasterman), GTK+ and Qt are the only two obvious choices, with GTK+ having the slight advantage of being more widely installed in the GNU/Linux world. However, nobody really contests the claim that Qt is a superior API in terms of design.

One important argument brought into the discussion was platform support. In line with the XMMS2 Vision, we want to respect the users’ freedom to choose their OS by making the official GUI client run on a wide selection of systems, at least GNU/Linux, MacOS X and Windows. In the current state, GTK+ integration in MacOS X simply isn’t good enough, and given the new Qt licensing as fully Free Software, the Qt toolkit is the most appropriate choice.

The other touchy subject is obviously the programming language. Most people agreed that using high-level interpreted languages would make for a more dynamic and simple development process. The most popular contestants for the code throne are Python and Ruby, with Python more widely used and installed on GNU/Linux.

Either choice would require bundling the runtime with the client on proprietary platforms. A recent article on Ars Technica analyzed the deployment of PyQt apps on Windows and OS X, and while it seems reasonably good on Windows, the OS X support was quite lacking according to the author.

An alternative solution suggested by DraX was to develop the client QtScript, which is the ECMAScript (think JavaScript) implementation embedded in the newer Qt. It remains uncertain, however, how much can be done purely in QtScript and where are the limits, both in terms of performance and design limitations. tru hinted at it, but I think it’d still be worth investigating and experimenting before we draw final conclusions.

If QtScript was too limited, we’d have to rely on more C++ code (which is somewhat annoying for rapid development) or switch to Python (which isn’t particularly my language of preference, as some may know). In case of a draw, it might also simply be up to the people who start working on the project, including a potential Google Summer of Code student with XMMS2!

As tru suggested, it’d probably be best to use the native C++/Qt XMMS2 bindings, and stabilizing them could be part of the preparatory work for this project (and a good motivation).

In short, the current idea is to write the new XMMS2 client using Qt and C++/QtScript or possibly PyQt, and to have it available at least on the main desktop platforms (GNU/Linux, MacOS X, Windows). Discussions still open, but avoid feeding the trolls please.

Darth Vader on the violin, by Houston Marsh

As I said earlier, what’s lacking to make an awesome XMMS2 GUI client is a common vision.

We need a dream!

I have a dream that one day this community will rise up and live out the true meaning of its creed: “when the music is end xmms power of the pc.”

I have a dream that one day on #xmms2, the sons of former GTK+ developers and the sons of former Qt developers will be able to work together on the code of a GUI client.

I have a dream that one day even the state of Microsoft, a state sweltering with the heat of instability, sweltering with the heat of dull interfaces, will be transformed into an oasis of freedom and music.

I have a dream that XMMS2 clients will one day live in a community where they will not be judged by their GUI toolkit but by the content of their user experience.

I have a dream today!

(Kudos to Martin Luther King for the original draft.)

In the end, all we want is an awesome music player.

But do we all share the same definition of what an awesome music player is? Probably not. In fact, I don’t think most of us even knows what an awesome music player is; if there were one, we wouldn’t have to build one, right?

So the goal here is to gather qualities that we expect of such a project, and refine them into a common vision. I won’t start dropping design mockups or fancy feature ideas until we have established what we all want, conceptually.

What I expect of this GUI client, and the vision for the project, is that it should be:

• Exciting
• Original
• Expandable
• Clearly focused

Nobody wants to work on boring project — at least nobody in the FOSS community. Elsewhere, people do code for money and dream of larger cars and larger breasts, but in the XMMS2 community, all we dream of are exciting coding projects. It should be exciting enough to make developers drop their own projects to work on it, and to make users fret about it. It should be exciting enough to compensate compromises by the quality of the end result.

One way to make it exciting is to make sure it is original. It’s way more thriving to build something new and unique than to try to replicate something everyone has seen before. Harder, yes, but more exciting! For the users, it will also help it stand apart from the variety of existing music players, either as a grand fiasco, or as a sexy newcomer.

Because people love to experiment and to make things their own, it should be expandable: rather than a static monolith, it should let developers and users play with it and customize it and adapt it to their needs. There will always be limits of course, but to remain true to the XMMS2 spirit, we should favor a modular design and bundle freedom inside.

Finally, it is primordial to establish a clear focus on what problem this client is meant to solve. The worst usability often comes from a blurry focus, or the wish to solve too many (or all) different problems.

I will come back to the first three qualities in future posts, and elaborate on the clear focus in the rest of this post.

Before anything else, we need to define what we want the target audience to be: newbies? your mum? the average random user? hardcore music fans? “everyone”?

It’s a tough call, but clearly, by trying to content everyone, we couldn’t provide the best solution for each group of users. But if we look at the XMMS2 demographics and, more importantly, the (brand new) Vision, it’s easy to see that we already target a particular niche of music listeners: demanding audiophiles, passionate fans who care about music. Which is very different from “everyone” or an average user.

Concretely, they might tend to have larger music libraries and more complete releases than scattered tracks. They might be more keen on browsing and organizing their music (using tags, folders, or their own semantics), on fine-tuning their audio setup (soundcard, equalizer, gapless playback), on joining music networks (e.g. Last.Fm) and discovering new music. They are the people who spend multiple nights getting complicated plugins and fancy themes working in foobar2000, to provide a full experience for their music; not the people who gaze in wonder at the atrocious WMP fullscreen visualization.

We can expect slightly more patience and curiosity from them but in return, we must provide them with powerful tools, with a great user experience and ways to make it their own.

Now, it doesn’t mean that the player should be unusable by anybody outside that niche. Simply, it should focus on filling it as best as possible, before anything else.

Therefore, my suggestion is to make this XMMS2 GUI client a great music player for people who care about and love music, and make it a rich experience for playing, browsing, searching, organizing, discovering and enjoying music.

It is an ambitious goal, but I believe it is one that is exciting, original and expandable!

Pearl, by Camila…

Addendum: FLACvest just posted a
very flattering post about XMMS2 and mentions similar attributes
that would benefit XMMS2: Totally Fresh and Unique, Beautiful,
Cross-Platform, some “magic” ingredient!

Ask anyone on the street why they are not using XMMS2 right now and they will tell you the same thing:

But, like, all your GUI clients suck lol!

You needn’t have punched the poor guy in the face, because I also believe it is quite true.

We have quite a few clients (you will have to check out the matrix by yourself), but none that can really be called polished, mature or attractive enough to motivate people to switch. No disrespect to the authors (I’m even one of them), some cool stuff has been done, but graphical clients aren’t really “ready for the desktop” yet. Which is really a pity, given the awesome framework we have in place to create a great music player.

I believe that the reason for this isn’t that we don’t have competent developers, but rather that all of these projects are one- or two-men effort and not really thrived by the community. Because of the freedom you have to write your own client, well, people do: they follow their personal vision of what they want their music player to be like and start coding. Unfortunately, this isn’t the most effective strategy.

Perhaps thanks to the physicality of our meeting at FOSDEM, or to all the Belgium beer, we acknowledged that and decided that, now that the new official command-line interface (AKA nycli) has been merged and is being worked on by various people (thanks greafine, AnthonyG, nesciens!), it was time to get serious about a graphical client.

The best way to get people to work together on a new project is to establish a clear and common vision of what we’re aiming at, what are the ideas that structure the project, and get people excited about it! The vision should be compatible with the XMMS2 vision we’re currently discussing (thanks to Debian’s Bdale Garbee for the inspiration).

I want to make it clear that even if members of the XMMS2 community start working together on a common client, it is in no way incompatible with other people writing their own client if they so wish. Simply, we hope to gather people working on similar clients and focus the effort to build something really, really cool!

So I’m going to start posting things on this blog to propose directions for a vision to follow, and everyone is welcome to comment, reply, blog and debate the ideas so we solidify the basis we will start from. There will be a lot of things to discuss, from technical choices to interface challenges, so let’s get started!

Over last weekend (February 6-8), I shared a fantastic time in Brussels with people from the XMMS2 team (and a few thousands over geeks and almost as many beers). We got together, mostly off our Google Summer of Code money, and spent some IRL time thinking of how to make XMMS2 better. Thanks to everyone who was involved, it’s been great fun!

Although tilman and DraX hacked fervently on GenIPC, most of what we worked on wasn’t code, but rather discussions on the organisation of the project, the status of projects waiting to be merged into the main tree (GenIPC, service clients, collections 2.0), as well as future projects we want to explore, such as mergestatus (a pimped up iteration of the merge page) and an official GUI (yes!).

The main issue we identified with organisation is the lack of clear vision of where we are heading. Projects like GenIPC or mergestatus are being worked on without a general and agreed upon (let alone written down) specification of all they entail. The decision we took was to try to get tru back into a role of project manager and require proper wiki pages to be written about all the major features that are being developed. I personally want to start blogging more about thoughts and projects I’m working on (I know I’ve said it before, but this time it is true).

Saturday Night Hacking, by theefer

I had several discussions with nesciens, who has been hacking on collections 2.0 last summer, and we identified different topics that should be explained on the wiki and possibly discussed with the rest of the team:

• Advanced (insane) queries (to retrieve all sorts of information and structures from the mlib using collections)
• Source goodness (server magic to make collections select the right value according to a global ranking of sources)
• Token operator (to match tokens, which is what you usually search for, using an external token table)
• Other new collection operators (in particular, related to treating medialists, i.e. keep collections ordered throughout the DAG)

Note: most of the links above are work in progress, meaning they’re not total rubbish, but don’t rely on them to launch your personal space capsule.

Other fancy stuff that has been debated:

• Proper serialisation of the collection DAG over IPC
• Usage of
  xmmsv_t

  dict to store collection attributes, possibly richer than strings (already started by anders)

• Idlist as a simple attribute in Idlist collections
• String table (optimisation of strings in the DB to avoid duplication of data)

nesciens has already started updating the todolist/roadmap.

And of course, the super exciting new project that nobody even expected: a common effort to work on an official GUI client!

I have a lots of thoughts about this, and I need to summarise all the discussions related to it at FOSDEM. And that will be the topic of my next post!

(Who said you couldn’t have cliffhangers in blog posts?)

Farewell group photo, by theefer