Blog

Transcribing the improvised solos of the greats is a core part of jazz education, but it's rare that people share their (often hand-written) notes on, say, exactly what Clifford was doing in that second chorus of "Step Lightly." 

Part of the point of transcribing solos is the fun and frustration of working through a recording yourself. But it would be nice to compare notes now and again. I just posted a pdf of an attempt to transcribe the solos one of my favorite Modern Jazz Quartet pieces. But I'm sure it's rife with errors, and there's no elegant way for people to point them out on these flat images.

This makes me wonder if it would be possible to use the WikiMedia framework for a collaborative music transcription project. There appear to already be some impressive tools for rendering musical notation in Javascript. I don't know exactly how the process of editing an existing transcription would work. Ideally, the "edit" tab would render the notation in a manipulative form common to most notation software, which instant playback and so forth. (I recommend SoundManager 2 for JS audio.) Like any wiki model, of course, all previous versions of a particular tune would be archived. A pure Javascript rendering of interactive notation would be a major undertaking, but I see no reason it isn't technically possible. 

Until about I year ago, I coded most of my projects in Flash and its cousin Flex, which is developed entirely in code without the benefit of Flash's visual interface. (It's sort of like the different between algebra and geometry, though since Flash moved to Actionscript 3, it resembles Flex a lot more.) Both platforms can draw very quickly and create enormously complex animations and interactives, though they can often eat up a lot of system resources in the process.

The general move away from Flash seems to have begun with the introduction of the first iPhone, which did not support it. While I admire the late Steve Jobs, I still resent that he closed down the development for his iProducts to a closed SDK, essentially doubling the work for people who want their applications to run smoothly on the Web and Apple products (not to mention other mobile platforms). 

Sheerly for the sake of efficiently, we started used the Javascript library Raphael for most visualizations. Raphael is based on the drawing platform SVG and is also compatible with Microsoft's equivalent structure, so it made life easier to do things once. I can't say enough about how useful that library has been to my data visualization projects. It has the added benefit of running natively in the browser, so it doesn't matter whether the user has updated his or her Flash player recently (or ever). 

I don't think this is a permanent solution, however. Raphael cannot draw as quickly as Flash, making projects like county-by-county maps over slow to the point that they're unusable. The obvious successor is HTML5, whose canvas feature makes for much more elegant drawing inside the browser. Unfortunately, the 30 percent or so of users who still use Internet Explorer 8 or below cannot view HTML5 content, and most of us can't afford to lose that audience. (IE9 can render many parts of HTML5, but requires an upgrade to Vista or Windows 7.) 

For intensive visualizations, I fear the short-term solution will be to code interactives twice, once in HTML5 for modern browsers with a Flash fallback for the IE crowd. A very clever developer might be able to write a meta language that could port to either platform with minimum recoding. This would be a major asset to anyone interested in reaching a maximum audience with powerful applications with dazzling capabilities.

You may notice that the tagline on this site refers to "hypermedia." The term was coined alongside "hypertext" in 1963 by a man named Ted Nelson, but only one would stick. "Hypertext"--literally "beyond text"--is etched in the 'H' and 'T' of HTML, but you don't often hear the second term. Nelson commented on this in his book Literary Machines.

"By now the word 'hypertext' has become generally accepted for branching and responding text, but the corresponding word 'hypermedia', meaning complexes of branching and responding graphics, movies and sound – as well as text – is much less used. Instead they use the strange term 'interactive multimedia': this is four syllables longer, and does not express the idea of extending hypertext."

Aside from syllable conservation, I like the term because it broadly encompasses the goal of Web-based graphics and interactives, which is to endow them with more information than is possible in print. Right now, this might mean a map that animate through time, changing color as datapoints change and allowing the user to mouse over counties to get an individual report for that locality. In the future, I look forward to "hypermovies" that let you mouse over or somehow point to an actor and get a list of what other movies you've seen him or her in. 

In a way, it's is a lazy phrase that serves as a catchall for whatever people are producing online. But as a concept, "beyond media" sounds as appealing as Web development gets.