[This is Part 2 of a set of posts covering “Database as Symbolic Form” by Lev Manovich (RTF, HTML).]

If you have read this article already, you might have noticed that I am skipping over what Manovich has to say about film for the most part. The reason for that is simple: I’m not currently prepared to discuss those ideas. While I think they are rather interesting, my field of study is more digital collections and interactions than film and I will not be highlighting his extended thoughts about how film is condensed from a larger set of shots into edited scenes, other than to note it in this sentence.

“To further understand how computer culture redistributes weight between the two terms of opposition in computer culture I will bring in a semiological theory of syntagm and paradigm. According to this model, originally formulated by Ferdinand de Saussure to describe natural languages such as English and later expanded by Roland Barthes and others to apply to other sign systems (narrative, fashion, food, etc.), the elements of a system can be related on two dimensions: syntagmatic and paradigmatic. As defined by Barthes, ‘the syntagm is a combination of signs, which has space as a support.’ …Now, lets look at the paradigm. To continue with an example of a langauge user, each new element is chosen from a set of other related elements. For instance, all nouns form a set; all synonyms of a particular word form another set. In the original formulation of Saussure, ‘the units which have something in common are associated in theory and thus form groups within which various relationships can be found.’ This is the paradigmatic dimension.”

Let me see if I can render this in a clearer way. Basically, objects put into a specific sequence carry meaning because of their proximity to each other. Not only are the objects important, but their overall meaning is a combination of the objects and their order. The example that Manovich uses is that of sentences. Words convey meaning through their placement in sentences. The totality of their meaning is derived by understanding what they stand for and how they connect to the words both before and after them; their sequencing contributes to how aspects of their meaning add to or subject from what they are used to build.

In contrast, a paradigm is a set whose relationships are equal to each other. They all have the same quality and are grouped together for this reason. In the comparison that Manovich is putting together, a paradigm is a database. It is a selection of signs that are associated because of what they share in common, not because they add up to a more complicated sign in connection with each other. His use of the “synonyms” is a great way to look at the concept: the objects all have a similar meaning, but might have different contexts in which to use them.

“The narrative is constructed by linking elements of this database in a particular order, i.e. designing a trajectory leading from one element to another. On the material level, a narrative is just a set of links; the elements themselves remain stored in the database. Thus the narrative is more virtual than the database itself…The paradigm is privileged over syntagm in yet another way in interactive objects presenting the user with a number of choices at the same time – which is what typical interactive interfaces do. For instance, a screen may contain a few icons; clicking on each icon leads the user to a different screen. On the level of an individual screen, these choices form a paradigm of their own which is explicitly presented to the user. On the level of the whole object, the user is made aware that she is following one possible trajectory among many others. In other words, she is selecting one trajectory from the paradigm of all trajectories which are defined. “

Without expressly mentioning it, Manovich is tying hypertext theory into this look at databases and narratives. In order to get a narrative, the user moves through a series of interface changes that allow certain selections from the database. Each movement is a narrowing of all present choices into a single destination that is linked to another set choices. Every action triggers a reaction and some decision (often electronic) in how to display another slice of possibilities. The interface, and its mechanics, underline how the database is accessed however. Every choice is one of interacting with the interface first while it processes a request to a database.

“Other types of interactive interfaces make the paradigm even more explicit by presenting the user with an explicit menu of all available choices. In such interfaces, all of the categories are always available, just a mouse click away. The complete paradigm is present before the user, its elements neatly arranged in a menu. This is another example of how new media makes explicit the psychological processes involved in cultural communication…The psychological processes of filling-in, hypothesis forming, recall and identification – which are required for us to comprehend any text or image at all – are erroneously equated with an objectively existing structure of interactive links.”

If I might again attempt at simplifying these ideas, we should not assume that just because a selection of links are presented to us that they are, in fact, all related to each other and placed there for us. While we do select one of them over the others, they might not all be valid paths. It’s the idea that, on a web page, one link may promise a destination but, until the user actually clicks, it remains only a possibility. From a narrative point of view, we might use any moment in the construction up to that point from the movements we have taken before, but this might lead to a sudden ending when we exhaust or even are lead outside the structure of some content.

“Why does new media insist on this language-like sequencing? My hypothesis is that it follows the dominant semiological order of the twentieth century – that of cinema. Cinema replaced all other modes of narration with a sequential narrative, an assembly line of shots which appear on the screen one at a time. For centuries, a spatialized narrative where all images appear simultaneously dominated European visual culture; then it was delegated to “minor” cultural forms as comics or technical illustrations. “Real” culture of the twentieth century came to speak in linear chains, aligning itself with the assembly line of an industrial society and the Turing machine of a post-industrial era. New media continues this mode, giving the user information one screen at a time.”

Although I said I was going to stay away from Manovich’s thoughts on film, I am including this because of its implications with information access. His observations that all information from a system was shown in the past and is now cut into chunks carries with it profound understanding of how mediums have shaped us over time. Our reliance on sequences could be said to be carried from radio into television. We came to understand that certain entertainment happened at certain times; programs and shows only occurred at fixed points in the transmission.

The use of multiple channels of access, first in television and now on the web, has brought with it not only a view of sequencing that is similar, but is also related to seeing information in different forms at the same time. Instead of just visual and audio that is authorial in construction, we might see multiple windows of content shown to us at the same time on the same page or screen. Advertising content might be shown to us alongside classical textual works on a web page. Even television shows might now deploy multiple point of views with the expectation that the audience is watching all of them simultaneously. The task of determining relevance is often solely the responsibility of the user herself.

“Modern media is the new battlefield for the competition between database and narrative….Digital computer turns out to be the perfect medium for the database form. Like a virus, databases infect CD-ROMs and hard drives, servers and Web sites. Can we say that database is the cultural form most characteristic of a computer?”

I find thoughts expressed like this to be fascinating. Using a word like “infects” places the form of the database as the corrupting force that is destroying traditional — and therefore best in many people’s eyes — media and works. The close proximity of sets of data that are unordered or even unrelated must seem quite strange to those who have grown up with and operated on human edited series. In much more traditional media, the preparation and placement of objects is as important as the object themselves: where they are on a printed page conveys much of their meaning. However, this is not true for digital media; their position when stored does not necessarily convey any meaning about their content.

As a programmer first and writer second (depending on the day), I have grown up with the idea of data neutrality. Nothing is as important as anything else on the low-level, the storage layer. It’s all just data that is loaded in bytes, chunks or some other measurement for the computer to process. The hardware only cares what what type it is, algorithms or data, during the moment of execution. Otherwise, it’s all just moved around from storage to cache when needed. Groups of data are only looked at if they are referenced by something else, if a pointer in an executable holds the address of instructions or data that is located in another position in the system.

In that light, databases are the native format of the digital platform. If thought of as a set of structures that hold addresses for other locations, they are the basis for how computers work. As memory is allocated and reclaimed, it works off an array, a list of locations. When different devices interrupt the system, like typing at a keyboard for example, their messages are placed in a queue. Operating systems are constantly parsing one collection after another, sorting and swapping data and instructions when needed. If data structures are loosely classified as databases, then they are the very root of everything digital.

To say that the database is in some way changing other media by invading it is very strange then. It’s the other way around. In order to be represented on a computer, other media must be placed in a database. Hard drives work on this principle, for example. The code that governs looking up data on a drive works by receiving an address and breaking it down into a series of locations: device, platter, cylinder and finally sector. Anything that is stored on some type of secondary system like this uses a similar method. Data is referenced from other locations and its position is often relative to how often it is needed or used, not necessarily how important it is culturally to a user. Such ordering is done for a practical reason: speed of access is prized over any other qualification. The user wants a response fast and systems are designed with this in mind.

“In the 1960s, artists working with computers wrote programs to systematically explore the combinations of different visual elements. In part they were following art world trends such as minimalism. Minimalist artists executed works of art according to pre-existent plans; they also created series of images or objects by systematically varying a single parameter. So, when minimalist artist Sol LeWitt spoke of an artist’s idea as “the machine which makes the work,” it was only logical to substitute the human executing the idea by a computer. At the same time, since the only way to make pictures with a computer was by writing a computer program, the logic of computer programming itself pushed computer artists in the same directions.”

In many ways, artists are beginning to push back at this notion that programs make art by creating displays that are in rejection or even anticipation of computers seeing them. This is the New Aesthetic movement’s way of making art that computers can see, while also making it visually interesting to humans. It’s the mix of letting the machine do the work for itself, yet also subversive in that the beauty of it is lost to the machine. It’s taking pixels, the smallest unit of machine acuity, and making pictures in the real world that hold meaning for humans, but are constructed of digital native frameworks.

In a simple way, it’s also beautiful QR codes. Readable by machine, yet constructed by humans for humans. It’s an odd juxtaposition of both the aesthetically blind machine parsing reality and the human making art of the result. The machine executes its fixed cycles and programs and the humans move in its shadows, making something from the output, input or even processes while the machine marches onward, unaware to what is being done around it. And, of course, in the very action of making the differentiation between to the two parties, it highlights the symbolic — and often symbiotic — nature of both to each other.

“Although database form may be inherent to new media, countless attempts to create ‘interactive narratives’ testify to our dissatisfaction with the computer in the sole role of an encyclopedia or a catalog of effects. We want new media narratives, and we want these narratives to be different from the narratives we saw or read before. In fact, regardless of how often we repeat in public that the modernist notion of medium specificity (‘every medium should develop its own unique language’) is obsolete, we do expect computer narratives to showcase new aesthetic possibilities which did not exist before digital computers. In short, we want them to be new media specific…How can a narrative take into account the fact that its elements are organised in a database? How can our new abilities to store vast amounts of data, to automatically classify, index, link, search and instantly retrieve it lead to new kinds of narratives?”

It’s hard not to laugh at the notion that the audience for media wants the same enjoyment but in the different package. “Give us something new,” they say “except not too different.” Still, it’s obvious for both Manovich and those who would read this that finding places for ‘interactive narratives’, in the face of everything being storied in databases on a digital level, is hard. There are moments when we can find something new for ourselves, some new visualization of data that leads to a different understanding, but it’s often looking at old data in new ways. And, in a way, that is good.

The use of the phrase “catalog of effects” reminds me of McLuhan’s “inventory of effects” and the idea that we are often shaped by how we process data. That is, we can get very different messages out of different mediums. From video, we can get understanding of chronologies faster: film is inherently a series of scenes arranged in a fixed order to invoke meaning between them. For text, we can get spatial relations of encoded information: words are placed in a certain order and placed on a page for a reason. Audio, like video, has fixed or even fluid intervals that invoke meaning according to what was before or after it in sequence. Yet, mix interactivity into any of those three and they start to lose their cohesion without careful maintenance. Their units relay meaning and give rise to narrative in the user; without that sequencing, it complicates the narrative process. It’s still possible, but requires that they be designed with that in mind.

I’m tempted to argue here that video games represent the pinnacle of interactivity for narratives, but for many games that premise would be false. Do I think that video games hold that potential? Absolutely. Do I think we are there yet? No. Too often, designers and developers are caught up in trying to use old ‘languages’ in a new package. Instead of trying to see where video games can go, many settle for telling stories that are native to other media in a new form. They try to take the plot of a movie and push it into a game. Or, often worse, try to reshape a book into something that is highly interactive while missing the fact that agency can often negate fixed storytelling.

More than anything else currently available, video games hold the unique position of holding other media within itself and integrating them into narrative potential. Books cannot hold video or audio. And, while a video might hold both auditory and textual elements, its interactivity is very low if not nonexistent in places. Video games can maintain the essence of all of them while also being highly interactive in the process. While in a virtual world, and hearing another person speaking and seeing them move, you can also read a book. Yet, they are also complicated works to construct and often the enjoyment of interactivity comes not from understanding its systems (exploration) but expressing mastery in connection with dominance of the system or other users within it; too often it becomes a matter of competition instead of cooperation.