The transmedia project

The Zeru transmedia project includes two components:

The IGN is accessible through an application for tablets or smartphones, with an enhanced version for the web.

(videos, texts, photos, audio). It includes several websites: an interactive card that immerses the user in a visual landscape; a video wall that displays the videos in a dynamic, integrated way; a personalized space for each user that is connected to social networks; and a film generator that is the background task for all the websites.

What makes the Zeru project unique is the dynamic bidirectional connection between its two components.

schema_technique

As each user reads the interactive graphic novel, a specific documentary film will be created in sync with their particular journey through the story. As the plot develops , the user will be able to select different interactive elements within the images. These elements will not distract or interfere with the reading experience, rather they will serve as a kind of reading list, a bit like bookmarks.

The role of these interactive elements is twofold: First of all, they will determine the pace and development of the plot (through an accordion plot). Secondly, the bookmarks will work as “lynchpins”, which will search for and collate appropriate footage from the documentary platform, creating a film at the end of the story.

Therefore, there is a strong link between the fictional world of the linear story and the nonlinear real-life worlds of a web documentary. This will heighten the emotional attachment and active interest of the user/reader in the issues involved.

When he “exits” the story, the reader will understand through the documentary film, that the fictional issues of the graphic novel are rooted in real events in today’s world. Since the documentary platform is linked to webstreams (news, video feeds and photographs), the films that emerge will also be rooted in the present day. The reader of the interactive graphic novel will become a user of the documentary platform, thereby having the possibility to actively engage with the subject matter by searching for complimentary information or other footage. Users may even add their own content (comments, images, videos) related to the cause.

This double mechanism, at once narrative and experiential, is made possible through Memoways platform software. The platform allows users to access, export and store content related to this project, drawing on the dynamic bidirectional structure between the two platforms used.

Conceptual elements

Fine granularity: the assets

  • each box of the graphic novel exists as a separate document, with multilayered access and several descriptors (tags and keywords). Users can determine the clickable areas in each layer because each area will also have its own metadata;
  • each file (video, photo, audio, etc.) on the documentary platform is stored individually. The video is stored as shots (unedited) or sequences; the audio as specific files (musical phrase, atmosphere, sound effects, etc.); the text in specific fields; and photographs with their specific metadata.

asset-layers

In this way, each document (bubble in the graphic novel, video shot, sound bite) forms an asset.
The advantage of storing the assets separately and not the plates (in the case of the graphic novel) or the films (documentaries) is that it allows users to access each element with complete freedom and precision (layers, metadata) and, most importantly, to combine the various assets in an automated way while ensuring a smooth continuity (of the storyline, subject matter, interactivity).

In order for a specific group of people to have access to this project, it will be necessary to provide access to the assets, which will be stored on a server, via Internet, and protected by a login name and password.
Once the assets are ready to be used, they can be made public through a Beam (see below).

Precise description: the metadata

An asset on its own, without descriptive metadata, is not locatable and therefore cannot be used.
By answering the following kinds of questions:

  • Who (created the asset)
  • With whom (as a protagonist, player, copyright owners, etc.)
  • How (with what device, in what way)
  • When (date, hour)
  • Where (location)
  • Why (the creative setting)
  • What (tags, categories, subjects, etc)
  • Other

The assets can be located through a query.

But, besides filtering/locating the assets, the metadata allows content to be grouped according to a filtering structure. So these could also become a playlist — and therefore a new story.
Therefore, the challenge is to create descriptions that are precise enough to meet the double need:
– locate, filter
– tell a story that moves and engages the reader/user

Once the assets have acquired this fine granularity and precise description (objective and subjective metadata), the asset placement can be automated in a linear framework (the story) and enable users to access and play with them interactively (the documentary platform).

Setup, workflow

Description of the content: the metadata structure

To ensure the bidirectional connection, a metadata structure (taxonomy) must be set up. It will allow the information (the issues, the story) and the content (through the descriptors) to be organized.
This requires the creation of a subject list (the issues), possible usage (how to use the content), subject matter (what the content deals with), objects (what content consists of), etc.
This structure will be gradually modified and updated by the structure editor of the Memoways OSX application: the choices (in the structure and the nomenclature), are not written in stone and can be modified throughout the project.

In essence, by adding keywords the assets will be enhanced with related information, allowing them to stand alone (in order to tell multiple stories).

Connections: the mapping structure

Once the metadata structure has been set up, a mapping structure must be created. This will “connect” the keywords (the metadata) with the user’s selections, the reading behavior, the calculation results, etc.
For example, the speed at which the interactive graphic novel is read will determine (trigger) the content or layers that will be activated in the next part of the story. The tilt of the tablet may change the colors or alter the sound.
So the mapping structure connects a usage mechanism with a narrative mechanism. Therefore, it consists of a formal connection in which one form (usage) brings about another (narration) through dedicated software for mapping structures.
The mapping structure is produced via Memoways’ web management console, through a form that is filled out and streamlined.

Rules of the game: the algorithms

In order to use (“play with”) the keywords and integrate the mapping structures, an algorithm must be designed (a program that calculates results according to the input data and the predefined instructions.)
The algorithm uses variables: thresholds (minimum / maximum), weighting (points given according to the calculation results), etc.
These variables will help define the game rules, which will provide a framework for possible usage between the interactive graphic novel and the documentary platform.
For example, it is possible to define an algorithm that takes into account reading behavior (speed, path chosen) and selections (clicks on elements or buttons) to calculate a unique, personalized documentary film, generated from the documentary platform at the end of the reading. This film will be the formal translation of the reader’s specific behavior and individual selections. The content (drawn from the “real” world) will obviously vary, but the layout will be an accurate reflection of the user’s individual reading of the interactive graphic novel.
The personalization of the algorithm(s) will be carried out like the mapping structure, through the web management console.

To sum up, by delinking the content indexing (the intelligence on the data) from the content usage (the “intelligent” connections between the different areas of content, and the editing), we can automatize a linear story and generate personal experiences while ensuring coherency between the two moments.

Use

First of all comes the reading of the interactive graphic novel. The storyline is linear, with a beginning, middle and end.
Interactivity is achieved through two “levers”:
– Selective: by using Memomarkers (see below)
– Behavioral: the reading pace and interactions determine the length of the reading. The plot develops through accordions, whereby boxes can be added or removed from the story. Some sequences can therefore be lengthened or shortened, allowing for a reading experience that is either deeper and more engrossing or else allows a faster skimming of the text.

Interactive uses: the creation of Memomarkers

As readers advance through the graphic novel, they can memomark or bookmark elements in the story according to their interests, questions, wishes, etc.
The elements can be:
– visual elements in the image (an object, a tree, a person, etc.)
– a word or group of words (underlined, highlighted)
– a button (in a dialogue or interface).

interaction-memomarking-2

By clicking on an element, the reader activates two things:
– The element is the “mouthpiece” of one (or several) tags, and of one (or several) keywords. By selecting an element, the related tags or keywords are memomarked. This operation is carried out invisibly, in the “backend” of the interactive graphic novel.
– In order to signal to the reader that the click has activated something, there will be different triggers such as a noise, a change in the sound mix, or a new color that appears.

These interactive elements must meet the following criteria:
– They should not take the reader out of the story. They should be presented in a plain, straightforward way that is smoothly choreographed with the unfolding of the story (quantity and appropriateness).
– The words assigned to the elements should not be ambiguous. The reader should be able to understand intuitively the reasoning behind the “coupling” of the element clicked on and the form + content of the resulting film.

Besides the possibilities and constraints described above, the following ideas may be added (depending on the interface design and the overall aesthetic):
– a timeline that displays the memomarkers throughout the reading (showing all the interactions). This timeline can be displayed or hidden like the dock.
– a visual effect that displays the keyword(s) above the element and then moves it (them) to the timeline (like a document that disappears from the dock).

zeru-timeline-2

Subsequently, the reader who has interacted with the selected elements will receive a video from the documentary platform. This video is the result of an automatic montage of all the elements memomarked during the reading. The video can be viewed through two main gateways:
– Directly in the interactive graphic novel. At the end of the reading, a message invites the user to watch the video generated by the interactive reading, without leaving the application. The reader can share the video throughgh social media.
– On the web. Once the video has been transferred from the application to the Internet (the publication can be personalized by setting up an account but it is not obligatory), it becomes “independent”: it can be remixed, embedded, and so on (see the Beams below).

Content flow: the Beams: the algorithms

The assets can be combined in the form of a beam, through a dynamic, multi-criteria filter, carried out by a production company.
A Beam is a coherent grouping of preexisting content, generated by a certain number of clicks on metadata (location, time, subject, author, etc.) By naming the filtered content, the production team can display the Beam publicly (on the web). The public will then be able to freely access structured, coherent content that can be filtered, remixed, aggregated, mashed-up and embedded.
It is important to remember that the Beam is dynamic, that the assets that make up a Beam can change (in their indexing); the number of assets can also vary. In fact, a Beam can be defined as a coherent connection to a specific filter of content, that is, those that vary according to their indexing (whether preexisting or new content).