Review of DocPerform 3: Postdigital

Thank you to David Bawden for taking notes during the sessions!

Temporary Poster

Over 50 participants took part in the third Documenting Performance symposium, held at City, University of London, on 16th May 2019, hosted by CityLIS, the Department of Library & Information Science. DocPerform is a part of the wider CityLIS project examining The Future of Documents.

The introductory session was opened by Sarah Rubidge, Professor Emerita at the University of Chichester, and long-standing member of the DocPerform team, who set the day into context. From the first symposium which focused mainly on conventional records of performance in libraries and archives, DocPerform has moved to deal more and more with the digital realm. This matches the changes in performance itself, which, from the 1980s onwards, has moved beyond the stage into other spaces, including the digital. In turn this leads to a number of questions;

– how do we document temporal media?
– how do we document improvised performances?
– how do we document performance systems and installations, especially immersive installations?
– how do we document processes?
– how do we document and preserve digital works, which are already disappearing as systems and software become obsolete?

Solving these problems requires research, not just applying existing archival methods.

The keynote presentation, The experience parlour, was given by Lyn Robinson from CityLIS, who reminded participants that the perspective of DocPerform is from that of library and information science, the remit of which is keeping the record of humankind. It is inspired in one way by Bruce Shuman’s vision of thirty years ago of a library as an experience parlour or experiencybrary, which would store immersive experiences which could be accessed by the library’s patrons. Virtual and augmented reality technologies, which now offer the prospect of an unreal reality, seem to offer the prospect of making Shuman’s vision a practical proposition. In order to bring this about, we will need to fully understand the nature of these new forms of documents, in order to be able to describe and use them properly. This will require extensive research, rooted in the concepts of document theory as originally outlined by Paul Otlet and Susanne Briet.

The first full session, devoted to Technologies, had four speakers.

Mark Underwood, an experienced sound designer now undertaking a PhD at the University of Surry, gave a presentation Exploring the extent to which sound design enhances temporal and somatic user experiences in mixed reality environments. This examined how intelligent sound design can help make such experiences as immersive as possible, with immersion understood as a deep mental involvement. Immersion has both psychological and sensory aspects. The former can be attained, for example by reading a book, while the latter is the aspect of immersion enhanced by sound. Mark showed, by playing a film clip with and without sound, that when sound is absent we may notice visual inputs more clearly; more sensory input is not, in itself, an advantage, it has to be carefully designed. By implication, this will influence the impact of documented experiences.

Hansjörg Schmidt and Nick Hunt described their interactive installation Traces, an aspect of their wider Library of Light project at Rose Bruford College, which is developing repository for lighting practices in various creative disciplines. Traces allows light effects to be recorded with a camera or smart phone, enabling an investigation of the relation between the ways we experience light, and the ways we can record and document it. The installation was made available in a separate room for participants to experience during the day.

https://performinglightblog.wordpress.com/schedule/traces/

The presentation by Tom Ensom and Jack McConchie, curators at the Tate galleries, Preserving virtual reality artworks, outlined the potential, and the challenges, of documenting digital installation artworks which employ virtual reality (VR), and which offer immersive and multi-sensory experiences. VR engines such as Unity, Source, Unreal Engine and CryEngine, originally developed for videogame development, can be used for this purpose. There remain considerable practical difficulties, not least cost and effort of collaborating with the VR industry. At a minimum, however, to stand the best chance of preserving such works, we can, and should:

  • start collecting now
  • create and archive a disk image of the running artwork
  • gather and create documentation that shows how to do this
  • create and archive a disk image of the production materials in a software environment in which they can be accessed
  • monitor the evolution of technologies, to identify problems and opportunities.

Joseph Dunne-Howrie, from CityLIS, gave a presentation on Trans-participation in the infosphere. Luciano Floridi’s concept of the infosphere, the contemporary digital information environment, is taken as a framework . The production and dissemination of media acts as the infrastructure of the infosphere, replicating our presence across platforms and communication networks. Audience participation in the infosphere and the condition of onlife, where the physical and virtual worlds fuse seamlessly, provides new forms of interaction and identity, as evidenced by examples from immersive theatre.

The second session, entitled Transcience, also featured four presentations.

Clarice Hilton, giving a presentation of behalf of herself and her fellow VR researcher Shivani Hassard, discussed Frictional forces in creating the effect of presence in immersive experiences. Presence in virtual reality is promoted by the effective illusions of place (creating the belief that we are somewhere we are not) and of plausibility (creating the belief that was is happening is natural and sensible). A contribution to plausibility is given by providing the sense of friction in VR experiences, making such experiences involve actual physical effort, so reinforcing the physicality of the actual body.

Piotr Woycicki from Aberystwyth University presented AR remediation/documentation of Our Lady of Shadows, an example of an augmented reality space which is at the same time enclosed but also open to the outside world. The work is an AR adaption of a radio play, giving a reimagining of Tennyson’s poem The Lady of Shalott. The discussion of this presentation led to the idea that, although we may think of audience interaction as a feature of modern and novel formats, it may be that radio drama allows audiences to be active participants [Tim Crook, Radio Drama, London: Routledge, 1999].

Harry Robert Wilson, Digital Thinker in Residence, for the National Theatre of Scotland, gave a presentation on Immersive media and the multi-modal document in PaR. This reflected on the documentation of immersive media in light of the definitions of document and documentation given by Angela Piccini and Caroline Rye [Of fevered archives and the quest for total documentation, in L. Allegue et al. (eds.) Practice-as-Research in performance and screen. London: Palgrave Macmillam, 2009), 34-39]. There are two distinctive features of VR documentation of performance. It puts the audience, in a sense, inside the document; and it integrates other forms of document, such as video, photographs, text, sound, and set design. However, it would wise not to claim uncritical and absolute “immersion” or “presence” from any currently feasible VR documents.

Sarah Rubidge presented an interactive artwork of which she was co-creator, Sensuous Geographies, a performative sound and video installation. Now that artwork itself is no longer active, would it be feasible to recreate the experience in VR? The presentation raised many questions about how to translate an experience in physical reality into virtual reality, while capturing sufficient of the essence of the experience to say that the VR version was, in some sense, the same. These questions are fundamental to the documentation of any interactive performance of artwork, and more broadly to new forms of digital document. See: Sensuous Geographies

The closing session was devoted to Structures and Interfaces.

A third CityLIS speaker, Deborah Lee, discussed Documenting interactivity and post-digital performances: exploring the application of data models and standards for augmented reality performance. This presentation examined the limitations of the standard library metadata models for describing performance documentation. The essential problem is that performance does not fit well into the bibliographic world described by standards such as the Functional Requirements for Bibliographic Records (FRBR) and the Library Reference Model (LRM). These models assume that there is a single creator of a work, and hence cannot adequately describe interactive or participatory works. Indeed, they do not cope well with works defined in space and time; they are fine for recording the text of a play, for example, but not a single performance. Nor do they recognise AR and VR among their defined media types. These problems had been noted a while ago, before experience had been gained in the used for FRBR and its associated standards [D. Miller and P. Le Boeuf, “Such stuff as dreams are made on”: how does FRBR fit performing arts? Cataloguing and Classification Quarterly, 39(3-4), 151-178]. Discussion of this presentation suggested that these were philosophical issues, of fundamental importance in keeping records of performance documentation. In turn, documenting performance is a good test bed for examining the issues which will arise with other new forms of document.

To close the day, the True Heart theatre company presented The Genie is out of the bottle: who’s got a story to tell? Taking comments from members of the audience on their impressions and experiences of the day, they presented them as Playback Theatre – short improvised performances interpreting and representing experiences – without any of the digital or technological means discussed throughout the day.

Installations and performances were available to the participants throughout the day. Apart from Traces, already mentioned, the Itinerant Poetry Library opened, and Rebecca McCutcheon’s virtual reality performances Affective bodies in dynamic spaces: documenting site-specific theatre practice were made available for participants to experience.

Further information about the symposium is available on the website. The full papers will appear in a special issue of Proceedings from the Document Academy.

 

Documenting Performance: Sensuous Geographies

Documenting Performance: Sensuous Geographies
Collaborative, exploratory research workshop held at CityLIS on March 26th 2019

This is a brief account of our collaborative, exploratory research workshop held on March 26th 2019. For further details, please contact Dr Lyn Robinson or Dr Joseph Dunne-Howrie.

***

IMG_6019CityLIS has a longstanding interest in the nature of documents, document theory and the processes of documentation. We understand a document to be something that stands as ‘evidence’, and follow a broad interpretation of those entities that may be classed as documents, including data sets, works of art and performance.

We have a current focus on novel forms of document, which are afforded by new technologies. These technologies, including multimedia, pervasive networks, multisensory transmission (taste, smell, touch), VR and AR, facilitate documents with which we can interact and participate to a greater or lesser extent, and which also promote the experience of immersion, so that a scripted unreality may be perceived as something approaching reality. In many cases, these documents have a temporal component, in that they are experienced at a particular moment in time. Additionally, at any given time, they may be experienced by multiple participants or members of the audience.

Examples include born-digital documents such as video games, interactive fictions or narrative, immersive artworks, and theatre and dance performances produced in VR.

Further, such technologies might also be used to recreate (document), and thus preserve, non-digital documents with a temporal aspect, such as performance.

Whilst the technologies used to create these new forms of document can also be used to preserve them for future access, it is important to note the difference between digitization as a process, and as a technique for preservation. Digital preservation is responsible for preserving digitization.

In order to document and preserve these novel document formats, we need to understand their nature, or documentality.

This one-day workshop brought artists and engineers together with library and information specialists, to explore how immersive, participatory, performance related works could be understood, and thus documented.

We took as our example, the immersive and interactive installation Sensuous Geographies created in 2003 by Professor Sarah Rubidge in collaboration with Professor Alistair Macdonald.

This choreosonic installation, although no longer extant, has been documented by texts, photographs, sound and video recordings. Several physical components remain, including the costumes and the materials used to create the flooring.

We would like to move beyond these forms of documentary evidence (themselves documents), to recreate a version of the original piece in VR. The original installation offers a range of multisensory aspects for documentalists to consider; not only how to recreate audio and visual aspects of the installation in VR, but how to include the elements of presence (participation) and immersion (a feeling of reality).

Our workshop, hosted by myself and Joseph Dunne-Howrie, began with a presentation from Sarah, which set out the nature of the installation, highlighting the features we would need to recreate in order to fully document this unique work, so that it could be made available to new audiences, or indeed again to those who interacted with the installation in 2003/4, in as close to the original format as possible.

We then heard brief presentations from each or our workshop members, on their areas of expertise, followed by informal group discussion around documentation of a multisensory, interactive, immersive, time-based installation.

As a starting point, we considered whether existing conceptual models for describing documentary works, including FRBR (IFLA LRM), and any metadata standards associated with artefacts or performance, offered existing work on which we could build. No model for works similar to Sensuous Geographies was known to the group, although this remains an area for further exploration.

We talked about the nature of the surrogate document, or simulacrum, and acknowledged that our recreation of the installation in VR, if successful, would be a new document, irrespective of how close to creating a work with the feeling of the original experience we came.

Working conceptually, we identified layers of the work that would need to be reproduced in our VR version of Sensuous Geographies:

  • The physical attributes of the installation, the space, the flooring, costumes and screens
  • Participation as either an audience member or a someone who entered the space
  • The soundscape: ambient sounds to give a sense of presence, then the interactive musical score reacting to participants’ movements and interaction (MacDonald ‘composed’ the underlying character of the soundscape in realtime by selecting sound strands and processing systems)
  • The rules embedded in the interactive system, governing the sounds, and the level at which participants could interact
  • The sensations experienced by the participants; sight, sound, touch, movement/direction

We then discussed in broad categories, the technological possibilities that we could employ to realize the work:

  • Static view
  • 360 video on screen
  • 360 video plus headset
  • 360 video plus headset and hand controllers
  • VR headset and empty space
  • VR headset and set pieces
  • VR headset and sensors
  • Sound: multichannel audio, or binaural audio
    (the latter would need to be connected to the participant’s head movement)

The discussions were engaging and constructive. We need now to describe in more detail each ‘layer’ of Sensuous Geographies, and to further explore the technologies available to render the work in as realistic a format as possible.

Our aim is to produce a specification for a project to render the work in VR, and we would welcome any thoughts as to how this could be achieved.

This project offers a case study of documentation, digitization, and subsequent preservation of a multisensory, interactive and immersive choreosonic installation. We believe this to be a novel undertaking, and one which will be of interest to artists and creators of such works, to library and information professionals, to archivists, scholars, engineers and those interested in cultural heritage.

Participants in our workshop included:

Astrid Breel (via Skype)
Joseph Dunne-Howrie
Matthew Freeman (via Skype)
Alistair MacDonald (via Skype)
Jorge Lopes Ramos
Lyn Robinson
Sarah Rubidge
Rebecca Stewart
Mark Underwood
Zhi Xu 

***

DocPerform 3: PostDigital
May 16th-17th 2019
City, University of London