"Illusory Interactive" Performance
with Saccade-based Display

As a cognitive scientist, my interest is to explore a stage design from the view of visual perception, and create a new kind of performance beyond the limits of conventional stages. I have been collaborating with media performance unit "cell/66b". The collaboration started from 2001. We had performed in ArsElectronica 2002, and Seoul International Dance Festival, 2003. We use images, sounds and sensor devices to synchronize body movement to visuals. In the interactive scenes, the visuals change according to body movements in real-time, resulting in expansion of the performance potential of the body.

Pictures at Ars Electronica in 2002

Ongoing inquiry goes into the interaction between stage and audiences. Stage performances are conventionally designed to present images in one way, namely from stage to audience. Audiences are fixed and basically see the same stage scenery and action. Given this convention, how do we realize an interactive relationship between stage and audience? 

The concept of our new performances is to make the stage more interactive (i.e. the stage could be composed to include audiences) so that audiences can have a more individualized experience. Consequently, we realized an 
gillusory interactiveh performance using Saccade-based Display.

When a flickering light array moves at a fast enough speed, we can perceive 2D images through retinal afterimages as in following figure left (ISM Inc. Fantazein). Conversely, if light sources are fixed on a vertical line and the flashing pattern is changed quickly during a horizontal saccade eye movement, 2D images can be perceived  as in the figure right. Saccade-based Display makes afterimage by viewer's eye movement. 

  ISM Inc. Fantazein                                  Saccade-based Display 

What's "Illusory Interactive" ?
 In most interactive installations or performances, interactivity occurs through obvious physical action of viewer with some kind of sensor device.
However, in our situation, we are exploring a kind of interactivity that occurs through non-obvious, unconscious physical action of the viewer and no sensor device. We call this interactivity as "illusory interactive".

 On performing art stages, audiences make eye movements at various times triggered by performerfs body actions and stage lights. When we use the Saccade-based Display on the stage, audiences can perceive different images according to their eye movements, even if performers move in the same way. Additionally, the shape of perceived images are different according to their own eye movements. Thus, their actions make the scene more interactive, and the display makes the stage an original experience for them.

We performed the interactive stages using four large LED arrays (1.8m in height). They are located behind the performers as in following figure left. Using four columns with flashing LEDs, we can see a 2D pattern when our eyeball moves at a fast enough speed across the LEDs as in the figure right. These patterns seem to hang in mid-air. We designed stage sets, images, sounds and choreography to effectively induce audiences' eye movements. The LED arrays function not only as illumination, but as a visual display, which can present different 2D images for each audience. 

Please see the VIDEOs !
Performance Only (4Mbyte)
Performance and Introduction (24Mbyte)

Performers and LED arrays during fixation            Perceived image during eye movement 
(we took right image by quickly rotating video camera, so stage performers appear smeared.)

In the performance above, the LED's power of illumination may overcome performer's body. Performer may be just a provocative of viewer's eye movement. For the next challenge in collaboration with Maria Adriana Verdaasdonk's project "Patchwork in Motion", performer's movement is synchronized with sound to enhance the expression of the body. Therefore, body and illumination can coexist for the expression.

Conceptual Picture by Tetsutoshi Tavata

contact     Junji Watanabe: junji(at)star.t.u-tokyo.ac.jp