The ORB Project - Teleaction Rooms

 
Summary

ORB is a collaborative project between Smart Studio, Stockholm and the Conceptual Information Arts Department at San Francisco State University.  The joint project tests the limits, possibilities, and implications of telecommunications via non-traditional, physical sensors and actions.  Plans called for two identical rooms (same size, same furniture, etc)  to be created in San Francisco and Stockholm.    The rooms are linked so that events in one room cause reactions in the other room.  The project allows experimental investigations of these linkages.


 

Description:

In the two places (SF - STO) a small (10sqm) room is furnished identically. This creates a physical intimacy between the spaces by giving the participants a sense of real presence of the remote space. The furniture and objects are configured with different kinds of sensory input and output which will enable communication between the participants in the two remote spaces. For example, touching a heat or pressure sensor in the armrest of a chair in one place will trigger a heat conducting actuator in the armrest of the same chair in the other place.   Activating a photo sensor on one table surface will trigger a light on the remote table surface.

The two spaces are built as lounge areas where people can sit down for a coffee or a talk and the remote action will act as an unobtrusive presence.  A person can choose to engage, observe or even ignore the remote presence, it will not be an imposed interaction but rather an addition to the everyday scenario of a meeting place.

The remote visit might come as a surprise while a person is drinking his/her coffee and reading the paper or talking to somebody else. Our hope is that one will learn to engage other senses than the ones commonly used when tele-communicating and explore and play with the possibilities that this creates.

The project works with low bandwidth and simple, easy-to-use technology and  with time delay as a conceptual asset rather than as a problem, using the time difference between the places as a memory bank and stretching time through the past-present-future of the two places.
 
 

 

Conceptual Challenges

The idea of this project is to explore and visualize interaction between two physically and temporally separate places. This interaction will take place through the use of other kinds of communication devices than those traditionally used - speech,  text based (e.g. phone, internet), and image based (e.g. video, webcam). The interaction will be based on the processing and interpreting of data that will be generated through the use of different kinds of sensor technology and on the relations that emerge out of this interaction.

Historically telecommunication has emphasized immediacy, bandwidth,  and optical and sonic fidelity as its main goals.  The assumption is that persons in physically remote spaces will experience a sense of being tele-present if they can see and hear the distant events in the same way a someone physically present.  The Orb project seeks to reexamine these assumptions by asking a series of questions about these topics.  The conceptualization of the problems and possible responses are major goals of the project.

Time Differences:

Clock time in Stockholm and San Francisco differ by 9 hours.  San Francisco actors are just coming into the space as Stockholm persons are leaving.  A 'real time' linkage would mean that the rooms would primarily be active when the other group is absent.  Thus the work day in San Francisco (9:00-17:00) would correspond with night time in Stockholm (18:00 - 4:00).  A variety of responses were proposed:


 
 
1. Disregard the difference.  The 'phantom' activity of the rooms (eg a very active room in the middle of the night) might be an appropriate response to the realities of being at different places on the globe.
2. Time Shifting: Record the day's activity and replay them at the corresponding clock time in the other location.  For example, the flow of actions in Stockholm would be recorded with time signatures.  Thus, when San Franciscans were in their room at 12 noon they would encounter the events that were sensed in Stockholm at noon the previous day (eventhough it would be 21:00 in 'real' Stockholm time.)
3.  Radical Time Shifting:  A prototype recording system was developed to implement the shifting.  Participants realized that once a day was recorded, more radical time mappings could be implemented.  For example, any dayís activities could be played or the dayís activities could be slowed or accelerated to any pace.  For example with a 8x acceleration an entire 8 hour day could be experienced in one hour.
4. External Conditions;  Provide web camera images of the outside environment (for example, night or day, weather) as base line 'real time' information as a background for differences between the two locations.
5 Philosophical questions:  Physicists and philosophers have questioned common sense notions of time and simultaneity.  Is the idea of 'real' time mapping as unproblematical as it seems?  Although real time synchronization is usually considered less mediated than other notions such as delayed mapping, the delayed clock time mapping might induce more intimate senses of the persons in the remote space.

 
 

Sensors/Actions/Linkages

New sensor and telerobotic technologies have helped inspire the project.  It is now possible to sense many new kinds of features of life and similarly possible to remotely activate many kinds of actions.  The Orb project asks how do these new possibilities expand the possibilities of creating the sense of life in a remote place.  What should be sensed?  What should be activated?  How should sensed activities be linked to teleactivated activities?

Obvious Linkages: The most common sense linkage would connect objects and their normal actions in one space with their corresponding object/actions in the other space.  For example, turning on a light in San Francisco would cause the same light to go on in Stockholm or a spot on the floor in one place would vibrate when someone stepped in the corresponding spot in the other location. These obvious physical linkages would provide interesting experiences to explore.  Would it be interesting to be in a space whose objects were activated in accordance with a remote personís actions on related objects? Would this panorama of change offer a strong sense of life in a remote location?  A prototype system was developed in which people moving past a certain designated doorway caused a variety of sounds to be broadcast in the other location.

Abstract Linkages: Project Orb participants were interested, however, in more abstract correspondences.  What if the linkage was disconnected from an exact correspondence?  What if sensual modalities were shifted?  For example, a prototype system was developed in which sound levels and frequency profiles in one location were linked to projected abstract digital video ëwallpaperí in the remote location.  Arrangements and thickness of lines of light changed with the sonic qualities of conversations in the other place.  Another prototype was developed in which sitting in a chair in one location resulted in changing heat in the chair surface in the other location.  A variety of similar experiments were proposed.

Dynamic linkages:  Researchers suggested that the linkage mapping did not have to be fixed - for example, turning on a light did not always have to cause vibrations in a specific table.  Once a communication system is developed, the mappings could be dynamically changed by project participants or even eventually by outsiders coming via the web.  There could be a kind of switchboard that allowed easy changing.  For example, at Time-1, the Stockholm light could control the San Francisco light; at Time-2 the STO light could cause sound in San Francisco; at Time-3 the STO light could result in bubbles in a SF aquarium.  This kind of platform would allow the ultimate range in linkage experimentation.  Some participants warned that a system that changed its mapping too frequently would be indistinguishable from a random system
 
 

Possible features to be sensed:

-motion
-presence
-sound levels
-voices
- specific sounds/voice
- use of items (light, coffeemaker, radio)
- light and temperature outside
- body signals - pulse, gsr, etc
-. amount of people in the room, length of time) and different qualities
(eg.personal characteristics)
- identity of persons (id tags)
- hand imprints and other body pressures exerted on things and architectural elements (eg foot pressure)


Possible teleactivated events:

- specific sounds
- degraded versions of speech in other place (eg syntax and general semantics preserved with identifying details masked)
- tube of smoke
- vibration
- heat
- indentation/impression
- color - addition, subtraction
- wall paper changed by sound frequency
- activate a fake radio/tv with signals from other space
- ghost avatars (shadows, diffused lights)
- decaying patterns from past actions -  light? sound?
- bubbles in an aquarium
- sound intensity mapped to light intensity in other place
-rotating color filter wheel device
-projected director movie (monochrome) (different color for different amplitude/frequency)
- transformed voice so it loses identity of person
- low frequency sound
- projection of 3-d images from remote place on local place
- additive and subtractive color changes
- decaying patterns indicating time of last presence

Communicating Intimacy/ Peripheral Awareness

An engineering approach would probably expand the historical focus on immediacy and fidelity to the new physical sensors/actuators.  Project Orb sought to question this approach.  What are the most powerful ways to communicate intimacy and the sense of a place?  Borrowing from the arts, the project sought to explore the impact of abstraction, nuance, and peripheral awareness for telepresence.   For example, a moving shadow might project a better sense of life in a place than high fidelity projected video.  Perhaps peripheral awareness of the vibrations of abstract video wallpaper might tell more about activity levels than actual sounds from conversation.  Similarly peripheral awareness of the life at a remote location might end up being significant in different way than intentional, directed communication.   The project also wanted to test the notion that the correspondence of physical arrangements and objects in two places might enhance telecommunication apart from any higher technology.
 
 

What is interstesting about Telepresence?

Although Project Orb  tested prototype rooms with only a few similar objects, it  made some headway in expanding ways of thinking about telepresence.  Both the arts and technology industries are pushing research in physical computing and telepresence. Why is telepresence interesting? What philosophical and technical questions require more interrogation?

Almost everyone who hears about the teleaction rooms finds the idea intriguing.  Yet, that interest must be inspected.  What is the goal of telepresence?  Research centers looking at work situations assume that more presence results in increased and deeper communication that ultimately advances work goals.  For example, video conferences are usually seen as richer and more productive than teleconferences.  Researchers at Xerox PARC and other research centers have spent much engergy on using enhanced telepresence to increase the creativity and output of work groups. Immersive VR researchers maintain that the richness of their environments allow unprecedented levels of remote communication and collaborative work.   Researchers in consumer based telepresence projects believe people will pay for more elaborated telecommunications.

Still, there are some anomalies in the supposed superiority of enhanced telepresence:  Many technology-enhanced, telepresence-oriented work experiments have been dropped because of participant dissatisfaction and lack of productivity gains.  Workers with shared physical history together are able to take better advantage of telecommunications than those who have never met face to face.  People still prefer face to face.  What is it about face to face contact? What gives a sense of place?  Why is there such interest in telepresence and telecommunications?  Some Orb participants wondered if the linked spaces might lose their interest after a while.  Why is linkage between San Francisco and Stockholm more interesting than a linkage across town?

The telecommunications industry cannot just assume that throwing more bandwidth at the question will result in an answer. The history of theatre, landscape painting and photography, and art installations suggests that the answer is not simple.  There is an art to conveying a remote place that must be part of future telecommunications research.
 

Future Questions:

Project Orb just scratched the surface of these questions.  Further research is necessary in topics of time shifting, actions to be sensed, objects to be activated, linkages between the two, the role of abstraction and nuance in telecommunication, and basic questions about the meaning of telepresence.  A collaboration between artists and technological researchers is an ideal environment to pursue these questions.
 

Links

Conceptual Information Arts Program - San Franicsco State University
Smart Studio at the Interactive Institute (Stockholm)
Smart Studio link to both sets of web cams    San Francisco Web Cam Site

Text contributed to by Stephen Wilson (CIA) and Arijana Kajfes (Interactive Institute) and other researchers at both locations.