Designing immersive and interactive experiences for positive change

Immersive and interactive technologies do not yet feature in existing design frameworks for eliciting positive change that draw on research across human-computer interaction, user centred design and psychology.

But recently researchers in Canada carried out a review of immersive and interactive experiences for positive change to distil common design strategies based on the experience’s inputs, outputs and desired outcomes.

Inputs

When the audience enter the experience, their inner states are measured and collected via three types of input

• • Biofeedback
    • Changes to blood flow
    • Skin electrical activity
    • Respiration rate
    • Brain electrical activation (EEG)
  • Physical movement
    • Arms
    • Body
    • Head
  • Controllers
    • Joystick
    • Screen

Measuring respiration rate is a common type of biofeedback- it’s relatively non-invasive and breath is important in mindfulness and meditation. But there can be limits to using physiological measures. For example, data can be noisy, especially EEG signals of brain electrical activation. Also, wearing physiological sensors might distract from the experience. Controllers are used least since they might cause distraction from the experience.

Outputs

Input data are fed into the system and represented as six types of outputs

• • changes in object appearance or animation
  • changes in music or audio
  • changes in light or colour
  • object movement
  • audience movement
  • levitation or floating

Changes in object appearance or animation and in music or audio are more likely to lead to relaxation or calm whereas object and audience movement are more likely to lead to engagement. Interestingly, other human senses tend to be underutilised, such as smell, touch, and temperature.

The audience responds to these representations of their inner states so that the system and audience can influence each another. This feedback loop reinforces positive experiences to make positive change in the audience

Mapping inputs and outputs onto desired outcomes

The researchers identified 16 types of outcomes:

• • Relaxed
  • Content or happy
  • Reflective
  • Mindfulness
  • Harmony or balance
  • Appreciation
  • Calm
  • Stress or anxiety
  • Connection or empathy
  • Clarity
  • Focus
  • Increased wellbeing
  • Emptiness or disembodied
  • Engaged
  • Presence and embodied
  • Increased risk perception

The researchers then mapped these input-output modalities onto their corresponding outcomes. Colour intensity and stroke breadth indicate number of experiences for that category going left to right.

The researchers used these results to propose the following framework to design future experiences based on the:

• • outcomes resulting from the audience’s interaction with the system (orange)
  • psychological theories to bring about these outcomes (purple)
  • design strategies informed by these psychological theories (green)
  • input (blue) and output (red) modalities informed by these design strategies.

The researchers also proposed the following design principles

• • Consider the outcome first then work backwards. Let a goal of the intended experience inform design choices.
  • Consider the type of technology last. Don’t get blinded by the latest technology if it’s not the most effective one for the experience.
  • Help navigate the audience through the experience. Monitor physiological signals to understand changes to the audience’s internal states and provide audio-visual (AV) feedback to help the audience understand how their internal states are affected by their body and environment
  • Use biofeedback to mediate changes to the virtual environment. Feedback that allows the audience to represent their internal states in virtual environments helps them to better understand and control their own internal states. The harder it is to notice a physiological change, the more obvious the feedback should be in the virtual environment, and vice versa.
  • Map physical and virtual movements together. Physical movement in a virtual environment allows the audience to feel more immersed in the experience.
  • Use sensory changes to support relaxation, contentment, and harmony (audience passive). Specific changes in music or audio, object appearance or animation, and light or colour can be associated with outcomes of relaxation, contentment and harmony.
  • Use movement to support calmness, clarity, and focus (audience more active). Big sweeping physical movements of the audience (and expanding or contracting virtual objects in rhythm with audience input) can release bodily tension and stress.
  • Help the audience become mindful of their bodies and remain present in the moment. Make audience members become aware of their breath, and use biofeedback and AV cues to refocus attention if the audience gets distracted
  • Use minimalist design. Abstract imagery that does not come with preconceived ideas or feelings is less likely to trigger unwanted emotional responses. However, replicating nature sounds and images can draw on the benefits of being in contact with nature.
  • Create feelings of connectedness. Support interaction with others directly or indirectly via their avatars or telepresence
  • Support curiosity and participation. Encourage movement and playfulness and help the audience express their emotions through AV mapping.