Providing alternative input for complex interactive learning resources


PhET Interactive Simulations Project

Taliesin Smith

Emily B. Moore, Jesse Greenberg, Michael Kauzmann, & Michael Barlow

CSUN, March 21, 2018

Overview

Introductions

PhET Interactive Simulations Project

Experimentation

Labs & demonstrations

  • Ideal for exploring & experimenting
Student and teacher wearing safety glasses examining a blue liquid in a jar.
  • Not available all the time
  • Pose a range of access issues for students with disabilities
Student working together to explore air flow with a fan and small windmill prototype.

Courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action. For more information about acceptable uses and licensing terms visit https://creativecommons.org/licenses/by-nc/4.0/.

Interactive Science Simulations

  • Are more readily available
  • Do not require special equipment
  • Create a safe lab-like exploratory experience
  • Support student learning (D’Angelo et 2014)
  • Allow students to experiment & discover
  • Help students to think about science

Images courtesy PhET's facebook page.

PhET Interactive Simulations being used in a classroom with desktop computers. PhET Interactive Sim being used in foreign country and language. PhET Interactive Simulations being used by small kids for fun on the couch.

PhET Interactive Simulations

Several New HTML5 PhET Simulations.
  • Started in 2002
  • 170 science & math simulations
  • Run 100 Million+ times/year
  • 87 languages
  • Flexible & exploratory
  • Used with a diverse age range
  • Free of cost & openly licensed

Balloons & Static Electricity {demo}

Design Features

  • Intuitive & easy to use
  • Highly interactive
  • Real-time feedback
  • Fun & game-like
  • Subtle connections to real world
  • Very flexible
The starting screen of the Balloons and Static Electricy Simulation. It is small room with a sweater, yellow balloon and wall. Each object shows its net zero charge with pairs of negative and positive charges. Negative charges are blue with minus signs. Positive charges are red with plus signs. There are a few other controls: buttons, radios, a checkbox and a reset button along the bottom of the screen.

Iterative Inclusive Design

Interview diverse users

  • Students & adults with vision impairments using screen reader software
  • Consultants with vision impairments & science expertitse

Goals

  • Align with standards
  • Design for simplicity & flexibility of use (true PhET fashion)
  • Minimize long-term maintenance costs & overall complexity
Image of PhET's iterative sim design process. The design process starts with learning goals. An intial design is then tested and improved iteratively through research. The design is finalized and published to the PhET website.

PhET’s Scene Graph - Scenery

A simplified piece of the scene graph's data tree structure, and the rendered result representing a negatively charged balloon against the wall with each object showing their positive and negative charges.
  • Open source sim architecture (by PhET) that runs all the HTML5 simulations
  • Data tree structure that represents each graphical element within parent and child nodes
  • Handles the logical & spatial arrangements of the graphical scene
  • For performance & flexibility Scenery uses multiple graphics renderers SVG, Canvas, and WebGL

Enhanced Scenery - Parallel DOM

The same simplified piece of the scene graph's data tree structure, showing the yellow balloon against the wall, but now with two rendered representations, the rendered graphics and the rendered HTML of the Parallel DOM.M
  • Renders nodes as HTML elements
  • Leverages structural & interaction semantics of HTML & ARIA
  • Creates a robust accessibility layer
  • Alternative input (& other a11y features) are agnostic to the graphics renderer
  • Keeps all accessibility features in one place, minimizing maintenance costs & reducing complexity of sim code

PhET's Challenge for Alternative Input

Wide range of interactive sim objects

Simple interactions: Buttons, radios, checkboxes

Simple: Maps directly to HTML elements.

Moderate interactions: Slider and a combobox in a phet control panel.

Moderately Complex: Maps to HTML enhanced with ARIA.

Array of weird complex interactive sim objects, some draggable: a water faucet, a stick man, a little girl weighing 30 kg, a yellow balloon, a circuit switch, salt shaker

Complex: Custom approach required.

Alternative Input (keyboard interactions)

Regardless, strive for interactions that

Ohm's Law: Sliders

Ohm's Law simulation on page load.

Code note

<input type=”range” role=”slider”>

Duplicate semantics needed for AT consistency.

Operation via keyboard

  • Tab key moves keyboard focus (pink highlight) to slider
  • Arrow keys adjust sliders
  • Pg Up & Pg Down keys adjust in larger steps
  • Home & End keys jump to minimum & maximum values

Pedagogical miss-step

Ohm's Law 3 values up on V sider and 3 values down on R slider, resultig in values with decimals.

Default step sizes

  • 1/100th of slider range
  • 1/10th of slider range

Result

  • Numbers not ideal for mental calculations
  • Single key presses do not result in a large enough changes to equation & circuit for qualitative comparisons

Solution

Ohm's Law 3 values up on V sider and 3 values down on R slider, with rouded values.

Two Additional Features

  • Customizable default step size
  • Shift key as a modifier key to reach all values

Demo: Accessible Ohm's Law

Arm & Leg Challenge (Hung, 2016)

John Travoltage, dressed to go outside, in mid-step as he crosses a rug and reaches for a doorknob.

Interaction Challenge:
Arm: 360 circle interaction
Leg: 180 degree semi-circle interaction

Slider interaction

Benefits

  • Leg: Intuitive movement with Arrows keys
  • Arm: Intuitive movement with Right & Left Arrow keys
  • Arm & Leg: Physically like moving or rubbing back & forth

Addressable limitations

  • Arm: Non-continuous interaction; fixed start & end positions
  • Arm: Visual disharmony with Up & Down Arrow keys

Pedagogical miss-step

Initial implementation

  • Leg swing range: 0 to 30
  • Arm position range: 0 to 60

Result

Too many positions! Screen reader users found it

  • Difficult to generate charge transfer (too many "Foot off rug" positions)
  • Unclear if they were getting closer to doorknob (too many positions with the same name)
A circle divided into 12 even-sized pie slices representing 12 regions for the arm slider, numbers 0 nd 60 mark the start and end of the slider and number 30 marks wher the doorknob is located. The same region name is read out for every position in the region.
12 region names identify positions within a region

Solution

  • Decreased the input value ranges for the arm and leg
  • Made significant changes to the description approach

Result

  • Effecient movement of Arm & Leg
  • Connection between relative hand position & discharge events
  • Supports the user to create & explore electron discharge events

Demo: Accessible John Travoltage

A circle divided into 12 even-sized pie slices representing 12 regions for the arm slider, numbers from -15 to 15 mark where the 9 landmarks were placed for orientation. Example landmarks, not shown, are Hand pointing at upper door; Hand pointing straight up; Furthest from doorknob, last stop; At doorknob.
9 landmarks added to the regions

How to grab & move a balloon?

User must be able to

  • Pick it up anywhere
  • Drop it anywhere
  • Move it in any direction
  • In addition, charged balloon just moves.

Custom interaction

  • No native HTML element; no native ARIA roles
  • Custom interaction with ARIA Application Role

Pedagogical miss-step

Information overload

  • Technical interaction semantics
  • Position & charge descriptions
  • Instructions on how to grab & move it

Result

  • Difficult to understand the most important sim interaction
  • Users tried other options (native interactions) first
  • Lead to more complicated or nuanced sim scenarios before playing with a single balloon
Yellow balloon has focus, and a huge speech bubble contains all the information the user would hear with a screen reader: Web application, grabbable, one of three, At center of play area. Has no more negative charges than positive charges. Press W, A, S, or D key to move balloon up, left, down or right. Press space to release.

Solution Part 1

Two-step interaction

  • Start with native button
  • Button press launches ARIA Application role
  • Balloon details & move instructions then provided after an intentional grab.

New Challenge

  • Confusion for sighted keyboard users
  • Balloon does not look like a button
Yellow balloon has focus, and a small speach bubble says, Grab yellow balloon, button.

Solution Part 2

Balloons and Static Electricity with visual grab hint about using Space to grab or release the balloon. Balloons and Static Electricity with visual move hint about using the W,A,S, or D keys.

Two visual cues

  • Communicate similar guidance
  • On focus, & after successful grab

Result

  • Intuitive user experience for visual & non-visual keyboard users for a fully custom interaction.

Demo: Accessible Balloons and Static Electricity

Concluding Message

As designers of learning tools, we want all students with or without disabilities to be able learn & explore with the tools we design. We know that:

Students do best when they are able to study what interests them. - Blind chemist

In the context of science, that means the ability to explore & experiment independently and with peers.

Ongoing design project

Find out more: phet.colorado.edu/en/accessibility

Finding PhET Simulations

PhET's Website

Demonstrated A11y Sims

Early work on A11y Sims

Big Shout Out

Collaborators at the IDRC

More about design process: John Travoltage simulation design

Resources & Funders

Resources

Funders

The National Science Foundation; The Hewlett Foundation; University of Colorado.

Contact us

Get involved! Follow PhET

References

← →

/