From the Field: Why Mix Media?

In an effective manner digital devices seem to be able to do it all, but is there an unintentional forging of dependencies in their screen magic?

On screen, all seems to be out in the open, and by fingertip gestures ready for thorough examination.  By fingertip we scroll in all directions, pause, zoom in and zoom out; mix video with still imagery, animate, and by learning to judge how far to stand from our target we can get its full dimensions into a single photo making part-to-whole examinations of a subject take on an unprecedented ease and degree of detail.

An inviting virtual depth is created by uniform, intense backlighting for high contrast black to white values combined with maximizing pixel distribution for rich, vivid colors which enhance the separating edges between forms; together these are adjustable through Settings making screen brightness highly individualistic.

Assumption of dependencies being forged is based on observation of three inseparable user behaviors: near screen viewing distance, range and frequency of head movement and maintenance of body trunk stability which is needed for the continuous exercise of precise fingertip movements. This observer considers these three behaviors as significant constraints compared to how these same behaviors are observed when students are working with traditional media. 

Scene editing options, Toolbars, can deeply engage us in the screen image.  Nothing outside the screen’s perimeter is needed.  We can move everything, color everything, size and pack everything while our heads remain nearly still.  Fingertip gestures require upper body, gross motor stability so the continuous use of our fingertips as editing tools strongly favors maintaining a relatively fixed head position. The head movements that would typically be part of working  - of drawing, writing and editing, cease to be integral with these actions. Head movement is the key prompt to dislodge our eyes from the relaxed position of a forward gaze (The Head-neck Sensory Motor System   A. Berthoz, ‎Werner Graf, ‎Pierre Paul Vidal Chapter 68)

We can shift our gaze from central fixation into the other eight positions of gaze but it takes effort to hold our eyes there, to observe something in the periphery without turning our head towards it.  Instead, rewarding shifts in gaze prompt us to turn our head in the direction of interest – the head turn momentarily dislodges our eyes – if we then maintain this new view our eye muscles return to a more relaxed tension as this becomes their central, forward gaze position.  These are subtle movements we are rarely conscious of.

The narrow span of our best visual discrimination (foveal vision, described as about the span of a thumbnail when viewed at arm’s length) facilitates our comfort with a gradual reduction in head movements. If the impetus to shift gaze is reduced for extended periods of time neural sensitivity, the peripheral retina’s attunement to quickly identify stabilizing elements in the visual scene may be decreased.  Such masking would be a form of suppression. The edges of our visual field shift in synch with our eye movements. The 20/800 acuity of our peripheral retinal is an important alerting mechanism.  Its visual cues inform us if may maintain our body weight in its present position or that stability is no longer maintainable, that a threat may loom.  Digital devices offer only photographic consideration of the environment outside of their skins.  This is not enough.

A Mixing Tool

These pictures illustrate and model some media mixing strategies.  The device is a folding slantboard with surfacing for use with dry and wet erase markers.  The board is called a kickstand.

Strategy:  integrate the digital device with a basic need 

Here, the need is being able to say with relative precision where things are located.

Picture of a suspended pocket watch on a beach used for teaching the relative position strategy of describing an object’s location in terms of its alignment with number positions on a clock face.  The picture’s surface is covered for use with markers. Pointing arrows from watch center to rim numbers and beyond may be drawn in.  The outline of hands grasping the board mark the hand position students assume when taking hold of the board. Holding the board with thumbs up, the student’s position corresponds with that of the suspended watch. The user than lowers the board to be parallel with the ground and sites objects in the surrounding space.

Photo of a pocket watch with a beach background. Blue arrow drawn from the middle of the watch to 11 & marker laid on top of the  photo.

The kickstand’s underside is covered with slip-resistant material which wraps around from bottom to front edge.  The kickstand unfolds so that the picture is hidden and a clear surface is presented to the user.

photo of a kickstand - 3 hinged pieces in the shape of a "z" with the bottom piece flat on a desk and a hand holding the top piece almost parallel to the table.

photo of the kickstand with bottom piece flat on the table, middle piece perpendicular to the desk and hand holding the top piece folded back- similar to a traditional slant board in a triangle shape.

The stand’s folding panel seats in a trough that locks its position so that it may pressed against.  The front edge wrap around of the slip-resistive material is pitched so that items rested against the panel cannot slip forward off of the board.

Strategy:  provide a working space supportive of integrating mixed media 

Student using a BrailleNote connected via VNC software with her iPad.  Using the iPad’s zoom function the VNC display of her text may be custom sized to her need. This accommodation is helpful in establishing a good working distance. The iPad is not the exclusive element in her visual scene.  She is reading from, and also editing, information from the large print copy of a recipe her class will purchase items for and then prepare.

Photo of a student inputting text using a BrailleNote paired with an upright iPad; a print recipe is upright (on the kickstand) beside the iPad.

Stategy:  stir the media:

Within the same working distance embellish the information appearing on the device’s screen so that gaze must shift between them.

Kickstand is upright on desk with student's handwriting (using Expo erasable blue pen) with upright iPad covering right side of kickstand.

Paul Del Frari   TVI    COMS

Read more about: Assistive Technology