Sonification: Sounds with Meaning Activity

Sightlings - students with vision - are able to glance at a chart and instantly identify the trend and general information without exploring each data point. Using digital materials, students with vision impairments have the same ability to quickly 'glance' - mmm 'listen' - to a chart and instantly identify the trend and general information. What? Just how is this possible? 

What is Sonification

Let's take a closer look at sonification. According to Wikipedia, "Sonification is the use of non-speech audio to convey information or perceptualize data. Auditory perception has advantages in temporal, spatial, amplitude, and frequency resolution that open possibilities as an alternative or complement to visualization techniques." Simply put, sounds have meaning. In the case of sonified charts and graphs, sounds can be used to provide critical information about a chart or graph.

How Does Sonification Work?

You will need to listen to the following samples using stereo headphones, earbuds or speakers. If using headphones or earbuds, be sure that the left headphone or ear bud is in the left ear and the right headphone or earbud is in the right ear. If using speakers, be sure to place the left speaker on the left side of the student and the right speaker on the right side. 

Sample #1

Listen to Line Chart Sample #1.

What can you tell me about the sounds in this audio clip? Listen again.

Teacher Hint: Use open ended questions! Only use leading questions if your student does not pick up the important pieces.

What did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds started low and progressively - in equal steps - went higher
  • Approximate number of tones (16)

What does this mean?

  • Panning: The direction the sound is heard from is important - it relates to the X-axis. Sounds heard in the left ear are data points located on the left side of the graph; sounds heard in both ears are data points located in the middle of the chart; and, sounds heard in the right ear are data points located on the right side of the chart. Remember, the X-axis runs from the left side to right side of the chart.
    • In Sample #1, you listened to the sounds from the left side moving across to the right side.
  • Pitch: The pitch of the sound correlates to the Y-axis. The lower pitches are located at the bottom of the Y-axis and the higher pitches are located at the top of the Y-axis. Remember, the Y-axis runs from the bottom to the top of the chart.
    • In Sample #1, the pitch started low and progressively stepped higher.
  • Size of the Chart: Each data point is one sound.
    • In Sample #1, there were approximately 16 data points. (We will compare the size of this chart to others charts.)
  • Summary of Sample #1: The pitch started low in the left ear and progessed in equal steps (pitch) while panning progressively to the right. This systematic rising pitch indicates that the data points created an "increasing trend".
    • Ask your student what the Line Chart would look like. If the student is not yet able to describe the Line Chart, create tactile versions of these charts and ask the student to find the tactile chart that corresponds with the auditory Sample #1. Sample charts are attached below. Note: Do not label the tactile charts the same as the auditory charts! 

Sample #1 is a line chart with a straight, diagonal line from the bottom left to the top right.

Sample #2

Listen to Sample #2.

What can you tell me about the sounds in this audio clip? Listen again.

What did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds started high and progressively - in equal steps - went lower
  • Approximate number of tones (16)

What does this mean?

  • Panning: Sounds moved left to right
  • Pitch: Progressive steps from high to low
  • Size of Chart: Approximately 16 data points
  • Summary: This chart started high on the left and progressively moved low to the right. The trend is downward/decreasing. Chart is approximately the same size as Sample #1.

The data points in the Sample #2 chart create a straight, diagonal line from the top left to the bottom right.

Sample Chart #3

Listen to Sample #3.

What can you tell me about the sounds in this audio clip? Listen again.

What did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds started high and progressively - in equal steps - went lower
  • Approximate number of tones (16)

What does this mean?

  • Panning: Sounds moved left to right
  • Pitch: Progressive steps from high to low then back to high
  • Size of Chart: Approximately 16 data points
  • Summary: This chart started high on the left, progressively in equal steps moving lower, then back up to the top right. The trend is a curve: starting high, dipping down and then moving back up. Chart is approximately the same size as Sample #1 and #2.

The data points in the Sample #3 chart create a curved line starting at the top left, curving down to the X-axis and then curving back up to the top right. 

Sample #4

By now, you are getting the hang of how sonification can provide an instant snapshot of chart. Ready for a challenge?

Listen to Sample #4.

What can you tell me about the sounds in this audio clip? Listen again.

What did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds stay in the middle range of the Y-axis: starting in the middle, progressing up a few points, then curving down, back up to the same upper height, then back down to the same lower height, then up a few points. Steps are progressively spaced
  • Approximate number of tones (16)

What does this mean?

  • Panning: Sounds moved left to right
  • Pitch: This chart started in the middle of the range curves a few steps higher, curves lower, curves back to same "high" pitch before dropping to the same "low" pitch then a couple steps high again. Compared to previous charts, pitch stayed in the middle range (did not go as high nor as low as the other sample charts).
  • Size of Chart: Approximately 16 data points
  • Summary: Progressive equal steps indicates a curved wavy line staying in the middle range moving slightly up, curving down, up, down and a slightly up. Chart is approximately the same size as Sample #1 and #2.

The data points in the chart create a curved, wavy line that stays in the middle range of the Y-Axis, moving up and down two times.

Sample #5

Let's change things up. Listen to Sample #5.

What can you tell me about the sounds in this audio clip? Listen again.

What did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds jumped around randomly with one anomaly - a high note in the middle of the chart. The last note was the lowest note.
  • Approximate number of tones (9)

What does this mean?

  • Panning: Sounds moved left to right
  • Pitch: Random jumps in pitch with one high "spike".
  • Size of Chart: Approximately 9 data points (smaller than previous charts)
  • Summary: The jumps in the pitches indicate that the line is jagged and the random pitch pattern indicates that there is not a trend. The one high note indicates a spike in the middle of the chart.

The data points in the Sample #5 create a jagged, random line - without a trend - with a spike in the middle.

Teacher Note: The corresponding tactile and print graphics depicts a simple bar chart not a line chart. For the purpose of learning to listen to information provided by sonification, the type of chart is not important. In these activities, we have not provided information about the X and Y-axis labels. To learn more about the different types of charts and why one type of chart is chosen over another, see the Chart and Graphs Skills Review post.

Sample #6 

Bet sonification makes more sense now with the basic charts we have looked at/listened to. What about more complex charts? Let's listen to a complex chart which has numerous data points.

Listen to Sample #6a. Whoa! Did you catch that? Let's listen to this one again!

What can you tell me about the sounds in this audio clip?

What did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds jumped around randomly with the last notes being the highest notes
  • Numerous data points - no idea how many!

Maybe, if it was slower, it would be easier to hear what is happening in this chart with numerous data points. With SAS Graphics Accelerator, you have the option of five listening speeds and the ability to manually navigate note-note-by-note through your graph. Sample #6a was the fastest speed.

Let's listen to Sample #6b (Fast Speed)

Still too fast for the amount of data points?

Listen to Sample #6c (Normal Speed)

Bet you can pick out more details at this speed! What can you now tell me about this chart?

Listen to Sample #6d (Slow Speed)

Which speed did you prefer for this chart? Is this the same speed that you used with the other charts?

What did you notice?

After listening to this chart several times at different speeds, what did you notice?

  • Sounds started in the left ear and traveled to the right ear
  • Sounds jumped around randomly up and down throughout the chart with lower notes on the left and higher notes on the right. notes
  • Numerous data points - too many to count!

What does this mean?

  • Panning: Sounds moved left to right
  • Pitch: Numerous random jumps up and down in pitch and overall moving higher towards the right side of the chart
  • Size of Chart: Huge (when compared to previous charts)
  • Summary: The jumps in pitch indicated jagged up and down line with an overall increase; chart has numerous data points.

The data points in the Sample #6 create a jagged, random up and down line with an overall upward (increasing) trend.

Teacher Note: The tactile/print version of this chart depicts a series chart with numerous data points.

Charts: Visual

No cheating! Listen to the audio clips above to determine each chart's layout before looking at the chart! The charts below can be printed and/or embossed. 

SAS Graphics Accelerator

The audio clips used in this post were actual recordings of sonified charts using the SAS Graphics Accelerator, a free, accessible tool that converts data visualizations into accessible charts and graphs. Students can use the Accelerator's automatic play - at the student's desired speed - to "glance" at a chart. Students can also manually navigate through a chart to hear the details of a desired data point.

Resources

Learn more about the SAS Graphics Accelerator and how to use the Accelerator for classroom activities.

Other posts in this series:

There are a number of additional Accelerator posts; here are a few to help you get started!

Collage of sonification