Strategy
Adventures of Karna: Multisensory App!
Check out this adventurous game that incorporates sonification and vibrations to make images accessible!
Can you guide Karna, the crown prince, through a mysterious land and back to Lanka to save his people? The Adventures of Karma is a free, fully accessible iOS game that includes using multisensory images to solve puzzles.
What makes this game so special?
- UNAR Labs are using cutting-edge technology (sonification and haptics) to solve a critical digital challenge for students who are blind or low vision. Digital educational materials often include images; these images are meant to be more interesting/interactive to keep students’ attention. General education students typically explore the images using vision to glean the information. For students who are visually impaired, providing image descriptions – the traditional way to adapt digital images – is often wordy and boring; students will frequently skip the image descriptions or tune them out. It is often challenging to provide succinct image descriptions without giving away the answer. With advancements in technology, developers such as UNAR Labs are harnessing sonification (sounds with meaning) and haptics (vibrations) to create images that are fully accessible to students who are blind or low vision.
- Through this app, UNAR Labs is gathering data about how students access and use these interactive, multisensory images.
- Adventures of Karna has gamified the technology. Students are playing a fun game with incorporated multisensory images – students are learning to use this new technology and data is being collected all while playing a game!
Multisensory Images
Dragging a finger around the screen, the student can find and explore the multisensory image. These images are interactive, meaning that when the student touches a line, the line makes a humming sound. If you are playing the game on an iPhone, the lines will also have haptics; the phone will vibrate when you are touching the line. Shape images have an additional tone which is heard when touching the corner – this is a helpful clue that there is a turn!
Image 1. Text in the screenshot: As you and Karna near the riverbank, you find an abandoned rowboat along the shore. “Caution friends, if we bring a boat into Lankan waters that is flying the enemy’s colors, they may sink us,” Karna says to you. Can you describe the symbol on the flag to Karna?
Note: The images in the text, such as the boat in the photo above, do have image descriptions. The puzzle images in the game are multisensory images.
Note: iPads do not have haptic motors and will not vibrate. Currently, my iPad running iOS 15.1 did not have sonified lines. It appears that the app is currently designed only for the iPhone and not for an iPad. The Adventures of Karna works well on my iPhone with sonified lines and haptics. Personally, I would like to see this game work on an iPad with sonification (no haptics), as many students have access to an iPad (and not an iPhone) at school.
The video below shows the app on an iPhone with the haptic vibrations. Note: The humming sound heard when the line is touched is not captured in video, even though I could hear the sound and could trace the line by following the sounds.
The video below demonstrates VoiceOver reading the text on the screen and then dragging a finger to explore the shape puzzle.
Once the student is finished exploring the multisensory image, three-fingers wipe right or activating the Back button at the top left will go back to the previous screen, where the student selects his/her answer.
Other images make included announcements such as the name of the town (on the map) or to identify the hour and minute hands on the clock.
Image. Text in the screenshot: You both begin walking in the direction you hope is Lanka. After several hours of travel, you come to a crossroads. There are no signs, but you think you can figure out where you are on the Karna’s map, and the road you must take toward Lanka. Image of the small map. Answer choices: Left, straight ahead. (There are a total of four answer choices, but two are not visible in this screenshot.)
Additional Information
Listen as the story unfolds and solve the puzzles to go on to the next section. There are numerous puzzles, each with a different type of multisensory image. This game cane be played more than once, as many of the puzzles change each time the game is re-opened! The game developers mention that additional episodes may be available in the future.
When exploring the shape puzzle (symbol on the ship’s flag), I did not always hear the tone on the corners and/or the hum of the lines (particularly on the square). I was using my iPhone with it’s smaller screen and wondered if the side lines on the square were too close to the edge of the iPhone screen? The triangle shape worked more consistently for me. When my finger dragged outside of the square I heard beeps; however, I did not hear beeps when dragging outside of the triangle.
Students use the multisensory images to answer questions. The game does not indicate if the answers are correct or not.
Thoughts
As an educator, I am excited to see this game that includes multisensory images! Multisensory images are the way of the future for students with visual impairments and there are several groups currently working on using sonification and/or haptics to make accessible images. UNAR Labs has put considerable thought into the Adventures of Karna multisensory images and have wrapped these images up into a fun game. The story is interesting to students and the interactive images keep the students engaged – while providing opportunities to learn and explore multisensory images! Can’t wait to see more episodes and to see more apps that embrace multisensory images!
Please support the developers! Try this app with students and take a few minutes to share your thoughts at the end of the game.
Resources
- UNAR Labs website
- Adventures of Karna in the App Store
- Sonification Summary Page (list and links to Paths to Technology posts about sonification)
By Diane Brauner
