Doore, S.A. – VEMI Lab

Multimodality as universality: Designing inclusive accessibility to graphical information

eblackwood — Wed, 10 May 2023 19:00:27 +0000

Graphical representations are ubiquitous in the learning and teaching of science, technology, engineering, and mathematics (STEM). However, these materials are often not accessible to the over 547,000 students in the United States with blindness and significant visual impairment, creating barriers to pursuing STEM educational and career pathways. Furthermore, even when such materials are made available to visually impaired students, access is likely through literalized modes (e.g., braille, verbal description), which is problematic as these approaches (1) do not directly convey spatial information and (2) are different from the graphic-based materials used by students without visual impairment. The purpose of this study was to design and evaluate a universally accessible system for communicating graphical representations in STEM classes. By combining a multisensory vibro-audio interface and an app running on consumer mobile hardware, the system is meant to work equally well for all students, irrespective of their visual status. We report the design of the experimental system and the results of an experiment where we compared learning performance with the system to traditional (visual or tactile) diagrams for sighted participants (n = 20) and visually impaired participants (�� = 9) respectively. While the experimental multimodal diagrammatic system (MDS) did result in significant learning gains for both groups of participants, the results also revealed no statistically significant differences in the capacity for learning from graphical information across both comparison groups. Likewise, there were no statistically significant differences in the capacity for learning from graphical information between the stimuli presented through the experimental system and the traditional (visual or tactile) diagram control conditions, across either participant group. These findings suggest that both groups were able to learn graphical information from the experimental system as well as traditional diagram presentation materials. This learning modality was supported without the need for conversion of the diagrams to make them accessible for participants who required tactile materials. The system also provided additional multisensory information for sighted participants to interpret and answer questions about the diagrams. Findings are interpreted in terms of new universal design principles for producing multisensory graphical representations that would be accessible to all learners.

Embedding Expert Knowledge: A Case Study on Developing an Accessible Diagrammatic Interface

eblackwood — Thu, 18 Aug 2022 14:23:40 +0000

When students with blindness and visual impairment (BVI) are confronted with inaccessible visual graphics in the geometry classroom, additional instructional supports are often provided through verbal descriptions of images, tactile and haptic representations, and/or kinetic movement. This preliminary study examined the language used by instructional experts to describe geometry images to students with and without access to a visual instructional image. Specifically, we investigated expert descriptions of geometry diagrams for 1) spatial information, 2) instructional concept information, and 3) overall description structure (e.g., length, vocabulary, image part/whole order/relationships). We found that experts used nearly twice as many words to describe diagrams in the no visual access condition. We consider the double-edged nature of this result for supporting BVI learners in classrooms and chart possibilities for future research.

Keywords: Students with Disabilities, Geometry & Spatial Reasoning, Mathematical
Representations

Download PDF

Natural-Language Scene Descriptions for Accessible Non-Visual Museum Exhibit Exploration and Engagement

eblackwood — Tue, 01 Oct 2019 14:15:47 +0000

Indoor navigation and exploration of museum environments present unique challenges for visitors who are blind or have significant vision impairments (BVI). Like other indoor spaces, museums represent dynamic indoor environments that requires the need for both guided and self-tour experiences to allow for BVI visitor independence. In order to fully engage with a museum and its exhibits, BVI visitors need assistive technologies that support natural-language (NL) spatial descriptions that provide flexibility in the way system users receive descriptive information about gallery scenes and exhibit objects. In addition, the user interface must be connected to a robust database of spatial information to interact with mobile device tracking data and user queries. This paper describes the results of an early-stage demonstration project that utilizes an existing graph database model to support a NL information access and art gallery exploration system. Specifically, we investigated using a commercially available voice assistant interface to support NL descriptions of a gallery space and the art objects within it. Future work involves refining the language structures for scene and object descriptions, the integration of the voice assistant interface with tracking and navigation technologies, and additional user testing with sighted and BVI museum visitors.

Citation: Doore, S.A., Sarrazin, A.C., and Giudice, N.A. (2019). Natural-Language Scene Descriptions for Accessible Non-Visual Museum Exhibit Exploration and Engagement. Stock, K., , Jones, C., & Tenbrink, T. (Eds.) In the Proceedings of Workshops and Posters at the 14th International Conference on Spatial Information Theory (COSIT 2019). Regensburg, Germany, Springer International Publishing, (Pp. 91-100).

Keywords and phrases: indoor environments, scene descriptions, voice assistant interface

Download PDF

Evaluation of an accessible, real-time, and infrastructure-free indoor navigation system by blind users in the Mall of America

eblackwood — Tue, 02 Apr 2019 14:34:07 +0000

Introduction: This paper describes an evaluation of MagNav, a speech-based, infrastructure-free indoor navigation system. The research was conducted in the Mall of America, the largest Mall in the country, to empirically investigate the impact of memory load on route-guidance performance.

Methods: Twelve participants who are blind and twelve age-matched sighted controls participated in the study. Comparisons are made for route-guidance performance between use of updated, real-time route instructions (system-aided condition) and a system-unaided (memory-based condition) where the same instructions were only provided in advance of route travel. The sighted controls (who navigated under normal visual perception but used the system for route guidance) represents a best-case comparison benchmark with the BVI participants who used the system.

Results: Results across all three test measures provide compelling behavioral evidence that blind navigators receiving real-time verbal information from the MagNav system performed route travel faster (navigation time), more accurately (fewer errors in reaching the destination), and more confidently (fewer requests for bystander assistance) compared to conditions where the same route information was only available to them in advance of travel. In addition, no statistically-reliable differences were observed for any measure in the system-aided conditions between the BVI and sighted participants. Post-test survey results corroborate the empirical findings, further supporting the efficacy of the MagNav system.

Discussion: This research provides compelling quantitative/qualitative evidence showing the utility of an infrastructure-free, low-memory demand navigation system for supporting route-guidance through complex indoor environments and supports the theory that functionally equivalent navigation performance is possible when access to real-time environmental information is available, irrespective of visual status.

Implications for Designers and Practitioners: Findings provide insight for the importance of developers of accessible navigation systems to employ interfaces that minimize memory demands.

Keywords: indoor navigation system; blind travel; nonvisual navigation; speech-based interfaces

Citation: Giudice, N. A., Whalen, W. E., Riehle, T. H., Anderson, S. M., & Doore, S. A. (2019). Evaluation of an Accessible, Real-Time, and Infrastructure-Free Indoor Navigation System by Users Who Are Blind in the Mall of America. Journal of Visual Impairment & Blindness. Advance online publication:

Download PDF

Spatial Prepositions in Natural-Language Descriptions of Indoor Scenes

eblackwood — Wed, 28 Feb 2018 19:32:02 +0000

In order to provide accurate automated scene description and navigation directions for indoor space, human beings need intelligent systems to provide an effective cognitive model. Information provided by the structure and use of spatial prepositions is critical to the development of accurate and effective cognitive models. Unfortunately, the use and choice of spatial prepositions in natural language is extremely varied, presenting difficulties for natural language systems attempting to provide descriptions of indoor scenes and wayfinding directions. The goal of the present study is to better understand how humans use spatial prepositions to communicate spatial relationships within virtual environment (VE) indoor scenes. A series of experiments investigates spatial preposition use and the influence scale, topology, orientation and distance within indoor scene descriptions and preliminary results are reported.

�侱�ٲ��پ��ǲ�:��Doore S, Beard, K., and Giudice, N.A. (2017). Spatial prepositions in natural-language descriptions of indoor scenes. In P Fogliaroni, A Ballatore and E Clementini (eds.) Proceedings of the 13^th International Conference on Spatial Information Theory (COSIT’17). Lecture Notes in Geoinformation and Cartography, (pp. 255-260). Springer.

Download PDF

Spatial Preposition Use in Indoor Scene Descriptions

eblackwood — Tue, 12 Jul 2016 18:17:59 +0000

In order to provide accurate automated scene descriptions and navigation directions for indoor space, humans need intelligent systems to support development of effective cognitive models. Information provided by the structure and use of spatial prepositions is critical to this process but also challenging as the use and choice of spatial prepositions in natural language is extremely varied. The goal of the present study is to better understand how English language speakers use spatial prepositions to communicate spatial relationships within virtual environment (VE) indoor scenes. A series of experiments investigates spatial preposition use and the influence of scale, topology, orientation, and distance within indoor scene descriptions and preliminary results are reported.

Citation:

Doore, S.A., Beard, K., and Giudice, N.A. (2016). Spatial preposition use in indoor scene descriptions. Paper presented at the Ninth International Conference on Geographic Information Science (GI Science ’16). Sept. 27-30, Montreal, CA.

Download PDF