An Audio-Based 3D Spatial Guidance AR System for Blind Users

Lecture Notes in Computer Science

We introduce CamIO, an AR app for Visually Impaired People (VIP) to reach objects in their immediate environment through real-time 3D audio guidance
Accessibility
Augmented Reality
Sensory Substitution
Auditory Interface
Computer Vision
Authors

James Coughlan

Brandon Biggs

Marc-Aurèle Rivière

Huiying Shen

Published

September 12, 2020

Doi

10.1007/978-3-030-58796-3_55

Abstract

Augmented reality (AR) has great potential for blind users because it enables a range of applications that provide audio information about specific locations or directions in the user’s environment. For instance, the CamIO (“Camera Input-Output”) AR app makes physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons by providing real-time audio feedback in response to the location on an object that the user is touching (using an inexpensive stylus). An important feature needed by blind users of AR apps such as CamIO is a 3D spatial guidance feature that provides real-time audio feedback to help the user find a desired location on an object. We have devised a simple audio interface to provide verbal guidance towards a target of interest in 3D. The experiment we report with blind participants using this guidance interface demonstrates the feasibility of the approach and its benefit for helping users find locations of interest.

Preprint Code

Link to the PDF (preprint)

Citation

BibTeX citation:
@inproceedings{coughlan2020,
  author = {James Coughlan and Brandon Biggs and Marc-Aurèle Rivière and
    Huiying Shen},
  editor = {Miesenberger Klaus and Manduchi Roberto and Covarrubias
    Rodriguez Mario and Peňáz Petr},
  publisher = {Springer International Publishing},
  title = {An {Audio-Based} {3D} {Spatial} {Guidance} {AR} {System} for
    {Blind} {Users}},
  booktitle = {Lecture Notes in Computer Science},
  volume = {12376},
  pages = {475-484},
  date = {20-09-12},
  url = {https://link.springer.com/chapter/10.1007/978-3-030-58796-3_55},
  doi = {10.1007/978-3-030-58796-3_55},
  isbn = {978-3-030-58795-6 978-3-030-58796-3},
  langid = {en},
  abstract = {Augmented reality (AR) has great potential for blind users
    because it enables a range of applications that provide audio
    information about specific locations or directions in the user’s
    environment. For instance, the CamIO (“Camera Input-Output”) AR app
    makes physical objects (such as documents, maps, devices and 3D
    models) accessible to blind and visually impaired persons by
    providing real-time audio feedback in response to the location on an
    object that the user is touching (using an inexpensive stylus). An
    important feature needed by blind users of AR apps such as CamIO is
    a 3D spatial guidance feature that provides real-time audio feedback
    to help the user find a desired location on an object. We have
    devised a simple audio interface to provide verbal guidance towards
    a target of interest in 3D. The experiment we report with blind
    participants using this guidance interface demonstrates the
    feasibility of the approach and its benefit for helping users find
    locations of interest.}
}
For attribution, please cite this work as:
James Coughlan, Brandon Biggs, Marc-Aurèle Rivière, & Huiying Shen. (20 C.E.). An Audio-Based 3D Spatial Guidance AR System for Blind Users. In Miesenberger Klaus, Manduchi Roberto, Covarrubias Rodriguez Mario, & Peňáz Petr (Eds.), Lecture Notes in Computer Science (Vol. 12376, pp. 475–484). Springer International Publishing. https://doi.org/10.1007/978-3-030-58796-3_55