Karp Library Fellow, Ayiana Crabtree ’22 joined the Studio X team in February 2021 as our inaugural XR Research Fellow. For her final research project, we asked Ayiana to focus on XR and accessibility. Technology in general creates many barriers for disabled users. As XR technologies are rapidly growing in popularity, they exacerbate these challenges. When creating an XR product, whether that be a VR (virtual reality) headset or an AR (augmented reality) game, etc., people tend to think more about their product’s aesthetic or its usability for the average user. What people fail to remember is that not every user will be “the average user.” The world is a diverse place, with people of all ages, genders, races, and abilities, and when creating XR, it is important to keep in mind this diversity. XR and accessibility is itself a new area that is a moving target.
The goal of this topic of research was to ensure that Studio X would be prepared to accommodate any person that walks into the space to try out technologies. While this is a long-term goal, the research done this semester is a good first step to making Studio X accessibility-friendly.
In order to make Studio X more accessible for the future, Ayiana conducted research to find ways in which accessibility is already being incorporated with immersive technologies. This involved putting together a resource guide, meeting with the director of the Office of Disability Resources on campus, and running a student survey through the Office of Disability Resources mailing list. View her findings and more in the semester of research recap presentation below!
Summary: This post reviews resources on XR (extended reality) and accessibility and summarizes best practices for centering accessibility when engaging with these technologies.
Technology in general creates many barriers for disabled users. As XR technologies are rapidly growing in popularity, they exacerbate these challenges. When creating an XR product, whether that be a VR (virtual reality) headset or an AR (augmented reality) game, etc., people tend to think more about their product’s aesthetic or its usability for the average user. What people fail to remember is that not every user will be “the average user.” The world is a diverse place, with people of all ages, genders, races, and abilities, and when creating XR, it is important to keep in mind this diversity. XR and accessibility is itself a new area that is a moving target. Because of this, many new developments are in the works, so these resources may be outdated in just a year’s time.
Before we dive into XR, let’s first define some terms: What are Accessibility and Universal Design?
Accessibility is the ability to access something and be able to benefit from its intended purpose. It sometimes refers to specific characteristics that products, services, and facilities have that can be used by people with a variety of disabilities.
Accessible Design is a design process that specifically considers the needs of people with disabilities.
Universal Design is the process of creating products that are accessible to people with a wide range of abilities, disabilities, and other unique circumstances.
Link to Webpage Education, Teaching, Research, Organization, Conferences, Resources
XR Access is a community committed to making virtual, augmented, and mixed reality accessible to people with disabilities. Their mission is to modernize, innovate, and expand XR technologies, products, content and assistive technologies by promoting inclusive design in a diverse community that connects stakeholders, catalyzes shared and sustained action, and provides valuable, informative resources.
The site provides a plethora of materials for those interested in their efforts. Their research network provides valuable information regarding accessibility research that’s happening across the XR access research network. They have workstreams, which are community-led efforts to inform the design, development, and production of accessible XR. In addition to these, they also have a wide variety of other resources that are there to aid people in their own research, some of which are their annual XR Access Symposium reports (see below for more about the symposium). XR Access also curates stories of disabled folks who have used technology both successfully and unsuccessfully to help advocate for accessible XR technology. Those interested can sign up for their newsletter or join their robust Slack community.
Accessibility Needs of Extended Reality Hardware: A Mixed Academic-Industry Reflection
This journal publication walks the reader through the process of and reasoning behind the need for accessible XR hardware and software. By starting out with an explanation of the benefits of XR, they then move on to show why the accessibility movement should start with hardware. If a user cannot wear a headset, then they cannot experience its software. The XR Access Symposium of 2019 allowed many people to connect and expand upon their individual ideas, which allowed them to establish their goals for XR hardware accessibility. They established a need to: understand related fields’ accessibility guidelines, determine the most pressing obstacles, consider industry guidelines, and increase public awareness of the issues at hand. With those needs in mind and a focus on a community-centered approach, they believe it is easily possible to succeed in overcoming the lack of accessible XR hardware.
Barriers to Supporting Accessible VR in Academic Libraries
Although XR technologies offer new opportunities to engage students, they also present more challenges for disabled students. Technology, in general, already tends to exclude these users, and XR’s rapid rate of development further complicates things. The article shares statistics as of 2019 from the U.S. Department of Education, National Center for Education: “19.4% of undergraduates and 11.9% of graduate students have some form of disability.” The authors argue that academic libraries, as leaders in supporting and sharing new technologies, are well poised to address accessibility challenges for XR and must create clear policies and service models that support all users. While no clear accessibility guidelines currently exist, there are several promising initiatives such as XR Access Symposium that are working towards this goal. They detail two accessibility initiatives occurring at Temple University and at the University of Oklahoma. The authors then conclude with a list of key takeaways:
Plan for Accessibility from the Beginning: Libraries can save time and resources by thinking about accessibility issues at the start of a program or project.
Lack of Standards: As of 2020, there are no standards for accessible VR design, but there are related standards that could lay the groundwork for their development.
Developer Support is Essential: Libraries that intend to develop VR experiences need to have sufficient developer support with accessibility expertise.
Importance of Auditing and Reporting: Out-of-the-box VR experiences will pose different accessibility challenges from one person to the next and should be audited to better understand these barriers to access. If a library lacks a developer to modify software or create new software, at the very least, available software needs to be audited and have a corresponding accessibility report produced.
VR is Not the Pedagogy: VR should be another tool in an educator’s arsenal, not the sole focus of a class (unless VR is the course subject). As Fabris et al. (2019) suggest “Having VR for the sake of having VR won’t fly; the VR learning resources need to be built with learning outcomes in mind and the appropriate scaffolds in place to support the learning experience” (74).
Acknowledge the Limits of VR Accessibility: There are limits to making VR accessible. The reality is that there will be students who are unable to use VR for a variety of reasons. Therefore, there should always be an alternative access plan developed so that students have access to non-VR learning methods as well.
XR Accessibility Initiatives in Academic Libraries
As libraries traditionally take the lead in accessibility initiatives, a survey was done to examine the accessibility of their digital resources. Three questions were asked and sent to various academic libraries, and they received responses from 30 universities:
Question 1: What is the level of development of accessibility support for XR technologies in academic libraries?
The majority of institutions surveyed did not have policies or dedicated staff to support the accessibility for XR resources
Question 2: What XR accessibility knowledge do library staff and administrators currently have?
Nearly all participating spaces had some awareness of the challenges that XR provides and are able to find resources to assist when needed.
Question 3: What are the main barriers to developing accessibility support for XR technologies in academic libraries?
The top three barriers to developing accessibility policies and processes were lack of staff knowledge, lack of funding, and lack of time.
The concluding result was that XR and accessibility in academic libraries is still developing, so policies and staff are not yet in place. It is also noted that many institutions have plans to begin progressing towards implementing strategies soon.
DLFteach Toolkit: Lesson Plans on Immersive Pedagogy
The digital library foundation (DLF) has put together a toolkit of lesson plans that facilitate interdisciplinary work engaged with XR technologies. The toolkit is focused on a decolonial, anti-ableist, and feminist pedagogical framework for collaboratively developing and curating humanities content for emerging technologies.
Located in the introductory materials section of the toolkit, there are three particularly useful resources. Recommendations for accessible pedagogy with immersive technology, an immersive technology auditing checklist, and instructions on how to create an equally effective alternate action plan for immersive technologies.
Recommendations for Accessible Pedagogy with Immersive Technology – serves to provide a background for the increasing need for creating educational resources for disabled learners. The list of materials provided are intended to guide educators on how to incorporate immersive technologies into their teaching while also keeping disabled learners in mind. It is split into three sections: accessibility and disability, readings on the accessibility of immersive technologies, and recommended administrative considerations. It ends with a series of questions to keep in mind when teaching.
Immersive Technology Auditing Checklist – serves to identify and document the various challenges of making immersive technologies accessible. It divides the workflow into three steps: purchasing software and hardware, providing technical support for software and hardware, and ensuring user access to software and hardware. The checklist then walks you through a series of important questions when considering each phase of the process, posing questions such as “What hardware is required?” and “Is there an accessibility page for the software?” It also dives into questions about ease of operation and perception, asks about the robustness of the technology, and asks about any documentation about the technology.
Creating and Equally Effective Alternative Action Plan for Immersive Technologies – serves to instruct the reader on how to create an Equally Effective Alternative Action Plan (EEAAP). An EEAAP is a document that is used when there is an accessibility barrier in a technology (i.e. when a technology is unable to be used by a person or group with a disability). The components of an EEAAP are a description of the issue, the person or group affected, the responsible faculty, how the EEAA will be provided, the additional EEAA resources required, repair information, and a timeline for unforeseen events. Some examples of EEAAP’s are listed at the end of the resource.
Exploring Virtual Reality Through the Lens of Disability
This resource comes directly from the DLF Toolkit. It provides a lesson in an interdisciplinary approach to introducing VR immersions through the lens of disability studies. They are not aiming to represent how all people experience disability, rather they are trying to create an activity that includes discipline–specific theory and criticism. They then talk about the different types of VR: cinematic VR uses filmmaking techniques; simulation VR simulates the real and fictional, while the user is an active participant; representational VR creates immersive experiences through sensory embodiment; and therapeutic VR is designed for various treatments.
The resource then becomes an instructional guide on how to try several disability-related experiences. They recommend the audience, curricular context, learning outcomes, materials needed, how to prepare for the experiences, and provide a long list of sample instructions. Following this, they list several important applications they recommend trying: Notes on Blindness, The Party, and InMind VR. Each experience is paired with a plethora of questions and other external resources they found to be relevant.
Notes on Blindness – This experience tells the story of a man who lost his sight and how he coped by keeping an audio diary. For three years, he recorded over sixteen hours of material.
The Party– A VR film by The Guardian that allows you to enter the world of an autistc teenager who is at a surprise birthday party. You will hear internal thoughts about how the experience affects her and share the sensory overload that leads to a meltdown.
InMind VR – A short adventure that allows the user to journey into a patient’s brain and search for the neurons that cause mental disorder.
When you are in the beginning stages of creating something in an XR medium, whether that be a device or an experience, it is important to keep in mind the various factors that might make something less accessible. Accessibility could mean anything from being differently abled than those around you in terms of motor function, sensory deprivation, or wealth and societal standing.
VR has a plethora of positive features that could be beneficial to differently abled users such as the ability to enhance spatial sound on one side of the body, render visuals with higher contrast, and enable those in wheelchairs to experience what it would feel like to “walk around” in VR. However, like with any technology, VR also presents many accessibility challenges such as the heavy emphasis on motion controls, the use of the body to control many experiences, and the requirement to stand during some VR experiences.
Considering these and other challenges, here are some things to keep in mind while trying to make XR design more inclusive:
Hardware – What equipment do people need to participate in a VR environment? Is a standalone headset and controllers all that’s required? Or is there some form of special equipment or a computer to run the experiences also needed?
Navigation and Interfaces – How understandable is the XR environment? If a user had no context or guidebook upon entering the space, would they know what to do and how to interact? Make things either clearly labeled or have a guide or some form of instructions available. This could involve an avatar that appears to give instructions along the way, an instruction dialog box, or a guidebook with your product.
Communication – How are speech and body language communicated? Do you have an avatar that represents you in an environment? Is there full body tracking, or does your avatar just float from place to place? Do you speak using a microphone, or are there pre-written text options to choose from? Is captioning available?
Customization and Interoperability – Allow users to customize the XR environment to their needs. Can you enable color contrast? Can you toggle on and off captioning when needed? Are there a variety of sound options?
Avatars and Embodiment – Make sure that there are a wide range of options so people can feel accurately represented. Is there a wide range of skin tones, hair colors, hairstyles, clothing, etc. that will enable any person from anywhere in the world to feel as if they are properly represented in the VR space?
Try out the space yourself and see if it works from several perspectives of ability, seated, standing, sound, no sound, etc. Think about the users that you want to be able to access the device and try to see it from their perspective. Another way to do this might be having testing where you have differently-abled people come to try out your device/program and offer feedback.
An Accessible Future – XR: Considerations for Virtual, Mixed, and Augmented Reality
There are many XR applications for the workplace, such as virtual orientation events and training sessions. Imagine being able to attend a conference with people from all over the world using VR: you could still get the experience of being among professionals in your field without ever having to leave your home or office. For example, the XR Access Initiative used VR during its annual symposium to foster engagement. They created virtual rooms that conference participants could explore and interact with their surroundings, held virtual demonstrations, and provided captioned rooms and rooms with ASL interpreters.
The XR Access Initiative emphasizes three key accessibility factors for virtual conferences: captions, sign language communication, and keyboard and screen reader usage.
Captions – Captions should follow a user and be legible regardless of what angle from which they view the environment.
Sign Language – Sign language interpreters should be located in high visibility areas, and those who need interpreters should be able to get easy access to them.
Screen Reader/Keyboard – For those who are unable to or do not wish to use VR to attend, they should be able to interact with the space in the same way a person in VR could, though with simplified controls. Having cross-platform capabilities is important.
This virtual symposium showcases how VR can make conferences and other virtual events accessible to many people.
Why VR/AR Developers Should Prioritize Accessibility in UX/UI Design
Link to Article UI/UX, Development, Inclusive Design, Accessibility Settings
An important thing that this article touches on is how a lack of accessibility in VR can make people feel left out or ignored. For example, the easier it is for people to understand a game, the more likely they are to play it. Some things that you might not think about for inclusive design are different hair types or people who experience arthritis. If you have long hair that’s in a ponytail or buns or even fluffy hair, putting on a headset might become difficult as you will have to rearrange your hair into a new position to get the headset on. People with arthritis may need to sit down in the middle of a game, or their fingers or hands get sore after a time. Making controls easier to change in the middle of a game or experience would be very helpful in these cases. Some ways to make VR more accessible for glasses wearers could include the ability to change vision settings or the creation of better glasses adapters for current headsets.
There is a huge importance in having a diverse group of people in your testing groups to ensure that people of all genders, ethnicities, abilities, socio-economic backgrounds, and other identities are able to interact with your product with ease. It may be impossible to accommodate every unique circumstance but taking the diverse voices of others into consideration while making your product will ensure a better end result. While it may take a little more time to try to make sure everyone is included, the end design will be more profitable and beneficial to a larger community, which is most important.
This link is to the proceedings of the International Conference on Computers Helping People with Special Needs (ICCHP). The 2020 ICCHP conference proceedings has a section on XR and accessibility. It has several articles on this topic that cover a wide range of subjects from vocational training for students with disabilities, AR for people with low vision, guidelines for inclusive avatars, and more.
This is a plugin offered on the Unity Asset Store that makes the UI for a Unity project accessible to blind and visually impaired players with just a few clicks.
This is Oculus’ guide for developers on how to create with accessibility in mind. The Accessibility VRCs (Virtual Reality Check Guidelines) focus on audio, visuals, interactions, locomotion/movement, and other aspects of accessible design. By deploying these guidelines, they ensure that every application officially available on their platform will meet certain accessibility requirements–something that might make their platform usable for more people. Link to the VRC Webpage: https://developer.oculus.com/resources/publish-quest-req/
Initiative aims to make virtual, augmented, and mixed reality accessible
This article links to a webinar about a new initiative to make XR accessible to more people. Larry Goldberg, Senior Director and Head of Accessibility at Verizon Media, discusses emerging technologies and how his company deals with this technological growth. The webinar highlights the importance of how we can use existing technologies as a jumping off point to create new accessible technologies from the beginning, or as Larry Goldberg says, have the technologies be “Born accessible.”
The XRA’s (XR Association’s) developer guide serves as a starter resource for developers looking to create XR experiences. The guide offers a series of industry-backed best practices to developing accessible platforms.
This is Oculus’ guide for those wishing to develop accessible content for their platform. They note the importance of accessibility as it pertains to widening the potential customer base.
This is Microsoft’s project that considers how to design mixed reality technologies in a way that makes them usable and useful to people of all abilities. This webpage links to those involved with the project, publications, and other news surrounding their efforts.
A custom locomotion driver for Steam VR applications introduces four new features for those with disabilities. The four features – virtual move, motion range boost, hand tracking, and Xbox controller move – can be adjusted to an individual user’s needs on the fly.
Virtual move allows players to use their controllers’ joystick to move, rather than having to physically move their arms.
Motion range boost changes the origin point of motion controllers to amplify movement. It translates a small movement into a large one.
Hand tracking allows the position of motion controllers to be emulated based on hand movements rather than having to use actual controllers.
Xbox controller move allows users to use a gamepad to emulate VR controller inputs.
This driver is free to download and is only available to users who use SteamVR headsets and applications. You must also have a Steam account to download the application. Link to the Steam store: https://store.steampowered.com/app/1248360/WalkinVR/
Using AI, people who are blind are able to find familiar faces in a room
Link to Article Microsoft, Developers, Software, HoloLens, AI
Project Tokyo is a Microsoft initiative that aims to help members of the blind and low-vision community with intelligent personal agent technology that leverages AI to extend their capabilities. The long-term goal of the project is to show that this XR technology can be used by anyone and even assist those with disabilities. Their focus is to create a way for those who are blind or have low vision to see the world or at least perceive it in a similar way to which sighted people do.
They provide several examples throughout the article. For example, they demonstrate the device’s AI ability to notify a user that someone is looking at them. If the wearer turns in the direction of another person, the AI is able to identify the other person’s name for the wearer. An individual working on the project states, “Whenever I am in a situation with more than two or three people, especially if I don’t know some of them, it becomes exponentially more difficult to deal with because people use more and more eye contact and body language to signal that they want to talk to such-and-such a person, that they want to speak now,” he said. “It is really very difficult as a blind person.” Social cues, whether conveyed verbally or physically, are so important for interaction. Rather than starting from scratch, the team is using a modified Microsoft HoloLens, as the HoloLens provides essential information to the AI for reading the environment.
Accessibility is a major priority for those in education fields. Approximately 15% of the world’s population has some form of disability, and one in four adults in the US have a disability that affects “major life activities.” As VR evolves, it provides a whole new range of opportunities and experiences for many people. For example, many visually impaired users can actually see better in VR due to the depth perception headsets provide. Moving forward, VR creators should consider the wide-ranging needs of users from the beginning of the development process.
Microsoft has developed several XR products with accessibility in mind:
Canetroller [Link]– The Canetroller, a Microsoft patented haptic device, works as a white cane that visually impaired people can use to experience a virtual environment.
Seeing VR [Link] – SeeingVR is a series of tools to make VR more accessible to those with low vision. The tools include a magnification lens, a bifocal lens, a brightness lens, a contrast lens, edge enhancement, peripheral remapping, text augmentation, text to speech, depth measurement, and more.
Braille Controller [Link] – The Microsoft-patented, braille-displaying controller attaches to the back of an Xbox controller, allowing for an alternative way for the visually impaired to experience games. The inspiration for this particular project was to make text-heavy video games more accessible to the visually impaired.
Hospitals are beginning to use VR to find new ways of relieving pain and offer palliative care to patients. While there is no technology currently in existence that would be able to restore someone’s sight, tools such as the IrisVision [https://irisvision.com/] can assist those living with such impairments by providing vision-aid features, a personal voice command assistant, a text-to-speech reader, and high contrast fonts. AR is also being studied to determine if such devices could be helpful with those who suffer from age-related macular degeneration.
The article also links to a variety of informational videos and links to accessibility groups and associations.
Inclusivity of VR and AR Accessibility for the Visually and Hearing Impaired
There are a plethora of companies working on creating applications for enhancing the experiences of differently abled users, and this article highlights a small sample of those projects. Microsoft has created the “canetroller,” which allows a blind or visually impaired person to access virtual reality through a controller that resembles a white cane that uses haptic and audio feedback. Nearsighted VR Augmented Aid is an Android application that uses a mobile device’s camera to display images in stereoscopic view. London’s National Theater did something similar with the help of Epsom’s latest smart glasses to display subtitles in the user’s field of vision, so even if a viewer looked away, they would still be able to see the subtitles. There are many more projects linked in the article.
