23 years ago, The Matrix introduced the idea of humanity living inside of a computer simulation. While moviegoers at the time wrote it off as a fictional piece of work, in the present day, we are much closer to that all-encompassing, technological world with the metaverse. Despite the promise of virtual worlds, there are still legal challenges and questions that echo those we face in the real world.
By way of this ontological thinking—that is, a metaverse is what we decide a metaverse is—we are introducing the concept of a “multiverse of metaverses.” Any single game or platform could be thought of as a metaverse. The move to name every virtual space a metaverse is in no small part due to companies attempting to utilize the buzzword to gain momentum for their products. While corporations have a long history of jumping on the bandwagon of the latest technical trend—think about the sheer number of apps generated in the last decade—the metaverse has the potential to shift in how we interact with technology. Take the digital economy, for example. Users can buy, sell, and create goods, and ideally, eventually be able to take virtual items from one platform to another. There are still challenges to this seamless virtual engagement: no singular company can solve the issue of interoperability, and collaboration often feels out of the question as it might be less profitable for companies to work together. Moreover, we are not at the level of computing to have “portals” from one metaverse to another. Technology development is not necessarily a linear progression like the development of the early internet. The existence of failed, and supposedly revolutionary, investments to further our “hybrid-verse,” [iii] such as the 3D TV, delivery drones, and Google glass is proof enough for companies to be wary.
In property law [iv], when referencing a physical piece of art, ownership is two-fold and can be attributed to the actual physical artwork. Depending on the terms of the sale, the buyer may/may not own the intellectual property of the artwork. With digital art, international law firm Reed Smith states that “ownership” in the metaverse—in this case, referring to the use of different platforms—“is nothing more than a form of licensing, or provision of services.” Here, true ownership lies with the owner, and the buyer cannot sell the item without permission from the true owner.
NFTs
Image via QuoteInspector.com
With the increased interest in the metaverse, we’re learning more about non-fungible tokens (NFTs) which seem to exist at the intersection of digital and physical property law arguments. An NFT can be an image, music, video, 3D objects, or other types of creative work. Because of their various forms, it’s difficult to determine if they count as regular pieces of digital art or something more. As individuals and companies continue to spend enormous sums to own “property” in the metaverse using NFTs, you begin to wonder what kinds of regulations are applicable to it. Can you apply land law? Can you have a mortgage? Can you sue others for damaging your property?
Data Protection
Internet Security Padlock for VPN & Online Privacy | Image via www.vpnsrus.com
In the same vein of the potential for personal grievances with extended reality (XR), there is also the issue of the general public’s data safety. With the continued modernization of daily life and companies’ compulsory desire to cater to the needs of their users (sometimes needs that the users themselves hadn’t even considered), new categories for personal data have come to fruition, namely facial expressions, gestures, and reactions. VR headsets collect large amounts of personal physical data from their users. With this expansion in data collection comes the fear of what could be lost during an inevitable cyberattack. Organizations and nations are not fully prepared to deal with the privacy and security issues facing the metaverse as there are not enough qualified people to deal with the complexity of the architecture to develop secure solutions. This fear also does not keep in mind user agreements where companies sell the data they obtained to third parties. In cataloging this information, users lose their right to privacy in microscopic areas of their lives.
Less Answers, More Questions
“Can someone be liable for their actions in the metaverse?” This is a critical question being asked with the development of XR. If one were to give avatars a legal persona, establishing rights and duties within a legal system, what might this mean for society? The distinction between a “legal” avatar and the true legal person who operates it becomes blurred, having a large effect on the ability to prove harm, loss, or injury suffered in the metaverse. Who owns our digital twin? Who is liable for the actions of our digital twin? We all have our place in the metaverse and these questions are just the beginning of much larger developments in the future.
[i] An online multiplayer game owned by Epic Games
Understanding how experience and exposure to trauma changes the brain could improve diagnosis and targeted care for conditions like anxiety and post-traumatic stress disorder (PTSD). Benjamin Suarez-Jimenez, Ph.D., assistant professor of Neuroscience, has been studying this topic for the past several years and was awarded a new $3.5 million grant to use virtual reality and MRI to look into the circuitry of threat, reward, and cognitive mapping in PTSD, trauma, and resilience.
For the next five years, this funding from the National Institute of Mental Health will allow the ZVR lab to build upon work that investigates brain areas that build spatial maps, specifically to discriminate between areas of an environment associated with emotions. Suarez-Jimenez’s most recent research identified changes in the salience network – a mechanism in the brain used for learning and survival – in people exposed to trauma (with and without psychopathologies, including PTSD, depression, and anxiety). His prior research has revealed people with anxiety have increased insula and dorsomedial prefrontal cortex activation – indicating their brain was associating a known safe area to danger or threat.
“This project the RO1 will support will probe whether the neural processes we have identified in the past are specific to threat or if they expand to reward processing,” Suarez-Jimenez said. “We are also looking at how attention allocation to some visual cues of the virtual reality tasks changes from pre- to post-task experience. We are hoping that understanding these brain processes can help us identify better ways to diagnose PTSD and to improve treatment.”
Suarez-Jimenez came to the University in January 2021. He is an active member of the Neuroscience Diversity Commission and has served as a mentor for the NEUROCITY program.
The University’s Goergen Institute for Data Science supports collaborative projects across all disciplines.
“I’m very excited about the wide range of collaborative projects we are able to support this year,” says Mujdat Cetin, the Robin and Tim Wentworth Director of the Goergen Institute for Data Science. “These projects tackle important and timely problems on data science methods and applications, and I am confident they will lead to significant research contributions and attract external funding.” (University of Rochester photo / Bob Marcotte)
Ten projects supported with seed funding from the Goergen Institute for Data Science this year demonstrate how machine learning, artificial intelligence (AI), and augmented and virtual reality (AR/VR) are transforming the way University of Rochester researchers—across all disciplines—address challenging problems.
“I’m very excited about the wide range of collaborative projects we are able to support this year,” says Mujdat Cetin, the Robin and Tim Wentworth Director of the institute. “These projects tackle important and timely problems on data science methods and applications, and I am confident they will lead to significant research contributions and attract external funding.”
The awards, approximately $20,000 each, help researchers generate sufficient proof-of-concept findings to then attract major external funding.
This year’s projects involve collaborations among engineers, computer scientists, a historian, a biostatistician, and experts in brain and cognitive sciences, earth and environmental science, and palliative care. Their projects include a totally new kind of computing platform, new virtual reality technologies to improve doctor-patient conversations and help people overcome color vision deficiency, and machine learning techniques to make it easier for people to add music to their videos and to enhance AR/VR immersive experiences based on the unique geometry of each user’s anatomy.
The 2022–23 funded projects and their principal investigators are:
Ising Boltzmann Substrate for Energy-Based Models Co-PIs: Michael Huang, professor of electrical and computer engineering and of computer science, and Gonzalo Mateos, associate professor of electrical and computer engineering and of computer science and the Asaro Biggar Family Fellow in Data Science
A Data-Driven, Virtual Reality-based Approach to Enhance Deficient Color Vision Co-PIs: Yuhao Zhu, assistant professor of computer science, and Gaurav Sharma, professor of electrical and computer engineering, of computer science, and of biostatistics and computational biology
Audiovisual Integration in Virtual Reality Renderings of Real Physical Spaces Co-PIs: Duje Tadin, professor and chair of brain and cognitive sciences and professor of ophthalmology and of neuroscience; Ming-Lun Lee, associate professor of electrical and computer engineering; and Michael Jarvis, associate professor of history
Personalized Immersive Spatial Audio with Physics Informed Neural Field Co-PIs: Zhiyao Duan, associate professor of electrical and computer engineering and of computer science, and Mark Bocko, Distinguished Professor of Electrical and Computer Engineering and professor of physics and astronomy
Computational Earth Imaging with Machine Learning Co-PIs: Tolulope Olugboji, assistant professor of earth and environmental sciences, and Mujdat Cetin, professor of electrical and computer engineering and of computer science, and the Robin and Tim Wentworth Director of the Goergen Institute for Data Science
Improving Deconvolution Estimates through Bayesian Shrinkage PI: Matthew McCall, associate professor of biostatistics
Building a Multi-Step Commonsense Reasoning System for Story Understanding Co-PIs: Zhen Bai, assistant professor of computer science, and Lenhart Schubert, professor of computer science
Versatile and Customizable Virtual Patients to Improve Doctor-Patient Communication Co-PIs: Ehsan Hoque, associate professor of computer science, and Ronald Epstein, professor of family medicine and palliative care
Machine Learning Assisted Femtosecond Laser Fabrication of Efficient Solar Absorbers Co-PIs: Chunlei Guo, professor of optics, and Jiebo Luo, Albert Arendt Hopeman Professor of Engineering Rhythm-Aware and Emotion-Aware Video Background Music Generation PI: Jiebo Luo, Albert Arendt Hopeman Professor of Engineering
There is a strong emphasis on fostering cross-disciplinary collaboration in extended reality (XR) at Studio X. Over 50 researchers across the UR use XR technology for their research and teaching, and many come to Studio X for consultation and advice in either program development or engineering. As an XR Specialist at Studio X, I got the opportunity to work on two XR-related research projects during the past summer, one in collaboration with the Brain and Cognitive Science Department (BCS), and the other with the Computer Science Department (CS). Through the Office of Undergraduate Research, these projects were supported by a Discover Grant, which support immersive, full-time summer research experiences for undergraduate students at the UR.
The research with BCS includes digitizing the Kodak Hall at the Eastman School of Music and bringing it into VR. The result will be used to provide a more realistic environment for conducting user testing to better study how humans combine and process light and sound. The visit to Kodak Hall was scheduled way back in March. Many preparations had been done before the visit that included figuring out the power supply and cable management, stage arrangement, clearance, etc. One discussion was had on what techniques will be used to scan and capture the hall. Three object scanning techniques were tested before and during the visit: photogrammetry, 360-image, and time-of-flight (ToF).
Photogrammetry creates 3D models of physical objects by processing photographic images or video recordings. By taking images of an object from all different angles and processing them with software like Agisoft Metashape, it is possible for the algorithm to locate and map key points from multiple images and combine them into a 3D model. I first learned about this technique by attending a photogrammetry workshop at Studio X led by Professor Michael Jarvis. This technique has been very helpful for the research since we are able to get great details on the mural in Kodak Hall, at which other techniques had failed.
Photogrammetry model of the mural in Kodak Hall
360-image, as its name suggests, is a 360-degree panoramic image taken from a fixed location. With the Insta360 camera borrowed from Studio X, the capturing session requires almost no setup whatsoever and can be quickly previewed using the app on a phone or smart device.
360 image of Kodak Hall, captured from the stage
The Time-of-Flight (ToF) technique shoots light and calculates the time it takes for the light wave to travel back from the reflection in order to get the depth information. Hardware using the ToF technique can be easily found on modern devices, such as iPhone and iPad with Face ID. I tested the ToF scanner on the iPad Pro at Studio X. It provides a great sense of spatial orientation and has a fairly short processing time.
We used the Faro Laser Scanner in order to get a scan with higher accuracy and resolution. Each scan took 20 minutes, and we conducted 8 scans to cover the entire hall. The result is a 20+ GB model with billions of points. In order to load the scene to the Meta Quest 2 VR headset, we shrunk down the size and resolution of the model dramatically using tools such as gradual selection, adjusting the Poisson distribution, material paint, etc. We also deleted excessive points and replaced flat surfaces with better quality images such as the stage and mural. The end result is a nice-looking model with decent details around 250MB, good for the headset to run.
The model was handed over to Shui’er Han from BCS as a Unity package, where she is going to implement the audio recording and spatial visualization before conducting the user testing. It is amazing to see many people working and bringing together their experience and knowledge in making this cross-disciplinary project to reality. I would like to thank Dr. Duje Tadin, Shui’er Han, Professor Michael Jarvis, Dr. Emily Sherwood, Blair Tinker, Lisa Wright, Meaghan Moody, and many more who gave me the amazing opportunity to work on this fun research and all the help they provided along the way. I can’t wait to see what they can achieve beyond this model and research project.
You can read more about this cross-disciplinary collaboration here.
This summer, I worked full time at Studio X. Even though the campus felt pretty empty with almost all the other undergrads home for summer, there was a lot going on in Studio X! For example, for two weeks in July, we held a pre-college program called “XR: Content Creation and World Building.” In this program, high schoolers came all across the country to learn about the world of extended reality or XR.
“Learn how XR (the umbrella term for augmented and virtual reality) experiences are created! Students will study the history of immersive technologies and gain technical skills by exploring both the basics of 3D graphics for asset creation and how to develop XR environments with Unity, a popular game engine. We will also discuss the applications and impact of XR across humanities, social science, and STEM fields. All learning levels welcome.”
It was really exciting to be a part of this program to teach passionate students about XR creation. As we prepared for the students’ arrival, we asked ourselves, “How can we introduce a dozen high school students to the complex and technically challenging world of XR development, all within two weeks of half-day sessions?” This was a challenge indeed. We knew that we wanted the students to walk away with a basic understanding of the fundamentals of Blender, a 3D modeling and content creation tool, and Unity, a game engine commonly used for VR development, but we did not want to overwhelm them with too much new material all at once. We decided that we would have to create a highly detailed plan, carefully crafting how we would use the two weeks that we have with the students.
Over the course of June and early July, we worked to create this plan, taking every little detail into consideration. The first major obstacle we faced was how we were going to ensure that each student would have the necessary hardware and software in order to complete the activities we were planning. Blender and Unity can both be very taxing on computers, and it is often the case that folks don’t have the necessary hardware, even for our undergraduates. It was very important that this program was open to anyone who was interested and that technical experience or personal hardware was not a limitation. We decided that instead of having each student bring in their own computer, we would use the high-powered workstations that we already have in Studio X. This, however, created the question of how to organize a dozen PCs in our space that each use a very large amount of power. With 12 high-powered PC’s running all at the same time in the same place, we actually ended up blowing a circuit and had to re-think our plans. We considered several options, including using another space or splitting up the group into different rooms, but we eventually decided to completely reorganize Studio X in order to keep the group together in one space. I really liked the way we eventually configured the space, as it allowed us to keep the whole group together and helped us build a stronger community as we worked.
Studio X configured to have all 12 PCs in the same place
After solving our issue of how to organize the computers, we could focus our energy entirely on planning out how to best use the two weeks with the students. The first week was focused on learning Blender. We wanted to give an introduction to 3D concepts, Blender basics, and character modeling. We felt that this would give our students a foundational understanding of how to navigate Blender, while still being realistic with the time that we have. Blender can be a very challenging program to learn. There are many different things that you can do using the software, and oftentimes it can be very overwhelming the first time that you try it out. Although we felt like we were trying to introduce a lot in a short amount of time, we were very excited to see what the students could make. At the end of this week, each student had their very own 3D modeled character. The students did an amazing job creating their characters in Blender. It was so impressive how fast they were able to learn, and it felt so good to see our planning pay off.
An example of Blender’s UIThe final renders of each student’s Antagonist
The second week of our program was focused on learning Unity. We wanted to teach the basics of Unity, get the students thinking about core game design principles, and introduce the world of VR development. The end goal for this week would be that each student would create their very own VR mini game, using the 3D character they modeled as the antagonist in their experience.
With so little time, it was really important that we had milestones to reach each day to make sure we stayed on track. On the first day working on their games, the students got an introduction to a template VR Unity project. I created this template using a beginner VR asset from the Unity Asset Store, a place where you can find free or paid packages to help you create games. The asset I used is linked here: VR Escape Room. This package handled a lot of the initial setup for a VR project that can be very complex, allowing the students to focus on their game concepts without being tied down or having to use too much coding. I also created a full VR mini game myself, giving the students an example of what their final project would look like. My game was called Jellyfishin, a game where the player has to go around catching Jellyfish. This game highlighted some of the main mechanics of the template and also was fun for the students to play around with.
Screenshot from the template project provided to the students
After being introduced to the template project, day 2 was all about environmental design. The students learned how to find resources to create their game world using a combination of free models, primitive objects, and their 3D characters that they made the week prior. By the end of day 2, the games really came together. I was really amazed at how much detail and care that each student put into their project, especially considering how little time that they had. The final development day was used to polish and finalize the games. We made sure that each student’s game could be playable start to finish and that there were no major problems with the experience. I think each project was really unique despite coming from the same template. It was so rewarding to see the tools we had created be used so well to create these awesome experiences.
On our final day with the students, it was time for the showcase. Staff members from all over the library came to Studio X, and each student had the opportunity to present their game. One-by-one they gave a quick introduction to their concept and then showed off some gameplay. In the world of game development, you never know if something is going to go wrong. One minor bug could throw off an entire demonstration. Thankfully, these students did an amazing job finalizing their games, and everything went off without a hitch. After two challenging weeks, our students left with a complete VR game, a 3D modeled character, and a set of skills they can continue to grow and use on their journey with XR.
XR Content Creation & World Building – Final Showcase
Being a part of this pre-college program throughout the summer has been an amazing learning experience for me. Through all of the preparation and thinking that went into making our goals possible, I really had to put my technical skills to the test. In the end, our planning really made all the difference and is what made the program run so smoothly. It was a great challenge to think about how we can teach so much information to the students in such a short amount of time, and I’m really proud of what we all accomplished. I can’t wait to see how this program continues to evolve and find more ways to lower the entrance barrier to the world of XR. Overall, it was a pretty great summer in Studio X.
A Wilmot Cancer Institute scientist published data that show a new microchip-like device that his lab developed can reliably model changes in the bone marrow as leukemia takes root and spreads.
Ben Frisch, PhD, holds the bone-marrow-on-chip device in his lab.
Ben Frisch, Ph.D., assistant professor of Pathology and Laboratory Medicine and Biomedical Engineering at the University of Rochester, and colleagues have been building what is known as a modular bone-marrow-on-chip to enhance the investigation of leukemia stem cells. The tiny device recapitulates the entire human bone marrow microenvironment and its complex network of cellular and molecular components involved in blood cancers.
Similar tissue-chip systems have been developed by others, but they lack two key features contained in Frisch’s product: osteoblast cells, which are crucial to fuel leukemia, and a readily available platform.
The fact that Frisch’s 3D model has been published in Frontiers in Bioengineering and Biotechnology and is not a one-off fabrication will allow others in the field to adopt a similar approach using the available microfluidics system, he said.
Rochester researchers will harness the immersive power of virtual reality to study how the brain processes light and sound.
A cross-disciplinary team of researchers from the University of Rochester is collaborating on a project to use virtual reality (VR) to study how humans combine and process light and sound. The first project will be a study of multisensory integration in autism, motivated by prior work showing that children with autism have atypical multisensory processing.
The project was initially conceived by Shui’er Han, a postdoctoral research associate, and Victoire Alleluia Shenge ’19, ’20 (T5), a lab manager, in the lab of Duje Tadin, a professor of brain and cognitive sciences.
“Most people in my world—including most of my work—conduct experiments using artificial types of stimuli, far from the natural world,” Tadin says. “Our goal is to do multisensory research not using beeps and flashes, but real sounds and virtual reality objects presented in realistically looking VR rooms.”
Members of the team begin the setup for audio and visual data collection. From left to right are Shui’er Han, a postdoctoral research fellow in Duje Tadin’s lab; brain and cognitive sciences major Betty Wu ’23; computer science and business major and e5 student Haochen Zeng ’23, who works in River Campus Libraries’s Studio X; and Victoire Alleluia Shenge ’19, ’20 (Take Five), who earned her degree in brain and cognitive sciences and is a manager in Tadin’s lab.
A cognitive scientist, a historian, and an electrical engineer walk into a room . . .
Tadin’s partners in the study include Emily Knight, an incoming associate professor of pediatrics, who is an expert on brain development and multisensory processing in autism. But in creating the virtual reality environment the study participants will use—a virtual version of Kodak Hall at Eastman Theatre in downtown Rochester—Tadin formed collaborations well outside his discipline.
Faculty members working on this initial step in the research project include Ming-Lun Lee, an associate professor of electrical and computer engineering, and Michael Jarvis, an associate professor of history. Several graduate and undergraduate students are also participating.
Many of the tools they’ll use come from River Campus Libraries—in particular, Studio X, the University’s hub for extended reality projects, as well as the Digital Scholarship department. Emily Sherwood, director of Studio X and Digital Scholarship, is leading the effort to actually construct the virtual replica of Kodak Hall.
The group recently gathered in the storied performance space to collect the audio and visual data that Studio X will rely on. University photographer J. Adam Fenster followed along to document the group’s work.
Senior Creative Writing major and Karp Library Fellow Ayiana Crabtree '22 was featured in this post for the UR admissions blog! Link to original post at the end.
Located on the first floor of Carlson Library, as the hub for extended reality at the University of Rochester, Studio X fosters a community of cross-disciplinary collaboration, exploration, and peer-to-peer learning that lowers barriers to entry, inspires experimentation, and drives innovative research and teaching in immersive technologies.
Studio X runs tons of fun workshops and events that aim to make XR fun and easier to understand. For example, I run an Intro to XR workshop every semester that teaches participants, no matter their skill level, all about the basics of XR with a fun hands-on learning experience. There are other workshops too, like Blender and Unity tutorials to teach you the basics of 3D modeling and game development. If workshops aren’t your thing, we also have events like our Beat Saber competition and a speaker series called Voices of XR, where you can learn about XR directly from professionals in the field.
Studio X has a wide range of XR technologies that students, faculty, and staff have access to using both inside and out of the space. Our most popular attractions are the Meta Quest 2 VR headsets, which can be borrowed and taken back to your dorm for up to three days at a time. On our VR headsets, there are a bunch of fun pre-downloaded games and experiences for you to play, like Beat Saber, Walkabout Minigolf, Job Simulator, and more! In addition to the VR headsets, we have 360 cameras and 360 audio recorders which can also be taken back to your dorm for a three-day period. If you don’t mind staying in the space, you can ask to try one of our Microsoft HoloLens 2’s (MR headsets) or use one of our high-end workstations for homework. You can also use any of the aforementioned technology in the space if you don’t want to take it back to your room.
Studio X’s main goal is to break down any barriers that may be preventing students from getting into XR technologies. Whether that be making resources readily available, or giving introductory tutorials, Studio X is here to help!
I first encountered the idea of virtual reality (VR) when I read the book Ready Player One by Ernest Cline. As an avid reader of science fiction books, I loved the idea of being able to escape to some virtual world through a VR headset. Soon after I read the book, the movie was released and seeing the concept executed in a visual form only increased my interest in the subject. Despite my fascination, I took it as the book genre labeled it. Fiction. I believed that there were no VR headsets, as I had never seen or heard of anyone having them.
In the fall of 2020, I happened across an advertisement for the Oculus Quest 1. My interest in the novel had not wavered, but nevertheless, I was shocked. I hadn’t realized that the concept introduced to me through a science fiction novel was real in the form of a readily available, and relatively affordable technology. I had been saving money for a while and prepared to make my purchase. Luckily, a friend encouraged me to wait a few months, as in October 2020, the Oculus Quest 2 was released. I ordered the headset and eagerly awaited its arrival.
When I finally got my hands on it, I was over the moon. It may not have looked like the vision Cline painted in his novel, nor like the version in the movie, but it was virtual reality nonetheless. In the time between ordering and receiving the headset, I researched various games and experiences that I wanted to try upon its arrival. Beat Saber, a rhythm game, was top of my list and was my first purchase on the device. I’d never been one to read instructions for consoles, games, or anything at all, so I dove right in and set up my account.
As soon as I began playing, I was hooked. Whether it was the idea of actually experiencing VR or the catchy songs of Beat Saber, I absolutely fell in love with my Quest 2. I played it every moment I had time. As I danced around my living room slashing to the beat of the songs, my parents asked me what I was doing.
I excitedly explained to them what VR was, and how it worked. It was at this point I had my first experience sharing VR with someone else. After a long tutorial on how to wear the headset, how to navigate the menu, and how to play the game, my parents tried out VR for the first time.
This was all back over the winter break of 2020-2021, just before I interviewed to join the Studio X team as a Karp Library Fellow to do XR research. This was during the time that the pandemic was still pretty bad, and VR provided an escape from the harsh reality around me. It helped my anxiety and allowed me to relax, even if just a little bit while I was immersed in the world of VR.
Ever since that initial experience, I made an effort to introduce as many of my family members as I could to virtual reality.
Family Experiences
Mother
It was the various times of having my mum try VR that really inspired me to explore the topic of user interaction with VR further. Her reactions to the various experiences I had her try really made me understand the impact that VR can have on people’s lives. Her first experience was with Beat Saber, which she thoroughly enjoyed due to its catchy songs, but it was Job Simulator that really captured her attention. “I thought it was going to be dumb,” she said. “When I saw you doing it, it looked silly, but when I tried it, it blew my mind. It was a strange experience because it really made me feel like I was in the room.” For me, it was especially funny watching her play Job Simulator. I had to make sure she wouldn’t forget about the guardian boundary, as she kept trying to walk down the virtual hallway when, in reality, she was about to crash into the coffee table. Another interesting thing was how she was worried about dropping the virtual coffee cup, because she didn’t want it to break or make a mess on the floor.
Father
While he didn’t try Job Simulator, my dad tried Walkabout Mini Golf. He’s not much of a mini-golf fan, but he was blown away by how realistic the physics were in the game. He said he kept feeling like he was going to fall off the edge of the map and even tried walking from hole to hole (which would have required a lot more space than we have in our living room). “You really don’t know what it’s like until you try it, and when you do, you can see all kinds of applications this technology may have in the future.”
Grandparents
I, of course, wasn’t going to have them do Beat Saber, and I didn’t have Walkabout Mini Golf at the time, so I had them watch a few Oculus TV videos.
Having my 96-year-old Great Grandma try VR was quite an interesting experience. She was in awe at the capabilities of the technology and loved the fly-over nature documentary about the ice caps.
My Gran tried a few different parachuting and paragliding videos. “It was amazing to feel like I was there. I feel like I could do paragliding now!”
My Grandpa watched a few shorter space documentaries and was thrilled to be immersed in the galactic environment.
Photo of my 96-year old Great Grandma trying VR for the first time
Running a Survey
After seeing the unique reactions from all my relatives, I was curious to know how others felt about their experiences with VR. I had joined several VR-focused Facebook groups to see the kinds of conversations people were having about VR and then decided to run a survey to directly ask the community about their experiences.
With the survey titled “How do Users Experience VR,” I asked a range of questions about age, their perception of VR, what they used it for, and if they had any stories they wanted to share. After about a week and a half of running the survey, I had 282 responses to go through.
One of the things that interested me the most was the age range distribution of those who responded. I’m not sure if this is directly related to the survey being run on Facebook and the demographic of Facebook users, but it doesn’t feel like a misrepresentation as the group was specifically for Oculus Quest users, and Oculus, now the Meta Quest, is owned by the same company.
Quotes
The following information is taken directly from the survey results. Some of the quotes have been reformatted a bit for coherency.
This first quote is from a friend who had some previous experience with VR. I had them try VR on my headset before asking them to fill out the survey. Their unique perspective on the potential threat VR poses to society is one that I haven’t seen discussed much elsewhere, which is why I believe it is important to include it here.
Jenna, 21, Non-Binary
“At first, I thought it was super cool, but a little bit scary. As I’ve had more experience with VR games, I still think it is an awesome technology with a lot of potential uses, but I fear that VR video game violence will further desensitize users to violence in the real world. I was at an arcade once and played a VR zombie game and had to ask the worker to stop the game because it felt too much like I was killing real living things. Hence my fear of it desensitizing people to violence.”
I included the next quote because it shows VR being used as a tool for long-distance interaction and also how people in your environment perceive you as you play VR.
Tracie, 49, F
“I saw it as an opportunity to stay connected and play with my friend who lives thousands of miles away from me. I used to live in my RV where there was very little space to play, so I would take my headset to the laundromat and play while doing laundry. Can't tell you how many times people were freaked out by what I was doing. I always tap the headset to bring up passthrough when someone came in and when I started interacting with one guy he was totally weirded out ... ‘you can see me!!!’ ... lol. Yes. Yes, I can. (I was in a closed RV community laundry place with the offices and rec center in the same building -- it was completely safe).”
These next two quotes are particularly interesting as they show the potential uses for the elderly and the health benefits of using VR.
Bonnie, 79+, F
“I became so enthusiastic, it was fun, and I moved my body. I bought a headset and began to realize all the possibilities. I finally got my Quest2 in September and found Fitxr and SuperNatural. I have continued to use my quest 2 every day and have barely explored all the apps. My enthusiasm prompted five other sales among my friends as they noted my weight loss and toning of my body. I never thought that an old person could gain strength and balance. Just thought we went backwards physically. I had given away my cross-country skis and now wish I had them back as I have gained strength in my entire body. My balance has improved so much and although I have “bat wings” on one side of my arms I actually have muscle “bumps” on the other side. I can do step ups - more and more each week or so. I can do squats, as many as 40 at a time. Every day my muscles ache, but I LOVE IT as I realize it is a good ache and I earned it. The technology allows me to socialize with others, visit sites I had traveled to previously and brings back happy memories. The technology allows me not to travel to a gym (not that I would have) and to have privacy.”
M, 75, F
“I am seeing more uses for homebound, elderly... seeing it as a way of connecting friends and family scattered around the country. wonderful experience taking my 85-year-old brother to the top of mount everest! Getting to play golf with a group of women every week. exploring worlds in altspace”
The following quote shows the emotional impact VR can have on people through the experiences it allows one to have.
Sherry, 57, F
“I bought an Oculus for my 11 year old granddaughter for Christmas. She brought me to the kitchen and told me to stand in a spot. She put it on me and told me to close my eyes. When it was on, she told me to open them. I was in a beautiful mountain lodge. Out the window were mountains. I was overwhelmed and began to cry. It was as if I had been transported to my home in the mountains 25 years ago. I could not believe my eyes. Literally! I just kept saying … is this real? I knew immediately that I must get my own Oculus and pretty much immediately ordered one for myself based on that 5 minutes of standing in a room looking at the mountains.”
These next quotes show an optimistic perspective for the future of VR technology.
Anonymous, 59, F
“At first, it was just a music game that I played. I've since added more experiences with various game types, the Multiverse, and more. Rather than this just being a gaming system, I can see a future for business, education, research, social interacting (that can actual involve talking to one another vs just texting), shopping, and so much more!”
Susan, 61, F
“I came to see it could be used for exercise and education and other non-gaming applications. I have come to see that it is a powerful educational tool, particularly for people who are limited, either physically or not able to travel to other parts of the world. Also, I believe it could be used to deepen educational experiences in a variety of ways. I also continue to believe it is probably pretty addictive and should not be used many hours of the day as it is basically an escape and not particularly productive in general. I think it’s a great tool for people who are disabled or otherwise housebound. I have a concern that entering a VR world takes away from the time that people spend outdoors, which in the end is far more important.”
Skye, 45, F
“It was more real than I thought it would be. And I immediately saw the potential applications to things that I cared about - like art, exercise, and experience with others. When my brother bought everyone in the family an Oculus for Christmas, that was a game-changer. I was SHOCKED with how much further the technology had come and am a total convert and trying to get others to get a headset so we can hang out in virtual worlds and other experiences. I now see VR as being something that is relevant for my life now and into the future. I see how it can improve my interactions with family and friends (we spend more time together...especially since Covid and distance limits our in-person opportunities), and it has given me new ideas for how to approach and use it for engagement for my wellness company and clients.”
The next several quotes show the potential therapeutic and mental benefits of using VR.
Audrey, 38, F
“I have ADHD. I was diagnosed at 37 years old, and I have found that the exercise component for VR allows me to keep engaged in a way no other exercises have previously. I still do other types of exercise (such as strength training or hiking) but when I’m not in the mood to workout, the menu of options in VR still brings excitement for me.”
Anonymous, 42, F
“I live in the Midwest, and it is dark by 4 o'clock in winter. In vr, I can hop into real fishing vr and spend time on the lake in sunshine. It doesn't matter that it's not real, your body still relaxes, endorphins are released. It has helped a lot with seasonal affective disorder this year.”
Anonymous, 41, F
“My mom passed away June 2020, she had prefrontal dementia, she slowly lost all her motor skills and eventually mobility. One of the last happy memories that I have with her was me putting the oculus quest on her face and guiding her through a tour of the African Sahara. She actually reacted and reached out to try to touch lions and I swear I saw her smile when she saw elephants. Looking forward to seeing what VR therapy for people with dementia, Alzheimer's and other debilitation can bring in the future.”
Susie, 46, F
“I bought a VR to study the exercise game Supernatural and its effect on learning and motivation for Neurodiverse individuals. (Specifically adhd) I realized pretty quickly that this is the platform of the future. Way beyond games. I see it used for mental health/therapy, exercise, social connection, work interaction, performance/skills enhancing (like public speaking) etc. I’m literally applying for a PhD program so I can study VR some more. It’s changed my life! Supernatural daily has decreased my adhd symptoms tremendously. I feel my brain starting to work better. I can see this tool being an alternative to meds for those who can’t take them.”
This next shows how giving VR a second chance can completely change your perspective on the technology.
Anonymous, 24, F
“My first experience was poor. I tried it at the mall when it was fairly new, and it was a video simulation of an amusement park ride. Sitting down, I got a very intense feeling of motion sickness and did not enjoy the video at all. It was a very bland video simulation. Although my 1st experience was bad, I gave it another try at a friend's house. This was a totally different experience compared to my first. I played beat Saber and it was an overwhelming, awe-inspiring time. From that point forward, I began thinking of VR as the future and one of the most advanced types of technology to exist yet. Almost all experience I have had after that has been incredibly immersive and entertaining. I look at VR as an opportunity to take a break from our physical world and enter another world.”
The last quote I leave you with is a pretty cool perspective on how the technology has changed over time, and how it has impacted this person’s life and social interactions.
Gnossos, 65, F
“Early 90s I was hired by a Space Museum to consult on a VR exhibit and traveled to Boston, Chicago and LA to test drive early concepts. First experiences were so bad that I told the Space Museum to hold off on purchasing VR until it was more developed. Oculus Quest's first experience did not disappoint. My perceptions shift with the technology development, of course. I still see it in its infancy - it’s the Pong Era of VR meaning it sucks but we don’t realize it yet. It’s going to be 100 times better in 10 years. I was surprised by having a crush on a guy in Rec Room who played Paint Ball like he was a trained assassin. Crushes are a distant experience for me, so having one with only a voice and a cartoon avatar really surprised me. I think the safety of my anonymous state helped create an openness to flirting that’s not my normal way. It inspired me to wonder more about the potential for intimacy in VR - especially if these spaces were developed by women.”
Reflection
VR is rapidly growing to be one of the most popular forms of XR. It is estimated that in 2020 nearly one in 5 people in the US, or 19% of consumers, used VR. Due to this increasing demand, it is expected that nearly 15 million AR and VR devices will be shipped to customers worldwide in 2022. Source
The quotes provided above shine a small spotlight on the many ways that people are being impacted by VR every day. From new ways of socializing to new methods of staying physically and mentally fit, VR has the ability to benefit everyone in some way shape or form due to its versatility. It is this social and emotional impact that allows VR to become so popular, as people feel directly connected to the experiences they are trying while in VR. The ever-present description of VR being the ultimate empathy machine is growing more and more accurate as the technology progresses and the range of possibilities expands.
Education from the sciences to the humanities, job training, interpersonal relationships, concerts, work meetings, all these fields can and are already benefiting from VR technologies. More and more people are being exposed to VR every day, and soon enough, it will become a household staple, much like cellphones and TVs. And why? Because of the ways we as users experience VR. It is the consumer perspective that shapes the industry, which is why it is so important to understand why people react the way they do to these technologies.
I personally believe that VR has shaped my perspectives on the world in ways I wouldn’t have been able to imagine due to some of the experiences I have tried. VR has opened my mind to new perspectives on personal space, human interaction, disabilities, and even the way I view myself as a person existing in the real world versus in the digital one.
VR has a unique ability to change perspectives and influence emotions, and it is up to the people using it to decide what path VR ultimately goes down.
Mary Ann Mavrinac, vice provost and dean of the University of Rochester Libraries, shares insight into how the campus community directed the development of Studio X, the library’s new extended reality hub featuring advanced technology and expert training
“I don’t believe in ‘if you build it, they will come.’ You can build something, but they won’t come if you don’t know what your users want,” Mavrinac said. It’s the guiding principle she and her team followed throughout the ideation and planning of the library’s new high-tech hub, Studio X. Located on the first floor of the Carlson Science and Engineering Library, the 3,000 SF space allows students and faculty to participate in immersive learning experiences.
Equipped with technology that supports virtual reality (VR), augmented reality (AR) and everything between (extended reality or XR), Studio X allows researchers to perform tasks such as visualizing large data sets and safely experimenting with hazardous materials by creating a virtual environment. Studio X broadens the range of possibilities for discovery and instruction, but what makes it truly special is its source of inspiration. CannonDesign collaborated with the university to design a facility that the campus community not only requested but also intimately shaped. From inception to completion, student and faculty preferences were integrated with expert knowledge to deliver a space tailored to serve the entire campus community.
We spoke with Dean Mavrinac to learn more about the process and impact of the project. She wanted to underscore that the success, to date, of Studio X is a team effort, much of it led by Digital Scholarship and Studio X director, Emily Sherwood.
There aren’t many academic libraries that offer a space like Studio X. What is it, and how did the project begin?
The project began in fall 2017 when Lauren Di Monte joined our team and learned from the faculty that there was a lot of research activity in extended reality and other immersive technologies. We thought it was something the library could get involved in since we had close to 50 researchers engaged in XR technologies. So, we set out to better understand that landscape and how the researchers would engage with any initiative we developed, whether it was a space or specialized expertise. We knew a generic cave wouldn’t work for them, so we thought about what we may be able to do to help them tackle specific research questions. As it turned out, we pivoted to a space and service that would provide an easy on-ramp to those less familiar with these technologies and related needs.
Today, Studio X is a collaborative hub for extended reality where students and faculty are immersed in learning and teaching in ways that just aren’t possible without advanced technology. It’s a high-tech space that allows exploration, experimentation and experience that truly brings education to life.
What was the goal of Studio X? Who is it for?
The overall goal was to offer physical space, a program, services, technology and expertise that students and faculty needed—and expertise was really big. The user research told us that they wanted a space and experts in the space to teach them how to use and apply the technologies. We approached this goal by providing an on-ramp that made it easy for people to gain access to and experience with XR technologies.
Whether a person is an advanced researcher or a novice user, we’re good at helping people feel comfortable to explore their questions. The library is an interdisciplinary crossroads at the university, so it could be someone studying history, biomedical engineering, neuroscience, religion, ethics—whatever it is—if they’re interested in using XR technologies, we provide the support they need to feel welcome.
