University of Rochester researchers combine freeform optics and a metasurface to avoid ‘bug eyes’
“Image” is everything in the $20 billion market for AR/VR glasses. Consumers are looking for glasses that are compact and easy to wear, delivering high-quality imagery with socially acceptable optics that don’t look like “bug eyes.”
The metasurface is a veritable forest of tiny, silver, nanoscale structures on a thin metallic film that conforms, in this advance, to the freeform shape of the optics—realizing a new optical component the researchers call a metaform.
By Ayiana Crabtree Karp Library Fellow, XR Research February 2021 – present Area of Focus: Conducting research to find ways humanities disciplines can benefit from XR technologies and developing future programming for Studio X that will engage humanities faculty and students
Although many humanists have embraced technologies in their research, immersive technologies are largely new territory for humanities and humanistic social sciences faculty and students. The technology is expensive, has a steep learning curve, and often comes off as a novelty rather than a viable research tool. However, as we progress into the future, it is important to recognize the benefits that immersive technologies could offer these disciplines.
Extended Reality, also known as XR, is the term that encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies.
With a VR headset, a user is immersed in a computer-simulated environment in which they can experience realistic sounds, images, and 3D content.
AR is an overlay of computer-generated images on the real world. AR uses our existing reality as the basis of the experience, and with the help of a device, this creates an interactive experience for users.
MR, sometimes referred to as a hybrid reality, is when there is a merging of the real and virtual worlds. The important distinction between AR and MR is that with MR the virtual and real worlds are able to interact in real time.
These technologies have the potential to create new experiences for the humanities and humanistic social sciences teaching, research, and learning communities. From getting an immersive language learning experience through VR tours of countries, to creating new interactive works of art, to exploring the worlds found in the pages of classic novels, the possibilities are endless!
Below are just a few examples of the many ways XR is used across these disciplines:
Allerin’s article shows how photogrammetry—the process of digitally overlapping photographs to turn them into 3D models—can be used to enhance research in the archaeological field by applying it to VR technology. From allowing recreations of archaeological sites and providing access to constricted areas to helping increase skillsets to educating archaeologists, we’re already seeing the benefits of VR to the archaeological field. These applications can allow archeologist to analyze and research more conveniently while giving greater detail than simple photographs of a site. Technology like this could be extremely beneficial and could encourage new people to join the field because of the accessibility it provides.
VIAR 360’s article provides a new take on ways art can be created and performed by using the VR space as a new medium for work. It answers important questions about how VR experiences are produced, why they could be beneficial to the field, and provides examples of ways people can use VR applications in practice. The question of whether or not VR could be the future of art is raised. Not only is it the future, but it is already here.
Using VR to create art gives artists more freedom in their designs while having ease of access. By slipping into the VR world, anything can be created in a three-dimensional space, allowing artists a new medium to think and create in. This could allow sculptors to visualize projects and can give painters the opportunity to create 3D paintings. These pre-project visualizations can help by allowing artists to test out new ideas before attempting them with materials. It also provides opportunities for them to expand and try new ways of conveying messages in their artwork. One popular application that would allow them to do this is Tilt Brush, a popular, open-source 3D painting platform. The possibilities are endless when it comes to all the different opportunities for new works that could be created in the VR medium.
Cultural Heritage, Archaeology, Anthropology, History, Teaching & Learning
YaleNews highlights the importance of VR in modeling cultural heritage sites for the purposes of student learning. Not only does this allow students to experience the site without needing to travel across the world, but it also allows them to visit a place that might not exist anymore. The students visited the ancient city of Nimrud, 30 kilometers south of Mosul in Iraq. Through immersive technologies, people are able to preserve heritage sites for future generations to experience, something that is important in the preservation of culture, especially in learning and understanding it. Visiting the sight in VR allows one to get a sense of the grandness of the space you are visiting, something that reading a book can’t do for you. This enhances the experience by allowing students to grasp how important these structures were, which could be relevant in their research.
While there was some initial skepticism about using VR, the benefits ended up outweighing the negatives. There is no comparison between visiting a site in person versus visiting it in VR, but for the purposes of the classroom, VR provides real opportunities for access in allowing instructors to showcase important sites to their students. Another benefit and future application could be to recreate places that have been destroyed, granting access to sites that no one has seen for decades or centuries.
From “Ready Player One” and the “Martian” to “The Lion King” and “Mission Impossible,” VR Sync’s article takes a close look into how VR can be used in the movie making process. Virtual cameras, application in animation, marketing and promotion, 360 movies, and VR movies are all different ways that the film industry uses VR to aid in their processes. Through the current use of VR in such a wide range of categories, it is evident that this technology is key to the future of the film industry. Seeing the technology develop from something in movies to something used to make movies is a fascinating growth and sets the stage for all kinds of use in the future.
History & Teaching & Learning
Yildirim, Gurkan, et al. “Analysis of Use of Virtual Reality Technologies in History Education: A Case Study.” Asian Journal of Education and Training, vol. 4, no. 2, 15 Feb. 2018, pp. 62-69, doi:10.20448/journal.522.2018.42.62.69. https://files.eric.ed.gov/fulltext/EJ1170733.pdf.
This case study looks at how VR can be used in the classroom to teach history classes. They found that students enjoyed the feeling of being present in the environment and being able to access destinations that are difficult to visit in real life. Going more in depth, the responses from each student seemed to be quite similar: they likened it to living in the places and enjoyed the more interactive aspects. VR experiences allowed students to focus on a particular topic and to not be easily distracted. One participant noted a downside: the long-term use of the headset and virtual experience caused eye strain.
As a whole, it is important to realize that while this may be the direction technology is heading, there is still some value to traditional teaching methods. The virtual reality classroom might not be good for an entire lecture, as it is important to see the instructor and interact in person, but is good for brief exercises and visiting foreign sights that wouldn’t be readily accessible in a classroom setting.
Virtual Speech’s article explains the benefits of learning a language in the VR environment. Not only does this allow for an immersive setting that forces the participants to adapt to the learning experience, but it also provides the same comfort of learning at their own pace. This style of learning is likened to the immersive experience of visiting a foreign country and trying to learn there. The article provides some examples of language learning apps as well as detailed descriptions of what learners will gain.
Learning a language in an immersive environment without needing to go abroad could be revolutionary for a college learning environment. VR can allow a student to challenge themselves without putting them out of their comfort zones. Sometimes being outside of your comfort zone can be beneficial, but for those with anxiety, this technology can help prepare them for the real situation, which leads to a better learning experience overall.
The Writing Platform gives a look into the world of literature and how VR can be used to enhance a reader’s experience. Digital literature has existed for a while but moving it into the XR fields is a newer development. Not only does this new medium allow for a more interactive experience, but it can also give people a different outlook on the importance of storytelling and how it impacts them. Reading can give people a certain level of immersion depending on the imagination they have, but by adding VR into the literary experience, the immersion into the world becomes even more intense, allowing for a new reading experience. Being able to step into the pages of one’s favorite book would not only allow readers to become more connected with the literature but also provide a new experience in terms of how they view the literature and potentially its meaning.
The potential of VR and literature have no bounds, though it is important to draw a line between digital literature and VR movies and games. There should be some interactive element as well but not enough to make it into a videogame, as digital literature should focus on the storytelling primarily. Another interesting application of VR and literature would be to create a 3D space that represents the world of an already existing novel. Classic literature could be explored in new depths using VR, and the potential is never-ending. One example of digital literature mentioned is “A Place Called Ormalcy.” It was entirely designed for and developed in VR constructed using MasterpieceVR to create each chapter.
The Scientific American dives right into the benefits of using VR to give insight into the working of human brains. Mental health has always been a predominant issue in society, and VR technology could help remedy mental health issues such as depression, anxiety, PTSD, and other things like fears. From applications in therapy to diagnosing symptoms, VR could be a great help to psychologists and other mental health professionals. VR has already proved successful in treating PTSD, and work is being done to see if it can be used for other issues such as addiction, claustrophobia, and teenage depression through exposure therapy. VR applications for diagnosis can test people for schizophrenia, ADHD, and autism.
The opportunities that VR provide for diagnosis and treatment of mental health disorders is something that will definitely be expanded upon in the future. The importance that this application holds is vital, as it could help people who don’t feel comfortable going to talk to a real person about their issues, as well as catching an underlying problem that might have gone undetected. While this technology still needs more experimentation, it is still a step in the right direction for creating new ways to help people with mental health issues.
This article by Film Independent talks about the importance of representing social justice in the immersive entertainment industry. Representation in the media is extremely important, so when a new form of media arises, the need for representation expands to this new platform. XR technology provides many different new ways of communicating concepts and ideas to a wide variety of audiences and ensuring some presence of representation for social justice is crucial. From art to movies to video games, there are many different ways that XR is used to communicate with the world, so using this as a platform to advocate for LGBTQ+ rights, BLM, and other social conscious ideas is a no brainer. The article includes a recorded talk by several exciting immersive storytellers as well as giving examples of projects that are already out there for people to experience. One story is Shogaolu’s Another Dream. It is the third installment of her Queer in a Time of Forced Migration multi-media series, which takes a look at the stories of politically displaced LGBTQ+ people. The potential for the advocation for social justice will grow with the VR platform, and it should not be ignored.
Immersion VR’s article speaks about how the tourism industry is being revolutionized by the VR medium. From the wide scale of flyover tours to the detail of visiting ancient sites, the possibilities that are achievable with VR technology are endless. Rather than simply using computer-generated images, 360 VR tourism is becoming more and more popular as it uses actual images from the sites, providing a more realistic experience. Another application besides interactive tourism entails 360 VR videos in which you can sit and look around while not needing to control anything.
The future of VR travel is growing rapidly, especially because of the COVID-19 pandemic, and the amount of resources available are on the rise. Some even go as far as to have VR flight experiences to pretend they are traveling from destination to destination. Other applications include VR booking interfaces, travel for senior citizens, and landmark destination experiences.
This past summer, due to COVID-19 distancing restrictions, a UR student inquired about virtual meeting spaces to foster community in lieu of our physical library spaces. We all miss the pre-pandemic opportunities to meet up and chat. The serendipitous catch-up on campus after class or in between meetings isn’t really possible these days and poses a considerable problem for community building.
This student’s inquiry inspired iZone and Studio X team members, including the Karp Library Fellows, to develop the Dream University Challenge, in which participants imagined, designed, and built virtual university spaces that provided these opportunities to connect. During the 2021 winter break, iZone and Studio X staff led student participants through design thinking and technical workshops. They then worked in teams using Mozilla Hubs, a free and open-source virtual reality platform, to create their own unique campus spaces.
Studio X and iZone staff members served as mentors during office hours and provided both conceptual and technical feedback throughout the week-long building phase. Two teams of four submitted links to their final projects and abstracts that described their concept and acknowledged sources.
Miguel Yakouma, Koshala Mathuranayagam, Joey Chan, & Sophea Urbi Biswas
Explore in VR
Share a virtual room with friends in your browser! Both submissions embody UR pride, consider facilitating connections thoughtfully, and inspire new ideas about space and community. Click on the buttons below to visit these spaces.
The public submitted over 200 votes and awarded the following:
Most Creative Concept
Most Desirable Campus Space
Tie! UR Haven & Dream Rush Rhees
Most Out of the Box
Most Likely to Facilitate Connections
Dream Rush Rhees
The judges awarded Best Overall Winner to…
We would like to thank our amazing judges:
Wendi Heinzelman, Dean of the Hajim School of Engineering and Applied Sciences
Julia Maddox, Director of the Barbara J. Burger iZone, River Campus Libraries
Nefle Nesli Oruç ’22, Public Programs Coordinator, Karp Library Fellow
Joe Testani, Assistant Dean & Executive Director of the Greene Center for Career Education & Connections
The UR Haven team will receive UR-branded Google Cardboards and a cash prize of $100 for each team member.
At the celebration event, the winners were announced, and students discussed their experiences during the challenge and what they learned. Students expressed enthusiasm for the interdisciplinary collaboration and getting to know others outside of their usual friend groups. They also felt that, regardless of their technical level, they all had something to contribute. One international student appreciated the opportunity to stay connected to campus and the community. Another student noted that the structure of the challenge helped her to maintain a learning mindset during the long break. Others described learning how to consider user needs and how to collaborate in teams:
The challenge taught me that teamwork is all about clear communication, understanding how to think from other people’s perspectives, and respecting each other’s ideas and viewpoints.
Thank you so much to everyone who participated! We are grateful for the support from the River Campus Libraries and the Hajim School of Engineering.
Thank you especially to those who made this event so successful!
Mary Ann Mavrinac, Vice Provost and Andrew H. and Janet Dayton Neilly Dean of the University of Rochester Libraries
Matt Cook, Senior Communications Officer, RCL
Claudia Pietrzak, Student Programming & Social Media Manager, RCL
Sarah Gerin, iZone Community Manager, RCL
Zoe Wisbey, iZone Program Initiatives Manager, RCL
By Emma Schaale Karp Library Fellow, Public Programs Coordinator for Studio X June 2020 – October 2020 Area of Focus: Varies between considering branding for Studio X, creating promotional materials, and conceptualizing workshop for Studio X’s fall programming
For the past month, I’ve been working on an exciting project… a Blender workshop for Studio X, which took place on Wednesday, September 23rd, 2020! This came about as a collaborative effort between Studio X and Tinkerspace to create introductory programming on immersive technologies for the UR community.
We first prioritized workshops for Unity, a video game development environment, due to faculty and student interest in this platform. This was not my area of expertise, as I primarily work in Unreal Engine 4, a similar but different development tool. Luckily, Sebastian Jakymiw, the Karp Library Fellow Immersive Technologies Developer for Studio X, has worked extensively with Unity and was able to take on the lead instruction for those workshops. However, I participated in the conversations between him; Meaghan Moody, Studio X’s Immersive Technologies Librarian; and Allegra Tennis, STEM Librarian.
Because students and faculty are also interested in 3D content creation, Meaghan and I started brainstorming a workshop focused on 3D modeling. Having completed recent coursework (CSC131: Recreational Graphics) for 3D modeling as well as having created my own personal gaming projects, I felt comfortable teaching a workshop on Blender, which is a popular, free, and open-source 3D creation software.
One of the most important considerations we had in our conversations is how to brand our programming. This is crucial for building an image and personality for Studio X, so that a community can form around it as well as pique students’ interest. We wanted these workshops to grab students’ attention and spark their creativity. Immersive technologies are still very new and might seem complicated to students. For example, if you’ve never heard of Blender before, an “Intro to Blender” workshop may seem irrelevant or uninteresting to you. While a “Create a 3D Animal with Blender” workshop might just catch your eye.
To prepare for this workshop, my first step was to scout YouTube for existing 3D modeling tutorials. Some of my favorites are by Grant Abbitt, who has hundreds of tutorials and workshops posted to his channel. I was inspired, in particular, by this video on creating low-poly animals and decided to use to use this as the basis for the workshop.
While working on the outline for my workshop, I considered how to make the process more interactive. During a YouTube video, the user can pause or speed up the instruction, but for a live session, I would need to ensure that my students have time to see everything on the screen. For new users, Blender’s user interface can be especially overwhelming–especially when you’re learning it virtually!
This was also mentioned during a practice workshop, which I ran for Meaghan, Allegra, and Sebastian. Allegra advised that I should take more time to make sure attendees can follow my cursor on the screen. I related to this frustration based on my own experiences learning new technologies in the classroom, so it was intuitive to incorporate this.
During this practice run, it was great to hear that my trial participants found my tutorial to be a great introduction. It’s a difficult software to teach to first-time users, but I was satisfied to hear that they had a fun time learning.
During the actual workshop, I had quite a fun experience introducing the class to Blender’s user interface. Before diving in though, I described how this platform is used for all kinds of projects such as animated films, video games, and virtual reality experiences. I also broke down some basic terminology and concepts. For example, I explained that a mesh is composed of vertices, edges, and faces.
I also explained that Blender’s user interface can seem even more intimidating than the user interface of software like Adobe Premiere Pro or Photoshop. I demonstrated how to perform basics edits in Object and Edit Mode, how to trace along a reference image using extrusions and edits in Edit Mode, how to apply color to the model, and finally, how to export the model.
Perhaps the most important advice I received was from Allegra, who suggested I really slow down my instruction. At times, I felt like I was waiting hours in between steps while hovering my mouse over a button or demonstrating keyboard shortcuts. Yet, a couple of times, students asked me to repeat these actions. I learned that going slowly is crucial while teaching a complex software like Blender, especially in a virtual teaching environment.
My previous experiences teaching at an all girl’s tech camp in 2018 and as a Publicity Manager for the UR Game Dev Club have made me a much more comfortable instructor. Certainly, having these experiences made me more understanding of the ways others learn and how to make it a comfortable learning environment. While I would say my Blender workshop went well, I’ve had many failures in the past that I have had to learn from. I was even comfortable joking around, which would have been unimaginable a couple of years ago!
Overall, I enjoyed myself during this entire process. I love teaching things I care about, and building this workshop from the ground up was a wonderful experience. I was supported by Studio X, Meaghan, and Allegra, who made me feel more comfortable and confident teaching to my peers. So thank you to Studio X for making this possible!
Are you interested in learning Blender? Check out my workshop slides! Stay tuned for more workshops, and here are some of my favorite resources in the meantime:
A $1.5 million grant from the National Science Foundation will provide additional impetus to a University of Rochester initiative applying augmented and virtual reality in health, education, product design, remote communication, entertainment, and other fields.
The grant will enable 62 doctoral students to be trained in the skills needed to advance AR/VR technologies and will also help them gain an appreciation for the broader cultural and societal implications of the technologies, says Mujdat Cetin, the principal investigator behind the grant. Other Rochester faculty supporting this initiative are Jannick Rolland, the Brian J. Thompson Professor of Optical Engineering; Michele Rucci, professor of brain and cognitive sciences; and Zhen Bai, assistant professor of computer science.
Andrew White, assistant professor in the chemical engineering department, and April Luehmann, associate professor and director of secondary science education at the Warner School of Education, are collaborating on research that explores how AR can enhance the way students learn about engineering. Studio X will provide a much-needed space where educators can develop new approaches to increase student learning and engagement.
Elizabeth Colantoni, associate professor of religion and classics, wanted to visualize ancient Roman topography to explore how the Temple of Jupiter fit within the larger context of Roman society. To do that, she collaborated with others at the University of Rochester to create the virtual reality experience seen here, which provides scholars and students new perspectives and expands scholarly conversation.
The experience is based on a model of the temple that Daniel Weiner ’16 (a dual major in computer science and classics) created as an undergraduate, using a program called Sketch-up. Then, in the fall of 2018, Blair Tinker, the Digital Scholarship Lab’s GIS specialist, built this virtual reconstruction, using combined spatial analysis and 3D modeling in Unity Gaming Engine.
In “Digital Elmina,” three University of Rochester faculty members—Renato Perucchio, professor of mechanical engineering and biomedical engineering and director of the archeology, technology, and historic structures program; Michael Jarvis, associate professor of history and director of Smiths Island archeology project; and Christopher Muir, professor of mechanical engineering—created 3D reconstructions of Elmina Castle, which were then visualized through Unity Gaming Engine.
Built in 1482, Elmina is the best-preserved and most complete example of early European masonry construction in Ghana and served as an active commercial outpost over four centuries. The castle was also one of the most significant stops on the Atlantic slave trade route. Immersive technologies help us understand Elmina Castle’s past, convey this knowledge in the present, and ensure the castle’s survival in the future.