Hello! My name is Matt Gottsacker. I am a Computer Science PhD Candidate at the University of Central Florida. I do research in the SREAL Lab advised by Prof. Greg Welch. My research interests include virtual/augmented reality, 3D user interfaces, collaboration, and embodiment. I take an interdisciplinary approach to research, drawing mostly on philosophy, psychology, and cognitive science (especially embodied cognition), as well as media studies to both inspire research questions and design experiments to understand related phenomena. I design and create software and hardware to investigate these questions. I collect and analyze both quantitative and qualitative data to get us closer to answering them.
I made a comic book that functions like a personal statement and describes how my interests in the humanities and computers have come together into my current academic studies in human-computer interaction and virtual reality. I generated all the images with OpenAI's DALL-E 2 except for some photos of the VR hardware on the last page.
Some profiles elsewhere on the web:
[ Google Scholar ] [ Curriculum Vitae ] [ LinkedIn ]
Email: mattg [at] ucf.edu
I am broadly interested in creating novel, intuitive interfaces for emerging computing technologies, especially virtual and augmented reality (VR/AR) systems. As these devices are used more frequently, for longer periods, and for more purposes, it is important that they provide a seamless experience for users when interacting with digital content and with other people across different contexts. Our current personal computers (laptops, tablets, smartphones) provide features for multitasking and quickly switching between different applications. This seamless switching of digital contexts is now a regular part of how we use computers. However, this is not yet the case for VR and AR systems. This may be due to the fact that immersive applications typically involve whole worlds, not just windows, and so interacting across them or switching between them is a more complex process.
To address this fundamental usability problem, my PhD research focuses on creating computer-mediated interactions and transitions across different realities, or cross-reality interactions/transitions. This is most easily illustrated using Milgram and Kishino's Reality-Virtuality (RV) Continuum, a seminal piece of virtual reality (VR) research that introduced the figure below to classify immersive virtual systems. On one end of the continuum is the physical world with no virtual content displayed in it. On the other end is a fully virtual environment (i.e., VR). These extremes are set on a continuum because they can be mixed in interesting ways. If your primary environment is the physical world, and you add virtual content to it, you get augmented reality (AR) (think: Pokemon Go). If your primary environment is a virtual world, and you add elements of the physical environment to it, you get augmented virtuality (AV) (imagine being in a VR headset and seeing real-time video of the physical world through a portal).
In my research, I have investigated using AV and other techniques to improve interactions between people on opposite ends of the continuum, specifically focusing on when an immersed VR user is interrupted by a non-immersed person nearby in the user's physical environment. More recently, I have explored methods for efficiently and effectively transitioning a single user between different environments.
Nature scene transition environment
Experiment environment: Camp
Experiment environment: City
Experiment environment: Funland
Experiment environment: Snowy
[ Paper PDF ] [ IEEE Xplore ] [ Presentation ] [ Google Scholar ]
In most cases, retaining memories of things we have experienced in the past is desirable, but in some cases, we want to clear our minds so that we may focus completely on subsequent activities. When someone switches from one task to another, they commonly incur some "cognitive residue" where some of their cognitive resources such as working memory and attention remain devoted to their previous task even after they try to switch their focus to their new task. This residue could have a negative impact on their performance in the next task, and in such circumstances, it is important to reduce that residue. In this paper, we explore the concept of cognitive residue in the context of switching between virtual reality (VR) environments. We conducted a human-subject experiment (N=24) with a spatial recall task to investigate how different visual transitions might reduce participants' spatial cognitive residue. In this instance, more errors on the recall task corresponds to less spatial cognitive residue.
Participants experienced a series of distinct virtual environments (VEs) in which a narratation directed them to look at a series of five objects in the VE. Examples of the experiment environments are shown to the left. Between each VE, participants experienced a transition for one of three durations: instantaneous, 20 seconds, or 60 seconds. The instantaneous transition was a cut transition, where the previous VE disappeared and the new VE appeared immediately. The 20-second and 60-second transitions either showed a nature scene (first image on the left) or nothing (i.e., a black screen).
We found that transitions that lasted one minute successfully reduced spatial cognitive residue: they significantly reduced participants' abilities to recall the positions of objects in their previous VE compared to an instantaneous cut transition.
Additionally, for transitions that showed the nature scene, greater head movement significantly correlated with more spatial memory errors (i.e., less spatial cognitive residue). We discuss how these findings can be applied to support users transitioning between virtual tasks and environments in VR task switching scenarios.
Participants seemed to like the nature scene transitions better as well. We measured participants' perceptions of the usability of the transitions with the User Experience Questionnaire. Both the 20s and 60s Nature transitions scored higher for the Hedonic sub-scale compared to the Blank transitions of the same respective duration, suggesting that participants found the nature transitions more stimulating and novel. Both of these transitions were also rated higher than the Baseline and 60s/Blank transitions for the Attractiveness sub-scale, suggesting that participants perceived them as better overall at clearing their minds when transitioning between different VEs. Both the shorter Nature and Blank transitions scored significantly higher on the UEQ Pragmatic sub-scale than the 60s/Blank transition.
Baseline: no virtual avatar
UI notification + passthrough view
Non-diegetic avatar
Partially diegetic avatar
Fully diegetic avatar
[ Paper PDF ] [ IEEE Xplore ] [ Presentation ] [ Google Scholar ]
When someone puts on a virtual reality headset, they are completely isolated from their physical environment, including the people around them. This is by design—VR presents to be the most immersive computing technology. However, there are many cases in which a person wants or needs to interact with someone immersed in VR. Some examples, where "you" are wearing a VR head-mounted display:
With the current state of the art, the interrupter cannot fully interact with the VR user unless they take off the headset. There is a lot of friction involved in that process, so it seems that there should be a communication channel that does not require the user to doff the headset. Additionally, the interruption is often jarring for the user. They are immersed in another world. When someone taps them on the shoulder or speaks to them, their physical environment abruptly calls them back. This process could be smoother.
With the help of some awesome people in my lab, I designed, implemented, and ran an experimental design human subjects study to examine ways to facilitate this interaction. Our work was published in the proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR) in October 2021. You can read the full paper here: Diegetic Representations for Seamless Cross-Reality Interruptions.
The word "diegetic" comes from describing narrative media elements. A story element is diegetic if it comes from within the context of the story world itself. For example, a sound in a movie is diegetic if it is produced by something in the scene (e.g., a radio playing a song). A different sound is non-diegetic if it is added to the scene as a narrative element (e.g., a musical score added to a scene where there is no orchestra plausibly nearby). I made a 2-minute creative explanation for this concept in my video presentation of this paper. You can watch the whole "live" presentation from that link if you want; my presentation starts around the 20:20 mark.
VR complicates the common definition diegetic because the virtual world completely surrounds the user. We can talk about audio mostly in the same diegetic dimensions, but the lines between diegetic and non-diegetic visuals blur a little. We explored how one might vary the diegetic degree of the appearance of an avatar to represent a non-VR interrupter to a VR user, and the different effects that might have on the VR user's virtual experience and cross-reality interaction experience.
I built a virtual office environment and tasked participants with stacking virtual blocks in a couple different formations—just an easy task that could help them fall into a rhythm pretty quickly. The experimenter interrupted them part-way through each block formation. The way the interrupter was represented to the VR user changed each time:
Based on a Cross-Reality Interaction Experience questionnaire we wrote, we found that participants rated the interaction experience with the interrupter highest for the partially and fully diegetic avatars. We also found that these avatars afforded a reasonably high sense of co-presence with the real-world interrupters, i.e., participants felt they were with a real person as opposed to a purely digital one. We found that participants more often preferred the partially diegetic representations. Their qualitative responses suggest why: several stated that the green outline helped them distinguish the avatar from the rest of the virtual environment; the outline suggested the avatar was not just an NPC (non-player character in a video game). I am interested in further exploring methods for representing cross-reality interactors, especially for interactions that may occur for longer periods (as opposed to brief interruptions).
Additionally, we asked participants about their place illusion, or their sense of "being there" in the virtual environment before, during, and after the interruptions. We found that the avatar conditions led participants to experience a consistent and high sense of place illusion throughout the interruption, where the conditions that caused participants to take the headset off led to a drop in place illusion that did not recover immediately after the interruption. I am interested in investigating further how VR users' senses of presence move and change throughout a VR experience.
Green: Low Virtual Activity
Yellow: Medium Virtual Activity
Red: High Virtual Activity
[ Paper PDF ]
When someone we want to interact with is wearing an HMD, our usual ability to attribute mental states (e.g., busyness, engagement, openness) to the other person is inhibited. We perform this attribution quite often in social interactions—in psychology it is referred to as developing a theory of mind about the other person. VR makes this process difficult because the VR headset blocks much of the user's face and does not share any information about their current task. This presents a problem for people in the VR user's physical environment who need to interact with (i.e., interrupt) the user. The interrupter does not have access to the cues they usually use to determine when and how to interrupt, nor the methods they traditionally use to signal an interruption. For example, they cannot read the VR user's full facial expression to determine how engaged they are with their task, and they cannot announce their intention to interrupt by slowly approaching. This disconnect is a problem that will become increasingly common as VR is used by more people, in more settings (e.g., the workplace), for more purposes, and for longer durations.
To improve this interaction from the interrupter's perspective, I am designing and implementing both hardware and software prototypes, and running user studies to test them. I aim to produce simple modifications to VR headsets that restore these broken communication links and improve these interactions for all parties. To this end, I have explored attaching visual cues to a headset to communicate the VR user's virtual activity level (i.e., their engagement with their virtual task). I have also explored signaling mechanisms such as a gesture sensor mounted on a headset to allow an interrupter to initiate an interruption through a wave gesture.
I presented a poster at IEEE ISMAR 2022 about an initial prototype and pilot study for this project. Through quantitative and qualitative data analysis, the pilot study suggested that the Virtual Activity Cues were helpful in providing interrupters with information about the VR user's mental states and useful for deciding when to interrupt. The gesture system did not seem to be helpful. Going forward, I am planning a user study to understand the information essential for interrupters to understand the VR user's mental states and interruptibility. Then, I will iterate on the design of this prototype to home in on cues that are most useful for communicating when and how one should interrupt.
These are some of the things I used to build this space:
e-mail me at matt [dot] gottsacker [at] gmail [dot] com. Feedback and collaboration ideas are welcome.
The only cookie used for this website checks whether you have already seen the banner about user data. If you have seen it, you won't see it again (unless you clear your cookies).
I use the free and open source GoatCounter for web analytics. I don't store any data on you, but I like seeing from where in the world people visit this webspace. In fact, you can see everything that I can see by visiting this website's public stats site.
I'm not tracking you. This website uses one cookie. More info.