projects_

When someone puts on a virtual reality headset, they are completely isolated from their physical environment, including the people around them. This is by design—VR presents to be the most immersive computing technology. However, there are many cases in which a person wants or needs to interact with someone immersed in VR. Some examples, where "you" are wearing a VR head-mounted display:

• You are playing a VR game in your home, and your roommate needs to ask what kind of pizza you want.
• You are working on a 3D design task in VR, and your collaborator needs to tell you about a specification update.
• You are passing time on a plane in peaceful virtual environment, and your seat neighbor needs to move around you to use the onboard lavatory.

With the current state of the art, the interrupter cannot fully interact with the VR user unless they take off the headset. There is a lot of friction involved in that process, so it seems that there should be a communication channel that does not require the user to doff the headset. Additionally, the interruption is often jarring for the user. They are immersed in another world. When someone taps them on the shoulder or speaks to them, their physical environment abruptly calls them back. This process could be smoother.

With the help of some awesome people in my lab, I designed, implemented, and ran an experimental design human subjects study to examine ways to facilitate this interaction. Our work was published in the proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR) in October 2021. You can read the full paper here: Diegetic Representations for Seamless Cross-Reality Interruptions.

The word "diegetic" comes from describing media elements (most often in movies). It refers to whether a story element comes from within the context of the story world itself. For example, a sound in a movie is diegetic if it is produced by something in the scene (e.g., a radio playing a song). A different sound is non-diegetic if it is added to the scene as a narrative element (e.g., a musical score added to a scene where there is no orchestra plausibly nearby). I made a 2-minute creative explanation for this concept in my video presentation of this paper. You can watch the whole "live" presentation from that link if you want; my presentation starts around the 20:20 mark.

VR complicates the common definition diegetic because the virtual world completely surrounds the user. We can talk about audio mostly in the same diegetic dimensions, but the lines between diegetic and non-diegetic visuals blur a little. We explored how one might vary the diegetic degree of the appearance of an avatar to represent a non-VR interrupter to a VR user, and the different effects that might have on the VR user's virtual experience and cross-reality interaction experience.

I built a virtual office environment and tasked participants with stacking virtual blocks in a couple different formations—just an easy task that could help them fall into a rhythm pretty quickly. The experimenter interrupted them part-way through each block formation. The way the interrupter was represented to the VR user changed each time:

• Baseline: interrupter taps the user on their shoulder, user takes off the headset to interact with interrupter.
• UI + passthrough view: UI notification alerted the user someone nearby wanted to speak with them, then the user activated the headset's passthrough view to see the interrupter through the headset's external cameras.
• Non-diegetic avatar: The interrupter was represented in the virtual environment as a floating green sphere.
• Partially diegetic avatar: The interrupter was represented in the virtual environment as an avatar in business clothes with a glowing green outline.
• Fully diegetic avatar: The interrupter was represented in the virtual environment as an avatar in business clothes.

Based on a Cross-Reality Interaction Experience questionnaire we developed, we found that participants rated the interaction experience with the interrupter highest for the partially and fully diegetic avatars. We also found that these avatars afforded a reasonably high sense of co-presence with the real-world interrupters, i.e., participants felt they were with a real person as opposed to a purely digital one. We found that participants more often preferred the partially diegetic representations. Their qualitative responses suggest why: several stated that the green outline helped them distinguish the avatar from the rest of the virtual environment; the outline suggested the avatar was not just an NPC (non-player character in a video game). I am interested in further exploring methods for representing cross-reality interactors, especially for interactions that may occur for longer periods (as opposed to brief interruptions).

Additionally, we asked participants about their place illusion, or their sense of "being there" in the virtual environment before, during, and after the interruptions. We found that the avatar conditions led participants to experience a consistent and high sense of place illusion throughout the interruption, where the conditions that caused participants to take the headset off led to a drop in place illusion that did not recover immediately after the interruption. I am interested in investigating further how VR users' senses of presence move and change throughout a VR experience.

This research follows on my work on using diegetic avatars to represent avatars that was published at ISMAR 2021. In running that experiment, I acted as the interrupter, and I interrupted participants immersed in VR. It felt strange to interact with someone wearing a head-mounted display (HMD). When someone we are interacting with is wearing an HMD, our usual ability to attribute mental states to the other person is inhibited. We perform this attribution quite often in social interactions—in psychology it is referred to as developing a theory of mind about the other person. This inhibition seems like a problem that will become increasingly common as VR is used by more people, in more settings (e.g., the workplace), for more purposes, and for longer durations.

To improve this interaction from the interrupter's perspective, I am designing and implementing both hardware and software prototypes, and running user studies to test them. I am exploring using basic cues about a VR user's virtual activity to communicate their interruptibility. I am also exploring signaling mechanisms such as a gesture sensor to notify a VR user of an interrupter's intention to interrupt.

I have a late-breaking work paper under submission about a preliminary study we ran, and we are planning to run a full study this semester (Spring 2022).

For CAP 6115 Mixed Reality Project taught by Dr. Ryan McMahan, I worked with UCF Computer Science Master's student Steven Perdomo and Prof. Tommy James from the UCF Architecture Department on his Pipeline to Better Placemaking project. Steven and I created a mobile AR app using Unity's ARFoundation to display virtual architecture models in the physical place in which they will be built. Users can cycle through different proposed models and complete a survey to provide public feedback to the architects. The app was designed to position the virtual objects based on image targets that would be part of a physical poster with information about the place. Positioning the models into a precise location requires many fine-tuned adjustments, so we built an authoring tool to help the architects update the position and rotation of models in real-time. Without this tool, each adjustment to the models' positions would require re-building and re-deploying the application.

All YouTube videos are embedded in privacy-enhanced mode, meaning YouTube will not store information about you for playing the videos here.

I took CAP 6110 AR Engineering with Dr. Ryan McMahan in my second semester of grad school at the University of Central Florida. Here are some of my class probjects. You can find the others on my YouTube page.

All YouTube videos are embedded in privacy-enhanced mode, meaning YouTube will not store information about you for playing the videos here.

I took CAP 5115 VR Engineering with Dr. Ryan McMahan in my first semester of grad school at the University of Central Florida. Dr. McMahan is a great teacher, and he taught this class very well. He got me to make some cool and fun VR interactions and environments. You can find all my demos on my YouTube page, but the ones shared here are probably the most entertaining. All of them except the first video build on some sort of story!

All YouTube videos are embedded in privacy-enhanced mode, meaning YouTube will not store information about you for playing the videos here.

As a summer intern at the workflow automation company Nintex, I developed a fully automated data analysis pipeline to gain insights about customer behavior and generate reports for company executives and the product team. It was an independent project supervised by the company's Principal Architect. At the end of the summer, I presented my work to the company CTO and other stakeholders.

The components of the pipeline and their purposes include:

• Azure Function and Azure Storage Queue: detect new and modified data
• Databricks: extract meaningful data, transform them into useful aggregated formats, and load to new tables
• Power BI: generate and share reports that summarize product usage and highlight interesting outliers

This is a web-based, interactive creative project in which the story is told through a series of found logs on a computer. The content is not finished, but most of the technical setup work is done. You can check it out here: https://mattgottsacker.space/alta_ie_v0

GitHub repo

Audio interacts with the listener's environment. This web application will give users the ability to easily create VR experiences for different audio tracks. They will be able to insert annotations in the form of text, shapes, environmental effects at different spacetime points in a track.

Upon completion, users will be able to share experiences with others by simply sending a web link.

I would love a collaborator on this project. It needs a lot of work. Here's the GitHub repo.

WikiLearn Spanish is an Alexa skill designed to help Spanish language learners with their reading comprehension. Users can choose a category (people, places, events, or nature), and WikiLearn Spanish will read the summary from a Wikipedia article fitting the selection. Alexa alternates between the Spanish and English articles paragraph by paragraph.

An important consideration in this Alexa skill is the idea of translation. Spanish and English Wikipedia articles often do not share the same authors, nor do they try to be "direct translations" of each other. They emphasize different elements. For this reason, my friend Graham Vogt and I manually select and process articles that focus on similar content in both Wikipedia communities.

dev status: in final testing
links: GitHub repo SLU article

I started working with Dr. Nathaniel Rivers at the SLU Computer Assisted Instruction Lab in the Fall of 2018. The mission of the Lab is to connect students and instructors with technologies that can help them produce new media compositions. It was previously abbreviated as the CAI Lab, but we rebranded it as the Compass Lab to more accurately portray the Lab's function. The compass is a technology that helps us navigate through technologically-defined environments.

We identified gaps in information distribution from the Lab to its users and kinks in the Lab's internal workflow. For example, there exists a webpage with information about the Lab, but it is difficult to update in a timely manner, and it does not serve a lot of useful content. Additionally, the process to reserve some of the Lab's resources are cumbersome for employees. Dr. Rivers and I determined that designing and implementing a website could address both of these issues.

The website displays the Compass Lab's resources in a user-centric and intuitive way. We organize the available technologies primarily according to the functions they serve (i.e., for podcast or video production) while also giving a full catalogue. We provide links to product pages and manuals for more detailed user instructions.

Some of the Compass Lab's most important resources are the physical spaces it offers to students and instructors. Its main office has workspaces for editing content and browsing among available technologies. The Lab also has a professionally-equipped recording studio. To connect users with these spaces, I am developing web-based virtual reality tours to orient visitors to their highlights and resources.

Here is a link to the website.
Look around the frame below to explore the Lab in VR!

The Digital_Native website is a portfolio for my Senior Seminar course on Rhetoric, Writing, and Technology. Throughout the semester, we studied how attention functions in humans and non-humans. In a series of publications, we pursued questions such as: What is attention? How does it work as a commodity? Is it better understand as a function of environmental factors and actors? How do networks of attention form and operate?

The small size of the class allowed us students to tailor our projects according to our interests. I chose to focus on digital attention, so each project implicates the field of Human Computer Interaction (HCI), even if I didn't realize it at the time. The portfolio is located at https://mott-lab.github.io/digital_native/.

My last and most HCI-intensive publication examined how humans attend to entirely digital worlds. I created a web-based mobile Virtual Reality experience using Mozilla's A-Frame technology. I observed some of my friends using the VR application on their phones in a basic headset. Through this project, I learned that human attention can be affected by VR experiences in serious and complex ways. And we don't understand all of those ways yet.

My portfolio required a lot of HTML/CSS and JavaScript coding. Here is the Github repo for my portfolio.

During the Summer of 2018, I worked as a Research Intern in the Cyber Analytics and Decision Systems Group at the MIT Lincoln Laboratory. My group researched software-defined networking (SDN) in the context of cybersecurity systems.

SDN allows network analysts to easily deploy access control rules across a network. A novel application for an SDN could leverage this feature to enforce dynamic, role-based policies on high-level information such as usernames at the network level. Such a system requires a database to keep track of the relationships among each device and its respective identifying information on the network. In order to enforce a policy based on a username, the system also needs knowledge of the network device’s hostname, IP address, and MAC address. We refer to the relationships between two network identifiers as bindings.

Network-level bindings are inherently dynamic, and bindings between usernames and hostnames complicate the state of the network even more. This dynamic nature makes it difficult for network analysts to validate and debug the policy enforcement system described above. Existing SDN verification tools, such as the web applications bundled with the Floodlight and ONOS controllers display the topology of network devices, but they do not visualize the bindings between network identifiers. Researchers currently test the system by examining database logs. Because of the scale of an enterprise network, this method is inefficient and may lead to change blindness.

We approached these issues with a web application displaying two different visual representations of the identifier binding database. Each method shows an overview of the network to allow top-down analysis and features that give users access to the granular data necessary for bottom-up analysis. The first representation is an enriched tabular view, in which the records from the database are fit into a sortable and filterable table. The bindings between each pair of identifiers are encoded as link icons in spaces between each identifier column. The icons are colored along a colorblind-safe gradient. The place on the gradient is calculated based on the current time and the difference between the bindings’ activation and expiration times. This view is intuitive and provides fast searching, but it does not scale perfectly. The second representation displays an undirected graph with the network identifiers as nodes and the bindings as edges between the corresponding identifiers. The edges are colored according to the same gradient as the link icons in the table view. While this view loses some simplicity, it provides a useful visual overview of the network and the analyst can quickly learn whether a certain identifier is bound to several others.

We have created a web-based application for the views described above. The tool updates in real-time to ensure the network analyst has up-to-date information. We have not evaluated its efficacy with real users or experts. The next step of our research is to conduct a user study to learn if our application provides insights valuable to the end user. I compiled my summer work into this poster, which I presented at the ACM Internet Measurement Conference in Boston, MA, on October 31, 2018.

My friend and sociology researcher, Andrew Dermott Smith, conducted research on land parcels in St. Louis. When he started his project, there was a less-than-ideal data source for parcel data in the City of St. Louis. For example, the City provides some downloadable datasets about land data, but these files exclude some of the fields he wanted to study. However, the City hosts a web application that provides complete data for each parcel entry when users search by address or by a unique parcelID. This parcelID field can sometimes be derived from information in the downloadable datasets.

This project involves a series of Bash commands and scripts that constructs parcelID identifiers from the datasets, queries the City's web application, and downloads the resulting web page. Scripts then process the raw HTML files to extract the desired fields. The final result is a tab-separated file with the parcelID and fields that Andrew wanted to study. This data can help produce maps of the city of St. Louis based on total parcel value and how each parcel is used.

Here is the Github repo.

I wrote this application as part of my Foundations of Rhetoric class my Sophomore year. My professor, Dr. Nathaniel Rivers, assigned us a project to map a part of the world in an interesting way. I created SLU+, an augmented reality app of Saint Louis University's campus. I used the Wikitude SDK to overlay markers on statues and historic buildings. Users can tap on the markers to learn more about each landmark.

This project was my first experience with the field of Human-Computer Interaction. I observed a few friends using the app. I learned that people experience an altered physical world when looking through the lens of an AR app. It shapes where they walk and look, and as a result, it shapes where other people walk and look. AR systems are rhetorical devices that affect how people network with physical objects and other people. These networks among humans and non-human devices run deep, and they affect how we attend.

Unfortunately, I do not have the code for the application on a Github repo because I was not very experienced with Git at the time of the project. Here is a demo video of it.

All YouTube videos are embedded in privacy-enhanced mode, meaning YouTube will not store information about you for playing the videos here.

This project originated as part of the Intern Idea Innovation Challenge at MIT Linoln Laboratory during my summer internship there over the summer of 2018. I worked on a team with Stevie Carnell, Toby Macaluso, David Gundana, and Robert Hakulin to develop the idea and present it to five of the Lab's executives. We won the challenge, and I have been able to work on it with Ricardo Saucedo and Hayden Clarke for our Senior Capstone project at Saint Louis University.

Implicit gender bias is unfortunately prominent in the workplace: 42% of women have experienced discrimination at their jobs because of their gender. While this bias can manifest in a variety of ways, a number of meeting-related bias behaviors have been identified: women are more likely to be interrupted by men and talk less frequently and for shorter periods than men. These trends in meeting communication should be particularly concerning for companies, given that the average organization spends 15% of their collective time in meetings. To mitigate these meeting trends, we propose a meeting analysis tool that serves as a bias intervention for individual meeting attendees and a company-wide meter for workplace culture.

The system we implemented records meeting audio and uses speaker diarization technology to differentiate among individuals speaking. We used pyAudioAnalysis for this task and slight modified its source code to meet our needs. Here is our forked Github repository. We are in the process of programmatically collecting data on a variety of conversational behaviors, such as speaking times and interruptions. By identifying an individual speaker and when he/she is speaking, we can compile the total speaking time for that meeting attendee. Similarly, overlaps or unusually short gaps between different speakers can be coded as interruptions.

The conversational data is then used in two primary visualizations: an individual view and a company-wide view. The individual view will show an attendee his/her own data from previous meetings, such as how long he/she spoke or how many times he/she interrupted other attendees. The individual view could also show historical trend data to demonstrate how his/her behaviors may change over time. By reviewing this information, the attendee should be prompted to and capable of engaging in self-monitoring regarding any biased behaviors he/she may exhibit. The second view is at a company-wide level and will allow companies to review conversational behavior at a larger scale. This view is similar to the individual view but will contain aggregate data from many meeting attendees broken down by various demographics of interest. This information can give upper-level management and Human Resources departments a concrete measure for creating a supportive workplace culture.

The system we implemented for the Capstone project is technically limited and falls short on some ethical considerations. From the technical perspective, we did not achieve full automation. After recording meeting audio, we had to manually run each file through the ML software. More glaring are the ethical shortcomings. The system only determines between male and female voices. A better implementation would account for non-binary genders and other identifiers that are personally constructed. This kind of system should allow users to create a personal profile, and construct an individualized voice model that would be used to classify their voice in a meeting. This design of course brings new privacy concerns, but they do not seem insurmountable.

This project is the frontend and (incomplete) backend for a Facebook Messenger chatbot that leverages the Noonlight API (previously SafeTrek). Noonlight connects users to emergency services in innovative and user-centric ways. Safe-Bot brings their technology to Facebook Messenger and allows users to easily request police, fire, or emergency medical services through a chat conversation. The Facebook Messenger platform is useful for these situations because it is an always-on application that allows users to share a precise location quickly and without having to think about it. It is also advantageous in events where users under duress need to remain silent.

I wrote the chatbot frontend in JavaScript. I wrote the backend using Node.JS on a Heroku server. Here is the Github repo for Safe-Bot. I wrote this application in a rapid development window for Noonlight, so its commit history is messy. I was not familiar with Heroku or writing backend JavaScript in general at the time. The chatbot has limitations in that it does not manage multiple users. To be complete, it needs a secure database to manage user tokens. I plan on working on this part in the future. Let me know if you want to help with this part. I would love to collaborate.

Here is a demo video of Safe-Bot.

All YouTube videos are embedded in privacy-enhanced mode, meaning YouTube will not store information about you for playing the videos here.