In Week 10, I finalized the draft of our final paper, added the abstract, limitations, discussion and conclusion section. We also had the final presentation and poster session where we shared our 10 week long progress and learnings from the project. This was my first poster session and it was exciting to see everybody else’s projects and present my work alongside the REU team.

During Week 9, I was running more participants and preparing for the results presentation. I had to present online as I was tested positive for covid. But we received the test accuracies of the sign language recognition system and compared it with the training and validation accuracy. We also collected qualitative feedback from the participants who shared their experiences with VR interaction, personal concerns and suggestions for improvisation. We were able to analyze the quantitative data and use different metrics to present the preliminary results. Both qualitative and quantitative data were helpful to assess the system’s performance and identify gaps in the model. This week was crucial for evaluating our progress and making necessary adjustments to enhance the system’s accuracy and overall effectiveness.

In Week 8, we faced challenges with the data collection process with the participants as the VR system did not function. This affected our trial as we could not use our participant’s data to train the system due to system failure. The VR system was also taking longer to save the sign data and we ran into other technical issues this week. Despite these setbacks, I revised the literature review and worked on incorporating feedback from our GRA. I realized the need to provide extensive background information and make our paper more clear by adding information about the purpose and scope of this technology. Using VR to learn ASL opens doors to so many people who do not have easy access to learning ASL in a traditional classroom setting. We provided example of how a hearing parent who just had a Deaf child could learn ASL during their free time at home within the VR environment and get real time feedback through the game. I also created a list of questions to answer questions about both the technology and background of the project and made the necessary changes to the paper. This week was focused on troubleshooting and making improvisations to both the study and our paper.

During Week 7, we focused on reaching out to different programs on campus and gathering more participants for the study. We had to prepare for next week’s results presentation so recruiting more participants for the data collection process was very important. It was difficult to find a diverse range of participants as most students were not on campus over the summer and we also realized there were geographical limitations as we required participants to come in person and train our model. Meanwhile, I also assisted Dr. Lorna Quandt and Dr. Shahinur Alam to create a research grant video for ASL Champ! We provided a general overview of the project and established the purpose, objectives and main features of the game. I modeled for the demo trial for the video to demonstrate the effectiveness of the sign language recognition system and the real time feedback provided by the signing avatar in the coffee shop simulation. This week was also significant in improving our research approach and preparing for the last three weeks of REU.

During Week 6, we continued our progress from week 5 and solidified our methodology. I also worked on the methodology paper draft and created powerpoint slides for the REU methodology presentation. We were concerned about the presentation because we decided to change the direction of our paper by focusing on future research possibilities and adding a comprehensive usability study. I worked on drafting a two part methodology section, one focusing on the data collection process and the other focusing on the future usability study. The data collection consisted of introduction, consent, background forms and the participants interacting with the VR interface to train the sign language recognition system. Whereas, the usability study would be more of a comparison between traditional methods of ASL learning VS learning ASL in VR and evaluating which method of learning is more effective and what improvements users would like to see.

In Week 5, we completed the IRB (Institutional Review Board) application and received approval from the board to proceed with our study. During the approval process, we created the presentation of instructions for participants to help them understand the purpose of the study, how to interact with the user interface in VR and the 9 different signs for the maker space environment. After receiving the official approval, we began advertising our study across campus by putting up flyers and sharing the study to potential participants digitally. We also got one participant for our data collection the same week. I continued with the literature review revision and reviewed the project’s methodology for next week’s presentation.

This week was filled with productive meetings that helped us refine our research methodology and develop a deeper understanding of the project scope. Our team had several discussions with our project mentors regarding limitations and scope of the study. We also continued our extensive literature review to gather background information and identify gaps in existing research in order to revise our research questions. We were also waiting for our IRB approval to begin recruiting participants for the data collection of our maker space simulation for ASL Champ! After the sign data collection stage, we are looking into conducting usability studies by recruiting participants to gather feedback on the VR game experience. If we have enough time, we hope to conduct trials with participants using the VR learning platform to observe their interaction and learning outcomes.

Week 3 was a unique experience for me as I was working on site only for the first half of the week. I worked on revising our literature review draft and preparing for the presentation of our research question and literature review. After completing these requirements, I left for the XR Access Symposium in New York City. This year’s focus was on “Insights from XR Assistive Technology: How can we learn from technologies aimed at assisting disabled people in order to make mainstream XR more accessible?”

This week, me and my team worked on the literature review draft. We looked into existing research in sign language recognition systems with a focus in the use of virtual reality (VR) platforms for teaching American Sign Language (ASL) and how it presents potential for enhancing the learning experience for users. We were able to assess various methods of powering the sign language recognition system and learned more about the use of deep learning algorithms for ASL Champ! I read a lot of insightful papers and had to choose the 15 most relevant research works for this literature review. I also looked into the future scope of research for sentences recognition in sign language as well as improvement in recognition accuracy.

The first week of my DREU internship comprised of introductions and orientations where I learned about ongoing projects in Accessible Information and Communication Technologies at Gallaudet University. I was also assigned to an exciting project called ASL Champ!