ABOUT JetXplore:

JetXplore uses mixed reality to create immersive and engaging training experiences for the next generation of aviation professionals. The project provides hands-on training for both normal and emergency aircraft operations. Students are able to meet specific learning objectives with higher retention and enhanced understanding.

Q&A WITH LORI

How did you come up with the idea for the project?

It was difficult to teach my students turbine engines in the classroom as they had no way of seeing inside of them. When I saw that Case Western was using HoloLens to allow students to see inside the human body, I realized that HoloLens was what I needed in my lab. Mixed reality gives students the ability to see inside the jet engine while also interacting with the engine components and full flight deck. Mixed reality allows me to create virtual engines and supplements full flight simulation. It is the perfect marriage of digital content overlaid onto real-world objects.

Describe the production process.

In the last 16 years that I’ve been teaching advanced aircraft systems, I have identified many gaps in the tools and equipment available to me as a professor. Mixed reality fills those gaps by providing an experience that bridges what students learn in the classroom and in the aircraft simulator or airplane. I first tried computer-based training, but that was not customizable and did not engage the students. I then tried virtual reality, but that restricted the students’ mobile ability (as they were tethered to a gaming computer). We were looking for something more interactive – When I saw what the medical community was achieving with HoloLens, I became excited about using mixed reality for aviation training.

We first created a PC-based app with Ihab Mohammed and Dr. Ala Al-Fuqaha at Western Michigan University. We later began prototyping the HoloLens aviation training applications with computer science students Gregory Ostroy, Dennis McFall, and Bill Cheu. We used Unity to create interactive turbine engines and interactive 3D cockpits. The students really enjoyed the prototypes we built, and we have seen improved assessment and retention. (We have even been able to replace some of our simulation training!) We also used Vuforia with the HoloLens to recognize images that trigger digital content. In doing this, we were able to design interactive textbooks with mixed reality overlays.

What was your biggest hurdle in creating the project and how did you overcome it?

There were two challenges in creating the project. The first was bridging the gap between technology, creativity, and subject matter. The second was getting project funding. I was able to overcome these challenges by collaborating with a diverse and extraordinary team and combining smaller grants for each phase of the project.

What advice do you have for someone who wants to start building for mixed reality?

Start with a small project so that you can see how the project goes from start to finish in a short timeline. You will be amazed at how immersive the experience is, and you will then be inspired to take on a larger project or maybe create several bite-sized micro simulations to fill specific training gaps.

What excites you most about the future of mixed reality?

I am most excited about how mixed reality will revolutionize training. Immersive training using devices such as HoloLens will enable students to learn and practice new skills while maximizing their retention of these skills. In addition, mixed reality promotes cost and space savings. (We won’t have to bring huge engines into the lab or spend money on expensive physical simulators.) I’m very excited to see what else will come of this in the training and education space! Mixed reality is the future, and this is just the beginning.

Microsoft provided mentorship and guidance to each creator as part of this initiative.