Q&A: Matthew Cornelius Gets in the Game

Photo: Matthew Cornelius speaking at “Put Yourself into a VR Game Using 3-D Scanning and Motion Capture” at Creative Tech Week.

There’s getting into video games—and then there’s getting into video games at NYIT.

From May 12 to 21, Creative Tech Week attendees learned about creating a virtual reality (VR) game at NYIT’s workshop, “Put Yourself into a VR Game Using 3-D Scanning and Motion Capture.” Held at the HIVE at NYIT-Manhattan, the workshop was led by Matthew Cornelius, art media technical director at NYIT College of Arts and Sciences, Terry Nauheim, associate professor and chair of Digital Art and Design, and Talal Choudhury, full stack developer at TimeWave VR Lab.

The presenters used a mix of technologies, including 3-D scanning, cloud-based rigging and animation software, a game engine, and a VR head-mounted display, to show participants how to create a 3-D avatar of themselves. Those avatars were then inputted into a game engine where users could control and interact with them.

Cornelius recently sat down with The Box to talk about the workshop, the role VR will have in the future, and how NYIT is leading the way when it comes to creative technology.

How did the workshop go?
We had a fantastic turnout and a great response from participants. Twenty-eight of 30 tickets were gone by the day of the workshop, and more people arrived that day. We were excited to find there is a large audience of creative professionals in New York City who want to develop skills in virtual reality (VR), augmented reality (AR), and motion capture.

Clockwise from left: Talal Choudhury setting up the Oculus, 3-D-scanned avatars of people in the game, and a default skeleton humanoid Mecanim animation rig.

What was the highlight of the workshop?
Showing participants how quickly you can create complex digital assets. For example, we used a 3-D scan of me that was taken that morning by Talal. Within an hour or so, it was rigged with a skeleton and had motion capture data applied to it to make it move. That's an incredible turnaround for having a scan running in a game engine. Everyone in the workshop was very surprised to see how Talal was able to put the pipeline together.

Your workshop was about creating a virtual reality game. What role will VR have in the future?
Google’s Clay Bavor [vice president of VR] recently posted this in a tweet: "VR/MR/AR/XR [Virtual Reality, Mixed Reality, Cross Realties, Extended Realities] are not separate and distinct things. They’re convenient labels for different points on a spectrum."

We are in the Atari "Pong" stage of these kinds of technologies. Think how video games went from Pong to where they are now. And the speed of development today is accelerated from those days. Soon, other realities (such as VR, AR, and MR hardware and software) will not be what we know them to be today. In the past, we have seen new consoles arrive every three to five years, depending on the technology, and this rate is increasing. New technology always shows a dramatic jump in capabilities. [Reality technology] is going to be used in many areas of life in our future, everything from cooking in the kitchen, to getting information like prices and directions while we are out and about, to working in fields such as engineering and medicine.

How is NYIT leading the way in creative technology?
We have many projects that use technology to creatively solve problems. In the Department of Digital Art and Design, we have the HIVE that features a long list of technologies, including 3-D scanning, 3-D printing, VR, and robotics, as well as the motion capture lab. NYIT School of Engineering and Computing Sciences and the Entrepreneurship and Technology Innovation Center (ETIC) utilize a host of technologies including sensors and 3-D printing. NYITCOM also has many ongoing research and development projects that use electronics, nanotechnologies, and high-precision robotics.

All of these projects enable us to explore ways to expand the capabilities of technology to creatively solve problems and achieve fantastic results. With social media and tech and science outlets aggregating information, the development of technology is becoming more of a social discussion--one in which people can truly understand the age we are living in. At NYIT, we continue to grow with technology, and we understand how rapid advancement, through sound research and development, is key to this growth. NYIT’s strength lies in its ability to coordinate the efforts of the leaders of each discipline within the university, fostering cross-curricular discussion and cooperative learning.

This interview has been edited and condensed.