Virtual Reality/Augmented Reality

Medical Simulations

SmartVCS: Beyond Big Studio Production

In the current production environment, the ability to previsualize shots utilizing a virtual camera system requires expensive hardware and large motion capture spaces only available to large studio environments. With accessible hardware such as multitouch tablets and the latest video game motion controllers, there exists an opportunity to develop a new virtual cinematography platform utilizing only consumer technologies and openly accessible game engines. The SmartVCS platform is designed for directors, both amateur and professional, who wish to embrace the notion of Virtual Production for films and game cinematics without a big studio budget. SmartVCS has potential applications to other digital media areas including game level design, real-time compositing & post-production, and architectural visualization.

Master's Student: Girish Balakrishnan
• Advisor: Dr. Paul Diefenbach

• 
• 
• 
• 

Motion Platform Simulator Games

The Drexel Ride is a multi-discipline effort to build games and a game architecture for a 2.5 ton, 3 DOF motion platform simulator donated by The Ride Works.

The game fLight was created as a joint senior design project by a team of Digital Media and Computer Science students. In this game, 5 participants outside of the vehicle play on iPads the two passengers in the vehicle. One drives the vehicle and the other uses a wiimote to light up beacons. The five players on iPads use their creature chraracters to try to stop the vehicle participants through various physical interactions such as bumping the vehicle or temporarily blinding them.

• Advisor: Dr. Paul Diefenbach

Operation Dino Museum Game

"Kollect" Cerebral Palsy Physical Therapy with Kinect

Music — Reactive Gaming — "Pulse" and "Pulse 2"

Pulse 2 is a music-reactive game which uses the player's own music library to dynamically incorporate music analysis together with web-accessible lyrics, text, and images based on the metadata included in the song files. They produced a web-deliverable 2D side-scroller game integrating these features that performs real-time analysis of songs in a user's music library to drive the gameplay, providing a novel form of game-music interaction. This work was presented at the IEEE Games Innovation Conference (ICE-GIC 2009) in London, England.

"Surge"

Surge expanded on this concept by using more analysis of the music to extract characteristics identifiable by the human ear to deform worlds and generate obstacles and environments which respond to the game world where each planet represents a song in the player's music libarary. This work was presented at SIGGRAPH Asia 2010.

• "Surge" Master's Student: Christian Hahn
• Advisors: Dr. Paul Diefenbach, Dr. Youngmoo Kim

Brain Interfaces

"Lazy Brains"

Lazybrains is a 3D game with both a bridge between a neuro-monitoring device and a gaming engine. The game explores the influence that new methods of interfacing may have on a user's immersion in an interactive 3D world, as well as new elements of game-play that may arise. The game is to be an exploratory, level-based experience where the user guides their character through different chambers, each offering a challenge which uses feedback from the player's brain as an additional input device. Utilizing the fNIR device, the user's brain activity becomes an additional "controller". In addition to standard controls on a traditional keyboard and mouse interface, the user's biofeedback output manipulates factors in the game's environment. For example, the user's particular concentration level may manipulate the height of platforms, slow down the motion of obstacles, or even change the color of virtual puzzle pieces to complete objectives. This game was designed and created by a collaboration between a digital media senior project team and a BioMedical Engineering Ph.D. student.

• Team Members: Jordan Santell (DIGM), James Borden (DIGM), Aaron Bohenick (DIGM), Zachary Brooks (DIGM), Keneth Oum (DIGM), and Hasay Ayaz of BioMedical Engineering
• Advisor: Dr. Paul Diefenbach

"Mind Tactics"

Augmented Reality — "crAR"

crAR is an ongoing Augmented Reality research project investigating novel gameplay mixing physical and virtual objects.

• Advisor: Dr. Paul Diefenbach

Behavioral Simulations — "Transition to Teaching

The TTT website hosts training material in the form of interactive 3D simulations which place the teacher in various classroom scenarios and permit them to safely explore alternative strategies for dealing with the situations presented. Numerous scenarios are presented which represent a range from everyday to extreme situations such as inattentive students to verbal and physical abuse.

• Advisors: Dr. Paul Diefenbach, Dr. Fredericka Reisman

Educational Multitouch Games— "e-Vironments" and "FISH"

Practical Applications for Multitouch Gaming Technology in the Middle School Classroom. Continuing research from Planet Diggum, this project adapts and builds upon our previous framework, resulting in a platform for creating multitouch applications for the classroom environment. By harnessing the current strengths of X3D and developing new extensions for the engine, we have created a modular system where educators can load learning simulations on the multitouch display, including an adaptation of Justin Dobies' FISH. Each simulation can communicate with a web portal allowing greater access to students and ease of administrator for educators.

• Team Members: Will Muto, Justin Dobies
• Advisor: Dr. Paul Diefenbach

Location-based Gaming — "Inversion"

Inversion is an Alternative Reality Game engine that allows users to play in both the real and virtual worlds in competitive, team-based games. A deterministic, central server acts as a bridge which links GPS-enabled mobile devices to a 3D representation of the physical game space in which the mobile users are playing. Users playing in the physical world, called Agents, must accomplish location based tasks given to them from their Operator (a user playing from the 3D interface) via in-game text messages. Domination is a specific game-type developed for Inversion. In Domination, Agents are guided by their Operators to physical locations, which they must occupy for a brief period of time in order to capture that location for their team. Teams are capable of capturing a location currently held by another team, and locations can become briefly unobtainable if opposing players attempt to occupy the same location for too long a period. Teams accumulate points by capturing and holding waypoints. The winning team is the one who reaches the point threshold first or has the most points when the time runs out.

• Advisors: Dr. Paul Diefenbach, Dr. Frank Lee

Multitouch Gaming — "Planet Diggums"

While devices such as the Wii and Playstation's Eye (formerly EyeToy) have expanded user interface possibilities in the gaming world, there is still a traditional disconnect between the user and the physical screen image. The advance of gesture computing and the new availability of multi-point touch screens permit a novel mechanism and universal vocabulary for direct interaction with a virtual world. Planet Diggum is a multi-user god-game which uses a novel combination of finger and hand-stroke gestures. The goal of this project is to create a testbed kiosk where multiple users can interact with the system without need of training. Planet Diggum uses a Drexel-built Frusrated Total Internal Reflection multi-point touch screen, stanadards-based software, and custom media assets, and brings together teams from Digital Media, Computer Science, and Engineering to provide a unique and fun user experience.

• Team Members: Dr. Paul Diefenbach, William Muto, Matthew Smith, Chester Cunanan, Justin Dobies, Arvind Neelakantan, Sara Colucci, James Grow, Dr. Frank Lee, Louis Kratz, Dr. Ko Nishino, Craig Polakoff, Boris Block, Dan Hennessey, Zenko Klapko, David Millar, William Morgan, Electrical Engineering, Dr. Youngmoo Kim, Timothy Kurzweg, Vijay Balchandani, Eric Effinger, Jeevan Kotha, Pannha Prak, Joseph Romeo
• Advisors: Dr. Paul Diefenbach, Dr. Youngmoo Kim

Humanoid Robots — "HUBO" & "Robonova"

There has historically been a disconnect between the engineering of robotic behaviors and the artistic creation of fluid, human-like motions. This work discusses an approach which leverages 3D tools originally created for computer graphics animators and repurposes them for humanoid robotics. Initial proof-of-concept work on a simple off-the-shelf humanoid built from simple servos and brackets provides a testbed prior to migration of the techniques to a larger, complex, reactionary biped robot, the HUBO.

• Team Members: Daniel Letarte (DIGM), Joyce Tong (DIGM), Robert Ellengberg (Mechanical Engineering)
• Advisors: Dr. Paul Diefenbach (DIGM), Christopher Redmann (DIGM), Dr. Paul Oh (Mechanical Engineering)