Integrating Web-Based 3D Modeling with Leap Motion Tracking and Oculus Rift VR
Kyle Davis - SURF 2015
The Objective
Create web-based applications to demonstrate how to integrate 3D and Virtual Reality with the web
Develop tools to enable the use of 3D graphics with the Oculus and Leap Motion
Overview
- Introduction to X3DOM, why it should be used and what it's good for
- Introduction to the Leap Motion tracker and how it ties in with web-based 3D graphics
- Introduction to the Oculus Rift and what we can do with it
- What I created
- Conclusion
The Purpose
- Integrating 3D with the web - takes advantage of web development tools to create web graphics
- Allows tools designed for web applications to work on graphics
- Does not require any additional plug-ins or software
- All you need is a link to the page and it works
- Manipulates 3D objects within the HTML DOM.
What is the DOM?
- Short for "Document Object Model"
- The data structure that a web browser uses to create the web page
- Paragraphs, fonts, styles, etc. are all information in the DOM
- The technology we use puts graphical objects into the DOM.
Tools Used:
- X3DOM
- HTML 5
- Javascript
- JQuery/Ajax
- Python
- Leap Motion
- Oculus Rift
Part I
Manipulating and mapping videos in X3DOM for VR
What is X3DOM?
- An open-source framework and runtime environment for 3D graphics
- Allows the integration of graphical (X3D) elements into HTML5 DOM tree
- Can manipulate the 3D content by adding, removing, or changing DOM elements.
Learning Web-Based 3D
Manipulating Models
Mapping Videos onto 3D Objects
Part II - Leap Motion and Virtual Reality
What is Leap Motion?
- Infrared camera sits in front of keyboard
- Provides computer interaction without keyboards or mice - only requires use of your hands' motions
- Can recognize a series of gestures that can be made with hands/pointers
Using Leap Motion
Why Use Leap Motion?
- When Oculus is being used, you can't see your environment (no keyboard/mouse)
- Needed to interact in a virtual world
- Interactivity
- Simplicity
- Inexpensive
Using Leap Gestures
Difficulties with Leap Motion
- Less precision
- Harder to control
- False positives (tracker will mistake one gesture for another)
Getting Leap Motion to Interact with X3DOM
- Translating Leap Motion data into something that the
X3D objects can understand - Determining how the two can be used together realistically (no wonky/confusing controls)
- Switching between normal and Leap Motion controls smoothly
Part III
Oculus Rift
What is the Oculus Rift?
- Virtual Reality headset with a separate display for each eye
- With proper graphics, user can see 360 stereo
- Reacts to the user's head position & orientation
Oculus and X3DOM In Action
X3DOM and Virtual Reality Tools
X3D/Oculus/Leap Motion Interaction
Tool (I created) to bring in any .x3d file and have it display in stereo with the Oculus rift - objects can be manipulated by sliders & Leap Motion
.X3D Files to Oculus
- Python script - takes any file with an X3DOM scene and creates a version with user-indicated attributes and Oculus Rift capabilities
Adding Oculus to X3D
Conclusion
- What did I learn and accomplish?
- How to use the web browser to manipulate 3D graphics
- Designed Leap gesture-controlled X3D objects
- Integrated Oculus Rift with X3D objects
- JavaScript (Ajax and jQuery)/HTML/Python
- Created several tools that will be useful for X3DOM developers
Acknowledgements
Thanks to:
- My mentor, Sandy Ressler
- My academic advisor, Deborah Knox
- The NIST SURF Directors and SURF Program Coordinator, Dr. Brandi Toliver
- NIST, for letting me come and and have a great research experience this summer
Questions!
Integrating Web-Based 3D Modeling with Leap Motion Tracking and Oculus Rift VR
Kyle Davis - SURF 2015