Introduction

Virtual Training System:

  • A training system that provides a safe and inexpensive education and training more effectively than actual training while realizing immersive experiences by computer simulation under virtual environment similar to actual manufacturing / defense / medical / disaster.
  • It is possible to overcome temporal and spatial limitations by minimizing the risk of work environment
Figure 1. MAR-based Virtual Training System

Components in Virtual Training System:

Components Issues
Instruments Fitness with virtual training systems
Safety of Instruments
Instrument Operation Manufacturing fitness with virtual training system
Accuracy of operation control program
Safety of Operating Equipment
Electrical equipment Manufacturing fitness with virtual training system
Safety of electrical equipment
Imaging System Manufacturing fitness with virtual training system
Performance of Imaging System
Network System Seamless Training Data Transmission System
Multi-modal display device Accuracy of multi-modal control
Program accuracy of multi-modal control
Safety of multi-modal devices
Content Production Object Model
Accuracy of multi-modal control
Safety of multi-modal devices
Simulation SW Accuracy of Simulation Interface
Accuracy of Training Simulation Action
Safety of Training Simulation Action
Virtual Training Evaluation System Virtual Training Evaluation Components based on Education Evaluation Tool

SC24 Components in Virtual Training System:

Components Issues Scope
Visualization 3D rendering CG/VR/AR/Simulation
Physics Engine Multi-Modal Interfaces
Special Effect -HMD/GTD/HUD
Behavior Control Movement/Status of Objects CG/VR/AR/Simulation
Behavior DB Control
Behavior Visualization
Interfaces Standard I/O devices Multi-Modal Interfaces
Experimental HW devices -Haptic/Controller
Real-time Data Processing
Sound 3D Sound Multi-Modal Interfaces
-2D/3D sound
Training evaluation Training DB Education
Training Items and Evaluation

Properties of multimodal interface in VTS:

  • Transparency
    • the condition for the human operator to perform the task in VE without perceiving the effect of presence of the mediating system generating the appropriate stumuli
  • Fidelity
    • the ability of the VE interface system to generate appropriate stimuli without being affected by disturbing effects during time.
  • Three Components
    • Vision (Rendering)
    • Audition
    • Haptics/Controller
  • Vision (Rendering): A large number of perceptual primitives.
    • Cortical visual pathways: where(dorsal) stream, what(ventral) stream
    • What: the analysis of form and color
    • Where: the analysis of motion and spatial relations
    • Perception: color, space, motion
    • HMD(head mounted display)
    • GTD(glasses type display)
  • Audition: the localization of sound in 3D spaces.
    • auditory analysis of scenes
    • Music and speech perception
    • Sound feedback
  • Haptics/Controller: exploits tactile and kinesthetic stimuli to acquire information from contact condition of the user with the external environment.
    • Kinesthetic: the sense of balance, sense of touch
    • Somato sensation
    • What stream: perceiving objects and their properties
    • Where stream: the localization of objects with respect to a user’s frame of reference.
    • Perception-action loop

Applications of multimodal interfaces in Virtual Training Systems:

Medical Training System

  • Vision
    • Projector-based augmented reality system for interventional visualization inside MRI scanners
  • Haptics/Controllers
    • Foot-based interaction: Navigation, Sensor Floor
    • Eye+Foot+Hands: multimodal interaction in the operating room
      • visualization: 2D Monitor, HMD
      • foot: navigation
      • eye: panning and zooming
      • hands: surgery instrument control)
    • Audio
      • sound: 2D/3D
  • Three Components
    • Vision (Rendering)
    • Audition
    • Haptics/Controller

Virtual Training in VR dome projection system

  • Vision
    • Multiple projection cameras
  • Haptics/Controllers
    • Gestures/Controller (glove)
  • Audio
    • sound: 2D/3D

Multimodal Interfaces in Virtual Submarine Operation Simulation

Multimodal Interfaces in Virtual Training System

Multimodal Interfaces Format Structure

Multimodal Interfaces Format Example

Output Type 3 Devices Initialization

361-763 충북 청주시 흥덕구 충대로1 충북대학교 E9동 311호(학연산) CGAC 연구실 (TEL : 043-261-2788)
Copyright 2015 CGaC (Computer Graphics & Contents Lab.) Allrights Reserved