Abstract
Maintaining or repairing satellites in orbit is a delicate task that requires expert skills. The planning, training and analysis of on-orbit servicing (OOS) missions performed by astronauts or through remote operation using a robot is often time consuming and costly. Virtual Reality (VR) enables simulation and training in a flexible and safe environment. This paper describes an interactive real-time environment that supports a number of OOS tasks within an immersive VR environment. The system simulates the dynamic and kinematic behaviour of satellite components and provides photo-realistic visualization of satellite parts and the space environment. It integrates user interaction with haptic force feedback through a bi-manual haptic human machine interface, as well as simulates and interfaces to a humanoid robot for tele-operation.In order to provide a realistic experience at interactive frame rates, we propose a distributed system architecture, where the load of computing the physics simulation, haptic feedback and visualization of the complex scene is transferred to dedicated machines. The modular architecture is designed to allow the inclusion of further simulation processes. Several mechanisms for reducing the communication traffic have been implemented. This paper gives an overview of the system architecture, outlines the software implementation and documents an evaluation of the real-time performance of our system in detail. We describe how system performance was measured in terms of simulation timings and distribution load, as well as report on latencies at several stages. Results show that our distributed system is capable of providing visual and haptic feedback at high frame rates required for user interaction with end-to-end latencies of less than 8 ms and 3 ms, respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Artigas J, Kremer P, Preusche C and Hirzinger G (2006). Testbed for telepresent on-orbit satellite servicing. In Proceedings of the Human-centered Robotic Systems Conference (HCRS); Munich, Germany.
Boening A and Bräunl T (2007). Evaluation of real-time physics simulation systems. In Proceedings of the 5th international conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia (GRAPHITE); Perth, Australia, pp 281–288.
Currie NJ and Peacock B (2002). International space station robotic systems operations—A human factors perspective. In Proceedings of the Human Factors and Ergonomics Society 46 (1): 26–30.
Ellery A, Kreisel J and Sommer B (2008). The case for robotic on-orbit servicing of spacecraft: Spacecraft reliability is a myth. Acta Astronautica 63 (5–6): 632–648.
Ellis SR, Young MJ, Adelstein BD and Ehrlich SM (1999). Discrimination of changes of latency during voluntary hand movement of virtual objects. In Proceedings of the Human Factors and Ergonomics Society. 43 (22): 1182–1186.
Gamma E, Helm R, Johnson R and Vlissides JM (1994). Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley Professional: Boston, MA.
Hinterseer P, Hirche S, Chaudhuri S, Steinbach E and Buss M (2008). Perception-based data reduction and transmission of haptic data in telepresence and teleaction systems. IEEE Transactions on Signal Processing 56 (2): 588–597.
Hirzinger G, Brunner B, Dietrich K and Heindl J (1994). ROTEX–The first remotely controlled robot in space. In Proceedings of IEEE Internationl Conference on Robotics and Automation (ICRA’94); San Diego, CA.
Hirzinger G, Sporer N, Albu-Schäffer A, Krenn R, Pascucci A and Schedl M (2002). DLR’ torque-controlled light weight robot III—Are we reaching the technological limits now? In Proceedings of IEEE Internationl Conference on Robotics and Automation (ICRA2002); Washington DC, pp 1710–1716.
Hulin T, Sagardia M, Artigas J, Schaetzle S, Kremer P and Preusche C (2008). Human-scale bimanual haptic interface. In Proceedings of the 5th International Conference on Enactive Interface; Pisa, Italy.
Hummel J, Wolff R, Gerndt A and Kuhlen T (2012a). Comparing three interaction methods for manipulating thin deformable virtual objects. In Proceedings of IEEE Virtual Reality; Costa Mesa, CA, pp 139–140.
Hummel J, Wolff R, Dodiya J, Gerndt A and Kuhlen T (2012b). Towards interacting with force-sensitive thin deformable virtual objects. In Proceedings of EGVE – EuroVR Joint Virtual Reality Conference of ICAT (JVRC’12); Madrid, Spain, pp 17–20.
Hummel J, Wolff R, Stein T, Gerndt A and Kuhlen T (2012c). An evaluation of open source physics engines for use in virtual reality assembly simulations. In Lecture Notes in Computer Science Vol. 7432. Springer: Berlin Heidelberg, pp 346–357.
Karkee M, Steward BL, Kelkar AG and Kemp II ZT (2011). Modeling and real-time simulation architectures for virtual prototyping of off-road vehicles. Virtual Reality 15 (1): 83–96.
Landzettel K et al (2006). ROKVISS verification of advanced light weight robotic joints and tele-presence concepts for future space missions. In Proceedings of the 9th ESA Workshop on Advanced Space Technologies for Robotics and Automation (ASTRA), ESTEC; Noordwijk, Netherlands.
Mark WR, Randolph SC, Finch M, van Verth JM and Taylor II R (1996). Adding force feedback to graphics systems: Issues and solutions. In Proceedings of ACM SIGGRAPH Computer Graphics and Interactive Techniques; New Orleans, Louisiana, USA, pp 447–452.
Marsh J, Glencross M, Pettifer S and Hubbold R (2006). A network architecture supporting consistent rich behavior in collaborative interactive applications. IEEE Transactions on Visualization and Computer Graphics 12 (3): 405–416.
Meehan M, Razzaque S, Whitton MC and Brooks Jr FP (2003). Effect of latency on presence in stressful virtual environments. In Proceedings of IEEE Virtual Reality, Los Angeles, CA, pp 141–148.
Ott C et al (2006). A humanoid two-arm system for dexterous manipulation. In Proceedings of IEEE-RAS International Conference on Humanoid Robots; Genova, Italy, pp 276–283.
Renz M, Preusche C, Potke M, Kriegel H and Hirzinger G (2001). Stable haptic interaction with virtual environments using an adapted voxmap-pointshell algorithm. In Proceedings of the Eurohaptics, Birmingham, UK, pp 149-154.
Roberts D, Marshall D, McLoone S, Delaney D and Aspin R (2005). Exploring the use of local inconsistency measures as thresholds for dead reckoning update packet generation. In Proceedings of IEEE Distributed Simulation and Real-Time Applications; Montreal, Canada, pp 95–102.
Rupp T, Boge T, Kiehling R and Sellmaier F (2009). Flight dynamics challenges of the german on-orbit servicing mission DEOS. In Proceedings of the 21st International Symposium on Space Flight Dynamics; Toulouse, France.
Sellmaier F, Boge T, Spurmann J, Gully S, Rupp T and Huber F (2010). On-orbit servicing missions: Challenges and solutions for spacecraft operations. In SpaceOps 2010 Conference, American Institute of Aeronautics and Astronautics; Huntsville, Alabama, USA.
van Reimersdahl T, Kuhlen T, Gerndt A, Henrichs J and Bischof C (2000). Vista: A multimodal, platform-independent vr-toolkit based on wtk, vtk, and mpi. In Proceedings of the 4th International Immersive Projection Technology Workshop (IPT); Ames, Iowa.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wolff, R., Preusche, C. & Gerndt, A. A modular architecture for an interactive real-time simulation and training environment for satellite on-orbit servicing. J Simulation 8, 50–63 (2014). https://doi.org/10.1057/jos.2013.10
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1057/jos.2013.10