Publications

Recently, more and more Virtual Reality (VR) and visualization solutions to support the factory layout planning process have been presented. On the one hand, VR enables planners to create cost-effective virtual prototypes and to perform virtual walkthroughs, e.g., to verify proposed layouts. On the other hand, visualization helps to gain insight into simulation results that, e.g., describe the various interdependencies between machines, such as material flows. In order to create truly effective tools based on VR and visualization, the right techniques have to be chosen and adapted to the specific problem. However, the solutions published so far usually do not exploit these technologies to their full potential.
To address this situation, we present a VR-based planning assistant that offers advanced visualization functionality that furthers the understanding of planning-relevant parameters, while also relying on established techniques. In order to realize a useful approach, the assistant fulfills three central requirements:

1. A smooth integration of the assistant into existing workflows is essential in order to not disrupt them. Consequently, existing tools need to be properly integrated and a mechanism for data exchange with these tools has to be provided.
2. Visualization is the main means of facilitating insight. Instead of only displaying factory models, advanced techniques to visualize more abstract quantities, like material flows or process chains, have to be provided.
3. VR systems vary in the degree of immersion they offer, ranging from non-immersive desktop systems to fully immersive Cave Automatic Virtual Environment (CAVE) systems. Scalability among these systems allows adapting high-end installations as well as cost-effective solutions. However, to ensure good scalability, devising a flexible system abstraction and a unified interaction concept are essential.

The purpose of this paper is twofold. First, our VR-based planning assistant is introduced and realization details are presented. Second, by presenting technical details of the used techniques we aim at providing VR and visualization guidelines for the creation or extension of VR-based factory layout planning tools.
The base for our planning assistant is an immersive VR (IVR) system in form of a CAVE. Our solution allows performing virtual walkthroughs and offers additional visualization techniques for planning relevant data.

In the recent past, efforts have been made to adopt immersive virtual reality (IVR) systems as a means for design reviews in factory layout planning. While several solutions for this scenario have been developed, their integration into existing planning workflows has not been discussed yet. From our own experience of developing such a solution, we conclude that the use of IVR systems-like CAVEs-is rather disruptive to existing workflows. One major reason for this is that IVR systems are not available everywhere due to their high costs and large physical footprint. As a consequence, planners have to travel to sites offering such systems which is especially prohibitive as planners are usually geographically dispersed. In this paper, we present a concept for integrating IVR systems into the factory planning process by means of a 3D collaborative virtual environment (3DCVE) without disrupting the underlying planning workflow. The goal is to combine non-immersive and IVR systems to facilitate collaborative walkthrough sessions. However, this scenario poses unique challenges to interactive collaborative work that to the best of our knowledge have not been addressed so far. In this regard, we discuss approaches to viewpoint sharing, telepointing and annotation support that are geared towards distributed heterogeneous 3DCVEs.

In this paper a single-point haptic rendering technique is proposed which uses a constraint-based physics simulation approach. Geometries are sampled using point shell points, each associated with a small disk, that jointly result in a closed surface for the whole shell. The geometric information is incorporated into the constraint-based simulation using newly introduced geometrically limited contact constraints which are active in a restricted region corresponding to the disks in contact. The usage of disk constraints not only creates closed surfaces, which is important for single-point rendering, but also tackles the problem of over-constraint contact situations in convex geometric setups. Furthermore, an iterative solving scheme for dynamic problems under consideration of the proposed constraint type is proposed. Finally, an evaluation of the simulation approach shows the advantages compared to standard contact constraints regarding the quality of the rendered forces.

This paper presents a software framework that supports the development of haptic-enabled physics simulations. The framework provides tools aiming to facilitate a fast prototyping process by utilizing component and flow-oriented architectures, while maintaining the capability to create efficient code which fulfills the performance requirements induced by the target applications. We argue that such a framework should not only ease the creation of prototypes but also help to effectively and efficiently evaluate them. To this end, we provide analysis tools and the possibility to build problem oriented evaluation environments based on the described software concepts. As motivating use case, we present a project with the goal to develop a haptic-enabled medical training simulator for a maxillofacial procedure. With this example, we demonstrate how the described framework can be used to create a simulation architecture for a complex haptic simulation and how the tools assist in the prototyping process.

System control is a crucial task for many virtual reality applications and can be realized in a broad variety of ways, whereat the most common way is the use of graphical menus. These are often implemented as part of the virtual environment, but can also be displayed on mobile devices. Until now, many systems and studies have been published on using mobile devices such as personal digital assistants (PDAs) to realize such menu systems. However, most of these systems have been proposed way before smartphones existed and evolved to everyday companions for many people. Thus, it is worthwhile to evaluate the applicability of modern smartphones as carrier of menu systems for immersive virtual environments. To do so, we implemented a platform-independent menu system for smartphones and evaluated it in two different ways. First, we performed an expert review in order to identify potential design flaws and to test the applicability of the approach for demonstrations of VR applications from a demonstrator's point of view. Second, we conducted a user study with 21 participants to test user acceptance of the menu system. The results of the two studies were contradictory: while experts appreciated the system very much, user acceptance was lower than expected. From these results we could draw conclusions on how smartphones should be used to realize system control in virtual environments and we could identify connecting factors for future research on the topic.

Recently, virtual reality (VR) and visualization have been increasingly employed to facilitate various tasks in factory planning processes. One major challenge in this context lies in the exchange of information between expert groups concerned with distinct planning tasks in order to make planners aware of inter-dependencies. For example, changes to the configuration of individual machines can have an effect on the overall production performance and vice versa. To this end, we developed VR- and visualization-based planning tools for two distinct planning tasks for which we present an integration concept that facilitates information exchange between these tools. The first application's goal is to facilitate layout planning by means of a CAVE system. The high degree of immersion offered by this system allows users to judge spatial relations in entire factories through cost-effective virtual walkthroughs. Additionally, information like material flow data can be visualized within the virtual environment to further assist planners to comprehensively evaluate the factory layout. Another application focuses on individual machines with the goal to help planners find ideal configurations by providing a visualization solution to explore the multi-dimensional parameter space of a single machine. This is made possible through the use of meta-models of the parameter space that are then visualized by means of the concept of Hyperslice. In this paper we present a concept that shows how these applications can be integrated into one comprehensive planning tool that allows for planning factories while considering factors of different planning levels at the same time. The concept is backed by Virtual Production Intelligence (VPI), which integrates data from different levels of factory processes, while including additional data sources and algorithms to provide further information to be used by the applications. In conclusion, we present an integration concept for VR- and visualization-based software tools that facilitates the communication of interdependencies between different factory planning tasks. As the first steps towards creating a comprehensive factory planning solution, we demonstrate the integration of the aforementioned two use-cases by applying VPI. Finally, we review the proposed concept by discussing its benefits and pointing out potential implementation pitfalls.

Four new structures based on CaCl2 and proline are reported, all with an unusual Cl–Ca–Cl moiety. Depending on the stoichiometry and the chirality of the amino acid, this metal dihalide fragment represents the core of a mononuclear Ca complex or may be linked by the carboxylate to form extended structures. A cisoid coordination of the halide atoms at the calcium cation is encountered in a chain polymer. In the 2D structures, CaCl2 dumbbells act as nodes and are crosslinked by either enantiomerically pure or racemic proline to form square lattice nets. Extensive database searches and topology tests prove that this structure type is rare for MCl2 dumbbells in general and unprecedented for Ca compounds.Four new structures based on CaCl2 and proline are reported, all with an unusual Cl–Ca–Cl moiety. Depending on the stoichiometry and the chirality of the amino acid, this metal dihalide fragment represents the core of a mononuclear Ca complex or may be linked by the carboxylate to form extended structures. A cisoid coordination of the halide atoms at the calcium cation is encountered in a chain polymer. In the 2D structures, CaCl2 dumbbells act as nodes and are crosslinked by either enantiomerically pure or racemic proline to form square lattice nets. Extensive database searches and topology tests prove that this structure type is rare for MCl2 dumbbells in general and unprecedented for Ca compounds.

The Human Brain Project is one of the largest scientific initiatives dedicated to the research of the human brain worldwide. Over 80 research groups from a broad variety of scientific areas, such as neuroscience, simulation science, high performance computing, robotics, and visualization work together in this European research initiative. This work at hand will identify certain chances and challenges for cognitive systems engineering resulting from the HBP research activities. Beside the main goal of the HBP gathering deeper insights into the structure and function of the human brain, cognitive system research can directly benefit from the creation of cognitive architectures, the simulation of neural networks, and the application of these in context of (neuro-)robotics. Nevertheless, challenges arise regarding the utilization and transformation of these research results for cognitive systems, which will be discussed in this paper. Tools necessary to cope with these challenges are visualization techniques helping to understand and gain insights into complex data. Therefore, this paper presents a set of visualization techniques developed at the Virtual Reality Group at the RWTH Aachen University.

Start-up procedures are non-routine tasks in process control, which are prone to skill decay. To mitigate skill decay, job performance aids (JPA) are used. A dynamic JPA was developed for the human-computer interface, which guides the gaze of the user by visual cues. Two gaze-guiding applications (spotlight, spotlight plus integration of information) were tested in terms of their impact in supporting the start-up performance after a three-week and six-month period of non-use (N = 46). Irrespective of the gaze-guiding format, participants with three weeks of non-use outperform participants with six months of non-use. Results show that the best indicator of skill loss is the length of the period of non-use. It is assumed that the advantage of gaze guiding plus integration of information was not shown due to the loss of interface-operating knowledge. In conclusion, gaze guiding for long periods of non-use needs to be accompanied by interface-operating knowledge.

Adaptable user interfaces (UI) have shown a great variety of advantages in human computer interaction compared to classic UI designs. We show how adaptable UIs can be built by introducing coloured Petri nets to connect the UI’s physical representation with the system to be controlled. UI development benefits from formal modelling approaches regarding the derived close integration of creation, execution, and reconfiguration of formal UI models. Thus, adaptation does not only change the physical representation, but also the connecting Petri net. For the latter transformation, we enhance the DPO rewriting formalism by using an order on the set of labels and softening the label-preserving property of morphisms, i.e., an element can also be mapped to another element if the label is larger. We use lattices to ensure correctness and state application conditions of rewriting steps. Finally we define an order compatible with our framework for the use in our implementation.

Immersive virtual environments (IVEs) are an appropriate platform for 3D data visualization and exploration as, for example, the spatial understanding of these data is facilitated by stereo technology. However, in comparison to desktop setups a lower latency and thus a higher frame rate is mandatory. In this paper we argue that current realizations of direct volume rendering do not allow for a desirable visualization w.r.t. latency and visual quality that do not impair the immersion in virtual environments. To this end, we analyze published acceleration techniques and discuss their potential in IVEs; furthermore, head tracking is considered as a main challenge but also a starting point for specific optimization techniques.

The available memory bandwidth of existing high performance computing platforms turns out as being more and more the limitation to various applications. Therefore, modern microarchitectures integrate the memory controller on the processor chip, which leads to a non-uniform memory access behavior of such systems. This access behavior in turn entails major challenges in the development of shared memory parallel applications. An improperly implemented memory access functionality results in a bad ratio between local and remote memory access, and causes low performance on such architectures. To address this problem, the developers of such applications rely on tools to make these kinds of performance problems visible. This work presents a new tool for the visualization of performance data of the non-uniform memory access behavior. Because of the visual design of the tool, the developer is able to judge the severity of remote memory access in a time-dependent simulation, which is currently not possible using existing tools.

In this paper, we present a first approach to a new test and diagnosis user interface (TDUI) as an application scenario for ambient intelligent systems in production environments, which represents one specific domain for urban areas. This method is based on a first evaluation study in 2012, identifying relevant requirements for the development of such TDUIs. It is planned to be implemented based on a formally defined Interaction Logic Layer (ILL), which connects the physical user interface layout with the test system. It allows the user to easily control and adapt the test and diagnosis application, to display and convey information characterizing the process state. Finally, this approach makes the TDUI implementation highly flexible and adaptive regarding the individual user and the requirements of the production environment.

This paper describes a metadata-based tour recommendation approach for on-site museum and outdoor visits. Recommendations are calculated utilizing spatial exhibit distribution, museum layout, user and navigation profiles, additional constraints as well as art work descriptors. We propose a service-oriented architecture for generating 3D virtual museums as well as a WebSocket layer to process sensor data during the museum visit.

A methodology for evaluating the ViMCOX metadata format for designing virtual museums is discussed. Two evaluation approaches are presented, addressing (1) design aspects of virtual museums, the completeness of the metadata from the visitor’s point of view as well as measuring the acceptance of virtual museums and (2) a qualitative survey in collaboration with museum experts to identify metadata requirements and feature sets for curator tool implementations.

Research in the field of Ambient Assisted Living (AAL) introduces assistive technologies in various contexts, e.g., medical applications for clinics, or activities in home environments. Assistive technologies have the potential to help prevent isolation and lack of social support arising from a growing number of older adults living in single person households in urban areas. In order to examine the role of such technologies and identify challenges and potentials of systems and services, we investigated existing literature of the past decade. Whereas many contributions support the individual user or social connection, only few integrate location-based information or urban structures. Future research is confronted with a multitude of challenges, e.g. considering network technologies, market uptake or adaptation, and potentials, e.g. the support or establishing of social neighborhood structures. This literature review contributes in achieving an overview of the state of the art and research areas that will draw an increasing focus according to demographic change.

Inefficient and error-prone interaction between human operators and technical systems was the reason for various catastrophic accidents in the past. User interfaces implement the communication between a human user and a technical system which is the reason why inaccurate design of user interfaces has been identified as one major factor for those errors. The use of adaptive user interfaces is one possible solution to reduce inefficient interaction by adapting the user interface to a specific user, task, or context. However, currently no self-contained formal approach exists that allows for the creation of adaptive user interfaces despite various advantages of formal methods: interaction becomes verifiable, formal methods close the gap between modeling and implementation by using executable formal languages, and they allow for using existing rewriting concepts making formal models adaptable. This paper introduces a new approach to a formal rule generation concept, which enables a flexible creation of adaptive user interfaces. This concept is based on a formal modeling and reconfiguration approach for the creation and adaptation of user interfaces. The applicability of this approach will be shown through an implementation of an adaptive user interface for adaptive automation. The main contribution of the presented work is a new concept for rule generation that is capable of adapting formally modeled user interfaces.

Simulations of geothermal reservoirs inherently contain uncertainty due to the fact that the underlying physical models are created from sparse data. Moreover, this uncertainty often cannot be completely expressed by simple key measures (e.g., mean and standard deviation), as the distribution of possible values is often not unimodal. Nevertheless, existing visualizations of these simulation data often completely neglect displaying the uncertainty, or are limited to a mean/variance representation. We present an approach to visualize geothermal simulation data that deals with both cases: scalar uncertainties as well as general ensembles of data sets. Users can interactively define two-dimensional transfer functions to visualize data and uncertainty values directly, or browse a 2D scatter plot representation to explore different possibilities in an ensemble.

Guided tours have been found to be a good approach to introducing users to previously unknown virtual environments and to allowing them access to relevant points of interest. Two important tasks during the creation of guided tours are the definition of views onto relevant information and their arrangement into an order in which they are to be visited. To allow a maximum of flexibility an interactive approach to these tasks is desirable. To this end, we present and evaluate two approaches to the mentioned interaction tasks in this paper. The first approach is a hybrid 2D/3D interaction metaphor in which a tracked tablet PC is used as a virtual digital camera that allows to specify and order views onto the scene. The second one is a purely 3D version of the first one, which does not require a tablet PC. Both approaches were compared in an initial user study, whose results indicate a superiority of the 3D over the hybrid approach.

Graphs play an important role in data analysis. Especially, graphs with a natural spatial embedding can benefit from a 3D visualization. But even more then in 2D, graphs visualized as intuitively readable 3D node-link diagrams can become very cluttered. This makes graph exploration and data analysis difficult. For this reason, we focus on the challenge of reducing edge clutter by utilizing edge bundling. In this paper we introduce a parallel, edge cluster based accelerator for the force-directed edge bundling algorithm presented in [Holten2009]. This opens up the possibility for user interaction during and after both the clustering and the bundling.

The visual discrimination of different structures in one or multiple combined volume data sets is generally done with individual transfer functions that can usually be adapted interactively. Immersive virtual environments support the depth perception and thus the spatial orientation in these volume visualizations. However, complex 2D menus for elaborate transfer function design cannot be easily integrated. We therefore present an approach for changing the color mapping during volume exploration with direct volume interaction and an additional 3D widget. In this way we incorporate the modification of a color mapping for a large number of discretely labeled brain areas in an intuitive way into the virtual environment. We use our approach for the analysis of a patient’s data with a brain tissue degenerating disease to allow for an interactive analysis of affected regions.

When dealing with free convection in a geothermal reservoir, it is preferable to detect regions of up flow, which locally increase the geothermal gradient. Free convection cells are likely to be found in a large reservoir layer (Yarragadee Aquifer), which is encountered in the entire Perth Basin, including the Perth Metropolitan Area (PMA). While the knowledge about the structure of the Perth Basin has been improved recently, the heterogeneity and spatial complexity of permeability was up till now mainly neglected. We set up a refined structural model of about 5000 km² comprising the region around the city of Perth up to a depth of 4.5 km, using an implicit modelling approach (3D GeoModeller by Intrepid Geophysics ). Based on the structural model we create a discretized numerical model for simulating fluid flow and heat transport in the Yarragadee Aquifer considering spatial heterogeneity of porosity and permeability. This heterogeneity is assessed by designing three different test cases: 1) constant porosity and permeability for the entire aquifer; 2) porosity and permeability decreasing with depth; 3) a conditional random permeability field within prescribed limits and for given correlation length. We calibrate a poro-perm relationship based on a fractal approach to the Yarragadee Aquifer, using over 100 measurement pairs of porosity and permeability from three boreholes. This data was also used for calibrating porosity decrease with depth, following Athy’s law. For the model with constant porosity and permeability, convection cells vary in size between 4 km and 6 km. Simulations with depth dependent decrease in porosity and permeability yielded a transition from conductive to convective heat transport in the Yarragadee Aquifer at a threshold permeability of around 1.7 × 10-15 m². There convection cells are much smaller, at the scale of 2.4 km to 3 km. Stochastic distributions of porosity and permeability in the Yarragadee cause the formation of convection cells to adjust to the permeability field, yielding a less distinct convection pattern. Where the Yarragadee Aquifer is in contact with overlying aquifers, stable regions of down flow develop. These in turn have a strong impact on the regional flow field and therefore on the temperature distribution. Temperatures drop to about 40 °C in 2 km depth. In order to improve the reliability of the model, as well as identification and comparison of convection cells in different simulations, we are developing a specialized visualization tool tailored to this purpose. By using tools like particle tracing and direct volume rendering, the recognition of the spatial distribution of convection cells and their change in the different cases facilitated.

Computer-assisted processing and interpretation of medical ultrasound images is one of the most challenging tasks within image analysis. Physical phenomena in ultrasonographic images, e.g., the characteristic speckle noise and shadowing effects, make the majority of standard methods from image analysis non optimal. Furthermore, validation of adapted computer vision methods proves to be difficult due to missing ground truth information. There is no widely accepted software phantom in the community and existing software phantoms are not flexible enough to support the use of specific speckle models for different tissue types, e.g., muscle and fat tissue. In this work we propose an anatomical software phantom with a realistic speckle pattern simulation to fill this gap and provide a flexible tool for validation purposes in medical ultrasound image analysis. We discuss the generation of speckle patterns and perform statistical analysis of the simulated textures to obtain quantitative measures of the realism and accuracy regarding the resulting textures.

The visualization of the progression of brain tissue loss in neurodegenerative diseases like corticobasal syndrome (CBS) can provide not only information about the localization and distribution of the volume loss, but also helps to understand the course and the causes of this neurodegenerative disorder. The visualization of such medical imaging data is often based on 2D sections, because they show both internal and external structures in one image. Spatial information, however, is lost. 3D visualization of imaging data is capable to solve this problem, but it faces the difficulty that more internally located structures may be occluded by structures near the surface. Here, we present an application with two designs for the 3D visualization of the human brain to address these challenges. In the first design, brain anatomy is displayed semi-transparently; it is supplemented by an anatomical section and cortical areas for spatial orientation, and the volumetric data of volume loss. The second design is guided by the principle of importance-driven volume rendering: A direct line-of-sight to the relevant structures in the deeper parts of the brain is provided by cutting out a frustum-like piece of brain tissue. The application was developed to run in both, standard desktop environments and in immersive virtual reality environments with stereoscopic viewing for improving the depth perception. We conclude that the presented application facilitates the perception of the extent of brain degeneration with respect to its localization and affected regions.