Self-Organizing Maps for Intuitive Gesture-Based Geometric Modelling in Augmented Reality

Modelling three-dimensional virtual objects in the context of architectural, product and game design requires elaborate skill in handling the respective CAD software and is often tedious. We explore the potentials of Kohonen networks, also called self-organizing maps (SOM) as a concept for intuitive 3D modelling aided through mixed reality. We effectively provide a computational "clay" that can be pulled, pushed and shaped by picking and placing control objects with an augmented reality headset. Our approach benefits from combining state of the art CAD software with GPU computation and mixed reality hardware as well as the introduction of custom SOM network topologies and arbitrary data dimensionality. The approach is demonstrated in three case studies.