The Internet has already democratized access to information worldwide, allowing every person to obtain almost any information. But what about the learning of skills? Instructrional videos on specific skills can help to a certain extent, but an expert showing something in person and giving feedback provides the best learning results. But what if the expert doesn’t even live close enough to take lessons with ease? Or even on another continent? This is where CeTI tries to provide an answer based on research, so that skills can be learned even over extremely long distances.
We advocate establishing the Centre for Tactile Internet with Human-in-the-Loop (CeTI) at Technische Universität Dresden (TUD) to achieve significant breakthroughs for enhancing collaborations between humans and machines or, more generally, cyber-physical systems (CPS) in real, virtual, and remote environments. CeTI’s vision is to enable humans to interact with co-operating CPS over intelligent wide-area-communication networks to promote equitable access to remote work and learning environments for people of different genders, ages, cultural backgrounds, or physical limitations. Thus, going far beyond the current state of the art, CeTI democratises the access to skills and expertise the same way as the current Internet has democratised the access to information.
Internet of Skills
The vision of the Internet of Skills development is to provide people around the world with access to scientifically based training opportunities using innovative Internet technologies. With this vision in mind, interdisciplinary teams at CeTI consisting of engineers, computer scientists and psychologists are researching how to capture the sensory impressions generated during movements and use them to design technically mediated feedback strategies.
Music
Capitalising on outstanding expertise within TU Dresden and associated institutions in the fields of communication, robotics, electrical engineering, computer science, psychology, neuroscience, and medicine, the innovations of CeTI are reflected in its structural and research objectives. CeTI conducts multidisciplinary research to (i) advance the understanding of the complexities and dynamics of human goal-directed perceptions and actions from the psychological and medical perspectives, (ii) develop novel sensor and actuator technologies that augment the human mind and body, (iii) develop fast, bendable, adaptive, and reconfigurable electronics, (iv) create intelligent communication networks that connect humans and CPS by continuously adapting and learning to provide low latency, as well as high levels of resilience and security, (v) design new haptic coding schemes to cope with the deluge of information from massive numbers of body sensors, (vi) design online learning mechanisms as well as interface solutions for machines and humans to predict and augment each other’s actions, and (vii) to evaluate the above solutions as well as to engage the general public about the societal and ethical changes and new opportunities the new technologies will bring by means of use cases in medicine (context-aware robotic assistance systems in medical environments), industry (co-working industrial space), and the Internet of Skills (education and skill acquisition for the general public).
This scenario addresses learning to play piano as a proof of concept, whereby an expert musician trains others remotely through the Internet of Skills in real-time. In order to achieve this remote training, novel wearable actuators in form of smart eGloves with haptic features and low latency are used. The piano teacher, wearing an eGlove with self-sensing capabilities or using a digital piano, would control each finger of the student in real time, in contrast to explaining each step verbally or demonstrating the movement visually. The student wears an eGlove with an integrated haptic force-feedback system. This would enable the student to experience the correct finger movements and it would be easier to reproduce the correct behaviour later on.
Demonstrator Sound and Science
The demonstrator illustrates that it is possible to transmit the movements of the piano player to a digital avatar in real time. Accurate capture of expert movements enables the further steps in which these movements are transmitted to the learner through a haptic force-feedback system, thus facilitating reproduction. The demonstrator was shown on the “Sound and Science” event as part of the Dresden Music Festival.
Sports
Capitalising on outstanding expertise within TU Dresden and associated institutions in the fields of communication, robotics, electrical engineering, computer science, psychology, neuroscience, and medicine, the innovations of CeTI are reflected in its structural and research objectives. CeTI conducts multidisciplinary research to (i) advance the understanding of the complexities and dynamics of human goal-directed perceptions and actions from the psychological and medical perspectives, (ii) develop novel sensor and actuator technologies that augment the human mind and body, (iii) develop fast, bendable, adaptive, and reconfigurable electronics, (iv) create intelligent communication networks that connect humans and CPS by continuously adapting and learning to provide low latency, as well as high levels of resilience and security, (v) design new haptic coding schemes to cope with the deluge of information from massive numbers of body sensors, (vi) design online learning mechanisms as well as interface solutions for machines and humans to predict and augment each other’s actions, and (vii) to evaluate the above solutions as well as to engage the general public about the societal and ethical changes and new opportunities the new technologies will bring by means of use cases in medicine (context-aware robotic assistance systems in medical environments), industry (co-working industrial space), and the Internet of Skills (education and skill acquisition for the general public).
In this scenario, a dynamic multimodal set-up in the domain of sports, specifically surfing, is chosen. Due to the dynamic and complex relevant motion sequences, mainly balancing on the board in the water, no supervised human haptic interaction is possible. Therefore, the input data has to be recorded and processed using machine learning methods and stored prior to the learning procedure: an expert / trainer wears an eBodySuit system, equipped with tactile and position sensing features to generate the complex movement specifications as a reference. For the learning procedure, the human trainee wears a multimodal eBodySuit (with auditory, visual, and haptic feedback) linked to the Internet of Skills, that teaches the learner the precise dynamic and complex motion sequences.
Surfboard Demonstrator
The demonstrator captures the movements of the person on the surfboard and transfers them to a digital avatar, which copies the movements in quasi-real time. On the screen, the surfers get immediate feedback through green and red colored areas whether they are in the right position on the board. The surfboard is built in such a way that it causes the slightest movements to wobble, which is supposed to resemble the real surfing experience. The demonstrator was showcased on various occasions, e.g. at the Juniordoktor event.
Collaborative Research Projects
Capitalising on outstanding expertise within TU Dresden and associated institutions in the fields of communication, robotics, electrical engineering, computer science, psychology, neuroscience, and medicine, the innovations of CeTI are reflected in its structural and research objectives. CeTI conducts multidisciplinary research to (i) advance the understanding of the complexities and dynamics of human goal-directed perceptions and actions from the psychological and medical perspectives, (ii) develop novel sensor and actuator technologies that augment the human mind and body, (iii) develop fast, bendable, adaptive, and reconfigurable electronics, (iv) create intelligent communication networks that connect humans and CPS by continuously adapting and learning to provide low latency, as well as high levels of resilience and security, (v) design new haptic coding schemes to cope with the deluge of information from massive numbers of body sensors, (vi) design online learning mechanisms as well as interface solutions for machines and humans to predict and augment each other’s actions, and (vii) to evaluate the above solutions as well as to engage the general public about the societal and ethical changes and new opportunities the new technologies will bring by means of use cases in medicine (context-aware robotic assistance systems in medical environments), industry (co-working industrial space), and the Internet of Skills (education and skill acquisition for the general public).
Teaching and learning skills over a long distance requires different technologies that are being tested in different research projects that complement and build on each other.
Textile Sensors and Actuators
Textile technology at CeTI deals for example with the functionalization of textiles and knitting technology. It gives a wide range of possibilities to enhance textiles for different applications, which is constantly expanding. Research is also focused on textile-based sensors and actuators and their conductive behavior. These sensors and actuators bring many advantageous properties concerning wearables. Wearables, smart textiles and devices are the hardware basis for software development and psychological examinations. The CeTI glove and Smart Kinesiotape are the result of research where, textile-physical and electromechanical investigations come together.
Human Motion Capturing
For an exact capturing of human motion the precision and latency of current 3D acquisition and tracking approaches are still insufficient. Immersive Modelling is a promising way to label, segment, and extract semantic properties of 3D objects in VR/AR. The goal is to bridge the physical world and the virtual world. Also providing AR/ VR environment for designing and implementing natural interaction in VR between humans and machines. It also takes a deep understanding of how humans perceive/recognize individuals by their body motions. In addition, the research examines whether humans and machines recognize human motion in the same way or whether they look for different features. Therefore motion data is collected via IMUmotion capture suits and AR/VR headsets. The anonymization of the collected behavioral biometric data is an important issue. By understanding the privacy risks of data collection, a privacy friendly pose estimation can be established.
Tactile Feedback
Feedback from an external source (augmented feedback) is a potent tool for promoting motor skill learning. Feedback can direct a learner towards efficient and task-relevant movement patterns/solutions. An example is lifting a heavy box, where a healthy execution with a straight back and bent legs is important. In this case passive tactile and auditory feedback can reduce spine flexion if the movement is performed incorrectly. Prescriptive feedback on movement processes is superior to feedback on movement outcome.
Room Leaders
