Combining engineering and the natural sciences is one of TUM’s particular strengths, like in the field of medical technology.Credit: Andreas Heddergott
“What comes after mobile phones?” ponders Matthias Nieβner, a visual computing scientist at the Technical University of Munich (TUM) in Germany. “A device worn on your wrist that takes you into virtual reality environments instantaneously?”
Working from home and speaking to relatives and friends online has been a lifeline to millions in 2020. But video calls lack the nuances of body language and facial expression, the atmosphere of a room full of people, or the power of shared emotions.
“Replicating the real world in virtual and augmented reality is the holy grail in machine learning,” says Nieβner. “To make the experience as authentic and immersive as possible, we must capture human communication in all its forms.”
Nießner’s team has successfully developed ‘Face2Face’, a software tool that enables real-time, accurate reconstruction of faces. These virtual avatars are used to create realistic videos, even switching from one language to another for movie dubbing, for example.
“Your avatar could be giving an online talk while you’re in a meeting,” says Nieβner. “Face2Face is a precursor to building realistic holograms.”
His team is also aware of the potential for misuse of these technologies, such as deepfake internet content. Hence, in tandem, they have created FaceForensics, a machinelearning algorithm trained to rapidly identify fake videos.
“My research is driven by the ways technology affects society,” says Nieβner. “As computer scientists, we have an ethical responsibility to understand the full impact of our creations.”
Nieβner’s group benefits from working closely with leading tech firms in Munich, including Google. The US company opened a research centre in Munich in 2016, and two years later entered into a long-term partnership with TUM. And it’s not the only one.
Munich has a long tradition as a science and innovation hub, hosting several institutes of the Max Planck Society, the Helmholtz Association and the Leibniz Association as well as technology companies such as BMW and numerous start-ups. This dynamic ecosystem has risen to prominence: Munich was recently heralded by the European Commission as the best ICT hub in Europe, beating London and Paris.
Assistant robot GARMI, developed by the Munich School of Robotics and Machine Intelligence, could help elderly people live a self-determined life. Credit: TUM; Astrid Eckert
Cultivating interdisciplinary research
TUM is at the heart of this ecosystem. “Besides disciplinary strength, interdisciplinary collaboration is a core component of our success,” says Thomas Hofmann, President of TUM. This philosophy also applies within the university, which boasts an impressive spectrum of subjects, spanning engineering, the natural sciences, medicine, economics, and social and educational sciences. “We leverage this enormous potential by intensively linking the different fields,” says Hofmann.
TUM’s Integrative Research Centers bring together scientists to investigate wide-reaching research fields like energy or robotics. At the Munich School of BioEngineering, for example, medical experts, biologists, physicists and engineers collaborate to improve disease diagnostics and treatments. One recent breakthrough is the ‘dark field’ X-ray method. The technique highlights near transparent structures that conventional X-rays miss, such as the fine, web-like structures formed in the lungs during the earliest stages of pneumonia.
Matthias Nießner develops software tools to create virtual avatars in real time, but also to rapidly identify deepfake internet content. Credit: TUM; Astrid Eckert
“We particularly encourage creativity and new initiatives,” stresses Hofmann. Teams can apply for specific funding to work in TUM Innovation Networks on high-risk, high-gain projects.
TUM’s strength in linking natural and technical sciences is also reflected in Germany’s national Clusters of Excellence programme. The current cohort includes the SyNergy Cluster, which combines systems biology, bioinformatics and neurosciences to investigate the onset of neurological diseases like Alzheimer’s.
Many tech companies, such as Siemens and General Electric, have settled on TUM’s campus, giving researchers the benefit of diverse, high-quality facilities and providing mutual inspiration. The German multinational software company SAP is the latest to build a state-of-the-art R&D centre here, which will house around 700 staff from SAP and TUM when it opens in 2023. The building itself is designed to foster collaboration, with plenty of meeting, innovation and exhibition spaces.
“This is a unique venture for SAP; locating at TUM was the obvious choice for us to synergize fully with the Munich ecosystem,” says Stefan Wagner, Managing Director of SAP Labs Munich. “We’re aiming for mutual knowledge transfer, which includes supporting lectures, seminars and new education formats, and developing wide-reaching projects in applied research.”
Boosting brilliant young minds
Besides the lab infrastructure, TUM has also revamped its career development. Its tenure-track programme is unique in Germany, and gives early career researchers the independence to pursue their own goals with the security of working towards a permanent post.
“This encourages young people to work to the best of their abilities, without the distraction or threat of having to move every few years,” says Hofmann. “We’re offering a fresh perspective on how research careers can progress.”
Since the tenure track programme began in 2012, the university has recruited 140 professors, 40% of whom are women, and 50% from outstanding institutions abroad. Nieβner, who came from Stanford University, cites the tenure track programme as a major reason for his decision to join TUM.
“With more young professors joining faculty, it really drives research forward, and in a lively way,” says Nieβner. “This is reflected in our publications and citations; TUM is climbing the metrics fast. It is a joy to work with such creative, focused people.”