
Nominee 2024
Democratization of Generative AI

This is made possible by the use of generative AI, artificial intelligence, which has become increasingly powerful in recent years. However, the increase in the performance of AI models is largely due to the exponential scaling of AI model size, thus entailing the computing power required for applying the AI to also increase at a prohibitive rate.

With the innovative and powerful AI model “Stable Diffusion” developed at LMU, it is now possible to run complex AI applications on conventional user hardware or even on an ordinary smartphone.
Generative AI learns the semantic detail of a scene by aiming at synthesizing content such as images. The goal is to generate local details of an image and the big picture, the meaningful context, as well as possible.

For an AI to be able to learn these relationships from training data, it usually must be very large, i.e. consist of a large artificial neural network. But that's exactly the catch. Such an artificial neural network requires powerful, expensive computing capacities in the application.

An innovative approach was found to minimize the storage and computing costs: Instead of describing images directly as a set of pixels, a new, efficient image description language for local image regions was first learned. What makes up the image of a dog? Ears, eyes and the fur on the various parts of the body should be consistent with each other. However, it is not necessary to know how each individual coat hair is curved in order to create a good image of a dog. Nevertheless, we can recognize whether the coat is short or long, smooth or curly. Local details are described efficiently, then the long-range context is captured. Stable Diffusion not only sees trees, but also the forest.

Stable Diffusion then learns a robust representation of objects or scenes by first adding noise to the image and then reconstructing it. This noise is removed in many small steps that gradually make more and more image details appear. The AI must therefore learn a robust representation of the image semantics in order to capture the global context and thus reconstruct the original as well as possible.
More Details
Resume
Prof. Dr. Björn Ommer
28.10.1979
Born in Cologne, Germany
1998 – 2003
Graduate studies in Computer Science with minor in Physics
Rheinische Friedrich-Wilhelms Universität Bonn2003
Diplom in Informatik (summa cum laude)
minor: Physics, Rheinische Friedrich-Wilhelms Universität Bonn2003 – 2007
Ph.D. student and teaching and research assistant, Inst. of Comp. Science, ETH Zurich
2007
Dr. sc. ETH Zürich, Schweiz, Thesis awarded with the ETH Medal
2008 – 2009
Postdoctoral Scholar, Computer Vision, Dept. of EECS, University of California, Berkeley, USA
2009 – 2013
Assistant Professor for Scientific Computing (W1), Heidelberg University, Interdisciplinary Center for Scientific Computing
Since 2011
present Director of the Heidelberg Collaboratory for Image Processing (HCI)
2013 – 2021
Full professor (W3) for Scientific Computing, Heidelberg University at the HCI/IWR, Department of Mathematics and Computer Science and cooptation at the Departments of Philosophy and Physics
2016 – 2021
Chairman (~acting dean) of the MSc Scientific Computing
2016 – 2021
Director of the Interdisciplinary Center for Scientific Computing (IWR) Heidelberg
Since 2021
Full professor (W3) & Head of Computer Vision & Learning Group, LMU Munich
Since 2024
Member of the Bavarian AI council
Patents
M.N.M. Afifi, M.S. Brown, K. Derpanis, and B. Ommer: Network for Correcting Overexposed and Underexposed Images, US Patent Application, 2020
Publications
More than 170 publications in den renown international Zeitschriften und Conference-Proceedings regarding Computer Vision and Machine Learning
Research interests: All aspects of semantic image and video understanding based on (deep) machine learning; esp.: generative approaches for visual synthesis (e.g. Stable Diffusion, VQGAN), invertible deep models for explainable AI, deep metric and representation learning, and self-supervised learning paradigms and their interdisciplinary applications in the digital humanities and neurosciences.
Associate Editor, Senior Area Chair and Program Chair of renowned journals and conferences about Computer Vision and AI (e.g. IEEE Transactions on Pattern Analysis and Machine Intelligence, NeurIPS, CVPR, ICCV, GCPR)
Honors and Awards
PhD-Thesis awarded with ETH Medal
Fellow of ELLIS Society
Falling Walls Science Breakthrough of the Year 2023 in Engineering and Technology: finalist
Best Paper awards on conferences on Computer Vision and AI
Dr.-Ing. Anna Lukasson-Herzig
21.01.1975
Born in Guttentag, Poland
1996 – 2001
Degree in metallurgy and materials engineering at the RWTH Aachen
2001 – 2005
Research assistant at the BFI - VDEh Research Institute, Düsseldorf
2007
PhD in engineering at the RWTH Aachen
2005 – 2014
Employed at Boston Consulting Group GmbH; last as ‘Principal’, projects in various industries, focus on manufacturing industry, several months of assignments in Brazil, Denmark, USA, and India
Since 2014
Preparation and foundation (2015) of nyris GmbH, serves as managing director
Further activities
Since 2018
Founding member of the national KI Bundesverbandes e.V.
Since 2021
Economic Advisory Council of the Green Party NRW
Scholarships
1997 – 2001
Scholarshipf VDEh, Düsseldorf
Patents
2005
Method and device for rolling a metal strip, EP1786577B1, withdrawn due to a lawsuit by Siemens AG
Publications
2008
“Optimization of steel strip geometry to reduce camber formation in hot wide strip mills”
Honors and Awards
2001
Springorum Medal of the RWTH Aachen University
Otto Junker Award of the Otto Junker GmbH,
VDEh Alumni Award2017
nyris selected for the first batch of the German Google StartUp Programme and the first batch of the German Microsoft Accelerator
2018
Forbes names nyris as one of the ‘100 most innovative start-ups in Germany’
2021
nyris receives a multi-million euro grant from the European Innovation Council for the development of the synthetic data pipeline and completes the project in 2023 with the highest rating of ‘excellent’
Contact
Press
Sascha Lindemann
nyris GmbH
Max-Urich-Str. 3
13355 Berlin
Mobile: +49 (0) 170 / 22 77 224
E-Mail: press@nyris.io
Web: www.nyris.io
Spokesperson
Prof Dr. Björn Ommer
Computer Vision & Learning Group
Ludwig-Maximilians-Universität München
Akademiestr. 7
80799 München
Phone: +49 (0) 89 / 21 80 73 431
E-Mail: b.ommer@lmu.de
Web: https://ommer-lab.com/people/ommer/
A description provided by the institutes and companies regarding their nominated projects
Stable Diffusion und nyris
The Ommer Chair at LMU Munich has developed an approach to democratising generative AI known as Stable Diffusion. Generative AI has quickly become a widely used enabling technology that is applied practically everywhere. Although its performance has continuously increased, its direct practical usability for users has decreased, as the gain in performance was mainly due to an excessive growth of the complexity of the models and the computing power required. As a result, generative AI quickly reached a point where the models could only be developed and operated by the largest (mostly American) technology companies. The possibilities for users and developers to use these models locally, without transferring their data, and to develop them further themselves have decreased more and more.
The Ommer Chair at LMU Munich recognised a critical problem: control over generative AI, which has become a widespread catalyst for new technologies, was in the hands of a few foreign companies. The goal was therefore to democratise generative AI and make the models powerful and at the same time compact enough for conventional, affordable user hardware.
To achieve this, the chair developed the innovative approach of stable diffusion, which was published in the most prestigious AI proceedings. Stable diffusion learns an efficient and compact description language for content, which focuses the AI on the essential details. Furthermore, the AI can implement natural language instructions. This resulted in an AI that is powerful and, at the same time, easy to use without computer knowledge. To promote democratisation, the software was made open source and is not patented. Already in its first two months, millions of users used the AI, which also formed the basis for many other projects, company start-ups and further developments, such as those of nyris GmbH.
nyris is a visual search platform that gives people a more natural way to find what they are looking for. Based in Berlin and Düsseldorf, nyris serves leading companies in more than 50 countries. Founded in 2015, nyris is financially supported by experienced deep tech investors such as the European Investment Bank, eCapital, Axel Springer and FlixFounders, as well as two long-standing customers, TRUMPF and IKEA.
The nyris technology is based on the use of 3D data from CAD files and their transfer files as input for the stable diffusion model to generate high-quality synthetic spare parts images for training and indexing the visual search engine. nyris is the only provider that can derive the necessary data completely from CAD data and index it for use in AI technologies. This capability puts nyris in a leading position in the market, as most OEMs and their suppliers have limited master data, which is a major obstacle to the use of AI in industrial applications.
The nyris technology enables machine operators to reduce unplanned downtime. By giving field engineers access to nyris visual search of their spare parts, they can identify parts from vast product catalogues rapidly and accurately, retrieve information and complete maintenance tasks. Tests show that the time to identify a part can be reduced from roughly 20 minutes on average to a few seconds. The nyris solution helps to minimize follow up visits by enabling these technicians to identify the correct spare part on the spot. Current processes involving multiple 1st and 2nd level service agents can be significantly streamlined and therefore operation costs can be reduced. Sending around emails with product photos or returning to base for checking with your colleagues or product catalogues manually is now a thing of the past.
The nyris team works closely with the Ommer Chair to further develop the Stable Diffusion model and expand its application. The long-term goal is to massively extend the currently very complex human-machine communication to the highly efficient visual level. Machines, like humans, are already capable of capturing and interpreting images very quickly. This is a huge potential that needs to be exploited.
The right to nominate outstanding achievements for the Deutscher Zukunftspreis is incumbent on leading German institutions in science and industry as well as foundations.
The project "Democratization of Generative AI – Stable Diffusion from Development to Practice” was submitted by Bundesministerium für Bildung und Forschung.
