Nominee 2011

6-D-Vision

6D-Vision – Recognizing danger faster than humans

Dr.-Ing. Uwe Franke (Spokesperson)
Dr. rer. nat. Stefan Gehrig
Dr.-Ing. Clemens Rabe
Daimler AG, Stuttgart

(f.l.t.r.) Dr. rer. nat. Stefan Gehrig, Dr.-Ing. Uwe Franke, Dr.-Ing. Clemens Rabe

The number of road fatalities has been declining in Germany for decades. Yet many people are still killed on the road – and every death is one too many. How can driving be made even safer in the future?

Dr. Uwe Franke, Dr. Stefan Gehrig, and Dr. Clemens Rabe are convinced: intelligent driver assistance systems can dramatically reduce the number of fatalities and injuries on the road. They have developed a technology that provides entirely new ways of assisting drivers. It can be used to recognize dangers, and many accidents can be avoided by the vehicle’s rapid response. Uwe Franke heads the “Image Recognition” Department at Daimler Research and Advanced Engineering in Sindelfingen to which Stefan Gehrig and Clemens Rabe also belong.

To be able to support the driver in complex traffic situations, the vehicle’s artificial senses must reliably recognize what’s happening around it. And the electronic assistant must be able to “sense” how other road users will continue to respond – for example, whether another car will collide with the driver’s vehicle or a child at play will run into the road. Previous systems were not able to grasp many critical situations and took too long to analyze measurements and data and still react fast enough.

The nominated scientists at Daimler succeeded in vastly improving the capabilities of the technical danger recognition system. They modeled their system on the function of the human eye and brain whose abilities they were even able to surpass. The new “6D vision” technology succeeds in identifying children at play at the side of the road in less than 0.2 seconds – a human being takes more than twice as long.

To achieve this remarkable feat, a stereo camera records three dimensional images in rapid succession of the surroundings in front and next to the vehicle. An algorithm developed just for this purpose analyzes the images virtually instantaneously. By comparing the sequence of images, the system also recognizes whether and how fast objects such as cyclists, pedestrians, or cars are moving. It even works very reliably in inclement weather and at twilight.

Daimler will soon be including 6D vision systems in the Mercedes vehicles series – as the basis for innovative assistance systems that recognize pedestrians, assist drivers as they pass through blind crossings or navigate narrow highway construction sites. The research team from Sindelfingen hopes that their innovations will find widespread acceptance in the automotive industry – so that as many road users as possible are provided with an additional safety feature. In attempt to ensure that this is the case, the company plans to make the technology available to other manufacturers. 6D vision has the potential to revolutionize electronic vision not only in cars, but also in service robots that act independently. These robots are designed to serve as household helpers or to assist in caring for the infirm. To do so, they must be able to monitor their surroundings and to recognize where and how their charge moves around. The six-dimensional look at the world provided by automotive research makes it possible.

The right to nominate outstanding achievements for the Deutscher Zukunftspreis is incumbent upon leading German institutions in Science and Industry as well as foundations.

The project “6D Vision: Recognising Hazards Faster than the Human Eye” was nominated by BDI - Federation of German Industry.

more details

Resumes

Dr.-Ing. Uwe Franke

Dr.-Ing. Uwe Franke

11.03.1958
Geboren in Witzenhausen
1976
Abitur am Zeppelin-Gymnasium Lüdenscheid
1977   1983
Studium der Elektrotechnik, Schwerpunkt Nachrichtentechnik an der Rheinisch-Westfälischen Technischen Hochschule (RWTH) Aachen
1983
Diplomarbeit „Untersuchung und Programmierung universeller Störkanäle für Satellitenkanäle“
1983   1988
Wissenschaftlicher Mitarbeiter am Institut für Elektrische Nachrichtentechnik, RWTH Aachen
1988
Dissertation „Regionenorientierte Bildbeschreibung – Algorithmen und Möglichkeiten“ bei Prof. Dr. H.D. Lüke, Rheinisch-Westfälischen Technischen Hochschule (RWTH) Aachen
Seit 1989
Wissenschaftlicher Mitarbeiter der Daimler AG
Seit 2000
Leiter der Arbeitsgruppe „Bildverstehen“ der Daimler Forschung und Vorentwicklung

Weitere Tätigkeiten:

seit 2007
Mitglied im Technischen Komitee der Deutschen Arbeitsgemeinschaft für Mustererkennung (DAGM)
 
Program Chair des IEEE Intelligent Vehicles Symposium IV 2002 in Versailles
 
Mitglied in verschiedenen Programm-Komitees, Gutachter für Konferenzen und Journale
1989   2004
Lehrauftrag der Berufsakademie Stuttgart

Ehrungen:

1988
DAGM (Deutschen Arbeitsgemeinschaft für Mustererkennung) Anerkennungspreis
1988
Preis der ITG (Informationstechnische Gesellschaft)
1989
1989: Wilhelm Borchers-Medaille der der Rheinisch-Westfälischen Technischen Hochschule (RWTH) Aachen
2010
Best Paper Award des IEEE Intelligent Vehicles Symposium IV 2010, San Diego
2010
DAGM Preis (Deutschen Arbeitsgemeinschaft für Mustererkennung)

Dr. rer. nat. Stefan Gehrig

Dr. rer. nat. Stefan Gehrig

07.08.1968
Geboren in Sindelfingen
1987
Abitur am Otto-Hahn-Gymnasium, Böblingen
1988   1991
Studium der Technischen Informatik, Schwerpunkt Prozessdatenverarbeitung an der Berufsakademie Stuttgart
1991
Diplomarbeit „Erstellung eines heuristischen Verfahrens zur Trendberechnung und Visualisierung am Beispiel „Niedrigwassermenge im Bereich Kernkraftwerk Neckarwestheim““
1990   1997
Studium der Physik an den Universitäten Stuttgart, Tübingen, San Jose und Berkeley
1997
Diplomarbeit „Design and Simulated Performance of the Level 1 Trigger System for the BaBar CP-Violation Experiment“, durchgeführt am LBNL, Berkeley, University of California, CA, USA
1997   2000
Doktorand bei der Daimler AG in Kooperation mit der Eberhard Karls Universität, Tübingen
2000
Dissertation „Design, Simulation, and Implementation of a Vision-Based Vehicle Following System“ bei Prof. H. Ruder, der Eberhard Karls Universität, Tübingen
Seit 2000
Wissenschaftlicher Angestellter bei der Daimler AG in Forschung und Vorentwicklung

Weitere Tätigkeiten:

seit 2005
Mitglied im Arbeitskreis „Bildverarbeitung im Sicherheitsversuch“
 
Mitglied in verschiedenen Programm-Komitees, Gutachter für Konferenzen und Journale
Seit 2002
Lehrauftrag an der Dualen Hochschule Baden-Württemberg Stuttgart

Ehrungen:

2009
Best Paper Award der Intelligent Computer Vision Systems Conference ICVS, Lüttich

Dr.-Ing. Clemens Rabe

Dr.-Ing. Clemens Rabe

08.02.1979
Geboren in Lahn-Gießen
1989   1998
Abitur am Wirsberg-Gymnasium Würzburg
1999   2005
Studium der Informatik, Schwerpunkt Technische Informatik an der FH Würzburg-Schweinfurt
2005
Diplomarbeit „Detektion von Hindernissen vor Fahrzeugen durch Bewegungsanalyse“
2005   2008
Doktorand bei der Daimler AG in der Arbeitsgruppe „Bildverstehen“
2008   2010
Wissenschaftlicher Mitarbeiter der Christian-Albrechts-Universität zu Kiel und Externer Mitarbeiter bei der Daimler AG
Seit 2010
Wissenschaftlicher Mitarbeiter der Daimler AG
2011
Dissertation „Detection of Moving Objects by Spatio-Temporal Motion Analysis“ bei Prof. Dr.-Ing. Reinhard Koch, Universität Kiel

Weitere Tätigkeiten:

1996   2004
Diverse Nebentätigkeiten u.a. als Gesellschafter der Firma Stefan Barwanietz & Clemens Rabe GbR, Programmierer und Wissenschaftliche Hilfskraft am Fraunhofer Institut für Silicatforschung in Würzburg.
2004
Referent zum Thema Musterprogrammierung im Rahmen des Workshops „Pattern CamPP“, Würzburg
2009
Referent zum Thema GPU-Programmierung der NVidia-Konferenz, NVision“, Kalifornien

Contact

Spokesperson

Herr Dr.-Ing. Uwe Franke
Leiter des Teams "Bildverstehen" in Forschung und Entwicklung der Daimler AG
Daimler AG
GR/PAP, HPC G024-BB
71059 Sindelfingen
Tel.: +49 (0) 70 31 / 43 89 873
Fax: +49 (0) 70 31 / 43 89 264
Mobil: +49 (0) 160 / 86 70 975
E-Mail: uwe.franke@daimler.com
Web: www.daimler.com

Press

Andrea Häussler
Research, Development and Environmental Communications
Daimler AG
Epplestrasse 225
HPC 000 | 1103
70546 Stuttgart
Tel.: +49 (0) 711 / 17 93 039
Fax: +49 (0) 711 / 17 79 01 10 00
Mobil: +49 (0) 160 / 86 39 074
E-Mail: andrea.haeussler@daimler.com
Web: www.daimler.com

Benjamin Oberkersch
Global Communications Mercedes-Benz Cars (COM/MBC)
Research, Development and Environmental Communications
Daimler AG
Tel.: +49 (0) 711 / 17 93 307
Fax: +49 (0) 711 / 17 79 02 26 21
Mobil: +49 (0) 151 / 58 62 00 19
E-Mail: benjamin.oberkersch@daimler.com
Web: www.daimler.com

A description provided by the institutes and companies regardingtheir nominated projects

6D Vision: Recognising Hazards Faster than the Human Eye
Although numbers have declined steadily in recent years, around 200,000 accidents involving personal injury still occur in urban road traffic alone in Germany every year. Now cars are learning to “see” in order to comprehensively understand the often highly complex situations faced when driving. This is a key requirement if we want to further reduce the number of injuries. . Other road users in particular must be identified within a fraction of a second, and potential collisions reliably determined. A timely warning or a fast reaction can help prevent serious accidents. This is why Mercedes-Benz’s intensive development activities in the field of safety focus on developing and refining assistance systems that help the driver avoid typical accidents at crossings or with crossing traffic.

With their new research project “6D Vision”, Dr Uwe Franke, Head of the “Image Understanding” team at Daimler AG’s research and development unit, Dr Stefan Gehrig, expert in stereo vision, and Dr Clemens Rabe, expert in real-time implementation are exploring new avenues in the field of risk identification and accident prevention. Using a pair of cameras (a “stereo camera”) and new algorithms, they have been able to identify any number of moving objects such as vehicles, cyclists and pedestrians quickly and calculate their position as well as their moving direction and velocity. This makes it possible to accurately predict the risk of collision even if road users are partially occluded by other obstacles, for example if a child is running out into the road and only their head and upper body are visible.

The principle behind it: we human beings perceive our environment in three dimensions with our two eyes. By looking more closely we are also able to estimate the trajectories of objects in order to recognise potential risks, even when we are moving at speed ourselves. 6D Vision allows us to recreate this essential human capability using compact, high-performance and automotive-compliant hardware. Like its human inspiration, 6D Vision uses a pair of cameras to calculate the three-dimensional geometry of the scene in front of the vehicle. Selected image points are also tracked from image to image. As the 3D position is recalculated for each image, the direction and speed can be calculated using a suitable filter. At the same time, 6D Vision precisely calculates the path of the own vehicle in order to be able to quickly and reliably differentiate between moving and stationary obstacles. The simultaneous calculation of the position (three dimensions) and motion (additional three dimensions) of relevant image points is what gave the new process the name “6D Vision”.

Three-Dimensional Vision
Spatial perception using stereo vision has been the subject of research since the beginnings of Computer Vision in the 1970s. It is based on the fact that objects are slightly offset in the images provided by the right and left cameras (or our retinas). This disparity becomes less with larger distance. Traditional methods of stereo image processing only calculate the distance to individual image points, and have substantial weaknesses when visibility is poor. In contrast, modern stereo algorithms provide a measurement for every image point using elaborate global optimisation techniques. These could not be calculated fast enough until recently due to the large amount of processing power required.

Following extensive investigations, Dr Uwe Franke’s research team opted for the stereo algorithm “Semi-Global Matching” (SGM) presented by the Deutsches Zentrum für Luft- und Raumfahrt (the German national research centre for aeronautics and space) in 2005, which performed top in an international comparison. Daimler’s researchers improved the algorithm and refined it so that it continues to work reliably at night and in poor weather. The world’s first successful automotive implementation in an FPGA (field programmable gate array) capable of performing 25 stereo calculations a second came in 2008 in collaboration with the hardware experts of the Swiss company Supercomputing Systems AG. FPGAs are programmable logic circuits capable of processing high volumes of input data simultaneously. They offer the processing power of high-performance PCs on one square centimetre of silicon, and millions of them are used in modern televisions and smartphones. The high degree of accuracy obtained by the stereo algorithm using these processors allows objects and obstacles to be located up to a range of 50 metres. Compare this with human stereo perception, which ends at around 10-12 metres.

From Static Images to moving Images
The perception of motion that makes 6D Vision possible requires precise values for the “optic flow”, i.e. the movement of image points through the sequence of images. The algorithms described in the scientific literature are unsuitable for calculating large image point displacements, even though these are exactly what we find in the immediate surroundings of a vehicle travelling at high speed or taking a bend. The “PowerFlow” algorithm developed by Daimler in 2004 is the first to solve this problem. The algorithm calculates image displacements of any magnitude while keeping computing time low, and provides reliable results even at high vehicle speeds, in poor weather conditions and with fluctuating light levels.

With the stereo camera system developed by the researchers, a subsequent processing step combines neighbouring 6D points following the same trajectory into objects. Using intelligent filter methods, it is even possible to calculate the acceleration and turning circle of approaching vehicles. This is necessary in order to predict the trajectory of other road users in the next one or two seconds in order to be able to distinguish potential accidents from uncritical situations.
The advantages of this process are readily apparent for partially occluded objects that are close together and for poor visibility. 6D trajectory information can be used to determine the potential danger posed by every single image point, and moving objects “reveal themselves” already at image point level. The computer is able to identify the approaching danger even before the dangerous or at risk road user appears in front of the vehicle, even if the road user is only partially visible. The driver can therefore receive the best possible support immediately in complex driving situations.

Recognising Hazards - faster than the Human Eye
Children that run out into the road unexpectedly are perceived by the camera’s entire field of view within 0.2 seconds. Due to the selective visual perception of humans, an attentive person requires about 0.5 seconds for recognition, and if the driver is distracted this adds at least a further 0.5 seconds. Because it takes time to assess the situation and react, there is always the famous “moment of shock” that everyone requires to recognise a hazardous situation and take action as appropriate, regardless of age. During this one second, a vehicle traveling at 50 km/h travels a distance of about 14 metres. Practical experiments have shown that when the safety system initiates an emergency braking, the vehicle comes to a stop seven meters earlier – a crucial advantage for preventing accidents in urban traffic.

Preventing Accidents, helping People
6D Vision is a base technology that opens up new possibilities for future driver assistance systems. It makes it possible to significantly reduce the risk of accidents and take another big step towards accident-free driving. On the basis of this technology, Daimler is working on potential driver assistance systems that include braking assistance systems that are activated when cyclists or pedestrians cross the road, crossing assistance systems to prevent collisions when turning by applying the brakes and an evasion assistance system that can help prevent accidents at the last second by quickly performing a swerve maneuver. In addition to driver assistance systems in the field of safety, 6D Vision can also help reduce the burden on the driver and boost comfort, for example in stop and go traffic or narrow lanes around construction sites. The range of potential applications for 6D Vision goes well beyond its use in motor vehicles, and offers potential in all fields where dynamic environments have to be monitored and interpreted, for example in the varied applications of robotics and human/machine interaction. Another field of application is emerging in the form of 3D television, which will in future require a stereo and potentially a motion analysis in order to enable a variable impression of depth.
The fact that modern cameras are constantly becoming more powerful and less expensive and 6D Vision uses hardware components taken from the field of consumer electronics means that this technology can spread rapidly to all vehicle classes. And because 6D Vision technology will significantly reduce road traffic accidents, Daimler also intends to make this technology available to other manufacturers, as it has done in the past with ABS and ESP®.

Daimler Company Profile
The company’s founders, Gottlieb Daimler and Carl Benz, made history with the invention of the automobile in the year 1886. 125 years later, in anniversary year 2011, Daimler AG is one of the world’s most successful automotive companies. With its divisions Mercedes-Benz Cars, Daimler Trucks, Mercedes-Benz Vans, Daimler Buses and Daimler Financial Services, the Daimler Group is one of the biggest producers of premium cars and the world’s biggest manufacturer of commercial vehicles with a global reach. Daimler Financial Services provides its customers with a full range of automotive financial services including financing, leasing, insurance and fleet management. As an automotive pioneer, Daimler continues to shape the future of mobility. The Group applies innovative and green technologies to produce safe and superior vehicles which fascinate and delight its customers. With the development of alternative drive systems, Daimler is the only vehicle producer investing in all three technologies of hybrid drive, electric motors and fuel cells, with the goal of achieving emission-free mobility in the long term. This is just one example of how Daimler willingly accepts the challenge of meeting its responsibility towards society and the environment. For Daimler, this also involves an obligation to lead the way in the field of safety innovations. It is now impossible to imagine modern vehicles without Mercedes inventions such as ABS/ESP, airbags and PreSafe. Daimler sells its vehicles and services in nearly all the countries of the world and has production facilities on five continents. In addition to Mercedes-Benz, the world’s most valuable automotive brand, Daimler’s brand portfolio includes smart, Maybach, Freightliner, Western Star, BharatBenz, Fuso, Setra, Orion and Thomas Built Buses. The company is listed on the stock exchanges of Frankfurt and Stuttgart (stock exchange symbol DAI). In the year 2010, the Daimler Group sold 1.9 million vehicles and employed a workforce of more than 260,000 people; revenue totaled €97.8 billion and EBIT amounted to €7.3 billion.

The right to nominate outstanding achievements for the Deutscher Zukunftspreis is incumbent upon leading German institutions in Science and Industry as well as foundations.

The project “6D Vision: Recognising Hazards Faster than the Human Eye” was nominated by BDI - Federation of German Industry.