A brief description provided by the institutes and companies regarding
their nominated projects
6D Vision: Recognising Hazards Faster than the Human Eye
Although numbers have declined steadily in recent years, around 200,000 accidents involving personal injury still occur in urban road traffic alone in Germany every year. Now cars are learning to “see” in order to comprehensively understand the often highly complex situations faced when driving. This is a key requirement if we want to further reduce the number of injuries. . Other road users in particular must be identified within a fraction of a second, and potential collisions reliably determined. A timely warning or a fast reaction can help prevent serious accidents. This is why Mercedes-Benz’s intensive development activities in the field of safety focus on developing and refining assistance systems that help the driver avoid typical accidents at crossings or with crossing traffic.
With their new research project “6D Vision”, Dr Uwe Franke, Head of the “Image Understanding” team at Daimler AG’s research and development unit, Dr Stefan Gehrig, expert in stereo vision, and Dr Clemens Rabe, expert in real-time implementation are exploring new avenues in the field of risk identification and accident prevention. Using a pair of cameras (a “stereo camera”) and new algorithms, they have been able to identify any number of moving objects such as vehicles, cyclists and pedestrians quickly and calculate their position as well as their moving direction and velocity. This makes it possible to accurately predict the risk of collision even if road users are partially occluded by other obstacles, for example if a child is running out into the road and only their head and upper body are visible.
The principle behind it: we human beings perceive our environment in three dimensions with our two eyes. By looking more closely we are also able to estimate the trajectories of objects in order to recognise potential risks, even when we are moving at speed ourselves. 6D Vision allows us to recreate this essential human capability using compact, high-performance and automotive-compliant hardware. Like its human inspiration, 6D Vision uses a pair of cameras to calculate the three-dimensional geometry of the scene in front of the vehicle. Selected image points are also tracked from image to image. As the 3D position is recalculated for each image, the direction and speed can be calculated using a suitable filter. At the same time, 6D Vision precisely calculates the path of the own vehicle in order to be able to quickly and reliably differentiate between moving and stationary obstacles. The simultaneous calculation of the position (three dimensions) and motion (additional three dimensions) of relevant image points is what gave the new process the name “6D Vision”.
Spatial perception using stereo vision has been the subject of research since the beginnings of Computer Vision in the 1970s. It is based on the fact that objects are slightly offset in the images provided by the right and left cameras (or our retinas). This disparity becomes less with larger distance. Traditional methods of stereo image processing only calculate the distance to individual image points, and have substantial weaknesses when visibility is poor. In contrast, modern stereo algorithms provide a measurement for every image point using elaborate global optimisation techniques. These could not be calculated fast enough until recently due to the large amount of processing power required.
Following extensive investigations, Dr Uwe Franke’s research team opted for the stereo algorithm “Semi-Global Matching” (SGM) presented by the Deutsches Zentrum für Luft- und Raumfahrt (the German national research centre for aeronautics and space) in 2005, which performed top in an international comparison. Daimler’s researchers improved the algorithm and refined it so that it continues to work reliably at night and in poor weather. The world’s first successful automotive implementation in an FPGA (field programmable gate array) capable of performing 25 stereo calculations a second came in 2008 in collaboration with the hardware experts of the Swiss company Supercomputing Systems AG. FPGAs are programmable logic circuits capable of processing high volumes of input data simultaneously. They offer the processing power of high-performance PCs on one square centimetre of silicon, and millions of them are used in modern televisions and smartphones. The high degree of accuracy obtained by the stereo algorithm using these processors allows objects and obstacles to be located up to a range of 50 metres. Compare this with human stereo perception, which ends at around 10-12 metres.
From Static Images to moving Images
The perception of motion that makes 6D Vision possible requires precise values for the “optic flow”, i.e. the movement of image points through the sequence of images. The algorithms described in the scientific literature are unsuitable for calculating large image point displacements, even though these are exactly what we find in the immediate surroundings of a vehicle travelling at high speed or taking a bend. The “PowerFlow” algorithm developed by Daimler in 2004 is the first to solve this problem. The algorithm calculates image displacements of any magnitude while keeping computing time low, and provides reliable results even at high vehicle speeds, in poor weather conditions and with fluctuating light levels.
With the stereo camera system developed by the researchers, a subsequent processing step combines neighbouring 6D points following the same trajectory into objects. Using intelligent filter methods, it is even possible to calculate the acceleration and turning circle of approaching vehicles. This is necessary in order to predict the trajectory of other road users in the next one or two seconds in order to be able to distinguish potential accidents from uncritical situations.
The advantages of this process are readily apparent for partially occluded objects that are close together and for poor visibility. 6D trajectory information can be used to determine the potential danger posed by every single image point, and moving objects “reveal themselves” already at image point level. The computer is able to identify the approaching danger even before the dangerous or at risk road user appears in front of the vehicle, even if the road user is only partially visible. The driver can therefore receive the best possible support immediately in complex driving situations.
Recognising Hazards - faster than the Human Eye
Children that run out into the road unexpectedly are perceived by the camera’s entire field of view within 0.2 seconds. Due to the selective visual perception of humans, an attentive person requires about 0.5 seconds for recognition, and if the driver is distracted this adds at least a further 0.5 seconds. Because it takes time to assess the situation and react, there is always the famous “moment of shock” that everyone requires to recognise a hazardous situation and take action as appropriate, regardless of age. During this one second, a vehicle traveling at 50 km/h travels a distance of about 14 metres. Practical experiments have shown that when the safety system initiates an emergency braking, the vehicle comes to a stop seven meters earlier – a crucial advantage for preventing accidents in urban traffic.
Preventing Accidents, helping People
6D Vision is a base technology that opens up new possibilities for future driver assistance systems. It makes it possible to significantly reduce the risk of accidents and take another big step towards accident-free driving. On the basis of this technology, Daimler is working on potential driver assistance systems that include braking assistance systems that are activated when cyclists or pedestrians cross the road, crossing assistance systems to prevent collisions when turning by applying the brakes and an evasion assistance system that can help prevent accidents at the last second by quickly performing a swerve maneuver. In addition to driver assistance systems in the field of safety, 6D Vision can also help reduce the burden on the driver and boost comfort, for example in stop and go traffic or narrow lanes around construction sites. The range of potential applications for 6D Vision goes well beyond its use in motor vehicles, and offers potential in all fields where dynamic environments have to be monitored and interpreted, for example in the varied applications of robotics and human/machine interaction. Another field of application is emerging in the form of 3D television, which will in future require a stereo and potentially a motion analysis in order to enable a variable impression of depth.
The fact that modern cameras are constantly becoming more powerful and less expensive and 6D Vision uses hardware components taken from the field of consumer electronics means that this technology can spread rapidly to all vehicle classes. And because 6D Vision technology will significantly reduce road traffic accidents, Daimler also intends to make this technology available to other manufacturers, as it has done in the past with ABS and ESP®.
Daimler Company Profile
The company’s founders, Gottlieb Daimler and Carl Benz, made history with the invention of the automobile in the year 1886. 125 years later, in anniversary year 2011, Daimler AG is one of the world’s most successful automotive companies. With its divisions Mercedes-Benz Cars, Daimler Trucks, Mercedes-Benz Vans, Daimler Buses and Daimler Financial Services, the Daimler Group is one of the biggest producers of premium cars and the world’s biggest manufacturer of commercial vehicles with a global reach. Daimler Financial Services provides its customers with a full range of automotive financial services including financing, leasing, insurance and fleet management. As an automotive pioneer, Daimler continues to shape the future of mobility. The Group applies innovative and green technologies to produce safe and superior vehicles which fascinate and delight its customers. With the development of alternative drive systems, Daimler is the only vehicle producer investing in all three technologies of hybrid drive, electric motors and fuel cells, with the goal of achieving emission-free mobility in the long term. This is just one example of how Daimler willingly accepts the challenge of meeting its responsibility towards society and the environment. For Daimler, this also involves an obligation to lead the way in the field of safety innovations. It is now impossible to imagine modern vehicles without Mercedes inventions such as ABS/ESP, airbags and PreSafe. Daimler sells its vehicles and services in nearly all the countries of the world and has production facilities on five continents. In addition to Mercedes-Benz, the world’s most valuable automotive brand, Daimler’s brand portfolio includes smart, Maybach, Freightliner, Western Star, BharatBenz, Fuso, Setra, Orion and Thomas Built Buses. The company is listed on the stock exchanges of Frankfurt and Stuttgart (stock exchange symbol DAI). In the year 2010, the Daimler Group sold 1.9 million vehicles and employed a workforce of more than 260,000 people; revenue totaled €97.8 billion and EBIT amounted to €7.3 billion.
The right to nominate outstanding achievements for the Deutscher Zukunftspreis is incumbent upon leading German institutions in Science and Industry as well as foundations.
The project “6D Vision: Recognising Hazards Faster than the Human Eye” was nominated by BDI - Federation of German Industry.