Auto makers have had the opportunity to establish realistic, minimum safety goals for assisted driving and autonomous driving. They have not. It is time to impose rigorous safety standards on an industry that keeps undermining its own case for less regulatory oversight.
What the industry is passing off as “safety” is simply inadequate.
On Semiconductor senior Vice President Ross Jatou bluntly noted in our recent interview that the “driver monitoring system today is a glorified checkbox.” DMS isn’t the only technology that’s true of, though. The same judgment applies to active advanced driver-assistance systems (ADAS) such as automatic emergency braking (AEB). Recent AAA test results on AEB, as Jatou said, warn the driving world that, “Hey, they don’t work consistently all the time.”
Recommended:
Lack of ADAS Benchmarks Is Haunting Car Industry
The industry should have rigorous programs to establish safety goals and meet them. Instead what we have are individual auto makers eager to tout new “life-saving” ADAS features, but who will only reluctantly acknowledge those features have suspect reliability. AEB seeing-eyes, for example, sometimes can’t see when the sun is too bright or the darkness too dark.
Perhaps they can’t hear the voices calling for more reliable safety standards, because that advice is being drowned out by “Autopilot” fanboys who tingle with every update from Tesla about “Full Self-Driving” (FSD).
For the longest time, consumers might have been led to believe that safety in the automotive industry was assured by regulation. But let’s not kid ourselves. Remember Ralph Nader’s “Unsafe at Any Speed: The Designed-In Dangers of the American Automobile” published 55 years ago? The U.S. automotive industry has always gotten a free pass through “self-regulation.” Today, in the same vein, we have autonomous vehicle enthusiasts and carmakers promoting “light-touch” regulation and feeding off each other.
Picture a pedestrian in white clothes crossing the street in daylight. Does your car run over him or stop just in time? Imagine driving out of a tunnel, from gloom to daylight and there’s that pedestrian again. Does your car see him and swerve life-savingly? What about a child who enters the roadway from behind a parked car at night? Your car can see the parked car, but can it spot the kid?
We are talking daily driving conditions. These aren’t “edge cases.” But the answer to each question is unsettled.
It’s time for carmakers, institutions that offer car assessment programs, and regulators who look after safety issues to raise the bar. They must conduct additional or gap tests and benchmark all vehicles’ capabilities.
“Pedestrian Detection is vital to ADAS and automated driving and is a fundamental capability for all of motor vehicle safety,” Phil Magney, founder and president of VSI Labs, told EE Times. “Until now, commercial systems have evolved to work reasonably well in daylight conditions using traditional methods that apply visible cameras with radar. But in low light conditions these commercial solutions fail too often.”
Magney made it clear: “Regulators have an opportunity to update testing protocols to be more representative of common driving conditions.”
Flir – VSI Labs partnership
Many ADAS and AV experts know this, but among them, Phil Magney might understand it most acutely. His firm recently issued a white paper on this very topic, based on tests VSI Labs ran in July at the American Center for Mobility (ACM) in Michigan.
VSI got involved because Flir Systems, Inc., engaged it to test the world’s first fused AEB sensor suite. It employs a thermal longwave infrared (LWIR) camera, a radar, a visible camera, and a convolutional neural network (CNN).
Flir’s thesis was that while no single sensor would solve for all the variables a vehicle will encounter on the road, thermal LWIR cameras could be effective, as they ‘see the heat’ given off by objects in the environment.
Flir has an obvious vested interest in promoting the adoption of low vision detection in test suites. The company hopes to see their LWIR cameras designed into active ADAS vehicles.
VSI did its tests according to Euro NCAP (New Car Assessment Programme) standards, said Magney. His firm also designed tests not currently conducted by NCAP. Additional tests included “driving toward a soft pedestrian target (SPT) heated to mimic a human at 25 MPH in a variety of common driving conditions, which are not currently tested as part of NCAP.”
He explained, “We conducted Euro NCAP protocols but did so under lighting conditions not currently tested. Since most pedestrian fatalities happen in low light conditions, we feel it is imminent that the testing protocol be expanded to include this. We simply repeated the test protocol at night and the results speak for themselves.”
This sort of extra tests redounds to Flir’s credit, if it passes. But they also serve to indict AEB systems that fail to perform reliably in many common driving conditions. One could argue that current testing setups not aligned with daily driving conditions such as darkness and sun glare are catering, by omission, to technology suppliers who can’t offer equal performance.
Thermal-enhanced fused AEB system
VSI Lab used a Ford Fusion based on a thermal-enhanced “fused” AEB system, to test against four 2019 production vehicles with AEB systems: Tesla Model 3, Subaru Forester, BMW X7, and Toyota Corolla. All four were equipped with the most current safety features, for that model year, to avoid pedestrian collisions.
The “fused AEB system” came with a thermal LWIR camera, radar, visible camera and CNN.
Asked about specifics, Magney explained, “We fused the data from each of these three sensors. As for cameras, one was a thermal camera (FLIR Boson) and the other was a RGB camera (FLIR Backfly). In terms of radar we used the Delphi ESR 2.5 which is both medium range and long range.” Magney noted, “Each of the two cameras has its own trained neural network for detection and classification. One is trained with thermal images the other with RGB images. The two vision pipelines detect and classify simultaneously with associated confidence values.” Radar data is then attached to classifications associated depth and movement. “The end result enables the system to know (with confidence) when to perform an AEB maneuver.”
So, who developed the neural networks?
“Both neural networks were supplied by Flir as they have their own data annotation department. VSI contributed to the training data sets as part of the development.” Magney reiterated that neural networks are only as good as their training process.
Test setups VSI developed five cases based on Euro NCAP testing protocols. Those tests included modified scenarios not included in standard AEB positive detection tests and identified as potentially challenging cases for standard AEB systems. Day Tests: In daylight, dry conditions at intersection
- Crossing adult soft pedestrian target (SPT) in white clothing that blends in with a light background
- Crossing adult SPT in dark oversized clothing
Night Tests: In dark, dry conditions at intersection
- Crossing child SPT emerging from behind a parked car
- Crossing adult SPT emerging from behind a parked car in dark oversized clothing
Sun Glare Tests: Vehicle emerging from a dark tunnel to sunrise glare in dry conditions
- Crossing adult SPT
Testing methodology and test results
Each test was repeated until the vehicle struck the SPT twice or completed a maximum of 5 runs. This was done to reduce potential damage to the SPT and the vehicle, VSI explained.
The fused thermal AEB system ran through entire test plan with five tests for each test case and was successful in 25 of 25 tests at preventing pedestrian injury. In only two instances did the vehicle contact, but did not knock down, the SPT, according to VSI.
All five vehicles did well with the standard dark-clothing/daylight scenario. For the pedestrian in white clothes, the Tesla failed on two of three tests. In general, the four commercially available AEB systems had a positive performance in daytime tests (42 passing grades out of 50 tests).
But tests performed at night were a whole different story. The four cars with four commercially available AEB systems knocked down the SPT in all but two tests. Driving out of the tunnel, the fused thermal AEB system stopped and touched the pedestrian target without knocking it down on one out five tests and stopped well clear on the other four.
How did all the commercial vehicles perform? The fused thermal AEB test vehicle went first when the sun was at its most extreme angle right after sunrise. The Subaru, Toyota and Tesla all did poorly, knocking over the target many times.
By the time the VSI team tested the BMW, the sun was higher in the sky and its system was able to avoid the target all five times.
VSI’s white paper concluded that the AEB system with added thermal LWIR camera “performed significantly better” than existing commercial AEB systems in several real-world scenarios. “The thermal-enhanced AEB system improves the AEB functionality in the most dangerous situations, including low-light conditions, darkness, and when exposed to blinding conditions such as emerging from a dark tunnel into bright light.”
Will AEB testing scenarios evolve?
In its white paper, VSI wrote, somewhat optimistically, “Automotive testing agencies can evolve their AEB testing scenarios to be more rigorous to align with other daily driving conditions such as darkness and sun glare.”
But when pressed, Magney acknowledged, “Well these things take time and these agencies don’t move that quickly.” Perhaps, more important, he added, “Also they are not getting enough push yet from the safety advocacy groups but we think this is going to change.”
So, who are in the business of assessing new cars’ safety? Magney explained, “There is usually an NCAP program for each region or country. Euro NCAP, United States NCAP, Japan NCAP, and so on.” He added, “VSI used Euro NCAP for its test protocol as it is considered more advanced than the US version, especially with AEB Testing.”
There is also the IIHS (Insurance Institute for Highway Safety), a non-profit safety agency formed by the U.S. insurance industry. “IIHS does not conduct AEB-P testing as of yet,” according to Magney.
Even if a safety advantage for thermal LWIR is tested and proven, the technology’s remaining hurdle to get designed into commercial vehicles is cost.
According to Flir’s spokesperson, the cost of its themal LWIRto be around $200 per camera at the start of production. The company expects it to decrease as volumes go up.
Magney observed, “Flir is pushing for the adoption of visibly challenged scenario testing and now they have the results to back it up.” He noted that there are RFQs out there calling for thermal for this purpose.
“Up until now, thermal has only been used for night-vision systems which carry a higher price tag because it is a sexy feature. Now we are talking about bringing thermal down-market and making it a vital component that can meet these uses cases for low light detection and emergency braking applications.”
The post Will AEB Systems Testing Scenarios Evolve? appeared first on EE Times Asia.
from EE Times Asia https://ift.tt/2Gdpw44
No comments:
Post a Comment
Please do not enter any spam link in the comment box.