Sensor cleaning is very challenging and still in its nascency within the autonomous vehicle ecosystem. While in transit, AVs need to remain spotless since they rely on a range of sensors. By preventing situations in which sensors might be obscured when making a turn or maneuvering, accidents can be avoided. The market for sensor cleaning systems technology is expected to cross USD 105 billion by 2030.
Given that the future belongs to AVs, an insightful dialogue between Miles Flamenbaum, CEO of Actasys, and a prominent leader in the sensor cleaning systems space, and Siddharth Jaiswal, Automotive Practice Head at Netscribes. They discuss what exactly sensor cleaning is, the reason for it to become a major concern in the AV universe, the market dynamics and the kind of opportunities and challenges ahead, and the possible solutions, technologies, and roadmap for commercialization.
Siddharth: Can you give us a quick snapshot of what sensor cleaning is and what Actasys does and your journey with Actasys?
Miles: There are several industry sectors highly dependent on the use of vision sensors. Vision sensors are nothing but cameras, LiDARs, infrared, and to a lesser extent even radar, which is not vision-based, yet a sensor.
These technologies are dependent on sensors for the success of a variety of products and applications, and ultimately the future capabilities and expansion of their particular sectors. And it’s something that I call ‘Sensorfication’, which is not a word in the Oxford English Dictionary. But I’m hoping that it becomes one, given how pervasive the deployment of sensors has become across several sectors and the rapid pace of vision sensor advancement uncovering new capabilities.
The issue is that sensors are sensitive electronics that are prone to function loss and data loss due to a variety of impacts. Those impacts can be the weather. So something like rain or mud gets on the lens of the sensor. Or they can get overheated in scorching weather and have thermal management issues. Then there are vibrations and other similar impacts that affect sensors. Ultimately, what ends up happening is that while those sensors are impacted, they’re still delivering data. And that data becomes unreliable. It can’t be trusted and becomes unreliable for perception. This is a significant issue as specific systems are being deployed and this data sort of has real-world impacts that affect the data streams.
So sensor cleaning is, at least from an Actasys perspective, a part of resolving, mitigating, and correcting those impacts so that that you’re able to get better data through those vision sensors and ultimately be able to trust and rely on that data for perception capabilities.
Interesting! Off late, we’ve seen a lot of interest coming in from autonomous vehicles with a gradual movement from Level 3 to level 4, Level 5 autonomy. Keeping these sensors running has become at least a thought in the supplier ecosystem. So how does Actasys see this space evolving? What’s your perspective on the automotive industry?
Miles: Yes. So, there are a lot of fully autonomous or almost fully autonomous vehicles like robo-taxis in development and testing. I believe right now there are 19 vehicle models having LiDARs to support Level 3 functionality. But, the operating domain and capabilities of these vehicles are limited. Some of that limitation comes from the perception stack, the software, and the systems operating that vehicle. But some of those limitations in the operating domain are due to what I just described, the impacts on the sensor, and the inability of those sensors to generate clean data under a wide range of operating conditions. So, Actasys’ viewpoint on this is a little bit different from the industry – in that the industry is very focused on cleaning in and off its sensors.
At Actasys, the first step we took is to at least remove whatever is on the sensor lens to enable data capture. Next, we take this further and break it down into three steps. First is detection – the ability to detect that there is something affecting the sensor; such as a physical impact of heat due to the weather. Next, we work to correct that impact through both sensor cleaning systems and cooling, since there could also be thermal management issues.
Finally, all this information needs to be reported to various stakeholders – from the OEM, the driver or the passengers of the vehicle, service providers or whoever else needs to know of the sensor fault. This is to understand under what conditions the fault occurred and ultimately gauge if those sensors need to be serviced.
In this manner, we take a much more holistic approach than the industry; in that, we understand the scope of the problem is something greater than just the simple removal of something on the lens. I think at a granular level of detail, how you remove things off the lens also starts to become very important.
We’ve seen several robo-taxis or tier-one vehicles have cleaning systems based on mechanical systems that are initially very effective. These generally tend to have compressors or be based on high pressure, and high-flow rate. For instance, liquid washes and sprays. They are effective, but to OEMs transferring these capabilities into a production vehicle doesn’t make sense, because they are pretty large, power-consuming, complex products, and difficult to integrate. That’s why they don’t really meet the OEM requirements.
Here again, our approach to cleaning is different from most of the industry. It’s air based – in that we have a unique way of generating an air jet out of an electronically-controlled actuator cartridge. So it’s a non-mechanical way of generating an air jet. That air jet becomes very capable of clearing things like rain and light, dust, and whatever else off the lens. We can also inject liquid for cleaning into this air jet. But because we’re leveraging an Aerojet with reduced power consumption and more efficiency we can provide liquid cleaning as part of the process using much less liquid. Also, we don’t need a big pump or a high-flow rate.
Ultimately, we aim to deliver a scalable approach that goes from cleaning one to multiple sensors on a vehicle. We believe our approach is not only unique but eminently meets the current and future needs of the market space. I think there’s general consensus in the industry that automotive sensor cleaning systems are a requirement. At the end of the day, it’s just how you go about satisfying those requirements. For our part we are developing different paths which we think have a significant set of competitive advantages.
Siddharth: Interesting! So we see a significant legacy mindset within the automotive industry where new technologies take relatively more time to become popular and make way into vehicles. In this light, how do you and Actasys perceive the reception of new technologies such as sensor cleaning by conventional auto players? How open are they? Do they consider sensor cleaning as one of the top priorities in AV development?
Miles: True. The legacy mindset is very prominent in aspects such as producing and installing products and technologies that have been well-tested and well-accepted in the automotive industry. In fact, a lot of components on a vehicle in essence are almost standardized. While they do improve over time, those improvements are incremental and some features and functions have become industry table stakes.
In the sensor cleaning space, that’s not the case. Firstly, the sensors themselves, the perception stack, and everything around it is a very new technology space. Next, since it involves both hardware and software it raises the level of complication.
When you add in a very new approach to enable sensor cleaning, particularly one that is ultimately multidisciplinary, at an industry level it’s difficult to manage. The general industry feedback is ‘We need help. We don’t necessarily understand all the parameters, inputs, and impacts that need to be characterized and well understood to really then develop a set of requirements for sensor cleaning.’
In this manner, most players seek support from a single vendor. But sensor cleaning, as a space, is new and has yet to reach that point. It’s not sort of an established technology or has established products to subdivide the system amongst several vendors. This again raises the level of complications.
I think this puts Actasys, at least from our perspective, in a very good position because we’re not just focused on cleaning a lens, but on holistically enabling sensor operation and really understanding how those different environments and operating conditions can affect the sensor function and ultimately how to mitigate them.
Again, I think we have an interesting approach. There are several tier-one players and you’re right – a lot of them originally had a wiper systems group. So they have a lot of experience spraying liquid onto a glass surface and clearing it off with a wiper and have transitioned from this space.
These groups include automotive sensor cleaning systems or have developed groups to include sensor cleaning that ejects out of the wiper systems groups. The limitations they run into in the first instance are a wiper in and of itself. It’s an object in front of the photonics package going back and forth, that can wreak havoc on the data capture. So our universal observation is that everyone recognizes the inefficiency of a viper for certain conditions and the need for air as a cleaning mechanism.
A lot of cleaning mechanisms rely on air. Again, high flow rate, high-pressure liquid pumps to help spray but work only in specific environmental conditions. But, you’re right. We’ve also seen estimates of a fully autonomous vehicle with a full sensor package have the potential to consume, under certain conditions, up to twenty-five liters of cleaning fluid in a day.
There’s no place to put that fluid. Thankfully, the future is leaning more towards electric vehicles that don’t have petrol tanks. Yet, you wouldn’t want to have a tank just carry just a large amount of liquid. So, it’s all about developing a system that is hyper-efficient, scalable, and can provide some liquid cleaning, but in a very controlled fashion.
Today, technologies even provide heat for ice and snow, covering the widest range of environmental conditions for cleaning and beyond. Thankfully, there is an industry-wide understanding of how critical it is to cool these instruments and report failures. Automotive players understand that sensors can lose data and reliability. Although they don’t completely understand how this happens, the issues occurring on the surface of the lens and the ways in which that impacts the data stream. That’s why the question we get asked most often is ‘How clean is clean? How do you not only qualify but also quantify that you need to clean a sensor?’
So we’ve developed a series of tools and capabilities born out of initially obscuration detection to measure it and to help use that measuring capability both as a trigger for the system and also to quantify cleaning capabilities under different conditions. In a nutshell, it does get very complex and the industry recognizes the need.
What’s occurring now are early deployments of sensor cleaning technologies and a significant effort amongst OEMs in tier-one to do more work and define the requirements and the technologies that will ultimately satisfy these requirements.
Siddharth: Essentially, we can say the industry is experimenting with four types of technologies. One – the conventional wiper liquid jet, second – the use of air with a combination of liquid, third – ultrasonic in a combination with air/liquid. There’s also aerodynamics, but not many are experimenting with it since it’s pretty expensive and complex. So, how do you see technology evolving? You mentioned the integration of software and hardware, but what would be the core technology for sensor cleaning systems in automotive going forward?
Miles: We appreciate the different avenues that have been pursued. You mentioned some of them. Just some quick comments on aerodynamics. It’s a sort of clever, yet obvious approach – to design the physical emplacement of the sensors in a way that funnels air towards the surface, which is great when the vehicles are in speed. But if it’s probably below 45 kilometers an hour, it’s not very effective, if effective at all. Therefore, you need a system that can handle zero to 150 kilometers an hour. That’s a challenge. We’ve seen several technologies – I mentioned the wiper in front of the sensor lens or ultrasonics. The feedback we’ve received is that in many cases, either the wiper going back and forth or the rapid vibration of the surface, for ultrasonics, can again affect the data stream or the photonics.
So, we see tier-one OEMs now primarily looking to deploy mechanisms with air combined with liquid as a cleaning requirement. I think the natural progression is one that ultimately Actasys is engaged in. It follows two paths. One – how do you most effectively and efficiently clear off the surface? In that, how we generate an air jet is different and how we generate a liquid spray is different. It’s much more efficient, it’s easily integrated into the vehicle. That’s the physical side and will be important as there are many constraints since every OEM is moving to an electric platform. You need a system that’s not power-intensive nor weight-intensive. That’s the general direction of the industry and the need is to meet those requirements. I think what we’ll also see and where Actasys is already headed are the tools, methodologies, and technologies to really understand various conditions, potential ways to optimize the system, sensor selection, and sensor location.
Even the design around the physical installation of the sensor is critical as a part of the overall effort. That’s why tools become really important in enabling OEMs to understand sensor operation under different conditions. Optimizing the selection, design, and sensor cleaning system that goes along with it and then developing the intelligence around it matters.
How to monitor the health of these sensors and how to really ensure optimal operations under the widest range of conditions? Here’s where it moves from a physical challenge. You have a surface to be cleared and it needs to be installed on a vehicle with limitations. But that quickly moves into data and intelligence and analytics.
Our approach has been to really focus on what’s occurring on the surface of the lens. Under what conditions and why is it occurring? Why is it happening both in real-world conditions and also in similar simulated conditions in a lab? We evaluate the data generated by the sensor and work to really understand where and how these two datasets converge and diverge, why? We take an analytical approach to utilize that information to help optimize the system.
Siddharth: Great! Given that this technology is relatively new, can you explain the commercial roadmap for it? Since players are still exploring this space, what could be the best and most sustainable solution? Would this tech eventually be an integrated solution by sensor makers or would OEMs directly choose an automotive sensor cleaning system supplier and get the tech? Or will it be a more conventional way where sensor cleaning companies coordinate and partner with tier-ones and that’s the route to commercialization? How do you see the dynamics play around here?
Miles: That’s a really good question. I think a lot of the activity right now is at the OEM level with some coordination already with tier-ones. But what’s important is the need was identified. Innovation groups at the various OEMs and the tier-ones as well started to investigate and evaluate different approaches, different technologies, and different products. We’re now seeing those move beyond the innovation space, into development, and ultimately into the commercialization space. So it’s moving through the pipeline.
I think at it’s at the outset it’s the OEM’s responsibility to address the problem. The first design requirements and the commercial thrust come from the OEMs. But, ultimately it will heavily rely on tier-ones for integration and production. I think it’s an open question on the integration side as to how these things get packaged together, whether they will be packaged together with the sensors or separately from the sensors. There are still some issues to be worked out. We’ve even seen some of the sensor companies, particularly some of the leading wiper companies begin this investigation to understand the scope of the need and the opportunities that exist to address that need.
So, I don’t think you’ll necessarily see sensor companies developing their own automotive sensor cleaning systems. But I think what we’ve already experienced and there will be more news to come on, is partnerships where sensor companies will have some level of provider preference. Sensor cleaning companies will work with the sensor makers in the automotive for some level of development and optimization, to present a more comprehensive package to the OEM who will then direct to develop and integrate it.
So, as we discussed, this is not part of the legacy set of products and capabilities in the industry. It’s new. I think we will see a bit of a non-traditional path at first, but ultimately things will settle down where you’ll have tier-ones who will get involved in selecting different automotive sensors and different cleaning systems and integrate and produce them on behalf of an OEM.
But we’re not there yet. I think, in the earlier days, again, a lot of this is going to be driven by the OEMs. We have several OEM partners where we’re now at a phase where they’re starting to either ask us to bring in a tier-one or to almost dictate that here’s a tier-one that we’ve been working with on this problem. We want you to work with that tier-one so that we can develop a commercial solution for production.
Siddharth: Interesting! My final question would be how soon do you see a sensor cleaning tech deployed in vehicles on the road and what happens next once the vehicle is on the road? It took nearly 120 years for us to figure out a service network, and troubleshooting codes. Who do we go to when there’s an issue? What will be the dynamics once the autonomous vehicles in the pipeline hit the roads? How will things pan out post the vehicles are purchased?
Miles: We’re starting to see cleaning systems get commercialized. They tend, at this point, to be liquid based and liquid only. So again, as I started off by talking about the limitation of the operating domains of these Level 3 capabilities, part of the restraint on that domain is the fact that there’s no single cleaning system that can address all operating conditions. Rain impacts the ability of these automotive sensor systems to generate clean data. But if you only have a liquid-based cleaning system, spraying water on top of water doesn’t really help. But we’re starting to see the early commercialization. We’ll see more comprehensive approaches, again that will combine the liquid with air.
The approach we’ll introduce will be liquid and air. Also the possibility of heat as well – all of the sort of software and data and analytics that can go around it. I think ultimately, there’s still a lot that the industry is learning. Once the vehicle is sold there’s a very urgent and open question as to how to service these sensor systems. How to service the sensor cleaning system to go along with it?
That’s again part of the toolset we’re developing. With it, we not only detect a problem with the sensor, but we can start to get into predictive maintenance, reporting out sensor health and functionality, understanding if the sensors are properly aligned, if they need to be calibrated, and communicating that information to key stakeholders.
So in the first instance, if you’re in a car that has Level 3 capabilities, you will want to know whether or not your optical sensors are operating properly. You want to know when you can rely on that Level 3 system and when it may not be reliable.
Such information becomes extremely important in understanding if the vehicle needs servicing. So as the owner of the vehicle, you want to know either predictively or immediately if there’s a malfunction, or if there’s something critical that needs servicing within the automotive sensor cleaning system. If you’re a service provider, you’re going to want the same notification.
One of your customers may get an alert. You want to see that alert and say – ‘Hey customer! You need to come in because your sensors aren’t functioning properly. They need to be recalibrated,’ or something along those lines. Even fleet managers need to understand how that sensor is operating across the fleet, and how to schedule maintenance, now that you have these new technologies.
I wouldn’t call them necessarily downstream. It’s again more of a holistic approach that’s not just clearing a surface. It’s issues around sensor function that need to be understood and managed to make these systems much more successful, thereby making the service and support of those systems as efficient as possible.
Miles Flamenbaum
CEO, Actasys
Miles Flamenbaum is the CEO of Actasys, a developer of mobility sensor-based technologies intended to maintain the optimal operation and optimization of sensors, electronics, and systems. Miles was trained as an attorney and spent over two decades in business and corporate development roles in the biopharmaceutical industry. The entrepreneurial itch moved him into the automotive industry, founding a company and subsequently joining Actasys as CEO in 2018. He is an active family man and skiing enthusiast.
