At the Future Work Summit held on January 12, 2022, listen to the opinions of CIOs, CTOs, and other C-level and senior managers on data and AI strategies. to know more information

Remote operation: Technology that enables humans to remotely monitor, assist and even drive autonomous vehicles.

Teleoperation is a seemingly simple capability, but to be implemented safely, it involves many technologies and systems. In the first article in this series, we determined what remote operation is and why it is critical to the future of autonomous vehicles (AV). In the second article, we demonstrated the legislative appeal and focus of this technology. In the third and fourth articles, we explained two of the many technical challenges that need to be overcome to achieve remote vehicle assistance and operation. The fifth article explains how to achieve all of this in the safest way. In this installation, we will reach the most important person in the entire cycle, the customer. 

The Israeli journalist Sarah Tuttle-Singer wrote the story she heard from a taxi driver. She has many, and she even wrote a book with the best of them. This makes perfect sense. Taxi drivers ride with different passengers all day, and when people are bored, they talk. Taxi drivers always have a million anecdotes to share. Unfortunately, with the imminent transition to autonomy, these stories will disappear.

There is an existential problem in removing the driver from the vehicle. You simply cannot provide 100% service availability or a satisfactory level of customer experience, whether it is today or 50 years from now. Interpersonal interaction is a key task. In many cases, autonomous systems are unable to respond to the level that customers expect and are entitled to. No one is driving, which not only means that the vehicle will stop due to the confusion of different situations, but also that the seemingly simple service cannot be provided. The reason is simple, machines don’t know how to interact with humans like humans. As far as autonomous driving is concerned, customer experience may be the biggest issue. They are divided into four main categories: passenger issues, emergencies, deliveries, and manned areas.

Above: The picture is from Stockphoto.com, authorized to Amit Rosenzweig

1. Passenger discomfort: Maybe they feel anxious or insecure. Since there is no human driver to respond to their needs, no one can communicate with passengers; no one can alleviate their fears, reassure them, or solve the root cause of the problem.

2. Control the air conditioning or infotainment system: Not every passenger has the same comfort or needs, and not everyone knows how to operate certain systems. Problems with temperature control or in-flight entertainment are inevitable. But no one can talk.

3. Specific drop-off point: Currently, when you arrive near your destination, you only need to say: "Can you get off at that door/that car/that tree/etc?" Usually the driver will do this . Not only will it not be possible to use an autonomous driving system, but it may also take you to a technically correct address, but not an ideal location for easy access. This is especially important for elderly passengers and those seeking medical care.

4. Passenger violation: Not all problems stem from the vehicle. What if the passenger is not wearing a seat belt or the back seat is full of people? The human driver will straighten them and the problem will be solved. Can robot taxis even recognize these problems, let alone deal with them?

5. Forgotten objects or worse, forgotten treasures: In 1999, the world-famous cellist Yoyo Ma forgot his 2.5 million US dollars cello in a taxi. The passengers even forgot the sleeping child. It is now easy to call the driver (his number is on the receipt). This is no longer an option when no driver can pull over and check whether the vehicle is occupied or property.

6. Vandalism: Unfortunately, not everyone is good. In fact, some people are quite bad, while others, such as teenagers, are just careless and indifferent. Passengers may damage or destroy the vehicle during the ride. Without the vigilance of the driver, this behavior will be uncontrollable. The result is higher maintenance costs, reduced profits and vehicle downtime.

Above: The picture is from Stockphoto.com, authorized to Amit Rosenzweig

7. Law enforcement: When the vehicle is driving irregularly or something needs to be investigated, the police use a loudspeaker to remind the driver to pull over. Without the driver in the car, the police cannot carry out their work, and the consequences may be unfortunate.

8. Medical/ambulance: The situation is similar to that of the police. Maybe the passenger has a medical problem and managed to call 911. Vehicles still need to know when and where to stop for medical professionals to provide passengers with the necessary treatment they need.

Above: The picture is from Stockphoto.com, authorized to Amit Rosenzweig

9. Wrong/damaged/missing package: When you receive the package from the courier, they will hand it to you and ask you to sign for it. If the received package is badly damaged, or just the wrong thing, you can tell them and they will handle it. If you lose something, they will go back and bring back what they forgot. The robot will automatically move to the next destination.

10. Find the right customers: Today, only a few suppliers have delivery robots. However, as technology advances and production costs fall, these numbers will increase exponentially. Soon there will be many robots and many recipients. The delivery robot may find itself getting off the truck to a location where multiple people are waiting for different orders. For computers, it is impossible to determine which person is required to deliver. This can cause equipment chaos and customer frustration.

11. Content verification: When the truck arrives to pick up or return the truck, the doorman needs to confirm the record, the delivery content, and guide the vehicle to its specific parking space or loading dock. The autonomous system does not respond well to directions or verbal commands, nor can it explain why there may be discrepancies between the information it has and the instructions given by the waiter.

12. Change order: Once in the facility, someone may need to transfer the vehicle from one task to another. This is especially true for construction areas where a vehicle may have many tasks and their order changes. People in this field cannot communicate these ever-changing needs.

13. Damage: After the vehicle arrives, a certain degree of vehicle inspection will be carried out. If there is a problem or damage to the vehicle, no one can tell.

For anyone who has read the first few articles in this series, the answer should be obvious. For those who have not done so, the solution is to involve remote personnel. At present, none of the above problems are a problem because there are human drivers and couriers. Autonomy cannot solve these problems. Humans still need to manage these problems in the most effective, efficient and safe way. This is why teleoperation is the only option. However, like most solutions, it has its own challenges.

When a remote operation session is triggered, it does not simply go to the first available remote operator (TO). For the same reason, when you call customer service, there is a routing system that will route you to an agent with the required expertise based on your input. However, unlike ordinary call centers, customers may not necessarily choose to "press three for traffic problems" or "repeat this message by pound."

The first possible solution is to have a remote operations manager (TM), which will first answer the call for help, and after determining the complexity and requirements, route the conversation to a specific operator. This will be very inefficient and means that TM cannot complete their actual work-management. On the contrary, there needs to be an automatic and intelligent way to meet the needs of manual intervention without... manual intervention.

When a remote operation session is triggered, the source of the request is automatically determined first. Did passengers trigger the demand for TO? Is it a first responder or a law enforcement officer? Or the vehicle itself? Each of these situations requires a different type of response, and therefore a different type of TO. In a particular remote operations team, some are junior, some are advanced, some are more customer-oriented, and some are more technical. Advanced TOs may be authorized for remote assistance and remote driving, while their junior TOs only allow assistance. Some TOs may be mainly used in customer interaction situations, so if the conversation is triggered by the customer, then they will be cyclic. On the other hand, if the vehicle triggers the conversation, it is not because of traffic chaos, but because of a technical problem, which requires a completely different response.

The seemingly simple aspect of determining who answers which call is complicated. For the challenge of establishing this connection, please refer to our previous article on network connections and video compression.

There is another challenge. Once the remote operation session starts, the TO must understand what happened. They need to receive a lot of information, which must be displayed hierarchically so that they can better understand the situation. When jumping between many vehicles, this problem will increase exponentially. There must be built-in processes and tools so that there is minimal delay between the start of the conversation and the goal of solving vehicle problems and customer satisfaction.

If self-driving car suppliers want to deploy robot fleets on a large scale, they must ensure that humans are involved. It is a complex and complicated process from beginning to end, a technology completely different from autonomy itself. This is why remote operation providers exist, and why industry leaders like Motional choose to rely on them for this mission-critical function. For anything else, they are not on the right track to the future of autonomous driving.

Welcome to the VentureBeat community!

DataDecisionMakers are experts, including technical personnel engaged in data work, where they can share data-related insights and innovations.

If you want to learn about cutting-edge ideas and the latest information, best practices, and the future of data and data technology, please join our DataDecisionMakers.

You can even consider contributing an article of your own!

On January 12, 2022, online listen to the opinions of CIOs, CTOs, and other C-level and senior executives on data and AI strategies.

Hear the opinions of CIOs, CTOs, and other C-level executives on data and AI strategies.

© 2021 VentureBeat. all rights reserved.

We may collect cookies and other personal information from your interactions with our website. For more information about the categories of personal information we collect and the purposes for which we use them, please review our collection notice.