Google News App

When designing a new platform, we work closely with our OEM partners to ensure we can integrate our Waymo Driver seamlessly into their vehicles and manufacture them easily. We begin this process by using our most recent self-driving system, comprised of radar, lidar, cameras, and compute and adapt it to the new vehicle, optimizing it for the various platform requirements, such as the variation in potential blind spots.

Each generation of our hardware suite is informed by more than a decade’s worth of our experience and millions of miles of on-road testing. This experience provides us an understanding of the edge cases that self-driving technology needs to handle to take people and things safely from A to B.

Compared to passenger vehicles, trucks spend a lot more time on freeways, which are higher-speed environments. They also have a lot more mass, are slower to accelerate and brake than passenger cars, requiring a distance of nearly two football fields to come to a stop, and they have trailers that can move independently from the tractor. Therefore, when configuring the self-driving system for our trucks, we take into account that our trucks require more time and space to maneuver and can have different blind spots than cars. To accommodate for these differences, we increased the number of sensors we apply to the trucks. The most noticeable difference is that our trucks feature two perception domes versus the single iconic perception dome on our passenger cars. The dual perception domes on our trucks help increase rear visibility by reducing blind spots caused by the trailer.

Five years ago this month, Steve Mahan, the retired director of the Santa Clara Valley Blind Center, who lost his sight years before, took the world’s first ride in a self-driving car on public roads. This little car, called Firefly, had a top speed of just 25 mph, and used a combination of lidar, radar, cameras, and massive on-board computing power to navigate the busy streets of Austin, Texas. 

Since that first ride, which some have called a Kitty Hawk moment of sorts for fully autonomous technology, Waymo has been hard at work to bring the Waymo Driver to more and more people. In 2017, we began our early rider program in the Metro Phoenix area with the next generation of our technology, using the full-size hybrid Chrysler Pacifica. This 4th-generation Waymo Driver was significantly more sophisticated than Firefly’s technology, and for the last year, has pioneered fully driverless, paid rides on high-speed roads across a service area larger than the city of San Francisco. 

A self-driving vehicle uses multiple sensors which work together as its eyes and ears, providing information about the world around it, from other vehicles and moving objects, to traffic lights and vegetation lining the sides of the road. In addition to this powerful sensor data, Waymo’s self-driving vehicles also have the advantage of knowing what the road ahead looks like. The Waymo Driver gets this knowledge from its highly-detailed, custom maps that provide our vehicles important context in every location we drive.

While wrestling with maps and folding them up may bring nostalgic memories for some, the maps we create for the Waymo Driver are orders of magnitude more complex. The information our vehicles care about is quite different from someone trying to find their way to a restaurant. For example, it’s far more important for the Waymo Driver to know the speed limit than the name of the road. We’ve built an incredibly detailed set of mapping technologies that help our cars navigate places even where GPS struggles, like tunnels or between skyscrapers.

These maps give the Waymo Driver a deep understanding of its environment, from road types and the distance and dimensions of the road itself, to other features like lane merges, stop signs, crosswalks, and so much more. This prior knowledge of what each mile ahead looks like, combined with our powerful sensing and perception capabilities, allows us to drive smoother and more predictably. For example, when the Waymo Driver approaches an intersection, not only can it sense a car that might cut across its path, but because of our custom maps, it also knows that vehicle has a stop sign. If the Waymo Driver detects the car is going too fast to stop, our robust neural nets can respond to it very quickly. This results in a safer and more comfortable experience for our riders and other road users.

Building custom maps for the Waymo Driver

To create a map for a new location, our team starts by manually driving our sensor equipped vehicles down each street, so our custom lidar can paint a 3D picture of the new environment. This data is then processed to form a map that provides meaningful context for the Waymo Driver, such as speed limits and where lane lines and traffic signals are located. Then finally, before a map gets shared with the rest of the self-driving fleet, we test and verify it so it’s ready to be deployed.

Just like a human driver who has driven the same road hundreds of times mostly needs to focus only on the parts of the environment that change, such as other vehicles or pedestrians, the Waymo Driver knows permanent features of the road from our highly-detailed maps and then uses its onboard systems to accurately perceive the world around it, focusing more on moving objects. Of course, our streets are also evolving, so if a vehicle comes across a road closure or a construction zone that is not reflected in a map, our robust perception system can recognize that and make adjustments in real-time.

Over 25 cities across the USA: What we learn through mapping new locations

While the process of creating our custom maps is similar for all geographies, every place is, in many ways, unique. Traffic laws differ from city to city, so we work closely with local officials and traffic engineers to become experts at local driving rules to convey that information to our vehicles.

For example, in San Francisco there are special areas called Safety Zones where buses and streetcars drop off and pick up passengers. If there is a bus stopped near a Safety Zone and it is not otherwise signed, it’s illegal for a car to drive more than 10 mph past the bus. We’re encoding Safety Zones into our map as a base layer, which helps ensure we are abiding by local laws.

There are many factors we look at when mapping new cities: the width of lanes, bicycle lanes, reversible lanes (think of the Golden Gate Bridge adjustable lanes that can change their direction with the traffic flow), and more. Some nuances are even more subtle. For example, many stores have roller shutter doors and curb cuts that make them almost look like driveways. Knowing which of them are actual driveways helps the Waymo Driver understand whether other cars could be emerging from these areas.

A store in Los Angeles that looks similar to a driveway (Image credit: Google Maps)

A crack or a lane separator? Our maps can tell the exact lane width of this street in SF

The situations we encounter mapping a new area also help improve other parts of our self-driving system. When our mapping vehicles come across a rare or complex situation, we can use this experience to help create our map and also train our perception and behavior prediction models. For example, in Los Angeles’ Fashion District, hundreds of mannequins stand outside the stores and could be mistaken for pedestrians who want to cross the street. Capturing this early means we can better prepare the Waymo Driver to handle unusual, real-world situations. Similarly, observing different traffic behavior patterns helps us evaluate and train our behavior prediction system.

Keeping our maps up to date

Our streets are ever-changing, especially in big cities like San Francisco and Los Angeles, where there’s always construction going on somewhere. Our system can detect when a road has changed by cross-referencing the real-time sensor data with its on-board map. If a change in the roadway is detected, our vehicle can identify it, reroute itself, and automatically share this information with our operations center and the rest of the fleet in real time.

We can also identify more permanent changes to the driving environment, such as a new crosswalk, an extra vehicle lane squeezed into a wide road, or a new travel restriction, and quickly and efficiently update our maps so that our fleet has the most accurate information about the world around it at all times.

We’ve automated most of that process to ensure it’s efficient and scalable. Every time our cars detect changes on the road, they automatically upload the data, which gets shared with the rest of the fleet after, in some cases, being additionally checked by our mapping team.

As city landscapes now face new challenges brought by the COVID-19 pandemic and continue to adapt to our new lifestyles with shared spaces and slow streets, we have also proactively gone out to map and capture these changes.

We’ve been building highly-detailed maps since we started driving more than a decade ago and, combined with our state-of-the-art onboard autonomy system, they’ve been instrumental to helping us get people and things where they are going safely. In the next blog posts we will take a closer look at the other parts of our self-driving technology stack, explain how they enable the Waymo Driver to safely cope with challenges on the road, and share interesting insights from behind the scenes. Stay tuned!

Spanning 113 acres, Castle is a state-of-the-art, closed-course testing facility — which has been specially designed and redesigned over the years for Waymo’s unique testing needs to prepare the Waymo Driver to handle challenges on the road, evaluate new software before it’s released to our self-driving fleet, and validate our system’s performance. While our closed-course testing at Castle has always been a critical component of our development process, over these past few months of shelter-in-place, Castle has produced even more benefits. It has helped test and scale the new policies we’ve put in place to keep our team and entire community safe, while unlocking some key technology advancements for our teams working at home. Now that our vehicles are back on the road, we can accelerate the Waymo Driver’s deployment by enabling the entire development cycle — from gathering real-world data and analyzing it for interesting encounters, to rigorously testing updates we make to the system and evaluating its performance.

Mastering the fundamentals to handle the one in a million

Our testing site within the former base is set up like an adaptable city, including everything from wide avenues and suburban driveways to a railroad crossing and roundabouts, similar to where our vehicles operate in the real world. At Castle, we stage complex and rare scenarios — such as a person walking out of a porta-potty onto a street or a pile of garbage falling out of the truck in front of us — in a safe, controlled environment and test them over and over again, changing different variables, to ensure that our self-driving technology can handle the wide range of situations it might come across.

Over the years, we’ve been able to amass a library of over 40,000 structured testing scenarios — not counting all the variations we create with each situation. These scenarios include things that we have never seen on public roads but could imagine happening and others that occur once every hundreds of millions miles. Additionally, we create some relatively mundane tests that can also be quite challenging for self-driving vehicles and humans alike. For example, driving behind a large garbage truck on a narrow street — that stops every hundred feet or so and leaves empty trash cans strewn about — may create a lot of complexity on the road.

Our teams develop these scenarios in a variety of ways. We use national databases to identify the most common traffic accident scenarios, recreate situations our trained drivers see while testing on public roads, create scenes specifically to test new driving functions developed by our engineers, and field suggestions from team members based on their driving experience. By exposing the Waymo Driver to a wide variety of scenarios and teaching our self-driving technology fundamental skills rather than only to handle individual situations, our vehicles become more equipped to handle any situation they encounter on the road, even if it hasn’t seen that specific scenario before.

How structured testing and simulation complement each other

Testing is not a one and done task, but rather a never ending feedback loop that includes structured testing, simulation, and public road operations. When we develop a scenario to test new software, we can use either one of these tools or a combination of them. For example, after executing a test on our private track, we can then create and run hundreds of variations of that scenario in simulation. Our simulation technology allows us to do this in a matter of seconds right at our developers’ desks as they work on new features for the Waymo Driver.

As much as simulation can help scale the value of structured testing, structured testing complements our simulation. While we maintain a high level of realism in simulation, structured testing allows us to evaluate both our software and hardware stack. By recreating scenarios our vehicles have already successfully completed in simulation, we can continuously verify our simulator and assess small nuances that may affect vehicle behavior. Adding this extra layer of testing and redundancy into our development cycles helps us to ensure we have done our due diligence before our vehicles drive on public roads.

Advancing our next-generation Driver

Testing Waymo’s fifth-generation hardware suite on the electric Jaguar I-PACE at Castle

Not only does structured testing help verify software updates, it also allows us to test and validate our hardware. With closed-course testing, we can evaluate our integrated system’s performance across our vehicle and sensing systems.

Over the past decade, we’ve changed the number of vehicle platforms and sensor suites we test and operate on five times — from Lexus 450 SUVs to our custom-built Firefly, from Chrysler Pacifica minivans to 18-wheeler trucks, and now to all-electric Jaguar I-PACES. All these platforms have different body builds and require different modeling for sensors, perception, motion control, and planning. Structured testing allows us to test various system capabilities, such as how our sensing and computer vision systems work together to detect and identify a speeding motorcycle at long distances. With structured testing, we can also measure how well our motion control system follows the intended path, and validate our system’s performance and reliability across more diverse operating domains through weather testing in rain tunnels and thermal chambers, across speed bumps and potholes.

Like the platforms that came before it, our custom-built 5th generation sensor suite has been rigorously tested at Castle to help ensure its safety and readiness for public roads. With each generation, we have matured our models and our methods, so that each new bring up is smoother and quicker, enabling us to get on the roads to serve our riders and Waymo Via partners sooner. And, if you are in the Bay Area, Phoenix, or Detroit area, you should see more of our next-generation Waymo Driver driving around.

Last week, we launched a new initiative called Self-Driven Women and hosted over 250 people at our inaugural virtual event. Our goal with this series is to build a phenomenal community of women and allies that are developing the future of mobility.

During the one-hour discussion I was honored to moderate, we heard from Michelle Avary, Head of Automotive Industry & Autonomous Mobility, World Economic Forum; Raquel Urtasun, Chief Scientist of Uber ATG and the Head of Uber ATG Toronto; and Tilly Chang, Executive Director, SF County Transportation Authority. These accomplished women shared how they began their careers in transportation and autonomous driving, the challenges they’ve faced, and lessons they’ve learned.

We constantly hear about the ‘pipeline problem’ in tech and engineering specifically. In 2019, only 26% of professional computing jobs in the U.S. workforce were held by women. In transportation, these numbers are even more disturbing: in 2019, women represented only 15% of the transportation workforce. During our discussion, Michelle shared that according to the World Economic Forum, it would take 257 years for the global gender pay gap to close and for women to have all the same economic opportunities as men. We want these industries to do better, and we’re starting by bringing awareness to the low representation of women in mobility by highlighting their contributions and illuminating an inclusive path forward to all.

Here are some of the key pieces of advice we left with from our panelists:

• Mentors and networking are gold: We heard from all of our panelists about the power of networking, mentoring, and having a strong support system around you to help you achieve your goals. Tilly emphasized the importance of informal networks and the benefits she’s received from seeking out and finding these groups herself. Raquel asked how we can create better mechanisms to ensure women have access to mentors, so they don’t always have to proactively seek them out themselves. To the women entering technical industries, Raquel emphasized: “You are not alone when you suffer from discrimination or biases. It’s important to know that all of us have arrived in successful positions and struggled with that journey. It’s important that you don’t give up and continue pursuing your dream.”
• Putting in the work increases pipeline diversity: One of the biggest barriers in tech is ensuring that recruiting efforts are targeting a diverse set of candidates. As Michelle highlighted, we need to be proactive, going above and beyond when it comes to filling the pipeline. And then, once we have diverse candidates in the door, it’s about educating our teams to understand that people have different ways of communicating and creating space so they can hear the different ways these people communicate. 
• More talent diversity leads to more inclusive mobility options: Tilly pointed out that mobility is a direct path to prosperity and shared her experiences in urban planning, a field that historically has largely been driven by men. She shared that there are “so many roots to it – not just for women. But for people of color, gender identities, physical abilities. Growing awareness through programs like this is critical…It is critical that we learn about these issues so that we can be a part of the solution.” Michelle reiterated this point, saying, “We need to ask the questions about how women, elderly, and low-income households travel. If we don’t understand these questions, we can’t solve for them. We also need leadership that represents the communities.”

At Waymo, we’re committed to fostering a diverse and inclusive culture and translating these values to the technology, products and experiences we design for all of our riders, partners and communities. We will continue this work and look forward to seeing you at one of our upcoming events. You can watch our discussion here and follow our hashtag #SelfDrivenWomen on Twitter and LinkedIn for more updates.

Back in 2016, we announced our very first OEM partner: Fiat Chrysler Automobiles (FCA). Since then, we’ve worked closely with FCA to integrate our Waymo Driver into FCA vehicles, and together we’ve made self-driving history in the proven, capable, L4-ready Chrysler Pacifica Hybrid minivan, including launching the first commercial autonomous ride-hailing service, beginning to offer fully driverless service to our riders, and driving in dozens of cities across diverse geographies and challenging weather conditions. Now, we’re pleased to share that we’ve strengthened our partnership with FCA in several important ways.

FCA has selected Waymo as its exclusive, strategic technology partner for L4 fully self-driving technology across FCA’s full product portfolio. We’ve already started to work together to imagine future FCA products for the movement of people and goods operated by the Waymo Driver.

In addition, Waymo will work exclusively with FCA as our preferred partner on the development and testing of L4 autonomous light commercial vehicles* for goods movement, including in Waymo Via. We will initially target integration of the Waymo Driver into the Ram ProMaster van, a highly configurable platform that will enable access to a broad range of global commercial customers.

The expansion of our partnership with FCA is the latest step towards scaling and deploying the World’s Most Experienced Driver across ride-hailing, trucking, local delivery, and personal car ownership. We’re fortunate to have the opportunity to build upon the deep roots we’ve established with FCA.

John Krafcik, Chief Executive Officer, Waymo: “FCA was our first OEM partner, and we’ve come a long way together. The Chrysler Pacifica Hybrid minivans were the first vehicles in our Waymo One fleet, and, guided by the Waymo Driver, have now safely and reliably driven more fully autonomous miles than any other vehicle on the planet. Today, we’re expanding our partnership with FCA with the Waymo Driver as the exclusive L4 autonomy solution for this global automotive company. Together, we’ll introduce the Waymo Driver throughout the FCA brand portfolio, opening up new frontiers for ride-hailing, commercial delivery, and personal-use vehicles around the world.”

Mike Manley, Chief Executive Officer, FCA: “Our now four-year partnership with Waymo continues to break new ground. Incorporating the Waymo Driver, the world’s leading self-driving technology, into our Pacifica minivans, we became the only partnership actually deploying fully autonomous technology in the real world, on public roads. With this next step, deepening our relationship with the very best technology partner in this space, we’re turning to the needs of our commercial customers by jointly enabling self-driving for light commercial vehicles, starting with the Ram ProMaster. Adding Waymo’s commitment to partner with us to deploy its L4 fully autonomous technology across our entire product portfolio, our partnership is setting the pace for the safe and sustainable mobility solutions that will help define the automotive world in the years and decades to come.”

* Class 1 – 3

The Waymo Open Dataset Challenges were held in five categories: 2D and 3D Detection, 2D and 3D Tracking, and Domain Adaptation. All submissions were evaluated through an automated process and participants could also publish their results to an online leaderboard. The Challenges were truly international with finalists from China, England, Singapore, the US, and many other countries.

Yu Wang, leader of the winning team that came in first in four out of five challenges, said: “We were really excited when the Waymo Open Dataset was first published last year. After years of frustrations with existing small and outdated benchmarks, we could finally move on to a large-scale multi-sensor dataset that was collected for real autonomous driving scenarios.”

Many finalists explicitly mentioned the size of Waymo’s dataset among the factors that made them participate in the Challenges. Shaoshuai Shi, a Ph.D. student in the Multimedia Lab of The Chinese University of Hong Kong, whose team developed the top-performing lidar-only solutions in the 3D Detection, 3D Tracking, and Domain Adaptation tracks, noted that “with such a large dataset, the ablation experiments are more reasonable and reliable since the model is hard to overfit.”

Some participants used this opportunity to try their skills in new areas of research. For example, Almaz Zinollayev, a researcher from Kazakhstan and one of the winners in the 3D Tracking Challenge, said he had mostly worked with 2D images before and wanted to hone his skills in building 3D algorithms. “The nature of 2D and 3D data is entirely different, though it was interesting to see that there are some common ideas that can be successfully applied to 3D object detection and tracking tasks,” remarked Zinollayev. “Learning these new concepts and then building and training the model was certainly challenging but very rewarding.”

The scientists from DAI-Labor, an AI laboratory at Berlin Institute of Technology, who set out to achieve state-of-the-art tracking quality by only using a combination of existing techniques landed third place in the 2D Tracking Challenge. The team said: “The field of machine learning and autonomous driving is changing rapidly. The Waymo Open Dataset is a great contribution to the research community, as it allows us to work on a real world problem without having an autonomous car with expensive sensors.”

While the official Waymo Open Dataset Challenges have come to an end, our leaderboards will remain open to new submissions. We also plan to continue to add to the dataset over time, in the hopes that we will aid the international research community in making advancements in machine perception and self-driving technology.

Congratulations to all the winners and thank you to everyone who submitted their work. We are excited to see what the next generation of researchers will accomplish!

Waymo is now the exclusive global L4 partner for Volvo Car Group, a global leader in automotive safety, including its strategic affiliates Polestar and Lynk & Co. International. Through our strategic partnership, we will first work together to integrate the Waymo Driver into an all-new mobility-focused electric vehicle platform for ride hailing services.