It was seven years ago when Waymo discovered spring blossoms were making his self-driving cars nervous on the brakes. Soap bubbles too. And street torches.

New tests over years of testing revealed more and more distractions for the driverless cars. Their road skills improved, but it was difficult to match the proficiency of human drivers. The crowded streets of America turned out to be a daunting place for a robot.

The wizards of Silicon Valley said people would now commute to work in self-driving cars. Instead, there have been litigation, injuries and deaths, and tens of billions of dollars spent on a frustratingly capricious technology that some researchers say is years away from becoming the next big thing in the industry.

Now the pursuit of autonomous cars is being reset. Companies like Uber and Lyft, worried about using their money looking for autonomous technology, have made it. Only the deepest outfits like Waymo, a subsidiary of Google’s parent company Alphabet, auto industry giants and a handful of startups manage to stay in the game.

The technology and car giants were able to work on their driverless car projects for years. Each will spend an additional $ 6 billion to $ 10 billion before the technology becomes commonplace – according to estimates by Pitchbook, a research firm that tracks its financial activities, sometime towards the end of the decade. But even that prediction might be too optimistic.

“This is a transformation that will take over 30 years and possibly more,” said Chris Urmson, an early engineer on Google’s self-driving car project before it became the Alphabet business unit called Waymo. Today he is the managing director of Aurora, the company that acquired the autonomous vehicle unit from Uber.

So what went wrong? Some researchers wouldn’t say anything – that’s how science works. You cannot fully predict what will happen in an experiment. The self-driving car project happened to be one of the most hyped technology experiments of this century that took place on roads across the country and was run by some of its best-known companies.

This hype attracted billions in investments, but sparked unrealistic expectations. In 2015, electric car maker Tesla’s billionaire boss Elon Musk said fully functional self-driving cars were only two years away. More than five years later, Tesla vehicles offered simpler autonomy designed solely for driving on the freeway. That, too, became controversial after several fatal crashes (which the company attributed to misuse of the technology).

Perhaps no company has seen the turmoil of driverless car development more restlessly than Uber. After poaching 40 robotics experts from Carnegie Mellon University and acquiring a self-driving truck start-up for $ 680 million in stock, the hailship company closed a lawsuit from Waymo, which was followed by a guilty plea by a former executive, who has been accused of stealing intellectual property. A pedestrian in Arizona was also killed in an accident involving one of his driverless cars. In the end, Uber was essentially paying Aurora to purchase its self-driving unit.

But for the deepest pocket companies, they hope, science is always driving one improved ride after another. Waymo reached a remarkable milestone in October, launching the world’s first “fully autonomous” taxi service. In the suburbs of Phoenix, Arizona, anyone can now drive a minivan with no driver behind the wheel. However, that doesn’t mean the company will deploy its technology in other parts of the country immediately.

Dmitri Dolgov, who recently took over the management of Waymo following the departure of John Krafcik, an automotive veteran, said the company is viewing its Arizona service as a test case. Based on what it learned in Arizona, Waymo is building a new version of its self-driving technology that it will eventually use in other regions and other types of vehicles, including long-haul trucks.

The suburbs of Phoenix are particularly good for driverless cars. The streets are wide, there are few pedestrians and there is almost no rain or snow. Waymo supports its autonomous vehicles with remote technicians and roadside assistance teams who can help get cars out of a difficult situation either online or in person.

“Autonomous vehicles can be used in certain situations today,” said Elliot Katz, a former lawyer who advised many of the major autonomous vehicle manufacturers before launching a start-up, Phantom Auto, that provides remote and self-driving software Vehicles when they get stuck in difficult positions. “But you still need someone up to date.”

Self-driving technology is not yet nimble enough to reliably cope with the various situations that human drivers encounter every day. They can usually get around the suburb of Phoenix, but they can’t copy the human chutzpah required to merge with the Lincoln Tunnel in New York or a turn off Highway 101 in Los Angeles.

“You have to pull back each shift before you can see the next shift,” said Nathaniel Fairfield, a Waymo software developer who has been working on the project since 2009, describing some of the distractions the cars are exposed to. “Your car has to be able to drive pretty well before you can really put it in situations where it can do the next more challenging task.”

Like Waymo, Aurora is now developing both autonomous trucks and passenger cars. No company has used trucks with no safety drivers behind the wheel, but Mr Urmson and others argue that autonomous trucks will hit the market faster than anything designed to transport regular consumers.

Long-haul trucks are not passengers who may not be without twitching brakes. The routes are also easier. Once you’ve mastered one section of the freeway, said Mr Urmson, it’s easier to master another. But even driving on a long, relatively straight freeway is extremely difficult. Delivering food orders in a small neighborhood is even more of a challenge.

“This is one of the greatest technical challenges of our generation,” said Dave Ferguson, another early engineer on the Google team who is now president of Nuro, a company that focuses on the delivery of food, pizza and other merchandise.

Mr Ferguson said many thought self-driving technology would improve, like an internet service or a smartphone app. Robotics is much more difficult, however. It was wrong to say otherwise.

“When you look at almost every industry that is trying to solve really tough technical challenges, the people who tend to be a little bit crazy and a little bit optimistic,” he said. “You have to have this optimism to get up every day and hit your head against the wall trying to solve a problem that has never been solved and it is not guaranteed to ever be solved.”

Uber and Lyft aren’t giving up driverless cars entirely. While this may not help for a long time, the bottom line is that they still want to deploy autonomous vehicles by working with the companies that are still working on the technology. Lyft now says autonomous driving could arrive by 2023.

“These cars will be able to travel at certain speeds on a limited number of roads in a limited number of weather conditions,” said Jody Kelman, Lyft general manager. “We’ll be able to use these cars very safely, but they won’t be able to go to that many places.”