Compilation/Youya
Apple's car building project "Titan" has been going through eight years. In the past eight years, four persons in charge have changed, and the development goals have also changed. In this way, Apple has experienced many challenges such as meaningless demonstrations, false hopes, and map obstacles in the process of making efforts to "fully automated driving" cars.
In August last year, Apple sent several autonomous vehicle prototypes for a "long-distance" trek of about 40 miles (64 kilometers) in Montana. The relevant Apple managers took this journey from Bozeman to Big Sky, a ski resort, with drones, so that they could make a beautiful documentary film. With the picturesque mountains as the background, they showed CEO Tim Cook that Titan, a money burning and time-consuming project, was making progress.
Inside Apple, executives praised the success of the demonstration: tests showed that these vehicles did not rely on high-definition 3D road maps for driving, and most of their friends' autonomous cars could not do without them. Titan's project managers hope that one day they can abandon these expensive and inextensible high-definition maps and build a truly fully automatic driving car - it can drive almost anywhere in the world without a steering wheel or pedal. Apple will be fully responsible for the behavior of the car and the safety of passengers.
The good times are not long. Two insiders involved in the project said that Apple's test vehicle was a modified Lexus SUV, and it was difficult to drive on the street near the Silicon Valley headquarters without a map. Sometimes it hit the kerb, and sometimes it could not drive across the intersection normally. An insider said that earlier this year, a test car almost ran into a jogger crossing the road.
All kinds of signs highlight that Apple Automotive is facing considerable challenges in safety. After eight years of project establishment, the objectives are constantly changed, and the person in charge is also frequently changed. Behind the turbulence, the software problem is much more difficult than expected, and the uncertainty of the listing date is one of the main reasons for the turnover of senior executives.
This demonstration also reflects a common business mistake made by Apple and other companies when developing autonomous driving projects: wasting a lot of time on technologies with minimal application scenarios, that is, designing demonstrations along specific routes and using techniques that are "effective here but almost useless elsewhere". This phenomenon is called "demoware". Some Titan R&D personnel said that Apple's involvement in "demos" is much more serious than that of many friends.
Arun Venkatadri, who was engaged in research on autonomous vehicles in Uber, said: "As long as the money is in place, autonomous vehicles can handle almost any fixed route. The problem is whether the corresponding software can be expanded and operated on more routes."
To make matters worse, Ian Goodfellow, a project manager, recently resigned. He is a famous machine learning expert who has been responsible for "injecting deep learning into Apple's autopilot software".
Apple's plan to develop autonomous cars is almost an open secret in Silicon Valley. Although many autonomous car developers are touting this "disruptive" technology, Apple has not yet publicly acknowledged it. Waymo under Alphabet, the parent company of Google, Cruise of General Motors and Zoox of Amazon are the giants who want to run robot taxis similar to Uber and launch them at different times, while Apple's goal is to sell cars. This means that Apple faces a much bigger challenge: its automatic driving can not only work in a specific area, but can run anywhere, so that people can buy it.
Apple automatic driving test vehicle modified based on Lexus SUV Based on public documents and statements of some autonomous vehicle developers, these companies have spent more than 30 billion dollars on related projects, but they have basically no revenue at present. This technology is far from ready for large-scale operation, and we don't seem to know how long it will take for commercial use. It is also unknown what breakthrough technologies are needed for cars to predict the movement of cars, bicycles and pedestrians around them. This is also a key challenge for autonomous vehicles to reach or surpass human drivers.
But Tesla and some traditional car companies can earn billions of dollars every year by using some automatic driving functions such as automatic cruise, lane keeping and automatic steering. This raises a question: Is it a major business mistake for Apple to put its chips on automatic driving?
"Cook gene" makes the foundation of car building unstable?
These bets mean a lot to Apple. In addition to the hybrid reality (MR) head display that is still under development, Titan is a great effort made by Apple to launch a new field since the death of Jobs in 2011. At present, the Titan project is led by John Giannandrea, director of Apple machine learning and AI, and has thousands of members.
Titan and MR head display projects face the same problem: Cook and Jobs have different styles. Jobs went deep into product design and pointed out the direction, while Cook and Titan kept a distance. Several people involved in the project said that Cook rarely visited Titan's California office. Some former employees said that this was a major reason why the project did not improve. Titan has never had a leader who can explain its products. Frequent strategic changes have exhausted its members. Cook's unwillingness to commit to mass production also discouraged some Titan executives.
The support of Apple's senior management team for Titan is also uneven. Craig Federighi, senior vice president of software engineering, has always expressed doubts about this. He has a good reputation at Apple and is responsible for software development, but he did not directly participate in the Titan project. Due to the frequent change of leadership and strategy, and the consequent layoffs, Titan became the object of ridicule of other business departments of Apple. Several former employees said that some managers had taken the initiative to warn employees to stay away from Titan.
Craig Federidge, Senior Vice President of Apple Software Engineering Titan is burning money. The annual cost is more than $1 billion, but for Apple, it is a drop in the bucket. After all, the annual R&D expenditure is more than $22 billion. In the eight years since Titan set up the project, Apple spent more than 430 billion dollars on repurchasing shares. An Apple spokesman declined to comment.
Sail toward the "physical vehicle", i.e. the inner bucket
In the past eight years, Titan has changed his coach four times, and each of them has different ambitions. In 2014, when Apple Watch was about to launch, Cook decided to extend his business to electric vehicles. At that time, Tesla had been selling cars for six years, and Google had been testing autonomous cars for five years. Two insiders said that from Apple's strength in engineering and design, it should not be difficult for Cook to build cars.
Titan has four core members, nicknamed "Four Heavenly Kings" by Apple, namely Steve Zadesky, Benjamin Lyon, John Wright and DJ Novotney, who reported to Dan Riccio, then Apple's director of hardware engineering. In the Apple executive team, Riccio is Titan's main supporter.
Although Riccio is the head of Titan, he does not have much "involvement". Zadesky is in charge of a lot of work, and the latter is the vice president responsible for Titan product design. Zadesky has always been a mechanical engineer of Ford Motor Corporation. He wants to take Apple to build a traditional electric car and overtake Tesla with more advanced "assisted driving" functions; But his colleagues have higher goals, such as fully automatic driving. The inner bucket is thus generated.
Apple has cooperated with Magna Steyr, a luxury car manufacturer, to build cars, initially designed similar to minivans. Apple focused on passenger experience rather than automatic driving. Apple has imagined how a car can detect a driver's heart attack and send him to the doctor, and also has thought about the surround sound that can eliminate noise.
In less than two years, Apple has made cars in many directions. Although the car building project was only regarded as a "technical investigation" at that time, Apple told its employees that it hoped to sell the car in 2019 as soon as possible. Zadesky also hired experts from the automotive industry to design doors, lighting and interior decoration, and began to contact with parts suppliers.
Half of Titan's employees are from other teams in Apple's consumer electronics department. Apple hopes to use new methods to build cars. Its industrial design team put forward some radical ideas, such as whether glass can be the main material.
Apple hopes to develop its automotive engineering knowledge without relying on decades of experience in the industry. This decision has brought major challenges. The engineers consider it a waste of time.
Diversion into software, with the goal of "iPhone in the automotive industry"
At the end of 2015, after Cook's vacillation to car building and facing new cost and safety risks, Titan stagnated, while Zadesky also chose to leave for personal reasons. About six months later, Riccio persuaded retired Apple executive Bob Mansfield to come back and restart Titan.
Early Titan leaders Steve Zadsky (left) and Bob Mansfield (right) After the change of leadership, Titan also suspended the development of the aforementioned cars and turned to full automatic driving software.
One of the major reasons for this change was the powerful influence of Jonathan Ive, then Apple's industrial design director, who firmly opposed the manufacturing of traditional electric vehicles. Ivy believes that autonomy is to Apple cars as multi touch is to iPhones. If Apple wants to build a car, it must create an iPhone in the mobile phone industry and turn the whole industry upside down.
It is worth mentioning that in mid July, Ivy's 30-year cooperation with Apple ended.
Jonathan Ivey, former chief designer of Apple (left) and Dan Riccio, senior vice president of hardware engineering (right) In 2016, when Mansfield terminated its cooperation with Magna Steyr, the appearance of Apple cars had been redesigned and became a compact car similar to BMW i3. But the autopilot software is not ready yet. Many Titan employees have been transferred to the "Automatic Driving Test Vehicle" department (code name "Baja") or the R&D department of key automotive components (battery, engine and powertrain).
BMW i3 A number of people involved in the project said that in 2017, Mansfield believed that Apple had a long way to go to improve the automatic driving software, so it decided to launch the software in stages and set relatively practical goals. He focused on the cooperation with Volkswagen to develop the self driving shuttle bus, which was from Palo Alto, California to Infinite Loop, the headquarters of Apple.
However, this goal has changed since then, to automate the multiple bus routes to and from work near the Santa Clara Valley. Finally, Mansfield stopped cooperating with Volkswagen, gave up manufacturing physical shuttles, and focused on preparing test vehicles for the above routes.
The stumbling block is hard to overcome, and the strategy of "software first" is stranded
At this time, Apple has already fallen into the "demo". The team invested in the development of autopilot software and then demonstrated it to the executives. However, two people involved in software development said that they could not make subsequent extensions or add functions to the software.
Apple has made some progress in real world testing. In 2017, Apple signed a lease purchase agreement to take over a test track that once belonged to Chrysler near Phoenix, Arizona. According to public documents, Apple acquired the track last year through a shell company for 125 million dollars. In addition to simulation and testing on public roads, Apple also tested automated driving software here. Apple built a simulated street on this track, using shipping containers as buildings, and hired contract workers to act as pedestrians or cyclists. This "three pronged approach" is similar to Waymo and other friends.
Apple's test track in Robo City, Arizona In the same year, Apple's first test cars began to run near the headquarters and on the Arizona track. The cost of each modified Lexus car is up to $1 million, mainly because of the equipment they carry, including 14 Velodyne laser radar sensors, which use lasers to determine the distance of objects. (People familiar with the matter said that Apple later developed its own customized laser radar.).
However, Mansfield's leadership over Titan is limited. A person familiar with the matter said that he was a chip engineer and had no experience in automatic driving software or cars, so he could only give the overall direction. Mansfield is only returning to Apple part-time this time, and the ultimate goal is to find a full-time leader for Titan.
Retreat from difficulties and re study physical vehicles
In 2018, Mansfield finally found this person: Doug Field. Field once served as the vice president of Apple Mac hardware engineering, and later worked in Tesla for five years. He participated in the design of Model 3 and supervised its production.
Like Zadesky, Field started his career from Ford, which is an old gun in the automotive industry. After returning to Apple, Field cut about 200 people from the Titan team, and once again took the design of physical cars as the team's primary task. Two people who worked with Field during this period said that his goal was to sell cars, not to operate robotaxi services. Field has automobile delivery experience, but lacks software experience.
Apple makes cars with immature software.
In 2018, a former Titan engineer was driving along a simple route with intersections and T-junctions in Sunnyvale, California, in a test vehicle, and found that the software basically handed over control to the standby human security officer throughout the whole process (this insurance measure would be taken when the system did not know what to do.)
Titan's managers prefer to use deep learning as a way to solve the problem of automatic driving software. Johnny Srouji, head of Apple's silicon business, helped them manufacture a customized chip, code named "Tinos", which is used to train the deep learning model to help Apple's automated driving software recognize objects and traffic signs (this process is called "perception").
So, under the leadership of the third person in charge, Titan began to embrace deep learning.
Apple programmers manually code the rules to be followed by the car according to the content detected by the perception software. This method is relatively risky and involves deep learning to identify patterns of human driving styles to determine the above rules and generate vehicle driving routes. Experts expressed concern that this method has not yet been verified and may have to deal with massive data to derive all possible rules.
In 2019, Apple recruited a widely respected researcher of in-depth learning, Goodfellow. He once worked at Google and introduced in-depth learning for Titan's "prediction and route planning" software. In 2020, Apple appointed AI director Janan Draya, who also worked at Google, to supervise Doug Field and Titan projects, highlighting the importance of deep learning.
Ian Goodfellow, Apple Deep Learning Researcher Executive presentations continue. Just before the outbreak of COVID-19 in 2020, Cook took a test car in Silicon Valley and completed a safe trip. An insider said that the car even automatically carried out a DMV (California Traffic Control Bureau) certified driving test to show its ability. Field told the team that the demonstration was successful.
Not every Titan member is shocked by this. In order to complete this demonstration, Apple engineers took a long time to "sort out" this route thoroughly with laser radar and other sensors. However, a person familiar with the matter said that the car could not deviate from the preset route. Once the automatic driving software fails, a human driver must take over.
The "landing" of Apple Car is far away
Under the leadership of the third pilot, Titan ushered in a period of stable development. Some former employees believe that Apple is likely to successfully develop a car and release it during the field tenure. But in September 2021, Field announced his resignation and returned to Ford as the director of advanced technology and embedded software. Two insiders said that Field, like Zadesky, was difficult to obtain commitments on mass production from Apple executives. Although he can understand that Apple's autopilot software is not yet ready.
Before Field resigned, two senior managers of Titan had left, Lyon in charge of sensors and Jaime Waydo in charge of operational security. Since Field resigned, at least three other Titan executives have left. In addition, there are also some middle-level personnel leaving. This has raised questions about whether Apple can continue to develop cars.
Apple appointed veteran Kevin Lynch to replace Field. Lynch was responsible for the software of Apple Watch. In May this year, researcher Goodfellow told the team that he would resign from Apple because the company required employees to return to the office.
Kevin Lynch (left), the fourth person in charge of Titan, and Doug Field (right), the third person in charge of Titan The physical design of Apple cars continues. An insider said that the designer Ive had told the Titan team that the focus should be on the unique design of the vehicle rather than hiding the sensor. The current design features that all four seats face into the car, so that passengers can talk to each other; And a curved roof similar to the Volkswagen Beetle.
Designers are still experimenting with a trunk. It can be raised to make it more convenient for the car owner to use the storage space in the trunk, and then it will automatically lower. The designers also discussed the large screen of the seat, which rises from the back of the seat and falls back when not in use. They also discussed the possibility of passengers sleeping on their back in the car, although safety concerns may be waived. People familiar with the matter said that Apple also hoped to obtain an exemption from the National Highway Traffic Safety Administration (NHTSA) for "no steering wheel and brakes", and hired Jonathan Morrison, the chief legal adviser of NHTSA, to help last year.
Titan employees are discussing how to modify the new version of the autonomous driving car so that Apple can finalize the version and achieve the goal of going on the road as early as next year.
"Joker Event"
Apple is testing vehicles in cities with more rain and snow, such as Los Angeles, San Diego, Seattle and around Lake Tahoe, to collect data on roads and local driving behavior. A person familiar with the matter said that earlier this year, Apple also planned to launch businesses in Portland, Las Vegas and Denver. But it is unclear when Apple will be ready to test its autopilot software in these new areas.
There are signs that Apple's autopilot software is still immature. According to the data submitted by Apple to California regulators, the cancellation of high-definition maps has brought many problems to the test vehicles. In the test vehicle, the human safety officer needs to take over the vehicle frequently. In the first quarter of this year, an Apple test car traveling at 15 miles per hour almost ran into a jogger crossing the road on an unmarked crosswalk. Apple's automatic driving software recognized joggers as "stationary objects", then reclassified them as "stationary people", and finally successfully identified them as "moving pedestrians". All this, before and after less than a second.
But the car didn't stop, just slightly adjusted the route. A person familiar with the matter said that the car stopped a few feet away from the pedestrian thanks to the safety officer's slamming on the brake at the last moment.
For this reason, Apple suspended the test to investigate the so-called "jogger incident". After solving the identification problem and adding specific crosswalks to the map database, the test was resumed a few days later.
