If you walked out of the headquarters of Google in sunny Mountain View, California, and drove north for about an hour, you’d eventually hit San Francisco, step out of your car, and shiver. Here it was typically several degrees colder, with gray clouds hanging low in the sky. While Google’s hometown had T-shirt weather, you needed a jacket in OpenAl’s urban microclimate. Another big difference: the researchers at OpenAI were giddily excited about the transformer technology that Google’s management wanted to keep in a metaphorical cupboard. For the researchers based in chilly San Francisco, an idea was about to bloom.
The nonprofit lab’s two dozen or so researchers were still busy trying to emulate the success of DeepMind and were hungry to make the next big breakthrough in AI. They had watched AlphaGo defeat the world’s top Go players, and now they were training their own AI agents to play Dota 2, a complex strategic video game similar to World of Warcraft. If an AI agent could steer an elf through a fantasy world, maybe it could capture the messy and continuous nature of the real world better than DeepMind’s AlphaGo could. That seemed, on the face of it, more impressive than moving some black and white stones around on a board.
A mini cold war was also brewing between Sam Altman and Demis Hassabis, and OpenAI’s convivial board member Reid Hoffman was looking for ways to get the two of them to “smoke the peace pipe,” according to someone who heard the comment directly. In 2017, both Altman and Hassabis took part in an AI safety conference in California, set up by the Future of Life Institute. Hoffman was there, and afterward, he tried to set up a dinner between the American start-up guru and the British neuroscientist. Altman didn’t like the idea, arguing that Hassabis was uncooperative and seemingly unconcerned about the existential risks of AI that Altman was trying to prevent. So Hoffman brought Mustafa Suleyman instead. The two got on well, both eager to make the world a better place, and it seemed for a while like their organizations might be making amends.
But behind the scenes, Altman and Hassabis were tussling for the best engineers. Thanks to his Big Tech benefactor, Hassabis now had the upper hand and could offer talented AI researchers far more cash than Altman could, as well as Google stock. Hassabis was known to send emails to OpenAI’s leadership, reminding them that he could outcompete them on acquiring talent. OpenAI managers would show them to engineers they were trying to recruit. “If we’re not going to be successful, why would he send these emails?” a former OpenAI staffer remembers.
Maybe it was because Altman himself was known to personally reach out to engineers at DeepMind to see if they would jump ship, according to someone close to OpenAI. But he generally took a careful, deliberate approach to recruiting, spending about 30 percent of his time on the task and speaking at length to every interviewee, another former employee says. “We went to his place and walked for one hour around [San Francisco’s] Prussian Hill,” says one former staffer about their experience being interviewed by Altman. Once you joined, Altman largely made himself accessible, sitting in the company’s open plan office on his laptop. “Anyone could message him on Slack and talk to him,” they remember. “It wasn’t frowned upon.” In the more hierarchical structure of DeepMind, Hassabis tended to be holed up in an office or meeting room and was harder to pin down. You had to go through other managers and gatekeepers to get his time.
OpenAI was about to differentiate itself from DeepMind in another way. Ilya Sutskever, OpenAI’s star scientist, couldn’t stop thinking about what the transformer could do with language. Google was using it to better understand text. What if OpenAI used it to generate text? Sutskever talked to a young researcher at OpenAI named Alec Radford, who’d been experimenting with large language models. Although OpenAI is best known today for ChatGPT, back in 2017 it was still throwing spaghetti on the wall to see what would stick, and Radford was one of only a handful of people at OpenAI looking at the technology that powered chatbots.
Large language models themselves were still a joke. Their responses were mostly scripted and they’d often make wacky mistakes. Radford, who wore glasses and had an overgrown mop of reddish-blond hair that made him look like a high schooler, was eager to improve on all the previous academic efforts that tried to make computers better at talking and listening, but he was an engineer at heart and wanted a quicker route to progress. For at least six months he’d been hitting brick walls with his experiments, spending weeks on one project and then moving on to the next. He’d trained one language model on two billion comments he scraped from the internet forum Reddit, but it didn’t work well.
When the transformer came out, he saw it at first as a crushing blow from Google. Clearly the bigger company had more expertise in AI. But after a while, it looked like Google didn’t have any big plans for its new invention, and Radford and Sutskever realized they could use the architecture to OpenAl’s advantage. They would just have to put their own spin on it. The transformer model that powered Google Translate used something called an encoder and a decoder to process words. The encoder would process the sentence coming in, perhaps in English, and the decoder would generate the output, like a sentence in French.
Bookmarked
Take a look at what’s new in the business section of Amazon’s bookshelf
Behold the Leviathan: The Unusual Rise of Modern India
Saurabh Mukherjea & Nandita Rajhansa
Published: October 2024
The book tells how modern India’s rise is unexpected and unusual. It argues that 1.5bn Indians are combining to create a range of social and economic outcomes which have no precedent in any emerging economy.
Behind The Startup: How Venture Capital Shapes Work, Innovation and Inequality
Benjamin Shestakofsky
Published: August 2024
The book shows how the demands of VCs create organisational problems that managers solve by combining high-tech systems with low-wage human labour.
Human at Work: Arm Yourself to Thrive in a Fast-Changing Workplace
Richard Lobo
Published: September 2024
The book highlights the need for companies to ensure collaboration and foster employee well-being while being inclusive and sustainable.
Booming Digital Stars: 11 Inspiring Adventures from India’s Creator Economy
Harsh Pamnani and Manish Pandey
Published: August 2024
In the crowded market of creators, the chances of getting lost are higher than those of getting noticed. This book covers lessons from the journeys of creators to navigate this economy.
Just one email a week