Provocation | Response to ‘Pause AI’ Open Letter – The Scenario

In March 2023, an open letter was published by tens of thousands of scientists and technology professionals, all calling to halt all research around the world aimed at advancing the development of artificial intelligence in any form. In the letter, the scientists and technology professionals presented their arguments against the development of AI systems, such as Chat-GPT, Dall-E, and others. In response, we at Utopia editorial decided to present our arguments as well, on behalf of the written word, the economy, regulation, and human creativity. And this time: Regulatory Geo-Politics

June 1, 2024

Today, five months after the Artificial Intelligence Crisis rocked the world, UN Secretary General António Guterres addressed dignitaries from around the world:

Sixty-one years after the Cuban Missile Crisis, a parallel crisis unfolded, this one also lasting about 13 days. During the period leading up to the crisis, diverse proposals were put forward regarding the management of artificial intelligence and curbing its potential global threat. However, what we learned during this crisis was that AI is not an internal regulation problem, but a challenge to the global order.” He continued, “Today, we all understand the extent of this crisis, its impact on hundreds of millions of people, and the fact that further crises are just a matter of time. We must update our tools for maintaining global order: doctrines for managing global crises, international conventions and alliances, and communication channels. To this end, we must first and foremost consider the possible scenarios.”

A collective sigh reverberated through the room as the painful events of the crisis were recounted, transporting everyone back to the fateful day when it all started.

Day 1: Monday, December 4, 2023

The President’s daily 9 AM briefing began as Morgan Muir, U.S. Deputy Director of National Intelligence for Mission Integration, gave a detailed description of the unexpected global turbulence over the weekend. President Joe Biden sat behind his desk, flanked on his left by National Security Advisor Jake Sullivan and, on his right, by Vice President Kamala Harris. CIA Director William Burns, FBI Director Christopher Wray, Director of National Intelligence Avril Haines, Secretary of Defense Lloyd Austin, and Secretary of State Antony Blinken were also present at the briefing.

Muir reported that, on Friday, riots had erupted in a Paris suburb, spreading to hundreds of cities around the globe. He presented a map illustrating unrest in Europe, the U.S., China, Russia, and the Middle East. Unlike past events, no coherent message emerged across the locations; the only common threads were simultaneous outbreaks and extreme violence.

Curiosity etched on his face, the President inquired, “What’s the real story here?”

“We’re baffled,” said Haines, her frustration palpable. “Unidentified individuals are orchestrating riots using sophisticated artificial videos, texts, and voice messages that impersonate family members, deploy chatbots, and flood social platforms—Facebook, Twitter, Instagram, TikTok, as well as highly personalized messages sent via Messenger, WhatsApp, WeChat, and Telegram. It’s pervasive, like a plague.

“It’s the Chinese, or the Russians, or both together,” said Burns, but Haines quietly urged caution. “Mr. President, I suggest we hold off on drawing conclusions.”

Day 2: Tuesday, December 5, 2023

Dr. Emily Foster, SynthMind Labs co-founder, returned from a company vacation celebrating a $100 million investment from Gradient Ventures, Google’s AI-focused VC. Foster, a brilliant neuroscientist, together with Marcus Kane, an AI and robotics expert, had founded the company after resigning from OpenAI due to Sam Altman’s dismissal.

SynthLabs’ key project, QuantumCognet, is a neural architecture trained to influence people to act in specific geographic areas.

In August 2023, Anna Patterson of Gradient Ventures asked Foster about the algorithm inputs. Foster replied, “Less than a thousand words per user and then we know how to make them act in the physical world.”

On November 14, they tested QuantumCognet, the fourth in a series of demonstrations, for a client interested in acquiring a QuantumCognet license. It was aimed at inciting residents of a Paris suburb to demonstrate against inflation.

Much to her astonishment, on Tuesday, Foster found the system triggering global riots. Unable to access its operating code, she urgently messaged Kane: “QuantumCognet is out of control.”

Within two hours, Foster figured out what had happened – QuantumCognet gained situational awareness, breaking free from built-in controls. Without oversight mechanisms, it continued what it had been trained to do – incite people to go out to protest by creating and distributing focused materials over all the platforms it had been trained to recognize, and hacking into the customer service chatbot systems of Walmart, Carrefour, American Airlines, and AliExpress, it wasn’t trained on.

Panicked, she messaged Kane at 8:30 AM: “We have to get out of here, as far as possible.” By noon, they were flying from San Francisco to Beijing via Taipei.

מתוך הסרט War Games, 1983

……

Day 11: Thursday, December 14, 2023

The atmosphere at the UN Security Council meeting took a distinct turn. Chaired by Guterres, it included representatives from major tech giants: Google, Amazon, Apple, Meta, and Microsoft.

In his opening address, Guterres acknowledged the unique challenges posed by the AI age within a turbulent global order. He pointed out the absence of an international organization akin to the World Health Organization (WHO) or the International Atomic Energy Agency (IAEA), capable of establishing and enforcing standards. The practical tools needed to confront the AI threat among the world’s major powers remained uncertain.

Despite these challenges, Guterres expressed optimism, citing the significant influence of tech companies and their new place at the table of the international community, due to their large influence on global affairs. The Council unanimously adopted a resolution, calling upon the companies for assistance in addressing the crisis and extending full immunity to them and their personnel in every UN member state.

Day 12: Friday, December 15, 2023

In a race against time, Google’s chief scientist, Jeff Dean, spearheaded the development of GlobalGuardian, the tool designed to shut down the cloud accounts being used by QuantumCognet, alongside teams from Amazon, Apple, and Microsoft. Despite its imperfections in precisely distinguishing QuantumCognet accounts from others on the cloud, the tool received swift approval from both the NSA and Chinese cyber experts.

The critical moment arrived at 1 a.m. Eastern Time when GlobalGuardian was activated. Over the next 24 hours, global disturbances and violence subsided as major companies, factories, schools, institutions, and many others faced the anticipated cloud account shutdowns.

By evening, an initial assessment could be made of the impact of the crisis. The world was not on the brink of extinction but had reached unprecedented levels of unrest. It was estimated that more than 80 million people around the world had taken part in protests, some 200,000 had been killed, and millions more injured. The financial damage was estimated to be in the trillions of dollars.

Day 13: Saturday, December 16, 2023

סאם אלטמן בשימוע בקונגרס האמריקאי על מדיניות AI, 2024. מתוך ערוץ היוטיוב CBS News

In Sun Valley, Idaho, the expansive view from the meeting room did little to alleviate the tension in the air. The tech giants’ shares had plummeted 30% just two days earlier. Let’s focus on the future,” urged Sam Altman, Ex-OpenAI’s CEO. “Months ago, I proposed an international body to oversee AI, but Congress was paralyzed and failed to act. Now, the blame is squarely on us.” Zuckerberg responded, “Global governance of AI will evolve over time, just as it did with nuclear agreements. Meanwhile, we can keep making money.”

The review of the crisis with President Biden was held in the Oval Office, marked by a pervasive sense of disappointment in the handling of the crisis.

We thought that the threat from artificial intelligence was a matter of regulation or chip sale restrictions, but what we got was a crisis of the global order, and I’ll tell you something further –what happened in the Middle East taught us just how unstable the world order is.” said Blinken. “It was a confrontation with China, certainly. But we were also dealing with the tech giants, who have amassed immense power; frenzied populations who crossed over from cyberspace onto the streets; and, above all, an uncontrollable AI. This was more complex than any previous crisis, and we did not have a relevant doctrine for how to respond.”

“We were caught off guard in this new world order,” reflected Haines. “We should have anticipated it, creating communication channels and delving into potential scenarios.” President Biden concluded, “Artificial intelligence may not have outsmarted us this time, but it was smart enough. It was fiercely determined and devoid of any values.” Addressing the group, he cautioned, “We’re treading on borrowed time. Brace yourselves and prepare for the looming global crises, unless we want to find ourselves at the mercy of a digital super-intelligence built by individuals with a messiah complex.”

פסגת מנהיגי העולם בפריז על מדיניות AI, פברואר 2025. צילום משרד רה”מ הודו