Explainer-Big Tech wants AI to be regulated. Why do they oppose a California AI bill? By Reuters

By Greg Bensinger

SAN FRANCISCO (Reuters) – California lawmakers are set to vote on a bill this week that would broadly regulate how artificial intelligence is developed and deployed in California even as a number of tech giants voice widespread opposition.

Here’s some background on the bill, known as SB 1047, and why it has faced backlash from Silicon Valley tech experts and some lawmakers:

What does the bill do?

Democratic Sen. Scott Wiener has proposed requiring safety checks on many of the most advanced AI models that cost more than $100 million to develop or require a certain amount of computing power. AI developers operating in the state would also have to come up with ways to shut down their AI models if something goes wrong, effectively a kill switch.

The bill also gives the state attorney general the authority to sue developers if they don’t comply with the law, especially if there’s an ongoing threat, such as AI taking over government systems like the power grid.

The bill also requires developers to hire third-party auditors to evaluate their safety practices and provides additional protections for whistleblowers who speak out against AI abuses.

What did the lawmakers say?

SB 1047 has already passed the state Senate by a vote of 32-1. Last week, it passed the Appropriations Committee of the state Legislature, setting it up for a vote by the full Legislature. If passed by the end of the legislative session on Aug. 31, it will go to Gov. Gavin Newsom for his signature or veto by Sept. 30.

Weiner, who represents San Francisco, home to OpenAI and many startups developing powerful software, said legislation is needed to protect the public before advances in AI become unmanageable or uncontrollable.

However, a group of California Democrats in Congress oppose the bill, including Nancy Pelosi of San Francisco; Ro Khanna, whose district includes much of Silicon Valley; and Zoe Lofgren of San Jose.

Pelosi this week called Senate Bill 1047 ill-conceived and said it could do more harm than good. In an open letter last week, Democrats said the bill could drive developers out of the state and threaten so-called open-source AI models, which rely on code that is freely available for anyone to use or modify.

What do tech leaders say?

Tech companies working on AI — which can respond to prompts with full text, images or voice, as well as perform repetitive tasks with minimal intervention — have called for stronger security barriers to deploying AI. They have cited the risk that the software could one day evade human intervention and invite cyberattacks, among other concerns. But they have also largely rejected Senate Bill 1047.

Weiner revised the bill to please tech companies, relying in part on input from artificial intelligence startup Anthropic — backed by Amazon (NASDAQ:) and Alphabet. Among other changes, he eliminated the creation of a government commission to oversee artificial intelligence.

Weiner also issued criminal penalties for perjury, though civil suits could still be filed.

Alphabet Inc.’s Google and Meta expressed concerns in letters to Weiner. Meta said the bill threatens to make the state unfriendly to the development and deployment of AI. Yan LeCun, the chief scientist at Facebook parent company Facebook, described the bill in a July post on X as potentially harmful to research efforts.

OpenAI, whose ChatGPT software has been credited with accelerating the AI ​​frenzy since its widespread release in late 2022, said AI should be regulated by the federal government and that SB 1047 creates an uncertain legal environment.

In a letter to Weiner, OpenAI said it opposes Senate Bill 1047 because it poses a threat to the growth of AI and could cause entrepreneurs and engineers to leave the state.

Of particular concern is the bill’s potential application to open-source AI models. Many tech experts believe open-source AI models are important for creating less risky AI applications more quickly, but Meta and others have expressed concern that they could be held responsible for policing open-source AI models if the bill passes. Weiner has said he supports open-source AI models, and one of the bill’s recent amendments would have raised the bar for open-source AI models under its provisions.

The bill also has support from the tech sector. Geoffrey Hinton, widely considered the “godfather of artificial intelligence,” Daniel Kokotajlou, a former OpenAI employee, and researcher Yoshua Bengio have all announced their support for the bill.