But in a separate Fortune editorial from earlier this month, Stanford computer science professor and AI expert Fei-Fei Liargued that the "well-meaning" legislation will "have significant unintended consequences, not just for California but for the entire country."

The bill's imposition of liability for the original developer of any modified model will "force developers to pull back and act defensively," Li argued. This will limit the open-source sharing of AI weights and models, which will have a significant impact on academic research, she wrote.

Holy shit this is a fucking terrible idea.

Wtf does a kill switch even mean? PCs have kill switches on them already, in the form of a power switch.

IRL, arms manufacturers claim they're not culpable when their products are used to blow up civilians. They point at the people making decisions to drop the bombs as the ones responsible, not them.

This legislature tries to get ahead of that argument, by putting reponsibility for downstream harm on the manufacturers instead of their corporate or government customers. Even if the manufacturer moves their munitions plants elsewhere, they're still responsible for the impact if it harms California residents. So the alternative isn't to move your company out of state. It's to stop offering your products in one of the largest economies in the world.

The intent is to make manufacturers stop and put up more guardrails in place instead of blasting ahead, damn the consequences, then going, oops 🤷🏻‍♂️

There will be intense lobbying with the Governor to get him to veto it. If it does get signed, it'll be interesting to see if it has the intended effect.

You would have assumed that legislators in California of all places would have access to experts that could explain to them why this won't work.

As we've previously explored in depth, SB-1047 asks AI model creators to implement a "kill switch" that can be activated if that model starts introducing "novel threats to public safety and security,"

A model may only be one component of a larger system. Like, there may literally be no way to get unprocessed input through to the model. How can the model creator even do anything about that?