California Releases Guidelines for Using Generative AI Tools: What You Need to Know

California has taken a significant step forward in regulating the adoption of generative AI technologies within its state government. New guidelines have been introduced, obliging state officials to conduct comprehensive audits and risk assessments before implementing these innovative systems. This initiative follows a September 2023 executive order by Governor Gavin Newsom, which aimed to establish clear protocols for the responsible use of generative AI.

### Key Guidelines for State Agencies

Under the new regulations, state agencies are required to demonstrate transparency when selecting vendors that offer generative AI products or services. Providers must clearly outline the generative AI features of their solutions. Additionally, the guidelines distinguish between “incidental” and “intentional” acquisitions of generative AI technologies. This distinction is crucial because it allows agencies to receive bids for generative AI responses even when the solicitation primarily concerns non-IT goods, IT services, or telecommunications.

To enhance accountability, only vendors who have successfully navigated a formal written solicitation process will be eligible to provide generative AI systems to state agencies. This ensures that only qualified suppliers are considered. State agencies are also required to engage generative AI experts to assist in drafting and reviewing contracts related to new systems.

### Conducting Risk Assessments

Before proceeding with any procurement, Chief Information Officers (CIOs) must conduct thorough risk assessments of the generative AI technologies being considered. This process is designed to ensure that no system poses a threat to citizens' safety or privacy. Staff responsible for conducting these assessments are guided by the California Generative Artificial Intelligence Risk Management Principles, which provide a structured approach to evaluating potential risks associated with generative AI.

### Workforce Training Requirements

As an essential aspect of this initiative, all state agencies are required to implement workforce training focused on generative AI technologies. Training will commence at the executive level, followed by specialized sessions for legal and privacy experts, with general staff receiving training just before the deployment of generative AI tools. Additionally, every employee must complete training on information privacy and security to ensure a well-informed workforce capable of handling sensitive information responsibly.

### Inventory and Toolkit for Compliance

Moreover, state entities are mandated to maintain an inventory of all generative AI applications and submit this information to the California Department of Technology for review. To facilitate compliance, a user-friendly toolkit has been provided to agencies, designed in plain language to help non-technical staff navigate the rules effectively. This toolkit will be continuously updated to reflect evolving best practices in the use of generative AI technologies.

In summary, California’s proactive approach sets a robust framework for the responsible adoption of generative AI in state government. By prioritizing transparency, risk management, and workforce readiness, the state aims to harness the benefits of these advanced technologies while safeguarding the interests and rights of its citizens.