California and New York lawmakers want to protect users, especially children, from interactions with chatbots that lead to self-harm or other dangers, Titus Wu and Zach Williams report.
Developers would have to warn users about the limits of such technology, and make clear they are not human. In California, first-of-its-kind legislation would establish additional restrictions for chatbot operators, such as third-party audits to ensure compliance. Policymakers are increasingly concerned about how social media and the internet affect children’s mental health. Their scrutiny has now focused upon AI companions, chatbots designed to offer emotional interactions in conversations.
“Our children are not lab rats for tech companies to experiment on at the cost of their mental health,” said state Sen. Steve Padilla (D), author of the California bill. Chatbot companies would have to report annually the number of times a minor user expressed thoughts of suicide, as well as if the software itself brings up the topic of self-harm if the bill is enacted. A New York bill would require warning labels on generative AI systems to warn that its outputs may not be accurate.
Proposals like California’s could have the same privacy concerns that social media-related measures to protect children have faced, said Megan Stokes, state policy director for the Computer and Communications Industry Association. Federal courts have blocked such laws over free speech challenges. Read More
If you or someone you know needs help, call or text the Suicide & Crisis Lifeline at 988.