A lawsuit filed on Monday against Elon Musk’s artificial intelligence firm xAI alleged that photographs of three Tennessee teenagers were used to generate child sexual abuse material (CSAM) with the Grok tool.
The suit, filed in a California federal court, accuses xAI of knowingly designing, marketing and profiting from Grok’s image and video generation, while “refusing” to implement measures aimed at preventing CSAM, attorneys said in a release shared with The Hill.
The suit takes aim at Grok’s “spicy” mode,” which was a controversial feature released last year as part of the AI chatbot. According to the filing, a Grok spokesperson has described spicy mode as a way for creators to explore “edgier, more visually daring narratives.”
“Their lives have been shattered by the devastating loss of privacy, dignity and personal safety that the production and dissemination of this CSAM have caused,” attorneys wrote. “xAI’s financial gain through the increased use of its image and video-making product came at their expenses and wellbeing.”
xAI did not respond to request for comment.
According to the attorneys, one of the victims was informed via direct message that AI-generated images of her and the other two plaintiffs were circulating on the platform Discord and on the messaging app Telegram. At least five of the files allegedly “depicted her actual face and body in settings with which she was familiar but morphed into sexually explicit poses.” Three of the photos used were taken when she was still a minor, per the filing.
The other two plaintiffs learned of the images depicting their likeness through the criminal investigation.
The three plaintiffs’ CSAM files were placed into a national database managed by the National Center for Missing & Exploited Children (NCMEC), attorneys said.
The case was filed as a class action, with attorneys stating it is for all individuals in the U.S. who had real images of themselves as minors altered by Grok to produce sexualized images and videos of their face and other features.
The filing follows a string of scrutiny on the AI company. Amid the backlash, Musk in January said he was unaware of any instances in which Grok generated nude photos of minors. The tech billionaire said Grok does not product illegal material, regardless of user requests, and said any bugs encountered are quickly fixed.
xAI limited Grok’s image and video editing and generation capabilities to paid subscribers to the X Premium+ or SuperGrok.
California Attorney General Rob Bonta (D) also sent a cease and desist letter to xAI in January, demanding it stop producing AI-generated sexualized deepfakes of girls and women online.
Earlier this year, a Reuters investigation found a majority Earlier this year, a Reuters investigation found a majority of lewd requests targeted young women, but did find several cases of Grok creating sexualized images of children.