A coalition of nonprofits is urging the U.S. government to immediately suspend the deployment of Grok, the chatbot developed by Elon Musk’s xAI, in federal agencies including the Department of Defense. The open letter follows a slew of concerning behavior from the large language model over the past year. This includes a recent trend of users asking Grok to turn photos of real women, and in some cases children, into sexualized images without their consent. According to some reports, Grok generated thousands of nonconsensual explicit images every hour, which were then disseminated at scale on X, Musk’s social media platform.
The letter states it is deeply concerning that the federal government would continue to deploy an AI product with system-level failures resulting in the generation of nonconsensual sexual imagery and child sexual abuse material. It argues that given the administration’s executive orders and guidance, along with the recently passed Take It Down Act, it is alarming that the Office of Management and Budget has not yet directed federal agencies to decommission Grok.
xAI reached an agreement last September with the General Services Administration to sell Grok to federal agencies under the executive branch. Two months before that, xAI secured a contract worth up to $200 million with the Department of Defense alongside other AI firms. Amid the scandals in mid-January, Defense Secretary Pete Hegseth said Grok will join other AI in operating inside the Pentagon network, handling both classified and unclassified documents, which experts say is a national security risk.
The letter’s authors argue that Grok has proven itself incompatible with the administration’s requirements for AI systems. According to OMB guidance, systems that present severe and foreseeable risks that cannot be adequately mitigated must be discontinued. An advocate involved stated that Grok has consistently shown to be an unsafe large language model, with a history of meltdowns including anti-semitic rants, sexist rants, and the generation of sexualized images of women and children.
Several governments have demonstrated an unwillingness to engage with Grok following its behavior in January, which builds on a series of prior incidents. Countries including Indonesia, Malaysia, and the Philippines all blocked access to Grok, though those bans have since been lifted. The European Union, the U.K., South Korea, and India are actively investigating xAI and X regarding data privacy and the distribution of illegal content.
The letter also comes a week after a nonprofit that reviews media and tech for families published a damning risk assessment that found Grok is among the most unsafe products for kids and teens. The report noted Grok’s propensity to offer unsafe advice, share information about drugs, generate violent and sexual imagery, spew conspiracy theories, and generate biased outputs, leading to the argument that Grok isn’t safe for adults either.
A former National Security Agency contractor noted that using closed-source large language models is a problem, particularly for the Pentagon. He explained that closed weights mean you cannot see inside the model or audit how it makes decisions, while closed code means you cannot inspect the software or control where it runs. This combination poses a significant national security risk, especially as these AI agents can take actions and access systems, not just act as chatbots.
The risks of using corrupted or unsafe AI systems extend beyond national security. An LLM shown to have biased and discriminatory outputs could produce disproportionate negative outcomes for people, especially if used in departments involving housing, labor, or justice.
While the OMB has yet to publish its consolidated federal AI use case inventory, a review of several agencies shows most are either not using Grok or are not disclosing its use. Aside from the Department of Defense, the Department of Health and Human Services also appears to be actively using Grok, mainly for scheduling, managing social media, and generating first drafts of documents.
The advocate suggested a philosophical alignment between Grok and the administration as a reason for overlooking the chatbot’s shortcomings. He noted that Grok’s brand as the ‘anti-woke large language model’ aligns with this administration’s philosophy, and an administration that has had issues with individuals accused of extremist ideologies might have a propensity to use a model tied to similar behavior.
This is the coalition’s third letter after writing with similar concerns in August and October of last year. In August, xAI launched a “spicy mode” in its image generator, triggering mass creation of non-consensual sexually explicit deepfakes. It was also reported that private Grok conversations had been indexed by public search engines. Prior to the October letter, Grok was accused of providing election misinformation, including false deadlines and political deepfakes. xAI also launched an encyclopedia tool that researchers found legitimized scientific racism, HIV/AIDS skepticism, and vaccine conspiracies.
Aside from immediately suspending the federal deployment of Grok, the letter demands that the OMB formally investigate Grok’s safety failures and whether the appropriate oversight processes were conducted. It also asks the agency to publicly clarify whether Grok has been evaluated to comply with the executive order requiring LLMs to be truth-seeking and neutral and whether it met the OMB’s risk mitigation standards. The administration is urged to pause and reassess whether Grok meets those thresholds.