When “The Devil Made Me Do It” Is Not a Defense: Lessons in AI Governance and Organizational Oversight from an SDNY Decision

A recent Southern District of New York decision involving the government’s use of ChatGPT to evaluate DEI-related grants is emerging as an important warning sign for companies deploying generative AI in operational decision-making. In American Council of Learned Societies v. National Endowment for the Humanities, the court rejected the argument that problematic AI-generated outputs could be blamed on ChatGPT itself, emphasizing that organizations remain responsible for the tools they choose to use. This article explores how the opinion highlights growing legal and regulatory expectations around AI governance, including meaningful human oversight, prompt design, accountability structures, and documentation practices.