AI Systems as State Actors

Article Source: Columbia Law Review, Vol. 119, No. 7, pp. 1941-1972, 2019
Publication Date:
Time to Read: 2 minute read
Written By:

 Jason Schultz

Jason Schultz



Governments often use artificial intelligence (AI) systems developed by private firms to make key decisions, but disclaim responsibility for problems with the software.


Policy Relevance:

Firms that develop AI systems for government should be liable for constitutional violations arising from the system’s operation.


Key Takeaways:
  • Governments often acquire AI systems used for public services such as welfare benefits or criminal risk assessment from private vendors; problems with these systems may result in violations of constitutional rights of due process.
    • Disabled patients whose cases were reviewed by cost-saving AI systems in Arkansas, Idaho, and other states suffered reductions in care without notice or explanation.
    • Fraud detection software in Michigan used to detect fraud in unemployment applications had an error rate of 93 percent.
  • When challenged to account for problems with these systems, state governments may disclaim the ability to explain or remedy the system’s operation; state governments insist they cannot be responsible for the operation of something they do not understand.
  • AI-based systems and algorithms are used by government to make decisions, but no mechanism exists to hold anyone accountable or liable for the system’s operation.
  • Private actors may be held accountable under federal and state constitutional law as state actors under some circumstances:
    • If the private actor serves a traditional public function at the behest of government.
    • If the private actor is acting under compulsion or at the direction of government.
    • When the private actor is a joint participant in government.
  • These principles should be applied to private entities that supply AI-based systems for use in government decision-making, so that someone is held accountable for problems with these systems.



Kate Crawford

About Kate Crawford

Kate Crawford is a Research Professor of Communication and Science and Technology Studies at USC’s Annenberg School for Communication and Journalism and a Senior Principal Researcher at Microsoft Research in New York. Professor Crawford is a leading scholar of the social and political implications of artificial intelligence. Over her 20-year career, her work has focused on understanding large-scale data systems, machine learning and AI in the wider contexts of history, politics, labor, and the environment.