AI Systems as State Actors

Privacy and Security and Artificial Intelligence

Article Snapshot

Author(s)

Kate Crawford and Jason Schultz

Source

Columbia Law Review, Vol. 119, No. 7, pp. 1941-1972, 2019

Summary

Governments often use artificial intelligence (AI) systems developed by private firms to make key decisions, but disclaim responsibility for problems with the software.

Policy Relevance

Firms that develop AI systems for government should be liable for constitutional violations arising from the system’s operation.

Main Points

  • Governments often acquire AI systems used for public services such as welfare benefits or criminal risk assessment from private vendors; problems with these systems may result in violations of constitutional rights of due process.
     
    • Disabled patients whose cases were reviewed by cost-saving AI systems in Arkansas, Idaho, and other states suffered reductions in care without notice or explanation.
       
    • Fraud detection software in Michigan used to detect fraud in unemployment applications had an error rate of 93 percent.
       
  • When challenged to account for problems with these systems, state governments may disclaim the ability to explain or remedy the system’s operation; state governments insist they cannot be responsible for the operation of something they do not understand.
     
  • AI-based systems and algorithms are used by government to make decisions, but no mechanism exists to hold anyone accountable or liable for the system’s operation.
     
  • Private actors may be held accountable under federal and state constitutional law as state actors under some circumstances:
     
    • If the private actor serves a traditional public function at the behest of government.
       
    • If the private actor is acting under compulsion or at the direction of government.
       
    • When the private actor is a joint participant in government.
       
  • These principles should be applied to private entities that supply AI-based systems for use in government decision-making, so that someone is held accountable for problems with these systems.
     

Get The Article

Find the full article online

Search for Full Article

Share