AI Systems as State Actors
Search for the full article on Bing
ARTICLE SUMMARY
Governments often use artificial intelligence (AI) systems developed by private firms to make key decisions, but disclaim responsibility for problems with the software.
POLICY RELEVANCE
Firms that develop AI systems for government should be liable for constitutional violations arising from the system’s operation.
KEY TAKEAWAYS
- Governments often acquire AI systems used for public services such as welfare benefits or criminal risk assessment from private vendors; problems with these systems may result in violations of constitutional rights of due process.
- Disabled patients whose cases were reviewed by cost-saving AI systems in Arkansas, Idaho, and other states suffered reductions in care without notice or explanation.
- Fraud detection software in Michigan used to detect fraud in unemployment applications had an error rate of 93 percent.
- Disabled patients whose cases were reviewed by cost-saving AI systems in Arkansas, Idaho, and other states suffered reductions in care without notice or explanation.
- When challenged to account for problems with these systems, state governments may disclaim the ability to explain or remedy the system’s operation; state governments insist they cannot be responsible for the operation of something they do not understand.
- AI-based systems and algorithms are used by government to make decisions, but no mechanism exists to hold anyone accountable or liable for the system’s operation.
- Private actors may be held accountable under federal and state constitutional law as state actors under some circumstances:
- If the private actor serves a traditional public function at the behest of government.
- If the private actor is acting under compulsion or at the direction of government.
- When the private actor is a joint participant in government.
- If the private actor serves a traditional public function at the behest of government.
- These principles should be applied to private entities that supply AI-based systems for use in government decision-making, so that someone is held accountable for problems with these systems.
QUOTE
TAGS
About Kate Crawford
Kate Crawford is a Research Professor of Communication and Science and Technology Studies at USC’s Annenberg School for Communication and Journalism and a Senior Principal Researcher at Microsoft Research in New York. Professor Crawford is a leading scholar of the social and political implications of artificial intelligence. Over her 20-year career, her work has focused on understanding large-scale data systems, machine learning and AI in the wider contexts of history, politics, labor, and the environment.
