Digital Dystopia

Artificial Intelligence, Privacy and Security, Innovation and Economic Growth and Networks, the Internet, and Cloud Computing

Article Snapshot

Author(s)

Jean Tirole

Source

American Economic Review, Vol. 111, No. 6, pp. 2007-2048, 2021

Summary

Autocratic states could use social credit scoring systems to discourage dissent. When social ties are strong, citizens are more likely to question a state’s social credit score.

Policy Relevance

Understanding social credit systems will help design safeguards against abuse.

Main Points

  • China’s social credit scoring system, launched in 2014, will lead other governments to adopt similar systems; analyzing how such systems could lead to a dystopic society could help us design legal safeguards.
     
  • A state might prefer to enforce the law through social scoring rather than traditional methods such as jail and fines, because enforcement through social scoring is less expensive and the necessary data might be easier to acquire.
     
  • States can use social sanctions to force citizens to adopt approved political, social, or religious attitudes; each agent decides to dissent or comply, depending on their intrinsic motivation to do the right thing and the psychological cost of compliance.
     
  • Autocratic regimes are more likely to use a “bundled” social score, tracking activities that everyone agrees are beneficial alongside compliance with the state’s controversial objectives; social scoring can enable increased political control.
     
  • In societies with strong social ties, agents’ knowledge of one another enables them to question state social scores, weakening the state’s ability to use the social score to enforce compliance; also, governments might seek to destroy competing private-sector social scoring systems.
     
  • Social scoring by private-sector actors such as digital platforms could be used to subvert democracy in non-autocratic states; the platform could rate officials to encourage them to grant favors to the platform.
     
  • States can use facial recognition to ensure that noncompliant citizens are ostracized by others, reducing the social scores of those who maintain relationships with dissenters.
     
  • Our digital society will benefit from developing guidelines to ensure that governments and private platforms use and disclose data productively.
     
    • Social scoring systems should not include information about divisive issues.
       
    • Social scoring systems should not rely on guilt by association.
       
    • Regulators should monitor private platforms' political coverage.
       

Get The Article

Find the full article online

Search for Full Article

Share