The Black Box Society

Privacy and Security and Artificial Intelligence

Article Snapshot


Frank Pasquale


Harvard University Press, 2015


Financial service providers, insurers, and Internet companies collect vast amounts of data, but we know little about how they use this data to make decisions. New legal strategies are needed to check abuses.

Policy Relevance

Disclosure requirements alone are not enough. Citizens should live in an intelligible society.

Main Points

  • Internet-based and financial firms accumulate vast amounts of data about consumers to make key decisions about credit, health care, and employment, but we know little about how their technologies work.
  • Search engines and social networks describe the algorithms they use to rank search results and distribute content as neutral, but we cannot verify those claims; critics note that Google has downgraded rivals in search results.
  • Opaque models are widespread in financial markets, and the failure of these systems caused financial collapse from 2007-2008; some obfuscation conceals illegal activity, while some arise from overwhelming complexity.
  • New legal strategies could limit the worst abuses of financial companies and large online platforms; however, transparency alone is not enough, especially in the financial sector.
    • Government surveillance should be subject to comprehensive oversight.
    • Antitrust investigations could spur Google, Apple, and other firms to reveal how their ranking systems operate.
    • Regulators could hire private-sectors expertise to analyze and monitor complex systems for failure.
  • First Amendment rights of free speech should not insulate online firms like Google from responsibility for activity that borders on fraud.
  • Platforms like Google, Facebook, and Microsoft should be required to address users’ claims of harm at independent arbitration panels.
  • Industry representatives gain power in administrative agencies and alter agencies’ agendas; to end this dynamic, services like credit reporting or online search could be provided by public agencies.
  • Educated citizens should understand the companies that influence our government and culture.

Get The Article

Find the full article online

Search for Full Article