Jonathan Zittrain Explores Whether to Trust or Resist Clever Computer Algorithms

By TAP Staff Blogger

Posted on August 12, 2015

Harvard law and computer science professor Jonathan Zittrain delves into the impact of computer algorithms on Internet users. He asks in what situations is manipulation of algorithms acceptable. And going further, Professor Zittrain questions the importance of transparency: should people on the receiving end of an algorithm’s output know the specifics of how it was designed to work?

On the occasion of his appointment as the George Bemis Professor of International Law at Harvard Law School, Professor Zittrain delivered a lecture entitled, “Love the Processor, Hate the Process: The Temptations of Clever Algorithms and When to Resist Them.” In this talk, Professor Zittrain raised a handful of scenarios that asked the following questions:
  • Once a search algorithm is created, are the results of the algorithm never to be tested or questioned? What about in cases where a search engine displays results to a key word search that may be inaccurate or illegal?  
  • Is it ever okay for a business or government leader to alter an algorithm with the intention of manipulating people’s actions?
  • Would it be okay for the results of a search algorithm to be altered if the results are believed to be harmful to individuals?

This post focuses on the first part of Professor Zittrain’s talk: his examination of search engine and Facebook newsfeed algorithms. Online searches rely on algorithms to provide a list of results based on the key words entered by a user. Algorithms also determine which of the web pages will be shown on the top of the list of results. Additionally, Facebook uses an algorithm for its newsfeed that determines what posts and ads to populate a user’s screen with. These are only two examples of algorithms most Internet users encounter daily.

Below are excerpts from the first part of Professor Zittrain’s lecture, “Love the Processor, Hate the Process: The Temptations of Clever Algorithms and When to Resist Them.”

What I thought I’d talk about today would be some of the joy and excitement I’m feeling in the face of a new set of really puzzling challenges that feels like overnight, but probably more accurately in the span of 6-18 months, have come into the view of multiple people simultaneously – not just within law, but across multiple disciplines – as we’re all trying to figure this thing out.

One kind of anxiety of our time that will definitely date this talk as circa 2015 are the worries around what artificial intelligence is going to do to us. I want to align myself with those who don’t see AI as a big threat. I’m actually more optimistic than I am pessimistic about it. But this kind of existential zonks, second only to zombies these days, is capturing a set of anxieties within us that have to do with technology, autonomy, power, and control. And those are feelings that aren’t coming from nowhere. Those are feelings that really are coming from the development of technology –wholly apart from whether there’s going to be The Terminator as a documentary.

And so I want to situate these anxieties in a much more present and concrete context. So here’s present and concrete [slide shows Google’s page-rank algorithm]. This is it: the page-rank algorithm. This is what Google originally used in its nut to figure out when crawling a ton of web pages all over the place, what to rank what so that when you put in a search term, something would come back.

Its parallel these days, the Facebook newsfeed algorithm. So secret, I don’t have anything to put up on the screen other than its result, rather than the recipe. And for those of you who have been working on that book for a while and not doing anything else, the Facebook newsfeed will populate your screen with all sorts of stuff, one thing after another that might interest you.

I want to now run through basically 5 ½ hypothetical or not so hypothetical situations and capture the zeitgeist of this room of 2015 as to how we feel about each of these.

1st Hypothetical – Facebook Newsfeed and a Presidential Election
This is Facebook-newsfeed oriented. And of course the feed doesn’t just show you stuff on the basis of being off in a corner. It’s learning about you as you act on or near Facebook. And in this instance, Facebook can predict when two Facebook members are about to enter into a relationship –possibly before even they know it.

Here's the hypothetical. Suppose that it's the 2016 presidential election and Facebook has a firm view as a company, as represented, say, through Mark Zuckerberg, about who should win the election. Mark decides to put reminders in the feeds of, take your pick, Republicans or Democrats, whoever you don't like most... and alerts them to go to the polls and it says nothing to those who you like and who Facebook doesn't. Let us suppose, as a result, we can document that the election outcome was changed. So here's when I'm going to call for a hum. At the count of three, if you think that that would be an awful thing to do, Facebook absolutely should not do that and if it did do it, they should possibly be in line for some form of punishment, let me know with a hum on the count of three. One, two, three. [humming] It's a somewhat uncertain hum but consistent. If you think that's perfectly fine, electioneering is the name of the game, let the good times roll, Facebook is entitled to do what it pleases, let me know with a hum. One, two, three. [humming – though much softer] it sounds like pretty lopsided, not happy with that. Let's move on.

2nd Hypothetical – Facebook Newsfeed and a Government Request to Show Less Violence in Video Feeds
Here's the next hypothetical and this was highlighted by a Berkman fellow, Zeynep Tufekci, thinking about the two events that consumed us last summer as a nation. First, the riots in Ferguson and the after effects. And second, the Ice Bucket Challenge.

Here's the hypothetical. Suppose it's the summer of 2015, and it's not Ferguson, it's some other town and we don't even know why, but there is unrest in that town. You are at Facebook and a law enforcement official, perhaps the new U.S. Attorney General calls and says, "I'm not gonna order you to do anything, but I want you to know that unrest is looking like it might spread to other cities. And we have reason to think that when people are uploading videos, well short of incitement, but still that are showing unrest in this town, it might spread to other towns. We are asking, as a civic duty and the interests of protecting safety, physical safety, we are asking you to do a few fewer shares in news feeds of video of violence taking place in this town and a little bit more of the ice bucket challenge of the day." Okay, that's the question. How many people say, "Absolutely not, I will not do that, whatever this roulette wheel of an algorithm that even I don't understand spits out is what they're gonna get by gum, I won't hand tweak it." How many say that? One, two, three. [humming] Alright, very confident humming now. How many people are like, "I am open to that discussion because I believe in saving lives." One, two, three. [laughter] If you're laughing while humming it's hard to hear, but... Less enthusiasm about that.

3rd Hypothetical – Google Search Results Showing an Anti-Semitic Site
For the longest time, if you performed a search in Google, not on the word, Jewish, but on the word, Jew, one of the top hits, in this case the second hit, would be the site, The most comprehensive and easiest to use, 'cause sometimes websites are hard to use, website dedicated to both current and historical Jewish news organizations, et cetera. You have the famous Jews list, Jewish entertainment, Zionist occupied governments, but wait a minute. This is an anti-Semitic site.

Some people noticed that and complained to Google. … Google took firm action. They bought themselves an ad, at their own prices, on their own service, and said, "We're disturbed about these results as well. Please read our note about it". If you click through you get a message from Google that's like, "We feel terrible, this is not the Google we strive to be. And yet, we couldn't possibly do anything about it because that would be messing with the secret algorithm that none of us understands anymore. We won't change the outcome".

Let me again ask. Each time I'm asking the first question it's in favor of non-intervention and the second is in favor of intervention. So how many people would say, "Google did the right thing here by refusing to adjust the position of the anti-Semitic site after the complaint?" One, two, three. [humming] How many people say, "Google should have taken action on the rankings?" One, two, three. [humming – softer than above] Very clear consensus in the room, not unanimity, but consensus that Google did the right thing here.

4th Hypothetical – Google Contradictory Search Results on Child Vaccinations
I Googled on Sunday, "vaccinate my child," and these are the results I got. The first of which is anti-vaccination, the second of which is pro, the third of which is anti, the fourth of which is pro, for which you might get a sense that it is completely up in the air as best humanity knows, whether vaccinations are a good idea. Should we ask Google? Should Google think about whether to change the ordering of these results? I'm going to take off the table the idea of buying an ad, being horrified by these results.

So how many people think, "This represents a problem that Google should fix? And I'll just put it on the table, in favor of pushing down the anti-vaccination woo. How many people would like to see that happen?" One, two, three. [very soft humming] Alright, not very much enthusiasm. How many people say, "Let it ride?" One, two, three. [louder humming] Alright, we are status quoists in the room. We like that roulette wheel even though we don't know the odds.

5th Hypothetical – Google De-emphasizing Results
This is, a website with a fairly clear business model. It gathers public domain mugshots that exist from police departments around the nation. It gathers them and information about the people who have been arrested and then it makes clear that a search engine should try to find it. And if you type in this guy's name odds are good, traditionally, that this would appear pretty high up because it's a "relevant" search result on his name. Now, I realize he looks pretty happy. He was booked for marijuana possession. You could imagine that later he regretted it and might not want this as a first hit on his name.

So again I ask, are you open to fixing the roulette wheel a little bit, so that this kind of site doesn't go down. And before you answer, let me remind you that there's a little button here to unpublish the mugshot and it costs only $400 to have your mugshot removed from Of course, that does not cover mugshots with a or any of the other 50 sites, potentially run by the same person, that do exactly this. What do you say? How many people think, "Google should finally violate what you appear to think is its prime directive and change the formula to push down the sites?" One, two, three. [humming] I've got some votes here. And how many people say, "Let it ride?" One, two, three. [humming] We are even on the Mugshots.

It may surprise and amaze you to know that Google finally decided, enough with the mugshots. And not because they were doing some undue search engine optimization, but because it basically did seem an extortion racket. The New York Times had written a story about it. And they just decided, enough. This greatly affected the business model of and similar sites, and made life better perhaps for people who had been arrested and had public mugshots about it.

5th-and-a-Half Hypothetical – Nest Thermostat Working with Local Utility to Lower Everyone’s Thermostat by Half a Degree
The half is grounded in the observation that around the corner, life is less and less going to be, "I search for something and I get results and then I click on stuff." That's very 2005. More and more, it's going to be concierge-like services where you just ask a question of your AI, and it gives you some form of answer back, and it's kind of the answer. … It's the model behind something like the Nest thermostat where you could set it to a temperature, good luck getting it to stay at that temperature, the Nest is just too smart for that. It's going to change the temperature of your house depending on what it infers you probably want at the time, according very little weight to what temperature you set the thermostat at.

So here's the half example. Suppose that Nest takes some money from an energy source … the local utility, to lower every person's thermostat in the jurisdiction by half a degree. Saving a lot of money for the utility, saving maybe some burden on the grid, and that's just one of multiple factors that go in. How many people think, "That's not fair, don't touch the thermostat in that circumstance?" One, two, three. [humming] Oh, anger at saving energy. How many people say, "Bless you, that's a great idea?" One, two, three. [humming] Alright, a few less than that.

Making Sense of the Hums
Alright, we've done a bunch of votes and case studies. Let's now try to sort them out along a spectrum. I've made a spectrum from "Leave the algorithm alone," over to "No, no. What just happened is not good, we should adjust or penalize the algorithm as a result." And here's where I think we ended up in our voting:
  • For the search on Jew within Google ... that's where people were most supportive of Google's action and in fact doing nothing, leave the algorithm alone.
  • For "vaccinate my child" which have results that include inaccurate stuff, there was a little more enthusiasm for it but not much.
  • For the "Mugshots being de-emphasized," I think I heard more support for Google's ultimate decision there, probably more after you found out that they made that decision.
  • And for Ferguson or another city having its updates about violence being de-emphasized, whoa, do not want to see Facebook even in the name of public safety making changes.
  • And finally, for "Facebook putting a thumb on the scale on election," that was the third rail. That was where people were like, "That's beyond the pale."

So how do we make sense of our range disclosed as a group of feelings about this? … Here are a couple of ways of thinking about it.

Tool and Friend – One is offered up by Randall Munroe. Randall Munroe is the creator of the xkcd comic. Randall has suggested that maybe the technologies that we're talking about and across these examples, some we think of as tools and some we think of as friends. And the things we think of as tools, we may expect to work a certain way and if they don't, we react in a way different from how we react if our friends don't act a certain way. And these technologies are evolving under our noses, sometimes pushing the line between tool and friend.

Toxicity – If you find yourself disagreeing with where I placed a given example along the spectrum, one account for that might be what I'm calling toxicity. There's a sense that some information or result is sufficiently toxic, however you want to define that, wrong, evil. ... For example, if the Google algorithm were turning up child abuse images in response to searches that you might infer were looking for exactly that, we would probably, I would assume, not have much objection to Google adjusting the algorithm to lose those images and in fact possibly to make reports for those who provide the images online. That's an example to me … of toxicity trumping.

As Professor Zittrain continues his lecture, he presents ideas from several academic colleagues and introduces the work they are doing with algorithms. These scholars include: University of Maryland law professor Frank Pasquale and his new book, The Black Box Society: The Secret Algorithms That Control Money and Information; Christian Sandvig, University of Michigan, focusing on algorithmic discrimination in online content; Nick Diakopoulos, University of Maryland, who has coined the phrase, "Algorithmic Accountability;” Microsoft researcher and visiting scientist at Berkeley’s Simons Institute Cynthia Dwork who is looking for formulas that could prepare data before it goes in to an algorithm to ensure that data of like people will be treated in like ways; and Yale law professor Jack Balkin's idea of information fiduciaries.

Watch the entire lecture: “Love the Processor, Hate the Process: The Temptations of Clever Algorithms and When to Resist Them.”

About Jonathan Zittrain
Jonathan Zittrain is the George Bemis Professor of International Law at Harvard Law School and the Harvard Kennedy School of Government, Professor of Computer Science at the Harvard School of Engineering and Applied Sciences, Vice Dean for Library and Information Resources at the Harvard Law School Library, and co-founder of the Berkman Center for Internet & Society.

Professor Zittrain's research interests include battles for control of digital property and content, cryptography, electronic privacy, the roles of intermediaries within Internet architecture, human computing, and the useful and unobtrusive deployment of technology in education.

He performed the first large-scale tests of Internet filtering in China and Saudi Arabia, and as part of the OpenNet Initiative co-edited a series of studies of Internet filtering by national governments: Access Denied: The Practice and Policy of Global Internet Filtering; Access Controlled: The Shaping of Power, Rights, and Rule in Cyberspace; and Access Contested: Security, Identity, and Resistance in Asian Cyberspace.