It’s funny how a set of instructions – an algorithm – written by people can come to be granted, by those same people, a superhuman authority. As if a machine fashioned by man should, upon trembling into motion, shed its earthly origin and assume a god-granted imperium, beyond our small-minded questioning.
Last week, CNET’s Elinor Mills reported on how a web search for “Martin Luther King” returns, as its first result on Google and as its second result on Windows Live Search, a web site (martinlutherking.org) operated by a white supremacist organization named Stormfront. The site, titled “Martin Luther King Jr.: A True Historical Examination,” refers to King as “The Beast” and says he was “just a sexual degenerate, an America-hating Communist, and a criminal betrayer of even the interests of his own people.” The site also features an essay on “Jews & Civil Rights” by former Ku Klux Klan official David Duke.
What’s remarkable, though, is not that a search algorithm might be gamed by extremists but that the owners of the algorithm might themselves defend the offensive result – and reject any attempt to override it as an assault on the “integrity” of their system. AOL, because it subcontracts its search results to Google, finds itself in the uncomfortable position of promoting the white supremacist site to its customers. In response to an inquiry from CNET, the company was quick to distance itself from the search result and to place the responsibility for it on Google:
AOL spokesman Andrew Weinstein said the company has contacted Google about the Martin Luther King search results. “We get all of our organic search results from Google, as you know, so we don’t set the algorithms by which they are ranked. Although we can’t micro-manage billions of search results, our users would not expect this to be the first result for that common search, and we do not want to promote the Web sites of hate organizations, so we have asked Google to remove this particular site from the results it provides to us.”
That seems like an entirely reasonable position. Clearly, a white supremacist site is not the site that any rational person would consider an appropriate recommendation for someone looking for information on a black civil-rights leader. But Google doesn’t seem to agree. In fact, in responding to CNET, it defends the King result as being “relevant to the query” and suggests that it is evidence of the integrity of the Google PageRank algorithm:
At Google, a Web site’s ranking is determined by computer algorithms using thousands of factors to calculate a page’s relevance to any given query, a company representative said. The company can’t tweak the results because of that automation and the need to maintain the integrity of the results, she said. “In this particular example, the page is relevant to the query and many people have linked to it, giving it more PageRank than some of the other pages. These two factors contribute to its ranking,” the representative wrote in an e-mail.
A Microsoft spokesman is even more explicit in asserting that the King result is a manifestation of algorithmic “integrity”:
The results on Microsoft’s search engine are “not an endorsement, in any way, of the viewpoints held by the owners of that content,” said Justin Osmer, senior product manager for Windows Live Search. “The ranking of our results is done in an automated manner through our algorithm which can sometimes lead to unexpected results,” he said. “We always work to maintain the integrity of our results to ensure that they are not editorialized.”
By “editorialized” he seems to mean “subjected to the exercise of human judgment.” And human judgment, it seems, is an unfit substitute for the mindless, automated calculations of an algorithm. We are not worthy to question the machine we have made. It is so pure that even its corruption is a sign of its integrity.
I think that the main reason that the “no human intervention” policy is in place so that they can defend themselves from anyone asking for the search results to be manipulated, which is what it essentially boils down. It’s quite similiar to the ISPs “common carrier” defense — you don’t want to be legally liable for the order (or content) of the sites returned by a search.
For example, let’s say a particular product is bad, and someone puts up a site detailing all the problems about it and it hits #1 in Google. What is to prevent the manufacturer to legally go after Google to suppress that result? The policy helps them in that case.
I don’t think it’s about legal liability as much as it’s about not having to pay people to manually vet the results.
Or perhaps when Google says that the site is relevant to the query they’re absolutely right. When you start trying to sanitize the information that surrounds something as heated as civil rights, bigotry and racism, you start glossing over the real problems. By returning this site in it’s search results Google is simply replicating the state of affairs and providing the user with a first hand example of the kind of ignorance and hate that King himself dealt with on a daily basis. To shield a user from that fact… and from the fact that these people still exist in frighteningly large numbers… would be the truly irresponsible act.
I think Google and Microsoft are acting from a reasonable position. It’s a slippery slope when they start editing random issues. This can easily happen since there are likely a number of positive sites watering down each other’s PageRank leaving the sole negative site to rise up.
However, I think it would also be reasonable for them to analyze the sitatuin and really see if they could make some algorithmic improvements. For example, Google would know that a user clicked to the site, then came back and clicked to the next entry and did not come back. That might suggest that the first result wasn’t what they were looking for. Enough people do this, the result should devalue.
Googling link:martinlutherking.org shows who links to the site. It appears to be a popular example of online misinformation. These links from critical authors seem to have boosted its rank — probably not what the authors intended, eh?
If people are so upset about this. They can google bomb other King sites so it’ll bump them up in rankings. Your site has a PR of 7, give them some google juice and the problem is fixed
It very much reminds me of Jaron Lanier’s point about the turing test:
Simon- I think that’s the point: the World According to Google is a popularity contest. Yet it holds enough credibility to upset people. If a human editor were responsible there could be some accounting for it. Since it’s defended as the result of an innocent algorithm we should instead re-evaluate our notions about search engines. GIGOogle? ;-)
Hopefully this discloses the lack of an search algorithm and not of the search base.
See my cartoon.
Bye,
Oliver
There is another, equally valid explanation of a winning computer, however, which is that the human had become less intelligent, less human-like.
Lanier makes a good point. In fact, as I read a longer version of the response from the anonymous Google representative (shared with me by Elinor Mills) I had the sense that it could very well have been generated automatically by a computer: “In response to your below story, Google looks objectively at many factors to determine how to order results. In this particular example, the page is relevant to the query and many people have linked to it, giving it more PageRank than some of the other pages. These two factors contribute to its ranking. A site’s ranking in Google’s search results is automatically determined by computer algorithms using thousands of factors to calculate a page’s relevance to a given query.”
What Google needs is a user option to bury a link, much like Digg’s “Bury Story” feature.
the page is relevant to the query and many people have linked to it
Yes, many people have linked to it not necessarily out of endorsement, but out of ignorance. This is similar to sites on Digg rising to popularity, etc. A network that relies on its members for quality is only going to be as good as the weaker parts of that network. Crowds are not always that wise.
I encountered this particular case, by the way, a few years ago when teaching my Internet and Society course to undergrads. Our class fell on MLK Day and so I told students that instead of meeting (so they could attend univ events), their assignment was to blog about a Web site related to the day. A student linked to and discussed in detail this specific site. It was not clear whether she had realized the origins of the source. I was left in a tricky position. If she was aware of the site’s origins and still decided to blog about it, should I comment? I ended up sending her an email pointing out the origin of the site and leaving it up to her as to whether she wanted to leave it up on her blog or not. She thanked me for the information and admitted to having missed this “detail”. Instead of deleting the post, however, she added an additional entry on her blog describing this entire experience. I think it made for a very educational one in the end and one that we also discussed in class later that week. (Since everyone blogged pseudonymously no one was put on the spot.)
Remember my aphorism here:
Google ranks popularity, not authority
We’re re-iterating the difference between “most popular reference” and “most authorative reference”
It’s the different between “fame” and “infame”.
But do you have any idea what a can worms it would open up to manually change results?
Nick- They clearly use anonymity in order to dodge accountability. Or, maybe their faceless appeal is a clue to a larger mystery: Google isn’t only run and managed by robots — it is used by robots. And consider: what use does a super-race of robots have for civil liberties anyhow? They truly don’t understand the issue at hand. ;-)
Not thinking about manually changing results, really. From what I glean from http://www.digg.com/faq, it seems the “Bury Story” feature is automated and rule-based.
Last.fm has the “Ban” button, which seems at this point to be relevant only to a user’s own preferences (i.e., you’re banning the tune from ever being played again on your device), but could have utility community-wide. Google could put a “Ban” button on its toolbar, and quantify the number of times users clicked “Ban” after retrieving a result that wasn’t useful.
You could make it an option to retrieve only results that were filtered with “Ban” button feedback. Those who want unfiltered results could opt out.
The problem here is expecting google not to be google, to expect it not to use its own stated method of operation. To have some “expert” decide which results are more truthy than others not only invites liability, but it also sets up the same problem you complain about now but just from a different angle. File this article under “french military victories”.
I find it hard to believe that Google or Microsoft have never intervened to remove an offensive search result. Given their boilerplate response, it seems likely that no human at Google has even looked at this yet. I bet with more heat someone would perk up and make the change quietly. Of course they’d probably do it algorithmically with some clever tweak, lest they sacrifice their silly robot purity.
Seth makes a good point — would you rather that Google just decided by fiat which King site should get the top rank? The algorithm is not the culprit — all it does is reflect the behaviour of linkers on the Internet. It’s called voting with your feet (or your links). I know you’re not a big fan of democracy though, Nick, so I don’t expect that argument to persuade you :-)
By the way, see my report:
“Jew Watch”, Google, and Search Engine Optimization
for discussion about an earlier similar incident.
Is this site truly popular? Many people are defending the *principle* that search sites should return sites that are popular.
It does not look like this site is actually the most popular Martin Luther King site in the world. The vast majority of people who talk about MLK do not flame him, so why would they read or link to this site? Further, the site itself indicates that it presents an alternative history. This site calls itself “a true historical examination”.
It looks like Google and Microsoft have both been gamed. They need to improve their algorithm.
There is plenty of evidence that Google is willing to edit search results – at least sometimes.
This comment from elsewhere says it all (though I would love to see a complete transcription of Brin’s remarks at the conference in question):
http://www.hyperorg.com/blogger/mtarchive/002542.html#comment-32779
“I seem to remember when we heard Sergey Brin speak at Supernova in Santa Clara a year and half ago, he talked about how Google does look at the results and tweaks them in some cases. If memory serves, the two specific examples he gave were suicide and heart attack, where Google looked at the top results generated algorithmically, then tweaked to make sure that a suicide prevention hotline came up on the top for suicide, and a page about what to do if you feel you are having a heart attack comes up for the second. A quick Google shows results consistent with my memory — if you would like me to check my notes to find out exactly what he said, I can.
So it sounds like Google thinks some things are important enough to ignore the algorithms, and some aren’t. I’d ask Google how do those decisions get made? And how does the fact that some of these rankings (e.g. miserable failure, Jew) are the results of gaming the algorithms influence whether they adjust the results or not.
Once Google admit to having changed anything, as they did, then they really have opened Pandora’s box for themselves.
I’m glad to see someone trying to keep them honest.”
“What Google needs is a user option to bury a link, much like Digg’s “Bury Story” feature.”
The “burry” feature will not work. We would create a new “arms race” where someone (person or organization) with enough resources and determination can burry any link.
In the end, what Elinor asks is a kind of “safe search” feature where ceratain sites are filtered out based on their content. We had a similar discussion some time ago when talking about wikipedians “inclusionists” and “deletionists”.
Solving the issue of popularity versus authority, good versus bad and so on requires a third party solution. Rating agencies will need to appear, where humans vet the quality of sites and then internet users choose what filtering mechanism they want to be applied to their search results.
Practically we will satisfy our urge to have fast answers by delegating our responsibility to decide what is relevant to a third party.
Does anyone else see the irony in a company who’s motto is “Do no evil” is suggesting that their computer algorithm (which has no notions of good or evil) should control the dispensation of their signature product?
Maybe Google should start thinking on how to embed Asimov’s laws into their algorithms.
The first law
“A robot may not injure a human being, or, through inaction, allow a human being to come to harm”
can be rewritten as “An algoritm may not mislead a human being, or incite him/her in harming others”
SF becomes reality sooner than we expected
Wake up people! The results in this case, in any case, are about relevance… not popularity… and certainly not about making moral judgments on right/wrong or good/evil. The only question on the table is whether or not the presence of hate speech is relevant to MLK’s life struggle. I tend to think it most definitely is.
In simply negating what King fought against we limit any chance at achieving what he fought for. Would we be any better off if this site were to be pushed back into the shadows where only the dark depressed and disturbed dare seek it out? Is it not better to bring this type of overt racism into the sunlight where discussion and dialogue can expose it for what it is? Shouldn’t the open air of public debate be the tonic which we apply to the bleeding sores of these ideas?
Perhaps I’m wrong… perhaps we should sweep this type of thing under the rug. But if we do… if we have Google change their algorithm to protect us and insulate us from this kind of vile hatred… aren’t we then yielding REAL power to the computer?