p>This month marked another milestone in game show history. On February 14-16, Jeopardy! champions Ken Jennings and Brad Rutter defended their trivial supremacy against Watson, a computer program created by IBM. Drawing on vast stores of
information and a complex algorithm for selecting the most probable answer, Watson all but ran the board on its way to a $1 million prize for IBM, which the company pledged to donate to charity. Yet the hype around Watson’s victory obscured a more important contest—that between humans and our ability to understand our own technology—and we’re not doing so well in that one, either.
The Jeopardy! spectacle brought with it the predictable man-versus-machine narrative that surrounded the 1997 victory of Deep Blue (another IBM creation—what are they up to over in Armonk?) over chess world champion Gary Kasparov. News outlets covered the match with tongue-in-cheek, Terminator-tinged headlines. “‘Jeopardy’ invites supercomputer Watson to destroy all humans,” wrote the Los Angeles Times a few days before the match started. Mashable wrote “IBM’s Watson Dominates Humanity in Jeopardy.”
Soon after the defeat, denial set in. Jennings, who good-naturedly welcomed “our new computer overlords” as he lost the match, said in Slate and elsewhere that Watson’s machine-driven reflexes on the buzzer certainly gave it an edge in the competition, though he stopped short of suggesting it should have been artificially handicapped. In the aftermath of the defeat, however, the Interwebs nonetheless buzzed with accusations of cheating, or, at best, a slanted playing field.
Jennings was probably right about the buzzer, but is this really the peg on which we’re going to hang our collective identity as a species? If Watson had had the reflexes of a mere human, it still would have been a good player. Maybe it would have lost, but even if it had, it seems likely that another year or so of tweaking the algorithm would somehow have managed to close the remaining gap. Buzzer or no buzzer, IBM has its proof of concept. Even if we carbon blobs had managed to hang on to our Jeopardy! supremacy this time, the writing was on the wall.
It’s easy to think of Watson as something like a person. It has not only a name, but a “face” (an avatar capable of displaying context-sensitive “moods”), and a reasonably pleasant synthesized voice. What’s more, it’s really good at Jeopardy!, which is one of those activities (along with playing chess, running mice through mazes, and taking the world hostage with a giant death ray) that are supposed to be the mark of superior intellect.
We humans jump to explain our surroundings in terms of things we know. A thing that talks, displays moods, answers tricky questions—there must be someone in there. Except there isn’t. Watson, like Deep Blue before it, and like your cherished TI-30 before that, is a fancy adding machine. Yes, its algorithm is way more complex. Yes, learns from its mistakes. Yes, when you hear a description of how it arrives at an answer, it sounds vaguely human. However, Watson does not have thoughts. It is not self-aware. It is not a “he,” despite what you may have gleaned from the narration of PBS’s NOVA segment, “Smartest Machine on Earth.” (PBS, I expect more from you.) And, contrary to what you may have heard on the CBS Evening News, Watson is most certainly not “able to understand language with all its nuances,” with “scary implications.” (CBS, I expect a little less from you, but come now.)
Watson is a system constructed and optimized to parse the type of clue encountered in Jeopardy!, come up with the most likely response, and phrase it in the form of a question. This was no small feat, but this formulaic interaction is still a tiny subset of human communication. Watson wouldn’t stand a chance in a five-minute Turing test—another endangered benchmark of human superiority.
Too Much Trust
Today, we user experience types strive to make our interactions with technology more natural and human. We “design for trust.” When you’re creating web content, being conversational is great. The ground rules are clear. Nobody’s going to mistake your “about us” copy for a human tour guide.
Off the web, however, the line is easier to cross. A friend of mine recently got an automated fundraising call from a police foundation. When he said “go on” at a conversational pause, the system decided it didn’t understand him, faked a coughing fit, apologized, and started in again. “Took me a couple times before I realized it was a computer,” he said—and he works in IT.
We’re on the verge of some mind-blowing technology. All the technical innovation of the last thirty years add up to some powerful things behind the interfaces that we design. But with these advances come new ethical dilemmas.
IBM has announced its intent to bring Watson’s artificial brainpower into healthcare as a resource for doctors. The possibilities are exciting, but this high-stakes use raises some questions. What should the experience of using such a system be like? Should you be able to talk to it? Should it have a human name? A face? How hard should the design work to get you to trust it? These belong to a class of issues that designers of IT systems don’t usually have to grapple with, because most systems couldn’t pretend to be intelligent if they tried. Which they can’t.
A medically styled version of Watson might play well with IBM shareholders, but if the trappings of intelligence artificially enhance the system’s credibility by playing off a flawed mental model of how it works, we’ll have
ventured into ethically murky territory. If “Dr. Watson” obscures the data and reasoning that led to a diagnosis with an anthropomorphic abstraction, it doesn’t even rise to murky. IBM clearly considers the ability to converse in natural language to be a core part of the experience. If the interaction is primarily verbal, will the design be
able to convey the level of confidence in its responses? Or will busy doctors learn to trust the calm, reassuring voice?
The Real Competition
The problem is not that we as a species lost a game to a computer. It’s that, by presenting Watson and Deep Blue as an opponents in a competition, we may start thinking
of them as quasi-human entities.
Our actual opponents are ourselves. We’re fooling ourselves into thinking that our technology is something more than a reflection of our own understanding of a problem. If we lose sight of that, we’re going to get what we deserve.
We need to insist on designing transparency into our smart systems. This may feel a little counterintuitive in an era when “simple” is the prevailing aesthetic. Many UXers have a story
in which a client says the new system should be “like an iPod,” not because it has anything to do with music, but because its user interface should be clean and intuitive. And, for our part, many of us have argued with a coder who wanted to base a user interface design on the structure of a database table. The urge to hide the details is hard to resist.
However, there’s a line between presenting information in a way that makes it easy to consume and obscuring the metadata that will allow the user to assess how it should be consumed. The latter is an important task, but one that seems skippable if we convince ourselves that our systems know better than we do.
Ten years ago, the notion that other people would be able to track your movements was a stock element of a dystopian future. Then location-aware services made it fun to let everyone know where you are. Today, most people would hesitate to hand over life-or-death decisions to a machine. However, we may yet find a way to make a game of it.